Add files using upload-large-folder tool

README.md

@@ -0,0 +1,96 @@
+---
+base_model: Qwen/Qwen3.5-4B
+library_name: peft
+license: apache-2.0
+pipeline_tag: image-text-to-text
+datasets:
+- sparklabuofu/MiniWeb
+tags:
+- lora
+- web-agents
+- gui-agents
+- browser-use
+- miniweb
+---
+
+# MiniWebAgent executor (Qwen3.5-4B LoRA)
+
+The executor of **MiniWebAgent**: a LoRA adapter on [Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B) trained on
+MiniWebGen's practice data, so the model performs single web interaction skills (primitives) such as filling a form,
+setting a slider, drawing a signature or uploading a file. In MiniWebAgent the untrained Qwen3.5-4B is the planner: it
+splits a task into primitive-level instructions and hands each one to this executor, which acts in the same browser;
+reasoning steps that only read the page stay with the base model.
+
+Code, benchmark and setup: [github.com/miniweb-site/MiniWeb](https://github.com/miniweb-site/MiniWeb).
+Data: [sparklabuofu/MiniWeb](https://huggingface.co/datasets/sparklabuofu/MiniWeb).
+
+## Results
+
+Task success (%) of Qwen3.5-4B (the paper's Table 3). MiniWeb: the 62 human tasks of the 13 held-out sites, from which
+no training data comes; the other three benchmarks are not trained on either.
+
+| | MiniWeb held-out | WebArena-Lite | WebVoyager | Online-Mind2Web |
+|---|---|---|---|---|
+| Untrained single agent | 38.7 | 27.1 | 45.5 | 17.9 |
+| Untrained planner + untrained executor | 45.2 | 30.7 | 59.9 | 21.8 |
+| **MiniWebAgent** (untrained planner + this executor) | **58.1** | **36.4** | **65.5** | **33.9** |
+
+On the 103 generated single-primitive tasks of the held-out sites, the trained model as a single agent goes from 55.3%
+to 80.3%, with gains in every skill family. On whole tasks it needs the planner: trained mostly on one-primitive
+episodes, the single agent often stops before reaching the skills it learned (39.2% on the held-out tasks, 38.7% untrained).
+
+## Use
+
+Serve the base model with the adapter (vLLM), then run MiniWebAgent with the repository's evaluator:
+
+```bash
+python scripts/fetch_adapters.py          # -> data/webmix/adapters/pooled_v4
+MODEL=Qwen/Qwen3.5-4B SERVED=qwen35-4b PORT=8400 MAX_LORA_RANK=64 \
+    LORA_MODULES=pooled_v4=data/webmix/adapters/pooled_v4 bash src/miniwebagent/scripts/serve.sh
+python -m miniwebagent.evaluate --human --arm agent_planner --agent-version v6 --planner-name qwen35-4b \
+    --adapters none --fallback pooled_v4 --vllm http://127.0.0.1:8400/v1 --ports 8300,8301 --out runs/miniwebagent
+```
+
+With PEFT directly: `PeftModel.from_pretrained(base_model, "sparklabuofu/MiniWebAgent")`.
+
+The executor expects the [browser-use](https://github.com/browser-use/browser-use) agent prompt it was trained on (the
+system prompt, the task, the agent's memory, the indexed interactive elements and the screenshot) and answers with
+its memory and one action. Thinking is off: the chat template renders an empty `<think></think>` block.
+
+## Training
+
+- **Data**: 27,646 steps of MiniWebGen (the dataset's `training_steps` table, rows with `is_target` and not
+  `validation`): 3,128 verified single-primitive trajectories on the 52 training sites, 857 chains of two trajectories
+  and 629 trajectories with an injected and corrected mistake, replayed through the agent harness with the base model
+  writing each step's memory.
+- **Recipe**: rank-64 LoRA (alpha 128, dropout 0.05) on the language model's attention projections (full-attention
+  layers) and MLP projections (every layer); no vision tower. One epoch, AdamW, learning rate 1e-4 with 3% warmup and a
+  cosine decay, 8 steps per optimizer update, cross-entropy on the whole target (memory and action); 4 GPUs, about 5.5
+  hours. `training_meta.json` and `train_log.jsonl` hold the settings and the loss curve.
+
+## Other checkpoints
+
+The paper's other trained executors, in `ablations/` (same base model and LoRA shape; each folder has its own
+`training_meta.json`). `python scripts/fetch_adapters.py --all` downloads them under the names the paper's run commands
+use (`scripts/reproduce.py show <arm>`).
+
+| Folder | What it is | Held-out task success |
+|---|---|---|
+| (root) | MiniWebAgent's executor | 58.1 |
+| `ablations/task_centric` | trained at matched size on task-centric trajectories: InSTA's pipeline reimplemented on the 52 training sites (the dataset's `task_centric_steps`) | 47.8 |
+| `ablations/specialists/{drag,io,select,rest,reasoning}` | one specialist per skill family group (supplementary material): `drag`, `io`, `select` and `rest` start from the main executor and train on their families' steps with a KL penalty toward it; `reasoning` starts from the base model | 52.7 |
+| `ablations/no_done` | the main recipe without each episode's final `done` step, a control for stopping early (single agent) | 34.4 |
+
+All numbers are with the untrained planner except `no_done` (single agent; 39.2 with the `done` steps). The specialists
+are served under their family names and picked by the planner's family for each subtask:
+`LORA_MODULES='drag=.../drag io=.../io select=.../select rest=.../rest reasoning=.../reasoning pooled_v4=...'` and
+`--adapters drag,io,select,rest,reasoning --fallback pooled_v4`.
+
+## Limitations
+
+Trained only on MiniWeb's synthetic sites and English instructions, mostly on single primitives: on whole tasks it
+relies on the planner for their structure. Outside MiniWeb it has been evaluated only on the three benchmarks above.
+
+## License
+
+Apache 2.0, as the base model.

ablations/no_done/adapter_config.json

@@ -0,0 +1,51 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "kasa_config": null,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.21.0",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "k_proj",
+    "v_proj",
+    "o_proj",
+    "gate_proj",
+    "up_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

ablations/no_done/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:791894dfa6b36d961a51eb4f1db276829363bdea70a3231fe24feb08bed4c1a5
+size 339776632

ablations/no_done/train_log.jsonl

@@ -0,0 +1,117 @@
+{"rows": 200, "of": 23288, "epoch": 0, "ce": 0.34716625526547434, "kl": 0.0, "lr": 2.9879619171899946e-05, "rows_per_s": 0.8101114764323301, "tokens_per_s": 4880.937847734317, "gpu_gb": 15.217637538909912}
+{"rows": 400, "of": 23288, "epoch": 0, "ce": 0.21411475408822297, "kl": 0.0, "lr": 5.857802768647396e-05, "rows_per_s": 1.0397079894041876, "tokens_per_s": 6375.042316591035, "gpu_gb": 15.331640243530273}
+{"rows": 600, "of": 23288, "epoch": 0, "ce": 0.18358522038906813, "kl": 0.0, "lr": 8.721332215053629e-05, "rows_per_s": 1.1027840850521553, "tokens_per_s": 6750.08992134694, "gpu_gb": 15.538420677185059}
+{"rows": 800, "of": 23288, "epoch": 0, "ce": 0.23809295322746038, "kl": 0.0, "lr": 9.970910665741333e-05, "rows_per_s": 1.1715920324553812, "tokens_per_s": 7171.8830527951295, "gpu_gb": 15.585730075836182}
+{"rows": 1000, "of": 23288, "epoch": 0, "ce": 0.20344223260879515, "kl": 0.0, "lr": 9.954572729398464e-05, "rows_per_s": 1.2154065614076905, "tokens_per_s": 7333.695128766566, "gpu_gb": 15.585730075836182}
+{"rows": 1200, "of": 23288, "epoch": 0, "ce": 0.20644701696932316, "kl": 0.0, "lr": 9.934628382092365e-05, "rows_per_s": 1.2396155210357884, "tokens_per_s": 7498.252476469066, "gpu_gb": 15.98822832107544}
+{"rows": 1400, "of": 23288, "epoch": 0, "ce": 0.20819568622857332, "kl": 0.0, "lr": 9.911092141222762e-05, "rows_per_s": 1.2517078860934598, "tokens_per_s": 7622.511208710358, "gpu_gb": 15.98822832107544}
+{"rows": 1600, "of": 23288, "epoch": 0, "ce": 0.2279452510178089, "kl": 0.0, "lr": 9.883981138712374e-05, "rows_per_s": 1.2695430899465547, "tokens_per_s": 7774.723142983125, "gpu_gb": 15.98822832107544}
+{"rows": 1800, "of": 23288, "epoch": 0, "ce": 0.2094112790375948, "kl": 0.0, "lr": 9.853315108536666e-05, "rows_per_s": 1.2832299127344662, "tokens_per_s": 7852.112352242481, "gpu_gb": 15.98822832107544}
+{"rows": 2000, "of": 23288, "epoch": 0, "ce": 0.2007575837522745, "kl": 0.0, "lr": 9.819116372359575e-05, "rows_per_s": 1.2958339655884625, "tokens_per_s": 7948.925445056197, "gpu_gb": 15.98822832107544}
+{"rows": 2200, "of": 23288, "epoch": 0, "ce": 0.20076242120936513, "kl": 0.0, "lr": 9.781409823285675e-05, "rows_per_s": 1.3042666099772364, "tokens_per_s": 8005.3560554443175, "gpu_gb": 15.98822832107544}
+{"rows": 2400, "of": 23288, "epoch": 0, "ce": 0.2138189042173326, "kl": 0.0, "lr": 9.740222907740604e-05, "rows_per_s": 1.3134456982565312, "tokens_per_s": 8058.943796011218, "gpu_gb": 15.98822832107544}
+{"rows": 2600, "of": 23288, "epoch": 0, "ce": 0.20103268582373857, "kl": 0.0, "lr": 9.695585605492922e-05, "rows_per_s": 1.313301536467747, "tokens_per_s": 8079.839186057835, "gpu_gb": 15.98822832107544}
+{"rows": 2800, "of": 23288, "epoch": 0, "ce": 0.20434669858776033, "kl": 0.0, "lr": 9.64753040783199e-05, "rows_per_s": 1.3177628174987666, "tokens_per_s": 8108.258621692475, "gpu_gb": 15.98822832107544}
+{"rows": 3000, "of": 23288, "epoch": 0, "ce": 0.19475500810891389, "kl": 0.0, "lr": 9.596092293917696e-05, "rows_per_s": 1.3269435737244792, "tokens_per_s": 8134.192415060631, "gpu_gb": 15.98822832107544}
+{"rows": 3200, "of": 23288, "epoch": 0, "ce": 0.18748346975073218, "kl": 0.0, "lr": 9.541308705319289e-05, "rows_per_s": 1.3306623015585655, "tokens_per_s": 8154.38008701686, "gpu_gb": 15.98822832107544}
+{"rows": 3400, "of": 23288, "epoch": 0, "ce": 0.19986288744956254, "kl": 0.0, "lr": 9.48321951876183e-05, "rows_per_s": 1.3356691030028494, "tokens_per_s": 8171.457006377412, "gpu_gb": 15.98822832107544}
+{"rows": 3600, "of": 23288, "epoch": 0, "ce": 0.18925629185512663, "kl": 0.0, "lr": 9.421867017100106e-05, "rows_per_s": 1.3390988681061815, "tokens_per_s": 8186.692521739483, "gpu_gb": 16.05839729309082}
+{"rows": 3800, "of": 23288, "epoch": 0, "ce": 0.21570354223251342, "kl": 0.0, "lr": 9.35729585854113e-05, "rows_per_s": 1.3444962621712806, "tokens_per_s": 8218.222080448166, "gpu_gb": 16.05839729309082}
+{"rows": 4000, "of": 23288, "epoch": 0, "ce": 0.17278008027002215, "kl": 0.0, "lr": 9.289553044137641e-05, "rows_per_s": 1.3468563560623668, "tokens_per_s": 8246.398758119409, "gpu_gb": 16.05839729309082}
+{"rows": 4200, "of": 23288, "epoch": 0, "ce": 0.19036508109420539, "kl": 0.0, "lr": 9.218687883576238e-05, "rows_per_s": 1.3502002420547135, "tokens_per_s": 8252.925382385458, "gpu_gb": 16.121848583221436}
+{"rows": 4400, "of": 23288, "epoch": 0, "ce": 0.17014498931355773, "kl": 0.0, "lr": 9.144751959285101e-05, "rows_per_s": 1.349605002111807, "tokens_per_s": 8247.607935814613, "gpu_gb": 16.121848583221436}
+{"rows": 4600, "of": 23288, "epoch": 0, "ce": 0.18903744788840413, "kl": 0.0, "lr": 9.067799088887364e-05, "rows_per_s": 1.3501924593846688, "tokens_per_s": 8258.176268932453, "gpu_gb": 16.121848583221436}
+{"rows": 4800, "of": 23288, "epoch": 0, "ce": 0.19422986574470996, "kl": 0.0, "lr": 8.987885286027511e-05, "rows_per_s": 1.3523096866134388, "tokens_per_s": 8236.151992340041, "gpu_gb": 16.121848583221436}
+{"rows": 5000, "of": 23288, "epoch": 0, "ce": 0.19265956941060722, "kl": 0.0, "lr": 8.905068719599311e-05, "rows_per_s": 1.3543201899718058, "tokens_per_s": 8261.81579460491, "gpu_gb": 16.121848583221436}
+{"rows": 5200, "of": 23288, "epoch": 0, "ce": 0.19548364244401456, "kl": 0.0, "lr": 8.819409671404933e-05, "rows_per_s": 1.3576527058530397, "tokens_per_s": 8287.257280931657, "gpu_gb": 16.121848583221436}
+{"rows": 5400, "of": 23288, "epoch": 0, "ce": 0.2127265632711351, "kl": 0.0, "lr": 8.730970492276097e-05, "rows_per_s": 1.3564331491641322, "tokens_per_s": 8295.966240359043, "gpu_gb": 16.121848583221436}
+{"rows": 5600, "of": 23288, "epoch": 0, "ce": 0.1779504157230258, "kl": 0.0, "lr": 8.63981555668919e-05, "rows_per_s": 1.359049691576303, "tokens_per_s": 8303.487829350606, "gpu_gb": 16.121848583221436}
+{"rows": 5800, "of": 23288, "epoch": 0, "ce": 0.19717831544578077, "kl": 0.0, "lr": 8.546011215907388e-05, "rows_per_s": 1.3597813880375607, "tokens_per_s": 8314.386110496467, "gpu_gb": 16.121848583221436}
+{"rows": 6000, "of": 23288, "epoch": 0, "ce": 0.18731762662529947, "kl": 0.0, "lr": 8.449625749683847e-05, "rows_per_s": 1.361221426000124, "tokens_per_s": 8324.466142456298, "gpu_gb": 16.121848583221436}
+{"rows": 6200, "of": 23288, "epoch": 0, "ce": 0.19575696093030273, "kl": 0.0, "lr": 8.350729316561209e-05, "rows_per_s": 1.3634717432634622, "tokens_per_s": 8329.869356688929, "gpu_gb": 16.121848583221436}
+{"rows": 6400, "of": 23288, "epoch": 0, "ce": 0.17651576897129417, "kl": 0.0, "lr": 8.249393902803499e-05, "rows_per_s": 1.3634043118580492, "tokens_per_s": 8323.509188783933, "gpu_gb": 16.121848583221436}
+{"rows": 6600, "of": 23288, "epoch": 0, "ce": 0.1740188127569854, "kl": 0.0, "lr": 8.145693269997658e-05, "rows_per_s": 1.365001131353449, "tokens_per_s": 8333.032433937322, "gpu_gb": 16.121848583221436}
+{"rows": 6800, "of": 23288, "epoch": 0, "ce": 0.1898169408738613, "kl": 0.0, "lr": 8.039702901362819e-05, "rows_per_s": 1.3647110914897769, "tokens_per_s": 8333.857942034942, "gpu_gb": 16.121848583221436}
+{"rows": 7000, "of": 23288, "epoch": 0, "ce": 0.19095598977059125, "kl": 0.0, "lr": 7.931499946806417e-05, "rows_per_s": 1.366227317354932, "tokens_per_s": 8332.956890818461, "gpu_gb": 16.121848583221436}
+{"rows": 7200, "of": 23288, "epoch": 0, "ce": 0.18435008518397808, "kl": 0.0, "lr": 7.821163166767115e-05, "rows_per_s": 1.3661488392754944, "tokens_per_s": 8342.520707061847, "gpu_gb": 16.121848583221436}
+{"rows": 7400, "of": 23288, "epoch": 0, "ce": 0.18096623800694941, "kl": 0.0, "lr": 7.708772874885446e-05, "rows_per_s": 1.3658879637709702, "tokens_per_s": 8348.403215884542, "gpu_gb": 16.398165225982666}
+{"rows": 7600, "of": 23288, "epoch": 0, "ce": 0.1753809541091323, "kl": 0.0, "lr": 7.594410879543877e-05, "rows_per_s": 1.3688325505510093, "tokens_per_s": 8362.458129500721, "gpu_gb": 16.398165225982666}
+{"rows": 7800, "of": 23288, "epoch": 0, "ce": 0.17033784243278205, "kl": 0.0, "lr": 7.478160424318858e-05, "rows_per_s": 1.3691197434145912, "tokens_per_s": 8365.049212539847, "gpu_gb": 16.398165225982666}
+{"rows": 8000, "of": 23288, "epoch": 0, "ce": 0.1664767140895128, "kl": 0.0, "lr": 7.3601061273882e-05, "rows_per_s": 1.3684969615298412, "tokens_per_s": 8372.328245191897, "gpu_gb": 16.398165225982666}
+{"rows": 8200, "of": 23288, "epoch": 0, "ce": 0.18192836705595256, "kl": 0.0, "lr": 7.240333919937893e-05, "rows_per_s": 1.3709520822951118, "tokens_per_s": 8382.637019167076, "gpu_gb": 16.398165225982666}
+{"rows": 8400, "of": 23288, "epoch": 0, "ce": 0.18850105555728078, "kl": 0.0, "lr": 7.118930983613183e-05, "rows_per_s": 1.3737566617739245, "tokens_per_s": 8394.65277495258, "gpu_gb": 16.398165225982666}
+{"rows": 8600, "of": 23288, "epoch": 0, "ce": 0.17768658720888197, "kl": 0.0, "lr": 6.995985687059452e-05, "rows_per_s": 1.3746494209450582, "tokens_per_s": 8392.891349732516, "gpu_gb": 16.398165225982666}
+{"rows": 8800, "of": 23288, "epoch": 0, "ce": 0.19153416998684406, "kl": 0.0, "lr": 6.871587521599073e-05, "rows_per_s": 1.376961601386744, "tokens_per_s": 8414.522217678666, "gpu_gb": 16.398165225982666}
+{"rows": 9000, "of": 23288, "epoch": 0, "ce": 0.19374615365639328, "kl": 0.0, "lr": 6.745827036091096e-05, "rows_per_s": 1.377440509605943, "tokens_per_s": 8433.137090532697, "gpu_gb": 16.398165225982666}
+{"rows": 9200, "of": 23288, "epoch": 0, "ce": 0.19472460869699718, "kl": 0.0, "lr": 6.618795771021142e-05, "rows_per_s": 1.3768767654228564, "tokens_per_s": 8431.822099817004, "gpu_gb": 16.398165225982666}
+{"rows": 9400, "of": 23288, "epoch": 0, "ce": 0.1575007288530469, "kl": 0.0, "lr": 6.49058619186949e-05, "rows_per_s": 1.3766073083224537, "tokens_per_s": 8435.2860550891, "gpu_gb": 16.398165225982666}
+{"rows": 9600, "of": 23288, "epoch": 0, "ce": 0.1743422680068761, "kl": 0.0, "lr": 6.361291621805889e-05, "rows_per_s": 1.3766548754924819, "tokens_per_s": 8435.030001812167, "gpu_gb": 16.398165225982666}
+{"rows": 9800, "of": 23288, "epoch": 0, "ce": 0.17363572431728244, "kl": 0.0, "lr": 6.231006173760033e-05, "rows_per_s": 1.3758807299589941, "tokens_per_s": 8430.879332767074, "gpu_gb": 17.460142135620117}
+{"rows": 10000, "of": 23288, "epoch": 0, "ce": 0.16873511981219053, "kl": 0.0, "lr": 6.099824681917201e-05, "rows_per_s": 1.377323048203962, "tokens_per_s": 8443.769299406351, "gpu_gb": 17.460142135620117}
+{"rows": 10200, "of": 23288, "epoch": 0, "ce": 0.20255186785012483, "kl": 0.0, "lr": 5.9678426326888716e-05, "rows_per_s": 1.3785188091225915, "tokens_per_s": 8448.30117531283, "gpu_gb": 17.460142135620117}
+{"rows": 10400, "of": 23288, "epoch": 0, "ce": 0.1683292843773961, "kl": 0.0, "lr": 5.83515609520861e-05, "rows_per_s": 1.378329969647078, "tokens_per_s": 8447.936530390514, "gpu_gb": 17.460142135620117}
+{"rows": 10600, "of": 23288, "epoch": 0, "ce": 0.18674072291702032, "kl": 0.0, "lr": 5.7018616514037726e-05, "rows_per_s": 1.3791718565013258, "tokens_per_s": 8453.130106365596, "gpu_gb": 17.460142135620117}
+{"rows": 10800, "of": 23288, "epoch": 0, "ce": 0.1757089355122298, "kl": 0.0, "lr": 5.56805632569397e-05, "rows_per_s": 1.3791833109986065, "tokens_per_s": 8447.995818284326, "gpu_gb": 17.460142135620117}
+{"rows": 11000, "of": 23288, "epoch": 0, "ce": 0.1819397346302867, "kl": 0.0, "lr": 5.4338375143674236e-05, "rows_per_s": 1.3787172772842933, "tokens_per_s": 8450.4178926753, "gpu_gb": 17.460142135620117}
+{"rows": 11200, "of": 23288, "epoch": 0, "ce": 0.1579248794168234, "kl": 0.0, "lr": 5.299302914686657e-05, "rows_per_s": 1.378576019989082, "tokens_per_s": 8447.57117588241, "gpu_gb": 17.460142135620117}
+{"rows": 11400, "of": 23288, "epoch": 0, "ce": 0.16158151485025882, "kl": 0.0, "lr": 5.164550453775103e-05, "rows_per_s": 1.3788756412783465, "tokens_per_s": 8458.524416992383, "gpu_gb": 17.460142135620117}
+{"rows": 11600, "of": 23288, "epoch": 0, "ce": 0.142861102912575, "kl": 0.0, "lr": 5.029678217336394e-05, "rows_per_s": 1.3801332520523086, "tokens_per_s": 8454.655373982898, "gpu_gb": 17.460142135620117}
+{"rows": 11800, "of": 23288, "epoch": 0, "ce": 0.14278876640833915, "kl": 0.0, "lr": 4.89478437825822e-05, "rows_per_s": 1.3803876998692788, "tokens_per_s": 8457.313970539813, "gpu_gb": 17.460142135620117}
+{"rows": 12000, "of": 23288, "epoch": 0, "ce": 0.17230502504855394, "kl": 0.0, "lr": 4.759967125152738e-05, "rows_per_s": 1.3813938160613035, "tokens_per_s": 8467.296679220663, "gpu_gb": 17.460142135620117}
+{"rows": 12200, "of": 23288, "epoch": 0, "ce": 0.15234414221718906, "kl": 0.0, "lr": 4.6253245908855215e-05, "rows_per_s": 1.3821762217539488, "tokens_per_s": 8468.969843887775, "gpu_gb": 17.460142135620117}
+{"rows": 12400, "of": 23288, "epoch": 0, "ce": 0.1532378008030355, "kl": 0.0, "lr": 4.490954781145097e-05, "rows_per_s": 1.3827274149719408, "tokens_per_s": 8469.366437540815, "gpu_gb": 17.460142135620117}
+{"rows": 12600, "of": 23288, "epoch": 0, "ce": 0.16508646093308926, "kl": 0.0, "lr": 4.3569555031050494e-05, "rows_per_s": 1.384038769048128, "tokens_per_s": 8476.243149393005, "gpu_gb": 17.460142135620117}
+{"rows": 12800, "of": 23288, "epoch": 0, "ce": 0.1611585654458031, "kl": 0.0, "lr": 4.22342429423062e-05, "rows_per_s": 1.3845351150622542, "tokens_per_s": 8474.658963192496, "gpu_gb": 17.460142135620117}
+{"rows": 13000, "of": 23288, "epoch": 0, "ce": 0.1255227701831609, "kl": 0.0, "lr": 4.090458351281634e-05, "rows_per_s": 1.384635272285666, "tokens_per_s": 8474.216248654044, "gpu_gb": 17.460142135620117}
+{"rows": 13200, "of": 23288, "epoch": 0, "ce": 0.1769755914248526, "kl": 0.0, "lr": 3.958154459563421e-05, "rows_per_s": 1.3850431830679024, "tokens_per_s": 8475.07504009194, "gpu_gb": 17.460142135620117}
+{"rows": 13400, "of": 23288, "epoch": 0, "ce": 0.16499804168008267, "kl": 0.0, "lr": 3.826608922477228e-05, "rows_per_s": 1.3858878825782739, "tokens_per_s": 8479.94843623467, "gpu_gb": 17.460142135620117}
+{"rows": 13600, "of": 23288, "epoch": 0, "ce": 0.16155777053907514, "kl": 0.0, "lr": 3.695917491421416e-05, "rows_per_s": 1.3864581520212165, "tokens_per_s": 8482.499814426239, "gpu_gb": 17.460142135620117}
+{"rows": 13800, "of": 23288, "epoch": 0, "ce": 0.16685899928212167, "kl": 0.0, "lr": 3.566175296094445e-05, "rows_per_s": 1.3864692051612593, "tokens_per_s": 8486.698567331649, "gpu_gb": 17.460142135620117}
+{"rows": 14000, "of": 23288, "epoch": 0, "ce": 0.169823180064559, "kl": 0.0, "lr": 3.437476775250409e-05, "rows_per_s": 1.387068275326287, "tokens_per_s": 8490.022189737709, "gpu_gb": 17.460142135620117}
+{"rows": 14200, "of": 23288, "epoch": 0, "ce": 0.15946113042766227, "kl": 0.0, "lr": 3.309915607957487e-05, "rows_per_s": 1.3881728786592193, "tokens_per_s": 8498.258283106172, "gpu_gb": 17.460142135620117}
+{"rows": 14400, "of": 23288, "epoch": 0, "ce": 0.15700333228334784, "kl": 0.0, "lr": 3.183584645409382e-05, "rows_per_s": 1.3879772449758208, "tokens_per_s": 8502.824170371883, "gpu_gb": 17.460142135620117}
+{"rows": 14600, "of": 23288, "epoch": 0, "ce": 0.16899112448096276, "kl": 0.0, "lr": 3.058575843339354e-05, "rows_per_s": 1.3873437547852159, "tokens_per_s": 8503.796142944246, "gpu_gb": 17.460142135620117}
+{"rows": 14800, "of": 23288, "epoch": 0, "ce": 0.14500759894028306, "kl": 0.0, "lr": 2.9349801950860654e-05, "rows_per_s": 1.3876271444249666, "tokens_per_s": 8504.30997632605, "gpu_gb": 17.460142135620117}
+{"rows": 15000, "of": 23288, "epoch": 0, "ce": 0.13505019856616854, "kl": 0.0, "lr": 2.812887665359938e-05, "rows_per_s": 1.3888674833021517, "tokens_per_s": 8507.526657565102, "gpu_gb": 17.460142135620117}
+{"rows": 15200, "of": 23288, "epoch": 0, "ce": 0.14296119790524245, "kl": 0.0, "lr": 2.6923871247582532e-05, "rows_per_s": 1.3895114672651956, "tokens_per_s": 8511.303668741593, "gpu_gb": 17.460142135620117}
+{"rows": 15400, "of": 23288, "epoch": 0, "ce": 0.14252571837976574, "kl": 0.0, "lr": 2.573566285076644e-05, "rows_per_s": 1.3905210486943524, "tokens_per_s": 8512.876045350544, "gpu_gb": 17.460142135620117}
+{"rows": 15600, "of": 23288, "epoch": 0, "ce": 0.1483127723261714, "kl": 0.0, "lr": 2.4565116354640867e-05, "rows_per_s": 1.3903137488791344, "tokens_per_s": 8508.532985520262, "gpu_gb": 17.460142135620117}
+{"rows": 15800, "of": 23288, "epoch": 0, "ce": 0.1424720217473805, "kl": 0.0, "lr": 2.3413083794678303e-05, "rows_per_s": 1.3898477736716512, "tokens_per_s": 8508.383471550556, "gpu_gb": 17.59785556793213}
+{"rows": 16000, "of": 23288, "epoch": 0, "ce": 0.12436808219179511, "kl": 0.0, "lr": 2.2280403730141285e-05, "rows_per_s": 1.389237208425678, "tokens_per_s": 8511.591726335924, "gpu_gb": 17.59785556793213}
+{"rows": 16200, "of": 23288, "epoch": 0, "ce": 0.14817202266305685, "kl": 0.0, "lr": 2.1167900633698983e-05, "rows_per_s": 1.3893334957196803, "tokens_per_s": 8516.671617328007, "gpu_gb": 17.59785556793213}
+{"rows": 16400, "of": 23288, "epoch": 0, "ce": 0.12732162108179182, "kl": 0.0, "lr": 2.0076384291297134e-05, "rows_per_s": 1.3898240388134717, "tokens_per_s": 8521.511518619369, "gpu_gb": 17.59785556793213}
+{"rows": 16600, "of": 23288, "epoch": 0, "ce": 0.1371114413300529, "kl": 0.0, "lr": 1.9006649212718724e-05, "rows_per_s": 1.390016857655212, "tokens_per_s": 8522.336040351904, "gpu_gb": 17.59785556793213}
+{"rows": 16800, "of": 23288, "epoch": 0, "ce": 0.14615049816668033, "kl": 0.0, "lr": 1.7959474053263738e-05, "rows_per_s": 1.3903809366367654, "tokens_per_s": 8526.45945120446, "gpu_gb": 17.59785556793213}
+{"rows": 17000, "of": 23288, "epoch": 0, "ce": 0.17076426083222032, "kl": 0.0, "lr": 1.693562104696967e-05, "rows_per_s": 1.3912789221834996, "tokens_per_s": 8530.329141358654, "gpu_gb": 17.59785556793213}
+{"rows": 17200, "of": 23288, "epoch": 0, "ce": 0.14331187158823014, "kl": 0.0, "lr": 1.5935835451784758e-05, "rows_per_s": 1.39137930130234, "tokens_per_s": 8528.189564463557, "gpu_gb": 17.59785556793213}
+{"rows": 17400, "of": 23288, "epoch": 0, "ce": 0.13005651479586958, "kl": 0.0, "lr": 1.4960845007098074e-05, "rows_per_s": 1.392283498750895, "tokens_per_s": 8526.882175904826, "gpu_gb": 17.59785556793213}
+{"rows": 17600, "of": 23288, "epoch": 0, "ce": 0.12835072754882276, "kl": 0.0, "lr": 1.401135940402144e-05, "rows_per_s": 1.3923495094780973, "tokens_per_s": 8527.836643762754, "gpu_gb": 17.59785556793213}
+{"rows": 17800, "of": 23288, "epoch": 0, "ce": 0.12834220239892602, "kl": 0.0, "lr": 1.3088069768808375e-05, "rows_per_s": 1.392523907109613, "tokens_per_s": 8528.233538457962, "gpu_gb": 17.59785556793213}
+{"rows": 18000, "of": 23288, "epoch": 0, "ce": 0.147457377910614, "kl": 0.0, "lr": 1.21916481597865e-05, "rows_per_s": 1.392358185985524, "tokens_per_s": 8530.806262531174, "gpu_gb": 17.59785556793213}
+{"rows": 18200, "of": 23288, "epoch": 0, "ce": 0.13536450020968915, "kl": 0.0, "lr": 1.1322747078169377e-05, "rows_per_s": 1.392885774389958, "tokens_per_s": 8535.406572423317, "gpu_gb": 17.59785556793213}
+{"rows": 18400, "of": 23288, "epoch": 0, "ce": 0.11933451063930989, "kl": 0.0, "lr": 1.0481998993103732e-05, "rows_per_s": 1.3933194427563378, "tokens_per_s": 8542.102563439881, "gpu_gb": 17.59785556793213}
+{"rows": 18600, "of": 23288, "epoch": 0, "ce": 0.13681555833667516, "kl": 0.0, "lr": 9.67001588129816e-06, "rows_per_s": 1.394760150548686, "tokens_per_s": 8542.14375317682, "gpu_gb": 17.59785556793213}
+{"rows": 18800, "of": 23288, "epoch": 0, "ce": 0.12398749938234686, "kl": 0.0, "lr": 8.887388781567957e-06, "rows_per_s": 1.3951444570469262, "tokens_per_s": 8544.995909322568, "gpu_gb": 17.59785556793213}
+{"rows": 19000, "of": 23288, "epoch": 0, "ce": 0.1345491605857387, "kl": 0.0, "lr": 8.13468736462073e-06, "rows_per_s": 1.3947813479417392, "tokens_per_s": 8546.170930975146, "gpu_gb": 17.59785556793213}
+{"rows": 19200, "of": 23288, "epoch": 0, "ce": 0.14193498470820487, "kl": 0.0, "lr": 7.412459518395631e-06, "rows_per_s": 1.3955095315900807, "tokens_per_s": 8547.851154457494, "gpu_gb": 17.59785556793213}
+{"rows": 19400, "of": 23288, "epoch": 0, "ce": 0.13867655330337583, "kl": 0.0, "lr": 6.721230949258223e-06, "rows_per_s": 1.3946476790060502, "tokens_per_s": 8549.936768260635, "gpu_gb": 17.59785556793213}
+{"rows": 19600, "of": 23288, "epoch": 0, "ce": 0.15316871294053272, "kl": 0.0, "lr": 6.061504799341272e-06, "rows_per_s": 1.3941571257325187, "tokens_per_s": 8558.30495016579, "gpu_gb": 17.59785556793213}
+{"rows": 19800, "of": 23288, "epoch": 0, "ce": 0.14444821476005018, "kl": 0.0, "lr": 5.433761280309751e-06, "rows_per_s": 1.3938826193673899, "tokens_per_s": 8560.535457101343, "gpu_gb": 17.59785556793213}
+{"rows": 20000, "of": 23288, "epoch": 0, "ce": 0.14456641616299748, "kl": 0.0, "lr": 4.838457323817086e-06, "rows_per_s": 1.3948737103888478, "tokens_per_s": 8566.22242206781, "gpu_gb": 17.59785556793213}
+{"rows": 20200, "of": 23288, "epoch": 0, "ce": 0.13804207751527428, "kl": 0.0, "lr": 4.276026248906578e-06, "rows_per_s": 1.3946265107781814, "tokens_per_s": 8570.532236062927, "gpu_gb": 17.59785556793213}
+{"rows": 20400, "of": 23288, "epoch": 0, "ce": 0.11691928330808878, "kl": 0.0, "lr": 3.746877446600505e-06, "rows_per_s": 1.3945357507157687, "tokens_per_s": 8569.790783088, "gpu_gb": 17.59785556793213}
+{"rows": 20600, "of": 23288, "epoch": 0, "ce": 0.1283823224902153, "kl": 0.0, "lr": 3.2513960819062384e-06, "rows_per_s": 1.3942777292372712, "tokens_per_s": 8564.439210973867, "gpu_gb": 17.59785556793213}
+{"rows": 20800, "of": 23288, "epoch": 0, "ce": 0.1312518483074382, "kl": 0.0, "lr": 2.78994281345637e-06, "rows_per_s": 1.3940719173360052, "tokens_per_s": 8566.052372149792, "gpu_gb": 17.59785556793213}
+{"rows": 21000, "of": 23288, "epoch": 0, "ce": 0.11590770972426981, "kl": 0.0, "lr": 2.362853530986969e-06, "rows_per_s": 1.3946194435503243, "tokens_per_s": 8570.57056759041, "gpu_gb": 17.59785556793213}
+{"rows": 21200, "of": 23288, "epoch": 0, "ce": 0.12887280928902328, "kl": 0.0, "lr": 1.970439110844924e-06, "rows_per_s": 1.3944247222274517, "tokens_per_s": 8571.135960718884, "gpu_gb": 17.59785556793213}
+{"rows": 21400, "of": 23288, "epoch": 0, "ce": 0.14048667300492526, "kl": 0.0, "lr": 1.612985189702515e-06, "rows_per_s": 1.3941458206175363, "tokens_per_s": 8577.36567851167, "gpu_gb": 17.59785556793213}
+{"rows": 21600, "of": 23288, "epoch": 0, "ce": 0.13064606766216458, "kl": 0.0, "lr": 1.2907519566437376e-06, "rows_per_s": 1.394028691461926, "tokens_per_s": 8577.780714085035, "gpu_gb": 17.59785556793213}
+{"rows": 21800, "of": 23288, "epoch": 0, "ce": 0.12414248663932086, "kl": 0.0, "lr": 1.0039739637738522e-06, "rows_per_s": 1.3943093995085893, "tokens_per_s": 8577.450395606284, "gpu_gb": 17.59785556793213}
+{"rows": 22000, "of": 23288, "epoch": 0, "ce": 0.13569101091939956, "kl": 0.0, "lr": 7.52859955490015e-07, "rows_per_s": 1.39432998318549, "tokens_per_s": 8579.565647568175, "gpu_gb": 17.59785556793213}
+{"rows": 22200, "of": 23288, "epoch": 0, "ce": 0.15178708683699368, "kl": 0.0, "lr": 5.37592716537133e-07, "rows_per_s": 1.395209728176862, "tokens_per_s": 8582.060983975125, "gpu_gb": 17.59785556793213}
+{"rows": 22400, "of": 23288, "epoch": 0, "ce": 0.12203105984255672, "kl": 0.0, "lr": 3.583289389597122e-07, "rows_per_s": 1.3954112975792552, "tokens_per_s": 8582.558667813764, "gpu_gb": 17.59785556793213}
+{"rows": 22600, "of": 23288, "epoch": 0, "ce": 0.1385567666590214, "kl": 0.0, "lr": 2.1519910804640442e-07, "rows_per_s": 1.395450903336351, "tokens_per_s": 8584.81074655324, "gpu_gb": 17.59785556793213}
+{"rows": 22800, "of": 23288, "epoch": 0, "ce": 0.1414877101685852, "kl": 0.0, "lr": 1.0830740735037204e-07, "rows_per_s": 1.3955025051560292, "tokens_per_s": 8583.399519663493, "gpu_gb": 17.59785556793213}
+{"rows": 23000, "of": 23288, "epoch": 0, "ce": 0.10061369412578643, "kl": 0.0, "lr": 3.773164285456043e-08, "rows_per_s": 1.39582193980082, "tokens_per_s": 8586.329963970142, "gpu_gb": 17.59785556793213}
+{"rows": 23200, "of": 23288, "epoch": 0, "ce": 0.12742840342223644, "kl": 0.0, "lr": 3.5231863370621675e-09, "rows_per_s": 1.3960771993339662, "tokens_per_s": 8586.359310973248, "gpu_gb": 17.59785556793213}
+{"rows": 23288, "of": 23288, "epoch": 0, "ce": 0.12263650904324921, "kl": 0.0, "lr": 0.0, "rows_per_s": 1.3960359730843455, "tokens_per_s": 8587.053957749244, "gpu_gb": 17.59785556793213}

ablations/no_done/training_meta.json

@@ -0,0 +1,36 @@
+{
+ "base_model": "Qwen/Qwen3.5-4B",
+ "families": [
+  "Discrete selection",
+  "Drag & gesture",
+  "Form transaction",
+  "Navigation",
+  "Out-of-page I/O",
+  "Reasoning base",
+  "Text entry"
+ ],
+ "training_rows": 23288,
+ "rows_trained": 23288,
+ "validation_ce": 0.12026296784519218,
+ "minutes": 280.7,
+ "args": {
+  "adapter": "pooled",
+  "epochs": 1.0,
+  "lr": 0.0001,
+  "rank": 64,
+  "alpha": 128,
+  "grad_accum": 8,
+  "beta": 0.0,
+  "anchor_ratio": 0.5,
+  "limit": 0,
+  "max_steps": 0,
+  "seed": 0,
+  "ce_on": "all",
+  "exclude_variants": "",
+  "frac": 1.0,
+  "episode_ids": null,
+  "init_adapter": null,
+  "drop_done": true
+ },
+ "gpus": 4
+}

ablations/specialists/drag/adapter_config.json

@@ -0,0 +1,51 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "kasa_config": null,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.21.0",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "k_proj",
+    "v_proj",
+    "o_proj",
+    "gate_proj",
+    "up_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

ablations/specialists/drag/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:47719d2fe473c91b01527aa78ed691762167a7ec34bcd498d8d42e1ad5c4a358
+size 339776632

ablations/specialists/drag/train_log.jsonl

@@ -0,0 +1,124 @@
+{"rows": 50, "of": 6182, "epoch": 0, "ce": 0.14114577851786808, "kl": 0.0004595453850016516, "lr": 1.5215123391891259e-05, "rows_per_s": 0.21905337483398613, "tokens_per_s": 1372.8425486245644, "gpu_gb": 14.386658668518066}
+{"rows": 100, "of": 6182, "epoch": 0, "ce": 0.10190403189044446, "kl": 0.00010337640434471274, "lr": 2.8244024726372086e-05, "rows_per_s": 0.2814809206682031, "tokens_per_s": 1763.4329310389858, "gpu_gb": 14.692233562469482}
+{"rows": 150, "of": 6182, "epoch": 0, "ce": 0.10261645056523902, "kl": 0.0323285159979544, "lr": 4.1248967980526154e-05, "rows_per_s": 0.31229843759734266, "tokens_per_s": 1957.3908353392806, "gpu_gb": 15.115210056304932}
+{"rows": 200, "of": 6182, "epoch": 0, "ce": 0.09023646466237933, "kl": 0.0002271195244692159, "lr": 4.987073511635124e-05, "rows_per_s": 0.3074600123869402, "tokens_per_s": 1964.578778448894, "gpu_gb": 17.690584659576416}
+{"rows": 250, "of": 6182, "epoch": 0, "ce": 0.1216916629999149, "kl": 0.0005196704034734428, "lr": 4.980133447469112e-05, "rows_per_s": 0.3116429456588969, "tokens_per_s": 2013.2009632386475, "gpu_gb": 17.690584659576416}
+{"rows": 300, "of": 6182, "epoch": 0, "ce": 0.12278362882618861, "kl": 9.442275334984689e-05, "lr": 4.9717148840184384e-05, "rows_per_s": 0.3239428980513708, "tokens_per_s": 2078.2167893008027, "gpu_gb": 17.690584659576416}
+{"rows": 350, "of": 6182, "epoch": 0, "ce": 0.1041526701534167, "kl": 0.0035170846912300213, "lr": 4.9618228399001675e-05, "rows_per_s": 0.33303874702400266, "tokens_per_s": 2125.8653000142176, "gpu_gb": 17.690584659576416}
+{"rows": 400, "of": 6182, "epoch": 0, "ce": 0.12467748245123673, "kl": 0.0003059996964365144, "lr": 4.948427721821654e-05, "rows_per_s": 0.33785920681778364, "tokens_per_s": 2160.356233194613, "gpu_gb": 17.690584659576416}
+{"rows": 450, "of": 6182, "epoch": 0, "ce": 0.1434574532788247, "kl": 0.0018394437791680596, "lr": 4.9353645332523144e-05, "rows_per_s": 0.34164718034600905, "tokens_per_s": 2191.8830384672005, "gpu_gb": 17.690584659576416}
+{"rows": 500, "of": 6182, "epoch": 0, "ce": 0.12070609011540287, "kl": 0.0007681466897595246, "lr": 4.9208495338040164e-05, "rows_per_s": 0.34615942281968837, "tokens_per_s": 2213.28104081298, "gpu_gb": 17.690584659576416}
+{"rows": 550, "of": 6182, "epoch": 0, "ce": 0.1339812695980072, "kl": 4.621991971998085e-05, "lr": 4.904891376404822e-05, "rows_per_s": 0.3517264599019929, "tokens_per_s": 2232.3758658652305, "gpu_gb": 17.690584659576416}
+{"rows": 600, "of": 6182, "epoch": 0, "ce": 0.14585241077597752, "kl": 0.0007430801488483653, "lr": 4.8844622740414894e-05, "rows_per_s": 0.35416278894976444, "tokens_per_s": 2252.6742591536286, "gpu_gb": 17.690584659576416}
+{"rows": 650, "of": 6182, "epoch": 0, "ce": 0.13545000398571186, "kl": 0.0019205574062652886, "lr": 4.8654110499928616e-05, "rows_per_s": 0.35841638271421566, "tokens_per_s": 2273.6429970582444, "gpu_gb": 17.690584659576416}
+{"rows": 700, "of": 6182, "epoch": 0, "ce": 0.10988286818409788, "kl": 0.002434414378482606, "lr": 4.844949716924128e-05, "rows_per_s": 0.35893174278657325, "tokens_per_s": 2276.9593987922453, "gpu_gb": 17.690584659576416}
+{"rows": 750, "of": 6182, "epoch": 0, "ce": 0.1420151364798538, "kl": 0.005905228854734116, "lr": 4.823090472592766e-05, "rows_per_s": 0.3603381868812451, "tokens_per_s": 2286.962214286525, "gpu_gb": 17.690584659576416}
+{"rows": 800, "of": 6182, "epoch": 0, "ce": 0.12600741705724172, "kl": 0.0018009582513263123, "lr": 4.795838647187818e-05, "rows_per_s": 0.36087082064396403, "tokens_per_s": 2292.9930422668826, "gpu_gb": 17.690584659576416}
+{"rows": 850, "of": 6182, "epoch": 0, "ce": 0.1369263383718889, "kl": 0.01614485661938398, "lr": 4.7709964000007044e-05, "rows_per_s": 0.3619419454830388, "tokens_per_s": 2303.5420403900916, "gpu_gb": 17.690584659576416}
+{"rows": 900, "of": 6182, "epoch": 0, "ce": 0.129010445756338, "kl": 0.0015281860415872668, "lr": 4.7448003278586086e-05, "rows_per_s": 0.3636467246703616, "tokens_per_s": 2311.95112470993, "gpu_gb": 17.690584659576416}
+{"rows": 950, "of": 6182, "epoch": 0, "ce": 0.15640339703957845, "kl": 1.5313197408609344e-05, "lr": 4.71726604720909e-05, "rows_per_s": 0.36484205477180637, "tokens_per_s": 2320.4576835201337, "gpu_gb": 17.690584659576416}
+{"rows": 1000, "of": 6182, "epoch": 0, "ce": 0.12346014158101752, "kl": 0.005113847240800995, "lr": 4.6834733150857426e-05, "rows_per_s": 0.3669941492022092, "tokens_per_s": 2326.9484226655595, "gpu_gb": 17.690584659576416}
+{"rows": 1050, "of": 6182, "epoch": 0, "ce": 0.1394481676453281, "kl": 0.0006134290922885088, "lr": 4.653096943175519e-05, "rows_per_s": 0.36801131934241876, "tokens_per_s": 2334.076744232211, "gpu_gb": 17.690584659576416}
+{"rows": 1100, "of": 6182, "epoch": 0, "ce": 0.1430460892663133, "kl": 0.0020534814617289964, "lr": 4.621437030537461e-05, "rows_per_s": 0.3692212233296078, "tokens_per_s": 2340.036507336428, "gpu_gb": 17.690584659576416}
+{"rows": 1150, "of": 6182, "epoch": 0, "ce": 0.13272530849739586, "kl": 0.0008814956891001202, "lr": 4.5885124508165475e-05, "rows_per_s": 0.3715670254430739, "tokens_per_s": 2347.9223407219465, "gpu_gb": 17.690584659576416}
+{"rows": 1200, "of": 6182, "epoch": 0, "ce": 0.13241521149682692, "kl": 0.0017724999370189655, "lr": 4.548528269061688e-05, "rows_per_s": 0.37172190736487354, "tokens_per_s": 2351.5954941145787, "gpu_gb": 17.690584659576416}
+{"rows": 1250, "of": 6182, "epoch": 0, "ce": 0.13404524534208967, "kl": 0.005608873526598973, "lr": 4.512931900282807e-05, "rows_per_s": 0.3727795761897121, "tokens_per_s": 2356.431852206404, "gpu_gb": 17.690584659576416}
+{"rows": 1300, "of": 6182, "epoch": 0, "ce": 0.13345753117981884, "kl": 1.276031924846155e-07, "lr": 4.476135548342666e-05, "rows_per_s": 0.37418447384256087, "tokens_per_s": 2362.4945570182003, "gpu_gb": 17.690584659576416}
+{"rows": 1350, "of": 6182, "epoch": 0, "ce": 0.10374875272361514, "kl": 0.0017316408443273318, "lr": 4.4381611489077026e-05, "rows_per_s": 0.37517040909952537, "tokens_per_s": 2365.576153599581, "gpu_gb": 17.690584659576416}
+{"rows": 1400, "of": 6182, "epoch": 0, "ce": 0.10423550059875616, "kl": 0.3251197911316395, "lr": 4.392399001569517e-05, "rows_per_s": 0.3762384477973141, "tokens_per_s": 2368.206841710596, "gpu_gb": 17.690584659576416}
+{"rows": 1450, "of": 6182, "epoch": 0, "ce": 0.13108731571119278, "kl": 0.022335486870899868, "lr": 4.35195074475946e-05, "rows_per_s": 0.3773103685617018, "tokens_per_s": 2371.7360263841433, "gpu_gb": 17.690584659576416}
+{"rows": 1500, "of": 6182, "epoch": 0, "ce": 0.16459643783136493, "kl": 0.010582552636512997, "lr": 4.310398471609416e-05, "rows_per_s": 0.37792117206269227, "tokens_per_s": 2375.829162391399, "gpu_gb": 17.690584659576416}
+{"rows": 1550, "of": 6182, "epoch": 0, "ce": 0.11549665239574604, "kl": 0.0025583334654769425, "lr": 4.267766952966369e-05, "rows_per_s": 0.3784058651325898, "tokens_per_s": 2380.1133232768334, "gpu_gb": 17.690584659576416}
+{"rows": 1600, "of": 6182, "epoch": 0, "ce": 0.136397794689756, "kl": 0.0009283299936528137, "lr": 4.216700075136953e-05, "rows_per_s": 0.37819506534703223, "tokens_per_s": 2383.725913747725, "gpu_gb": 17.690584659576416}
+{"rows": 1650, "of": 6182, "epoch": 0, "ce": 0.15308213529642672, "kl": 0.005073661378941302, "lr": 4.171818213432548e-05, "rows_per_s": 0.37931820534013194, "tokens_per_s": 2385.5181701049833, "gpu_gb": 17.690584659576416}
+{"rows": 1700, "of": 6182, "epoch": 0, "ce": 0.1161827460432855, "kl": 0.0003898738240414161, "lr": 4.125939719052927e-05, "rows_per_s": 0.3799940427937666, "tokens_per_s": 2387.2258457997186, "gpu_gb": 17.690584659576416}
+{"rows": 1750, "of": 6182, "epoch": 0, "ce": 0.12339889399103217, "kl": 0.003731375374381206, "lr": 4.07909194186557e-05, "rows_per_s": 0.38010499326206765, "tokens_per_s": 2388.2607066673463, "gpu_gb": 17.690584659576416}
+{"rows": 1800, "of": 6182, "epoch": 0, "ce": 0.1306552495678059, "kl": 0.0005049576034340887, "lr": 4.023248425866597e-05, "rows_per_s": 0.3808565617757476, "tokens_per_s": 2389.0467737079325, "gpu_gb": 17.690584659576416}
+{"rows": 1850, "of": 6182, "epoch": 0, "ce": 0.12639638231840075, "kl": 0.0025931622481572486, "lr": 3.974397091158336e-05, "rows_per_s": 0.38126866872509363, "tokens_per_s": 2391.4930921049177, "gpu_gb": 17.690584659576416}
+{"rows": 1900, "of": 6182, "epoch": 0, "ce": 0.17681284190288613, "kl": 0.0025755984571720534, "lr": 3.9246668138081757e-05, "rows_per_s": 0.3814663640177399, "tokens_per_s": 2391.693515734191, "gpu_gb": 17.690584659576416}
+{"rows": 1950, "of": 6182, "epoch": 0, "ce": 0.11859757501793944, "kl": 0.00023241362002331123, "lr": 3.874087239873895e-05, "rows_per_s": 0.3825150835467113, "tokens_per_s": 2395.3327963976817, "gpu_gb": 17.690584659576416}
+{"rows": 2000, "of": 6182, "epoch": 0, "ce": 0.14864671975374222, "kl": 0.002888255933891606, "lr": 3.814044574153217e-05, "rows_per_s": 0.3831552518061061, "tokens_per_s": 2397.5537568752693, "gpu_gb": 17.690584659576416}
+{"rows": 2050, "of": 6182, "epoch": 0, "ce": 0.12226391824462064, "kl": 4.470009717557251e-06, "lr": 3.7617289474088725e-05, "rows_per_s": 0.38327051406160945, "tokens_per_s": 2399.4110214882844, "gpu_gb": 17.690584659576416}
+{"rows": 2100, "of": 6182, "epoch": 0, "ce": 0.12477974519133568, "kl": 0.012983836885970546, "lr": 3.708661157369411e-05, "rows_per_s": 0.3835545204103049, "tokens_per_s": 2401.2350386482863, "gpu_gb": 17.690584659576416}
+{"rows": 2150, "of": 6182, "epoch": 0, "ce": 0.13662995785676144, "kl": 0.022018204272037163, "lr": 3.654872839707375e-05, "rows_per_s": 0.38405047466244785, "tokens_per_s": 2401.759317796902, "gpu_gb": 17.690584659576416}
+{"rows": 2200, "of": 6182, "epoch": 0, "ce": 0.1366547176370486, "kl": 0.0002575464320947636, "lr": 3.591251936920861e-05, "rows_per_s": 0.38458932655053735, "tokens_per_s": 2403.993234975392, "gpu_gb": 17.690584659576416}
+{"rows": 2250, "of": 6182, "epoch": 0, "ce": 0.12514410663123537, "kl": 0.0019742345963630952, "lr": 3.536013024012762e-05, "rows_per_s": 0.38464370888460037, "tokens_per_s": 2405.525342426516, "gpu_gb": 17.690584659576416}
+{"rows": 2300, "of": 6182, "epoch": 0, "ce": 0.11839392025722191, "kl": 0.004419480437723999, "lr": 3.480156505420396e-05, "rows_per_s": 0.3846691852322851, "tokens_per_s": 2407.2301583811245, "gpu_gb": 17.690584659576416}
+{"rows": 2350, "of": 6182, "epoch": 0, "ce": 0.131934127457706, "kl": 0.0010739699229240691, "lr": 3.4237156792805455e-05, "rows_per_s": 0.38498109441344563, "tokens_per_s": 2406.950948795553, "gpu_gb": 17.690584659576416}
+{"rows": 2400, "of": 6182, "epoch": 0, "ce": 0.11856837021852178, "kl": 0.0017562982520043062, "lr": 3.3571744553159e-05, "rows_per_s": 0.38489747284513537, "tokens_per_s": 2408.6237543642087, "gpu_gb": 17.690584659576416}
+{"rows": 2450, "of": 6182, "epoch": 0, "ce": 0.13094268235354087, "kl": 0.008594318517839112, "lr": 3.299583492376044e-05, "rows_per_s": 0.38527083096909587, "tokens_per_s": 2409.874394327308, "gpu_gb": 17.690584659576416}
+{"rows": 2500, "of": 6182, "epoch": 0, "ce": 0.10048015381639394, "kl": 0.003943277600745196, "lr": 3.241515868140921e-05, "rows_per_s": 0.38539435697642604, "tokens_per_s": 2410.809580669444, "gpu_gb": 17.690584659576416}
+{"rows": 2550, "of": 6182, "epoch": 0, "ce": 0.09488553983943634, "kl": 0.001990981712622422, "lr": 3.183006198869177e-05, "rows_per_s": 0.3856497230413954, "tokens_per_s": 2411.5899162077344, "gpu_gb": 17.690584659576416}
+{"rows": 2600, "of": 6182, "epoch": 0, "ce": 0.13199111116118728, "kl": 0.014373885557359897, "lr": 3.1142327692122916e-05, "rows_per_s": 0.385980603346371, "tokens_per_s": 2412.807654746768, "gpu_gb": 17.690584659576416}
+{"rows": 2650, "of": 6182, "epoch": 0, "ce": 0.09389652701657872, "kl": 0.001709859581934576, "lr": 3.054885315370568e-05, "rows_per_s": 0.3861269495117955, "tokens_per_s": 2414.412328352609, "gpu_gb": 17.690584659576416}
+{"rows": 2700, "of": 6182, "epoch": 0, "ce": 0.12107294789319367, "kl": 0.0009544604437288171, "lr": 2.9952070738679312e-05, "rows_per_s": 0.3863484853047578, "tokens_per_s": 2415.001993514266, "gpu_gb": 17.690584659576416}
+{"rows": 2750, "of": 6182, "epoch": 0, "ce": 0.1301670425815419, "kl": 0.0031185491042353027, "lr": 2.935233621111552e-05, "rows_per_s": 0.3865810709829129, "tokens_per_s": 2416.461201290618, "gpu_gb": 17.690584659576416}
+{"rows": 2800, "of": 6182, "epoch": 0, "ce": 0.10466484443561451, "kl": 0.001197442674518797, "lr": 2.8649391849130437e-05, "rows_per_s": 0.38679116166634164, "tokens_per_s": 2416.712758141182, "gpu_gb": 17.690584659576416}
+{"rows": 2850, "of": 6182, "epoch": 0, "ce": 0.11597714083036408, "kl": 0.0028139715697209217, "lr": 2.80444896350671e-05, "rows_per_s": 0.3868210849197639, "tokens_per_s": 2417.6596047212993, "gpu_gb": 17.690584659576416}
+{"rows": 2900, "of": 6182, "epoch": 0, "ce": 0.13245788954484922, "kl": 0.00858452558562148, "lr": 2.7437772488120944e-05, "rows_per_s": 0.3869519169080167, "tokens_per_s": 2418.134448442063, "gpu_gb": 17.690584659576416}
+{"rows": 2950, "of": 6182, "epoch": 0, "ce": 0.11487443593525609, "kl": 0.010283363993325082, "lr": 2.6829602094824864e-05, "rows_per_s": 0.3871971540812303, "tokens_per_s": 2419.3336905879214, "gpu_gb": 17.690584659576416}
+{"rows": 3000, "of": 6182, "epoch": 0, "ce": 0.11175610243808479, "kl": 0.004219171304220351, "lr": 2.6118716949116635e-05, "rows_per_s": 0.38712692812779687, "tokens_per_s": 2420.403883959959, "gpu_gb": 17.690584659576416}
+{"rows": 3050, "of": 6182, "epoch": 0, "ce": 0.13322830083262588, "kl": 0.005222299185785317, "lr": 2.5508642468552775e-05, "rows_per_s": 0.3875520074626594, "tokens_per_s": 2422.2470611474446, "gpu_gb": 17.690584659576416}
+{"rows": 3100, "of": 6182, "epoch": 1, "ce": 0.09897869676351548, "kl": 0.0017405983973975785, "lr": 2.4898264767399446e-05, "rows_per_s": 0.38331177916945114, "tokens_per_s": 2397.7348708991867, "gpu_gb": 17.690584659576416}
+{"rows": 3150, "of": 6182, "epoch": 1, "ce": 0.07597046926413185, "kl": 0.0005387187047598305, "lr": 2.4287947714381313e-05, "rows_per_s": 0.38365977629231024, "tokens_per_s": 2398.919957743805, "gpu_gb": 17.690584659576416}
+{"rows": 3200, "of": 6182, "epoch": 1, "ce": 0.07298260375759319, "kl": 0.0010005762257499565, "lr": 2.367805514206845e-05, "rows_per_s": 0.3839248447425447, "tokens_per_s": 2399.450135329564, "gpu_gb": 17.690584659576416}
+{"rows": 3250, "of": 6182, "epoch": 1, "ce": 0.0794545347976964, "kl": 0.006408645264236665, "lr": 2.3068950629982254e-05, "rows_per_s": 0.38413276591968415, "tokens_per_s": 2400.5638489293124, "gpu_gb": 17.690584659576416}
+{"rows": 3300, "of": 6182, "epoch": 1, "ce": 0.0629229514696006, "kl": 0.00502883654115374, "lr": 2.2359809110118356e-05, "rows_per_s": 0.38387500035156247, "tokens_per_s": 2400.4733254938687, "gpu_gb": 17.690584659576416}
+{"rows": 3350, "of": 6182, "epoch": 1, "ce": 0.07447583189059515, "kl": 0.004204478245856888, "lr": 2.1753656802405954e-05, "rows_per_s": 0.38424401882789266, "tokens_per_s": 2401.280807298179, "gpu_gb": 17.690584659576416}
+{"rows": 3400, "of": 6182, "epoch": 1, "ce": 0.07111075534996372, "kl": 2.431219241051509e-07, "lr": 2.1149439759951277e-05, "rows_per_s": 0.38427692797376334, "tokens_per_s": 2402.5695407426365, "gpu_gb": 17.690584659576416}
+{"rows": 3450, "of": 6182, "epoch": 1, "ce": 0.07913623872445896, "kl": 0.07715891133917586, "lr": 2.0547518178882457e-05, "rows_per_s": 0.3846019234023002, "tokens_per_s": 2402.7733156241866, "gpu_gb": 17.690584659576416}
+{"rows": 3500, "of": 6182, "epoch": 1, "ce": 0.06851981869457584, "kl": 0.024767653221030594, "lr": 1.9848656170292554e-05, "rows_per_s": 0.3844893985519206, "tokens_per_s": 2403.4184033183205, "gpu_gb": 17.690584659576416}
+{"rows": 3550, "of": 6182, "epoch": 1, "ce": 0.06393960744548494, "kl": 0.0005088893800535162, "lr": 1.9252936944488264e-05, "rows_per_s": 0.3844572365992976, "tokens_per_s": 2404.1499382518364, "gpu_gb": 17.690584659576416}
+{"rows": 3600, "of": 6182, "epoch": 1, "ce": 0.08692521565152626, "kl": 0.017267390729455667, "lr": 1.8660643755547097e-05, "rows_per_s": 0.3848690448564119, "tokens_per_s": 2404.4223181905063, "gpu_gb": 17.690584659576416}
+{"rows": 3650, "of": 6182, "epoch": 1, "ce": 0.052859315451039654, "kl": 0.00010004467278041622, "lr": 1.8072129691347057e-05, "rows_per_s": 0.3850662167874263, "tokens_per_s": 2405.6708060724773, "gpu_gb": 17.690584659576416}
+{"rows": 3700, "of": 6182, "epoch": 1, "ce": 0.07934485286306073, "kl": 0.014502070121126759, "lr": 1.73907742593293e-05, "rows_per_s": 0.3850910707481005, "tokens_per_s": 2405.9081915966394, "gpu_gb": 17.690584659576416}
+{"rows": 3750, "of": 6182, "epoch": 1, "ce": 0.06558437061734325, "kl": 0.003287055389349964, "lr": 1.6811648561546003e-05, "rows_per_s": 0.38521793783597724, "tokens_per_s": 2407.070238242302, "gpu_gb": 17.690584659576416}
+{"rows": 3800, "of": 6182, "epoch": 1, "ce": 0.051078009618712325, "kl": 0.001095887473983842, "lr": 1.6237404242930698e-05, "rows_per_s": 0.38568942151345686, "tokens_per_s": 2406.7292929951254, "gpu_gb": 17.690584659576416}
+{"rows": 3850, "of": 6182, "epoch": 1, "ce": 0.048799248835712204, "kl": 0.0021325419046857724, "lr": 1.566838363176219e-05, "rows_per_s": 0.38585784932207856, "tokens_per_s": 2407.3520884977474, "gpu_gb": 17.690584659576416}
+{"rows": 3900, "of": 6182, "epoch": 1, "ce": 0.08252570917829871, "kl": 0.00042919761841947457, "lr": 1.5011580802768046e-05, "rows_per_s": 0.38592937869873994, "tokens_per_s": 2407.1406891074917, "gpu_gb": 17.690584659576416}
+{"rows": 3950, "of": 6182, "epoch": 1, "ce": 0.05587935655494221, "kl": 0.0012137149708899286, "lr": 1.4455037482281103e-05, "rows_per_s": 0.3860043787961368, "tokens_per_s": 2407.742574463451, "gpu_gb": 17.690584659576416}
+{"rows": 4000, "of": 6182, "epoch": 1, "ce": 0.0721158667399389, "kl": 0.02290166817894355, "lr": 1.3904780403994202e-05, "rows_per_s": 0.3863231879556876, "tokens_per_s": 2408.4314712808655, "gpu_gb": 17.690584659576416}
+{"rows": 4050, "of": 6182, "epoch": 1, "ce": 0.056466463190089494, "kl": 0.000514499562657115, "lr": 1.3361137596504936e-05, "rows_per_s": 0.3862072341203991, "tokens_per_s": 2408.1816672646014, "gpu_gb": 17.690584659576416}
+{"rows": 4100, "of": 6182, "epoch": 1, "ce": 0.06856435998863737, "kl": 0.13822977245309662, "lr": 1.2735679493836009e-05, "rows_per_s": 0.38627483535013696, "tokens_per_s": 2408.592899842037, "gpu_gb": 17.690584659576416}
+{"rows": 4150, "of": 6182, "epoch": 1, "ce": 0.068175776491467, "kl": 0.001820291015415355, "lr": 1.2207473871250872e-05, "rows_per_s": 0.3862968335896368, "tokens_per_s": 2408.8992093077545, "gpu_gb": 17.690584659576416}
+{"rows": 4200, "of": 6182, "epoch": 1, "ce": 0.05692100692837638, "kl": 0.0020757193912383323, "lr": 1.1686894346668512e-05, "rows_per_s": 0.3862596097414902, "tokens_per_s": 2410.5665813437745, "gpu_gb": 17.690584659576416}
+{"rows": 4250, "of": 6182, "epoch": 1, "ce": 0.055340510385576636, "kl": 0.009431862183461362, "lr": 1.1174251256799017e-05, "rows_per_s": 0.3863881981907736, "tokens_per_s": 2411.641120773882, "gpu_gb": 17.690584659576416}
+{"rows": 4300, "of": 6182, "epoch": 1, "ce": 0.07031077940094595, "kl": 3.4809091784340773e-07, "lr": 1.0586605861964805e-05, "rows_per_s": 0.38648525717597343, "tokens_per_s": 2412.288987718034, "gpu_gb": 17.690584659576416}
+{"rows": 4350, "of": 6182, "epoch": 1, "ce": 0.06872095134131974, "kl": 0.0019238851753987528, "lr": 1.0092200212349162e-05, "rows_per_s": 0.3866454252310281, "tokens_per_s": 2413.0420106420574, "gpu_gb": 17.690584659576416}
+{"rows": 4400, "of": 6182, "epoch": 1, "ce": 0.06917678606017337, "kl": 0.0013458646766534796, "lr": 9.606681653605342e-06, "rows_per_s": 0.3866832867014387, "tokens_per_s": 2412.8629315067437, "gpu_gb": 17.690584659576416}
+{"rows": 4450, "of": 6182, "epoch": 1, "ce": 0.05219243298827981, "kl": 0.002460014024126045, "lr": 9.130339621304852e-06, "rows_per_s": 0.3868561174800913, "tokens_per_s": 2414.0808431192168, "gpu_gb": 17.690584659576416}
+{"rows": 4500, "of": 6182, "epoch": 1, "ce": 0.07713174625323752, "kl": 1.19284498176929, "lr": 8.586583887392546e-06, "rows_per_s": 0.38671953313399693, "tokens_per_s": 2414.607928811779, "gpu_gb": 17.690584659576416}
+{"rows": 4550, "of": 6182, "epoch": 1, "ce": 0.052040698887801475, "kl": 0.0018177946060404793, "lr": 8.131090953848227e-06, "rows_per_s": 0.38677251477355573, "tokens_per_s": 2415.93575127659, "gpu_gb": 17.690584659576416}
+{"rows": 4600, "of": 6182, "epoch": 1, "ce": 0.05817273516795072, "kl": 8.764592634991075e-08, "lr": 7.685654200943379e-06, "rows_per_s": 0.3868704337080164, "tokens_per_s": 2417.210287089417, "gpu_gb": 17.690584659576416}
+{"rows": 4650, "of": 6182, "epoch": 1, "ce": 0.07038103444173406, "kl": 0.001767905029305355, "lr": 7.250539170002477e-06, "rows_per_s": 0.38706851593793024, "tokens_per_s": 2417.624373432061, "gpu_gb": 17.690584659576416}
+{"rows": 4700, "of": 6182, "epoch": 1, "ce": 0.04757744694451013, "kl": 0.009295638220851972, "lr": 6.756296178746282e-06, "rows_per_s": 0.3867617524313027, "tokens_per_s": 2418.0055925036736, "gpu_gb": 17.690584659576416}
+{"rows": 4750, "of": 6182, "epoch": 1, "ce": 0.05796706701059722, "kl": 1.2895735765528732e-05, "lr": 6.3444263005578795e-06, "rows_per_s": 0.38677052271006157, "tokens_per_s": 2417.8606655353533, "gpu_gb": 17.690584659576416}
+{"rows": 4800, "of": 6182, "epoch": 1, "ce": 0.05242275988327011, "kl": 0.057145555804856314, "lr": 5.94367769989497e-06, "rows_per_s": 0.38657376643792946, "tokens_per_s": 2417.5463226397783, "gpu_gb": 17.690584659576416}
+{"rows": 4850, "of": 6182, "epoch": 1, "ce": 0.06099311683352131, "kl": 0.007050861684532557, "lr": 5.554289277821528e-06, "rows_per_s": 0.3864800807765603, "tokens_per_s": 2417.4006321784746, "gpu_gb": 17.690584659576416}
+{"rows": 4900, "of": 6182, "epoch": 1, "ce": 0.05949995981573011, "kl": 0.03272153615600455, "lr": 5.11467009024216e-06, "rows_per_s": 0.3866148089160708, "tokens_per_s": 2417.6973644772584, "gpu_gb": 17.690584659576416}
+{"rows": 4950, "of": 6182, "epoch": 1, "ce": 0.07069506667321548, "kl": 0.0002738478993271204, "lr": 4.750682492360278e-06, "rows_per_s": 0.38680233099217337, "tokens_per_s": 2417.9652129521605, "gpu_gb": 17.690584659576416}
+{"rows": 5000, "of": 6182, "epoch": 1, "ce": 0.03925501098080228, "kl": 0.00010956108881288174, "lr": 4.398766261638271e-06, "rows_per_s": 0.38713260278985506, "tokens_per_s": 2419.0153346464936, "gpu_gb": 17.690584659576416}
+{"rows": 5050, "of": 6182, "epoch": 1, "ce": 0.07771768247123895, "kl": 0.0005179948601829822, "lr": 4.05913118835817e-06, "rows_per_s": 0.3873594243907977, "tokens_per_s": 2419.4185839554125, "gpu_gb": 17.690584659576416}
+{"rows": 5100, "of": 6182, "epoch": 1, "ce": 0.06569815051550858, "kl": 0.0420349938901133, "lr": 3.6786819903062068e-06, "rows_per_s": 0.3874157336714275, "tokens_per_s": 2419.6015462707796, "gpu_gb": 17.690584659576416}
+{"rows": 5150, "of": 6182, "epoch": 1, "ce": 0.047528109410344514, "kl": 0.0027340290847762843, "lr": 3.3663407378900196e-06, "rows_per_s": 0.38773193185132615, "tokens_per_s": 2420.220685846626, "gpu_gb": 17.690584659576416}
+{"rows": 5200, "of": 6182, "epoch": 1, "ce": 0.04388658805793628, "kl": 1.5153696010873827e-05, "lr": 3.066896109958392e-06, "rows_per_s": 0.38773856735619355, "tokens_per_s": 2420.7995894858573, "gpu_gb": 17.690584659576416}
+{"rows": 5250, "of": 6182, "epoch": 1, "ce": 0.08377320564184816, "kl": 0.0012684626215120217, "lr": 2.780526616530077e-06, "rows_per_s": 0.3876788731159956, "tokens_per_s": 2421.1233848398197, "gpu_gb": 17.690584659576416}
+{"rows": 5300, "of": 6182, "epoch": 1, "ce": 0.06482889569943419, "kl": 0.009472533491275874, "lr": 2.4631817057111595e-06, "rows_per_s": 0.3875475344134892, "tokens_per_s": 2421.1173758132927, "gpu_gb": 17.690584659576416}
+{"rows": 5350, "of": 6182, "epoch": 1, "ce": 0.048750881147053504, "kl": 1.488654977788162e-06, "lr": 2.205716779212849e-06, "rows_per_s": 0.3876213159020461, "tokens_per_s": 2421.982093029655, "gpu_gb": 17.690584659576416}
+{"rows": 5400, "of": 6182, "epoch": 1, "ce": 0.06898807637143607, "kl": 0.005231621825102475, "lr": 1.9618403680707056e-06, "rows_per_s": 0.3876321962892411, "tokens_per_s": 2422.422705896349, "gpu_gb": 17.690584659576416}
+{"rows": 5450, "of": 6182, "epoch": 1, "ce": 0.05086816659119601, "kl": 0.004377716740202446, "lr": 1.7316978560340646e-06, "rows_per_s": 0.3875310148549606, "tokens_per_s": 2422.481474491064, "gpu_gb": 17.690584659576416}
+{"rows": 5500, "of": 6182, "epoch": 1, "ce": 0.055116085967581184, "kl": 0.006416618930027447, "lr": 1.4807389566860674e-06, "rows_per_s": 0.3873723726582333, "tokens_per_s": 2422.2012020140505, "gpu_gb": 17.690584659576416}
+{"rows": 5550, "of": 6182, "epoch": 1, "ce": 0.05217106928669869, "kl": 0.0004756948460350542, "lr": 1.2808128500070322e-06, "rows_per_s": 0.38732634850015957, "tokens_per_s": 2422.842437984517, "gpu_gb": 17.690584659576416}
+{"rows": 5600, "of": 6182, "epoch": 1, "ce": 0.04638890630179426, "kl": 0.0009230164826759227, "lr": 1.0950266281272548e-06, "rows_per_s": 0.3874311452120671, "tokens_per_s": 2423.324667207561, "gpu_gb": 17.690584659576416}
+{"rows": 5650, "of": 6182, "epoch": 1, "ce": 0.0826175806237521, "kl": 0.002194308799462875, "lr": 9.234910450855672e-07, "rows_per_s": 0.38738310033671913, "tokens_per_s": 2423.8028536235133, "gpu_gb": 17.690584659576416}
+{"rows": 5700, "of": 6182, "epoch": 1, "ce": 0.04952725685106065, "kl": 0.0013720305105392359, "lr": 7.415133710424793e-07, "rows_per_s": 0.3873056733623303, "tokens_per_s": 2423.942092690127, "gpu_gb": 17.690584659576416}
+{"rows": 5750, "of": 6182, "epoch": 1, "ce": 0.07640978489787532, "kl": 0.0017552813508707845, "lr": 6.011935581761774e-07, "rows_per_s": 0.3874507166609641, "tokens_per_s": 2424.0257750703227, "gpu_gb": 17.690584659576416}
+{"rows": 5800, "of": 6182, "epoch": 1, "ce": 0.08123546589810687, "kl": 0.0006525151126942932, "lr": 4.754187763071488e-07, "rows_per_s": 0.387499722679312, "tokens_per_s": 2424.073062585172, "gpu_gb": 17.690584659576416}
+{"rows": 5850, "of": 6182, "epoch": 1, "ce": 0.04623103610091758, "kl": 0.054546445416483645, "lr": 3.642640044350998e-07, "rows_per_s": 0.3876560700751212, "tokens_per_s": 2424.787790595955, "gpu_gb": 17.690584659576416}
+{"rows": 5900, "of": 6182, "epoch": 1, "ce": 0.049153984285800194, "kl": 0.0025435278770979336, "lr": 2.5314942152586953e-07, "rows_per_s": 0.38769860099075465, "tokens_per_s": 2425.1244692337214, "gpu_gb": 17.690584659576416}
+{"rows": 5950, "of": 6182, "epoch": 1, "ce": 0.07885093773559977, "kl": 0.004603828486814676, "lr": 1.7388697641500251e-07, "rows_per_s": 0.38774285682679077, "tokens_per_s": 2425.1473427194337, "gpu_gb": 17.690584659576416}
+{"rows": 6000, "of": 6182, "epoch": 1, "ce": 0.0627570175906507, "kl": 0.003614848319651679, "lr": 1.0942429556026201e-07, "rows_per_s": 0.3878279527553652, "tokens_per_s": 2425.617767648786, "gpu_gb": 17.690584659576416}
+{"rows": 6050, "of": 6182, "epoch": 1, "ce": 0.07288269579933002, "kl": 0.003465794367637174, "lr": 5.979980755009273e-08, "rows_per_s": 0.3879684555299671, "tokens_per_s": 2426.012731500391, "gpu_gb": 17.690584659576416}
+{"rows": 6100, "of": 6182, "epoch": 1, "ce": 0.0694127718392664, "kl": 0.004462148105772118, "lr": 2.0697372865235986e-08, "rows_per_s": 0.38791520072923097, "tokens_per_s": 2426.2220458192164, "gpu_gb": 17.690584659576416}
+{"rows": 6150, "of": 6182, "epoch": 1, "ce": 0.051383548687359816, "kl": 2.1825850267364304, "lr": 3.311963569402332e-09, "rows_per_s": 0.3879146246896524, "tokens_per_s": 2426.5256317497165, "gpu_gb": 17.690584659576416}
+{"rows": 6182, "of": 6182, "epoch": 1, "ce": 0.05753069252629454, "kl": 0.017210277611863153, "lr": 0.0, "rows_per_s": 0.387977373218742, "tokens_per_s": 2426.4077307052207, "gpu_gb": 17.690584659576416}

ablations/specialists/drag/training_meta.json

@@ -0,0 +1,26 @@
+{
+ "base_model": "Qwen/Qwen3.5-4B",
+ "families": [
+  "Drag & gesture"
+ ],
+ "training_rows": 2473,
+ "rows_trained": 6182,
+ "validation_ce": 0.15570124087472711,
+ "minutes": 267.4,
+ "args": {
+  "adapter": "drag",
+  "epochs": 2.0,
+  "lr": 5e-05,
+  "rank": 64,
+  "alpha": 128,
+  "grad_accum": 8,
+  "beta": 0.1,
+  "anchor_ratio": 0.25,
+  "limit": 0,
+  "max_steps": 0,
+  "seed": 0,
+  "ce_on": "all",
+  "exclude_variants": "",
+  "init_adapter": "the main executor (repository root)"
+ }
+}

ablations/specialists/io/adapter_config.json

@@ -0,0 +1,51 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "kasa_config": null,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.21.0",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "k_proj",
+    "v_proj",
+    "o_proj",
+    "gate_proj",
+    "up_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

ablations/specialists/io/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6f2241616ffe739f41a89cd8477a0285ec741a1a12943f18a710c710e3586493
+size 339776632

ablations/specialists/io/train_log.jsonl

@@ -0,0 +1,97 @@
+{"rows": 50, "of": 4805, "epoch": 0, "ce": 0.08602714834687038, "kl": 0.0001946775850565125, "lr": 1.9439647114666836e-05, "rows_per_s": 0.20859978171381313, "tokens_per_s": 1278.0533145998245, "gpu_gb": 16.012765407562256}
+{"rows": 100, "of": 4805, "epoch": 0, "ce": 0.08636152779799885, "kl": 0.0027777784412321427, "lr": 3.6075482596621576e-05, "rows_per_s": 0.2608457337755108, "tokens_per_s": 1623.4125510119577, "gpu_gb": 16.012765407562256}
+{"rows": 150, "of": 4805, "epoch": 0, "ce": 0.10782670399004762, "kl": 0.012027078248175496, "lr": 4.9889049115077005e-05, "rows_per_s": 0.29253193199934674, "tokens_per_s": 1846.8262445884359, "gpu_gb": 16.179973602294922}
+{"rows": 200, "of": 4805, "epoch": 0, "ce": 0.12043155184606226, "kl": 0.0183020033906901, "lr": 4.9786121534345265e-05, "rows_per_s": 0.31050867096931717, "tokens_per_s": 1977.5831491029357, "gpu_gb": 16.184417724609375}
+{"rows": 250, "of": 4805, "epoch": 0, "ce": 0.11903609485099358, "kl": 0.001348626736702302, "lr": 4.9671392910170185e-05, "rows_per_s": 0.32610168171979764, "tokens_per_s": 2057.9677106242066, "gpu_gb": 16.343734741210938}
+{"rows": 300, "of": 4805, "epoch": 0, "ce": 0.11769211776554585, "kl": 0.0009666763491038982, "lr": 4.9532316599806124e-05, "rows_per_s": 0.3356242148520967, "tokens_per_s": 2117.40282786965, "gpu_gb": 16.343734741210938}
+{"rows": 350, "of": 4805, "epoch": 0, "ce": 0.11725400494178757, "kl": 0.0012186336495273453, "lr": 4.936902985478055e-05, "rows_per_s": 0.34316199547774306, "tokens_per_s": 2159.2968529386426, "gpu_gb": 16.343734741210938}
+{"rows": 400, "of": 4805, "epoch": 0, "ce": 0.12584537257616585, "kl": 0.002219905391672887, "lr": 4.914814565722671e-05, "rows_per_s": 0.3469136887398252, "tokens_per_s": 2188.6090795218092, "gpu_gb": 16.343734741210938}
+{"rows": 450, "of": 4805, "epoch": 0, "ce": 0.11229788962130745, "kl": 0.0012886176550705056, "lr": 4.893298743830168e-05, "rows_per_s": 0.35099078002837075, "tokens_per_s": 2211.8596579515856, "gpu_gb": 16.343734741210938}
+{"rows": 500, "of": 4805, "epoch": 0, "ce": 0.1418607813705291, "kl": 0.001721094902154205, "lr": 4.869421025023965e-05, "rows_per_s": 0.3548053520306521, "tokens_per_s": 2237.482735914851, "gpu_gb": 16.343734741210938}
+{"rows": 550, "of": 4805, "epoch": 0, "ce": 0.11016227715220805, "kl": 0.0022455191134570208, "lr": 4.843204973729729e-05, "rows_per_s": 0.3569290900863634, "tokens_per_s": 2252.1557153608096, "gpu_gb": 16.343734741210938}
+{"rows": 600, "of": 4805, "epoch": 0, "ce": 0.12122889395271029, "kl": 9.463228651816295e-05, "lr": 4.8096988312782174e-05, "rows_per_s": 0.35987657075093366, "tokens_per_s": 2262.964456104614, "gpu_gb": 16.343734741210938}
+{"rows": 650, "of": 4805, "epoch": 0, "ce": 0.1280615866099569, "kl": 0.0005717197266709902, "lr": 4.7785081915886134e-05, "rows_per_s": 0.3633954400742014, "tokens_per_s": 2279.3134783093496, "gpu_gb": 16.343734741210938}
+{"rows": 700, "of": 4805, "epoch": 0, "ce": 0.10683306705738817, "kl": 0.0018723622799549783, "lr": 4.745068939401539e-05, "rows_per_s": 0.36484826136566484, "tokens_per_s": 2294.1330311237775, "gpu_gb": 16.343734741210938}
+{"rows": 750, "of": 4805, "epoch": 0, "ce": 0.12962917967631324, "kl": 0.001015078666040608, "lr": 4.709414075221734e-05, "rows_per_s": 0.3663949228531724, "tokens_per_s": 2305.081814136739, "gpu_gb": 16.343734741210938}
+{"rows": 800, "of": 4805, "epoch": 0, "ce": 0.13292814248605914, "kl": 0.00031961324276297276, "lr": 4.665063509461097e-05, "rows_per_s": 0.36780878098432335, "tokens_per_s": 2319.727683658314, "gpu_gb": 16.343734741210938}
+{"rows": 850, "of": 4805, "epoch": 0, "ce": 0.12035210953326896, "kl": 0.0012915700082216652, "lr": 4.6247317324671605e-05, "rows_per_s": 0.3681565274538961, "tokens_per_s": 2328.060756997012, "gpu_gb": 16.343734741210938}
+{"rows": 900, "of": 4805, "epoch": 0, "ce": 0.11073057440808043, "kl": 9.72073914659588e-05, "lr": 4.5823031017752485e-05, "rows_per_s": 0.3699959719331366, "tokens_per_s": 2334.558239720251, "gpu_gb": 16.343734741210938}
+{"rows": 950, "of": 4805, "epoch": 0, "ce": 0.13753625779197767, "kl": 0.0068689294825748475, "lr": 4.537819489321386e-05, "rows_per_s": 0.3718962854355414, "tokens_per_s": 2340.5386676631592, "gpu_gb": 16.343734741210938}
+{"rows": 1000, "of": 4805, "epoch": 0, "ce": 0.15512315002647606, "kl": 0.11251606029672212, "lr": 4.4833833507280884e-05, "rows_per_s": 0.37175856975266364, "tokens_per_s": 2343.6686290860434, "gpu_gb": 16.52502202987671}
+{"rows": 1050, "of": 4805, "epoch": 0, "ce": 0.1442013806066452, "kl": 0.0025443030373730417, "lr": 4.434600524316266e-05, "rows_per_s": 0.3723488710459603, "tokens_per_s": 2348.203615914479, "gpu_gb": 16.52502202987671}
+{"rows": 1100, "of": 4805, "epoch": 0, "ce": 0.12619986641220748, "kl": 1.1251534512979333e-05, "lr": 4.383908480754095e-05, "rows_per_s": 0.3739675540592181, "tokens_per_s": 2355.6274025175776, "gpu_gb": 16.52502202987671}
+{"rows": 1150, "of": 4805, "epoch": 0, "ce": 0.12623497821088578, "kl": 0.00020509865929137976, "lr": 4.3313572469684474e-05, "rows_per_s": 0.37365886799147346, "tokens_per_s": 2358.5581690565245, "gpu_gb": 16.52502202987671}
+{"rows": 1200, "of": 4805, "epoch": 0, "ce": 0.11506305815918105, "kl": 0.0017093990933488474, "lr": 4.267766952966369e-05, "rows_per_s": 0.37415673101059366, "tokens_per_s": 2360.1005273149335, "gpu_gb": 16.52502202987671}
+{"rows": 1250, "of": 4805, "epoch": 0, "ce": 0.13507370395879997, "kl": 0.002966944433639753, "lr": 4.211367764821722e-05, "rows_per_s": 0.37430762596135936, "tokens_per_s": 2362.6818386896343, "gpu_gb": 16.52502202987671}
+{"rows": 1300, "of": 4805, "epoch": 0, "ce": 0.09945599765617501, "kl": 0.00031120618288823853, "lr": 4.1532796633091296e-05, "rows_per_s": 0.3748988726572576, "tokens_per_s": 2365.2159355810195, "gpu_gb": 16.52502202987671}
+{"rows": 1350, "of": 4805, "epoch": 0, "ce": 0.1260320895362827, "kl": 0.0022714128640585443, "lr": 4.093559974371725e-05, "rows_per_s": 0.3752552619974217, "tokens_per_s": 2369.2886189673463, "gpu_gb": 16.52502202987671}
+{"rows": 1400, "of": 4805, "epoch": 0, "ce": 0.11876631107491752, "kl": 6.925721401598861e-05, "lr": 4.021903572521802e-05, "rows_per_s": 0.3762959392370958, "tokens_per_s": 2374.495378637951, "gpu_gb": 16.52502202987671}
+{"rows": 1450, "of": 4805, "epoch": 0, "ce": 0.1290086665876994, "kl": 9.223961501068911e-05, "lr": 3.958853028390294e-05, "rows_per_s": 0.3764341624715119, "tokens_per_s": 2376.639151594104, "gpu_gb": 16.52502202987671}
+{"rows": 1500, "of": 4805, "epoch": 0, "ce": 0.13341115951174642, "kl": 0.00048145186873318726, "lr": 3.894362772449226e-05, "rows_per_s": 0.37651282374820627, "tokens_per_s": 2379.7154163464006, "gpu_gb": 16.52502202987671}
+{"rows": 1550, "of": 4805, "epoch": 0, "ce": 0.12405679622091927, "kl": 0.002763456657046894, "lr": 3.828496448795207e-05, "rows_per_s": 0.37654001286121863, "tokens_per_s": 2382.323962429468, "gpu_gb": 16.52502202987671}
+{"rows": 1600, "of": 4805, "epoch": 0, "ce": 0.13424859549372625, "kl": 0.0005612914170502217, "lr": 3.7500000000000003e-05, "rows_per_s": 0.3762619304710462, "tokens_per_s": 2384.1098810259296, "gpu_gb": 16.52502202987671}
+{"rows": 1650, "of": 4805, "epoch": 0, "ce": 0.12958910010222877, "kl": 0.006393607735600826, "lr": 3.681376912172636e-05, "rows_per_s": 0.37695917830820075, "tokens_per_s": 2387.2763078029175, "gpu_gb": 16.52502202987671}
+{"rows": 1700, "of": 4805, "epoch": 0, "ce": 0.13314329775748773, "kl": 0.0028233122412523737, "lr": 3.611587947962319e-05, "rows_per_s": 0.3770909491354674, "tokens_per_s": 2389.621130125555, "gpu_gb": 16.52502202987671}
+{"rows": 1750, "of": 4805, "epoch": 0, "ce": 0.13098562972094768, "kl": 0.0031766026694391132, "lr": 3.540701980651003e-05, "rows_per_s": 0.3774288729920741, "tokens_per_s": 2390.8771143787208, "gpu_gb": 16.52502202987671}
+{"rows": 1800, "of": 4805, "epoch": 0, "ce": 0.11381365333218127, "kl": 0.00016535488479195448, "lr": 3.456708580912725e-05, "rows_per_s": 0.3778966586617272, "tokens_per_s": 2392.7780300418485, "gpu_gb": 16.52502202987671}
+{"rows": 1850, "of": 4805, "epoch": 0, "ce": 0.12338092419600417, "kl": 0.0006300521330742025, "lr": 3.383687109448143e-05, "rows_per_s": 0.37860850137806146, "tokens_per_s": 2395.6399714853405, "gpu_gb": 16.52502202987671}
+{"rows": 1900, "of": 4805, "epoch": 0, "ce": 0.12542712756178595, "kl": 0.001098281008343564, "lr": 3.309793545495374e-05, "rows_per_s": 0.3786067417701924, "tokens_per_s": 2399.128698777431, "gpu_gb": 16.52502202987671}
+{"rows": 1950, "of": 4805, "epoch": 0, "ce": 0.106699604625729, "kl": 0.0021750875750333374, "lr": 3.23510081308076e-05, "rows_per_s": 0.3786320734779549, "tokens_per_s": 2402.1810942450093, "gpu_gb": 16.52502202987671}
+{"rows": 2000, "of": 4805, "epoch": 0, "ce": 0.13178050508722663, "kl": 0.0006399955536364743, "lr": 3.147047612756302e-05, "rows_per_s": 0.37943332750617154, "tokens_per_s": 2403.3805919066576, "gpu_gb": 16.52502202987671}
+{"rows": 2050, "of": 4805, "epoch": 0, "ce": 0.11574270468401282, "kl": 0.0010900345375344674, "lr": 3.0708771752766394e-05, "rows_per_s": 0.3797791825746351, "tokens_per_s": 2404.0289028692896, "gpu_gb": 16.52502202987671}
+{"rows": 2100, "of": 4805, "epoch": 0, "ce": 0.13457851182846797, "kl": 0.011422056110934158, "lr": 2.9941433509478156e-05, "rows_per_s": 0.37980655718637507, "tokens_per_s": 2406.117041938038, "gpu_gb": 16.52502202987671}
+{"rows": 2150, "of": 4805, "epoch": 0, "ce": 0.12962375564331358, "kl": 0.0009690414605171544, "lr": 2.916921866790256e-05, "rows_per_s": 0.37983615166091705, "tokens_per_s": 2405.0341783281688, "gpu_gb": 16.52502202987671}
+{"rows": 2200, "of": 4805, "epoch": 0, "ce": 0.1427700865225723, "kl": 0.00012294368984069125, "lr": 2.8263154805501297e-05, "rows_per_s": 0.380139543983249, "tokens_per_s": 2405.686321539211, "gpu_gb": 16.633502960205078}
+{"rows": 2250, "of": 4805, "epoch": 0, "ce": 0.13575956278613635, "kl": 0.0036402833863604287, "lr": 2.7482993743590978e-05, "rows_per_s": 0.3800022364151225, "tokens_per_s": 2405.386965236586, "gpu_gb": 16.633502960205078}
+{"rows": 2300, "of": 4805, "epoch": 0, "ce": 0.12829517616945155, "kl": 8.409870674697291e-05, "lr": 2.6700382266631206e-05, "rows_per_s": 0.3794780177405293, "tokens_per_s": 2404.832713757672, "gpu_gb": 16.633502960205078}
+{"rows": 2350, "of": 4805, "epoch": 0, "ce": 0.11949703888967633, "kl": 0.0020544007294776012, "lr": 2.591609271766391e-05, "rows_per_s": 0.3803478404857871, "tokens_per_s": 2405.8593516151727, "gpu_gb": 16.633502960205078}
+{"rows": 2400, "of": 4805, "epoch": 0, "ce": 0.10921200581101907, "kl": 0.00027792147261784805, "lr": 2.5e-05, "rows_per_s": 0.3806032988295618, "tokens_per_s": 2408.5423592274565, "gpu_gb": 16.633502960205078}
+{"rows": 2450, "of": 4805, "epoch": 0, "ce": 0.11857849301006167, "kl": 0.0012941636291745708, "lr": 2.4214731023046793e-05, "rows_per_s": 0.38067263680304936, "tokens_per_s": 2409.1057379516446, "gpu_gb": 16.633502960205078}
+{"rows": 2500, "of": 4805, "epoch": 0, "ce": 0.1231865936091968, "kl": 0.00011086609606785619, "lr": 2.3430237011767167e-05, "rows_per_s": 0.3809627892256242, "tokens_per_s": 2410.4253218265185, "gpu_gb": 16.633502960205078}
+{"rows": 2550, "of": 4805, "epoch": 0, "ce": 0.10426508846717912, "kl": 0.000889719753116025, "lr": 2.2647292167037144e-05, "rows_per_s": 0.3813130655756024, "tokens_per_s": 2410.690658872456, "gpu_gb": 16.633502960205078}
+{"rows": 2600, "of": 4805, "epoch": 0, "ce": 0.129819553981467, "kl": 0.00046119710398053983, "lr": 2.173684519449872e-05, "rows_per_s": 0.38101907161727666, "tokens_per_s": 2411.7780366221605, "gpu_gb": 16.633502960205078}
+{"rows": 2650, "of": 4805, "epoch": 0, "ce": 0.1319959158214127, "kl": 0.0011903562325068754, "lr": 2.0959904472415977e-05, "rows_per_s": 0.38140088752530305, "tokens_per_s": 2412.748059329322, "gpu_gb": 16.633502960205078}
+{"rows": 2700, "of": 4805, "epoch": 0, "ce": 0.1017441232688725, "kl": 0.0002819695422713875, "lr": 2.018695083685232e-05, "rows_per_s": 0.38146580538184566, "tokens_per_s": 2413.598434703326, "gpu_gb": 16.633502960205078}
+{"rows": 2750, "of": 4805, "epoch": 0, "ce": 0.10915328126866371, "kl": 0.004004245639250392, "lr": 1.941874709972622e-05, "rows_per_s": 0.3819160618445792, "tokens_per_s": 2414.5022297235623, "gpu_gb": 16.633502960205078}
+{"rows": 2800, "of": 4805, "epoch": 0, "ce": 0.1270165562902282, "kl": 0.005176704357219795, "lr": 1.852952387243698e-05, "rows_per_s": 0.38209887039449375, "tokens_per_s": 2415.704113769246, "gpu_gb": 16.633502960205078}
+{"rows": 2850, "of": 4805, "epoch": 0, "ce": 0.10900447165241113, "kl": 0.00075658452786067, "lr": 1.7774205076388206e-05, "rows_per_s": 0.382188775941173, "tokens_per_s": 2415.569444995628, "gpu_gb": 16.633502960205078}
+{"rows": 2900, "of": 4805, "epoch": 0, "ce": 0.11996678574498565, "kl": 0.0005498259879993701, "lr": 1.702601726754825e-05, "rows_per_s": 0.38249764821837084, "tokens_per_s": 2416.615129405518, "gpu_gb": 16.633502960205078}
+{"rows": 2950, "of": 4805, "epoch": 0, "ce": 0.1103495519004158, "kl": 0.006639791636532549, "lr": 1.6285698816954624e-05, "rows_per_s": 0.38279051925837804, "tokens_per_s": 2416.663729423533, "gpu_gb": 16.633502960205078}
+{"rows": 3000, "of": 4805, "epoch": 0, "ce": 0.11051605468881982, "kl": 0.0007683531629627446, "lr": 1.5432914190872757e-05, "rows_per_s": 0.3827304719169647, "tokens_per_s": 2416.1582051113783, "gpu_gb": 16.633502960205078}
+{"rows": 3050, "of": 4805, "epoch": 0, "ce": 0.10751782339066267, "kl": 0.026312432329801044, "lr": 1.4712141034872282e-05, "rows_per_s": 0.3827452689060034, "tokens_per_s": 2416.9704907590585, "gpu_gb": 16.633502960205078}
+{"rows": 3100, "of": 4805, "epoch": 0, "ce": 0.10954811533865257, "kl": 0.0018470598737594832, "lr": 1.4001520753602121e-05, "rows_per_s": 0.38215446843972495, "tokens_per_s": 2412.7065951621275, "gpu_gb": 17.41025400161743}
+{"rows": 3150, "of": 4805, "epoch": 0, "ce": 0.12056146329268813, "kl": 0.0020209783978116513, "lr": 1.330175464348567e-05, "rows_per_s": 0.3820494773315145, "tokens_per_s": 2413.910974898803, "gpu_gb": 17.41025400161743}
+{"rows": 3200, "of": 4805, "epoch": 0, "ce": 0.10239862964341515, "kl": 0.0007717651149781341, "lr": 1.2500000000000006e-05, "rows_per_s": 0.38223548817459585, "tokens_per_s": 2414.360837804501, "gpu_gb": 17.41025400161743}
+{"rows": 3250, "of": 4805, "epoch": 0, "ce": 0.09482319156737312, "kl": 0.010629187128166802, "lr": 1.1826105112583061e-05, "rows_per_s": 0.3826996946110347, "tokens_per_s": 2414.6360691544314, "gpu_gb": 17.41025400161743}
+{"rows": 3300, "of": 4805, "epoch": 0, "ce": 0.08952410409360059, "kl": 0.0012268220493023493, "lr": 1.11652112689164e-05, "rows_per_s": 0.38299184735175823, "tokens_per_s": 2415.89168263049, "gpu_gb": 17.41025400161743}
+{"rows": 3350, "of": 4805, "epoch": 0, "ce": 0.09270484868902713, "kl": 0.0002554617213230692, "lr": 1.0517970691433035e-05, "rows_per_s": 0.383058541760543, "tokens_per_s": 2416.6145721755743, "gpu_gb": 17.41025400161743}
+{"rows": 3400, "of": 4805, "epoch": 0, "ce": 0.09283079765737057, "kl": 0.0003019437882237491, "lr": 9.780964274781984e-06, "rows_per_s": 0.38349467060512893, "tokens_per_s": 2417.380876291815, "gpu_gb": 17.41025400161743}
+{"rows": 3450, "of": 4805, "epoch": 0, "ce": 0.0905792395185147, "kl": 0.0003616955569720262, "lr": 9.16547818431125e-06, "rows_per_s": 0.38361945980222867, "tokens_per_s": 2418.734704502888, "gpu_gb": 17.41025400161743}
+{"rows": 3500, "of": 4805, "epoch": 0, "ce": 0.10580950789153576, "kl": 0.00046570411970184067, "lr": 8.565618855147603e-06, "rows_per_s": 0.38387253940557386, "tokens_per_s": 2418.982129683038, "gpu_gb": 17.41025400161743}
+{"rows": 3550, "of": 4805, "epoch": 0, "ce": 0.09163989548881848, "kl": 9.403243897504465e-06, "lr": 7.981978276027055e-06, "rows_per_s": 0.3843070534544131, "tokens_per_s": 2420.4207081984573, "gpu_gb": 17.41025400161743}
+{"rows": 3600, "of": 4805, "epoch": 0, "ce": 0.11685588437972999, "kl": 0.0010777893395931086, "lr": 7.3223304703363135e-06, "rows_per_s": 0.3845154259349837, "tokens_per_s": 2421.3061338639354, "gpu_gb": 17.41025400161743}
+{"rows": 3650, "of": 4805, "epoch": 0, "ce": 0.0981252357745782, "kl": 0.0007672295101955102, "lr": 6.775784314464717e-06, "rows_per_s": 0.384595656572058, "tokens_per_s": 2421.8058037666697, "gpu_gb": 17.41025400161743}
+{"rows": 3700, "of": 4805, "epoch": 0, "ce": 0.11468390340451151, "kl": 0.0002968026597268969, "lr": 6.247223259238511e-06, "rows_per_s": 0.38454883889470876, "tokens_per_s": 2422.494095881957, "gpu_gb": 17.41025400161743}
+{"rows": 3750, "of": 4805, "epoch": 0, "ce": 0.10285064672776074, "kl": 0.03539560242552207, "lr": 5.737168930605272e-06, "rows_per_s": 0.38482051790657773, "tokens_per_s": 2423.6092679432754, "gpu_gb": 17.41025400161743}
+{"rows": 3800, "of": 4805, "epoch": 0, "ce": 0.11344841504469513, "kl": 0.000977099261271236, "lr": 5.166166492719124e-06, "rows_per_s": 0.3849827574851969, "tokens_per_s": 2424.2710723324585, "gpu_gb": 17.41025400161743}
+{"rows": 3850, "of": 4805, "epoch": 0, "ce": 0.10722692810544153, "kl": 0.00033676227604407003, "lr": 4.6979118277048426e-06, "rows_per_s": 0.38519077541514674, "tokens_per_s": 2424.970327517993, "gpu_gb": 17.41025400161743}
+{"rows": 3900, "of": 4805, "epoch": 0, "ce": 0.10017820305960334, "kl": 6.525680912305987e-08, "lr": 4.2496928726158155e-06, "rows_per_s": 0.3856707196200146, "tokens_per_s": 2426.4540570044333, "gpu_gb": 17.41025400161743}
+{"rows": 3950, "of": 4805, "epoch": 0, "ce": 0.10914792331229699, "kl": 0.0014828807664768462, "lr": 3.821951965446577e-06, "rows_per_s": 0.3858927995696698, "tokens_per_s": 2426.599335929713, "gpu_gb": 17.41025400161743}
+{"rows": 4000, "of": 4805, "epoch": 0, "ce": 0.09310039771764084, "kl": 1.910545646291847e-07, "lr": 3.3493649053890326e-06, "rows_per_s": 0.38599135842518706, "tokens_per_s": 2427.5469370774085, "gpu_gb": 17.41025400161743}
+{"rows": 4050, "of": 4805, "epoch": 0, "ce": 0.10774478973711238, "kl": 0.005401443111193501, "lr": 2.967413698375196e-06, "rows_per_s": 0.3861269928492347, "tokens_per_s": 2426.9699420344436, "gpu_gb": 17.41025400161743}
+{"rows": 4100, "of": 4805, "epoch": 0, "ce": 0.10233193307706132, "kl": 0.015932930374073076, "lr": 2.6072059940146775e-06, "rows_per_s": 0.3860478225224352, "tokens_per_s": 2427.6773621612456, "gpu_gb": 17.41025400161743}
+{"rows": 4150, "of": 4805, "epoch": 0, "ce": 0.08947834717791255, "kl": 0.00035421041075480196, "lr": 2.269097273823287e-06, "rows_per_s": 0.38622881602772946, "tokens_per_s": 2428.444113743708, "gpu_gb": 17.41025400161743}
+{"rows": 4200, "of": 4805, "epoch": 0, "ce": 0.09941394574707374, "kl": 0.00039840244160667854, "lr": 1.9030116872178316e-06, "rows_per_s": 0.3860117491198172, "tokens_per_s": 2429.450784745969, "gpu_gb": 17.41025400161743}
+{"rows": 4250, "of": 4805, "epoch": 0, "ce": 0.0789804194953192, "kl": 0.000581689607462299, "lr": 1.6138992292533183e-06, "rows_per_s": 0.38594374246473734, "tokens_per_s": 2429.4823498176465, "gpu_gb": 17.41025400161743}
+{"rows": 4300, "of": 4805, "epoch": 0, "ce": 0.10094443928789008, "kl": 0.0007254295433808492, "lr": 1.3478660293113676e-06, "rows_per_s": 0.3857829324681441, "tokens_per_s": 2430.563102444581, "gpu_gb": 17.41025400161743}
+{"rows": 4350, "of": 4805, "epoch": 0, "ce": 0.10133733936365591, "kl": 0.0010733331467564975, "lr": 1.105174630041747e-06, "rows_per_s": 0.38559461464132666, "tokens_per_s": 2430.3883187240976, "gpu_gb": 17.41025400161743}
+{"rows": 4400, "of": 4805, "epoch": 0, "ce": 0.12300259061157703, "kl": 0.00025951686538877583, "lr": 8.51854342773295e-07, "rows_per_s": 0.38595265750354973, "tokens_per_s": 2431.376921674266, "gpu_gb": 17.41025400161743}
+{"rows": 4450, "of": 4805, "epoch": 0, "ce": 0.11402995124435386, "kl": 0.016768689516305224, "lr": 6.605274281709928e-07, "rows_per_s": 0.386150048315068, "tokens_per_s": 2431.5537928538242, "gpu_gb": 17.41025400161743}
+{"rows": 4500, "of": 4805, "epoch": 0, "ce": 0.07664989593046669, "kl": 0.048717232224052553, "lr": 4.932206344569562e-07, "rows_per_s": 0.3861258112151464, "tokens_per_s": 2431.3741692066988, "gpu_gb": 17.41025400161743}
+{"rows": 4550, "of": 4805, "epoch": 0, "ce": 0.0861509177761729, "kl": 0.014684346029274885, "lr": 3.5009907323737825e-07, "rows_per_s": 0.3861005851038494, "tokens_per_s": 2431.982669756487, "gpu_gb": 17.41025400161743}
+{"rows": 4600, "of": 4805, "epoch": 0, "ce": 0.0966363332104371, "kl": 0.0005315006876702563, "lr": 2.1387846565474045e-07, "rows_per_s": 0.38616556272305647, "tokens_per_s": 2432.754982617148, "gpu_gb": 17.41025400161743}
+{"rows": 4650, "of": 4805, "epoch": 0, "ce": 0.0996280340729533, "kl": 0.051464440686500884, "lr": 1.2361075096934406e-07, "rows_per_s": 0.38599998804961416, "tokens_per_s": 2432.724913501136, "gpu_gb": 17.41025400161743}
+{"rows": 4700, "of": 4805, "epoch": 0, "ce": 0.11196290842520121, "kl": 0.002237863708445304, "lr": 5.7893029109867823e-08, "rows_per_s": 0.386083287539002, "tokens_per_s": 2433.019497122522, "gpu_gb": 17.41025400161743}
+{"rows": 4750, "of": 4805, "epoch": 0, "ce": 0.08573512508510014, "kl": 0.006358081641883198, "lr": 1.67901555335942e-08, "rows_per_s": 0.38627049914048983, "tokens_per_s": 2433.2456279710295, "gpu_gb": 17.41025400161743}
+{"rows": 4800, "of": 4805, "epoch": 0, "ce": 0.09952633676314077, "kl": 0.7836807063572451, "lr": 0.0, "rows_per_s": 0.3862629838361425, "tokens_per_s": 2434.094856334122, "gpu_gb": 17.41025400161743}
+{"rows": 4805, "of": 4805, "epoch": 0, "ce": 0.11093618161976337, "kl": -2.8383165262546584e-09, "lr": 0.0, "rows_per_s": 0.3862668187278562, "tokens_per_s": 2434.078405435397, "gpu_gb": 17.41025400161743}

ablations/specialists/io/training_meta.json

@@ -0,0 +1,26 @@
+{
+ "base_model": "Qwen/Qwen3.5-4B",
+ "families": [
+  "Out-of-page I/O"
+ ],
+ "training_rows": 3844,
+ "rows_trained": 4805,
+ "validation_ce": 0.1317719433363527,
+ "minutes": 209.9,
+ "args": {
+  "adapter": "io",
+  "epochs": 1.0,
+  "lr": 5e-05,
+  "rank": 64,
+  "alpha": 128,
+  "grad_accum": 8,
+  "beta": 0.1,
+  "anchor_ratio": 0.25,
+  "limit": 0,
+  "max_steps": 0,
+  "seed": 0,
+  "ce_on": "all",
+  "exclude_variants": "",
+  "init_adapter": "the main executor (repository root)"
+ }
+}

ablations/specialists/reasoning/adapter_config.json

@@ -0,0 +1,51 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "kasa_config": null,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.21.0",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "k_proj",
+    "v_proj",
+    "o_proj",
+    "gate_proj",
+    "up_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

ablations/specialists/reasoning/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:05dbdee573f43b46cd270eda66d382b9c8c5bb11ecf34b2538cb26b33a9fdcd0
+size 339776632

ablations/specialists/reasoning/train_log.jsonl

@@ -0,0 +1,29 @@
+{"rows": 200, "of": 5760, "epoch": 0, "ce": 0.2577789610044824, "kl": 0.16837326706633707, "lr": 9.970281691111598e-05, "rows_per_s": 0.6307757672736115, "tokens_per_s": 3947.8362946353523, "gpu_gb": 15.256054878234863}
+{"rows": 400, "of": 5760, "epoch": 0, "ce": 0.2544304948532954, "kl": 0.03746419742431802, "lr": 9.881480035599667e-05, "rows_per_s": 0.810804043759091, "tokens_per_s": 5223.078029289501, "gpu_gb": 15.81239128112793}
+{"rows": 600, "of": 5760, "epoch": 0, "ce": 0.26670797203074803, "kl": 0.12841644801055202, "lr": 9.73465064747553e-05, "rows_per_s": 0.8946281392943494, "tokens_per_s": 5702.097335607989, "gpu_gb": 15.81239128112793}
+{"rows": 800, "of": 5760, "epoch": 0, "ce": 0.28882892028643536, "kl": 0.0543268827702074, "lr": 9.53153893518325e-05, "rows_per_s": 0.9577780459271861, "tokens_per_s": 6050.3126494634125, "gpu_gb": 15.81239128112793}
+{"rows": 1000, "of": 5760, "epoch": 0, "ce": 0.2832969942688942, "kl": 0.0968433364352677, "lr": 9.274559353364734e-05, "rows_per_s": 0.9814088165838669, "tokens_per_s": 6191.390247371043, "gpu_gb": 15.81239128112793}
+{"rows": 1200, "of": 5760, "epoch": 0, "ce": 0.3009799416176975, "kl": 0.03366438765927645, "lr": 8.966766701456177e-05, "rows_per_s": 1.0096780020469922, "tokens_per_s": 6375.850701052884, "gpu_gb": 16.716597080230713}
+{"rows": 1400, "of": 5760, "epoch": 0, "ce": 0.31191022992134093, "kl": 0.04753414500737563, "lr": 8.611819810298778e-05, "rows_per_s": 1.0293943863266701, "tokens_per_s": 6520.79285624447, "gpu_gb": 17.835293769836426}
+{"rows": 1600, "of": 5760, "epoch": 0, "ce": 0.2816520441127451, "kl": 0.12532968321465887, "lr": 8.213938048432697e-05, "rows_per_s": 1.0407932081002564, "tokens_per_s": 6678.473390315037, "gpu_gb": 18.370572090148926}
+{"rows": 1800, "of": 5760, "epoch": 0, "ce": 0.24644574816479828, "kl": 0.14326527389076707, "lr": 7.777851165098012e-05, "rows_per_s": 1.049856546921072, "tokens_per_s": 6741.348191147784, "gpu_gb": 18.72653341293335}
+{"rows": 2000, "of": 5760, "epoch": 1, "ce": 0.2466322773232542, "kl": 0.08163721362333967, "lr": 7.308743066175172e-05, "rows_per_s": 1.0426453482833684, "tokens_per_s": 6745.886209323641, "gpu_gb": 18.72653341293335}
+{"rows": 2200, "of": 5760, "epoch": 1, "ce": 0.1897313885133842, "kl": 0.06584541735243138, "lr": 6.812190191418508e-05, "rows_per_s": 1.0555948426347834, "tokens_per_s": 6843.49045828191, "gpu_gb": 18.72653341293335}
+{"rows": 2400, "of": 5760, "epoch": 1, "ce": 0.1459376883293901, "kl": 0.10906424961285666, "lr": 6.294095225512603e-05, "rows_per_s": 1.0649558234977126, "tokens_per_s": 6932.935182951382, "gpu_gb": 20.68865966796875}
+{"rows": 2600, "of": 5760, "epoch": 1, "ce": 0.1854311903672559, "kl": 0.09118593924722518, "lr": 5.7606169309495836e-05, "rows_per_s": 1.071274579992721, "tokens_per_s": 7007.978345429982, "gpu_gb": 20.68865966796875}
+{"rows": 2800, "of": 5760, "epoch": 1, "ce": 0.17243211964766184, "kl": 0.3702436981299851, "lr": 5.218096936826681e-05, "rows_per_s": 1.070039968218501, "tokens_per_s": 7056.956391999743, "gpu_gb": 20.68865966796875}
+{"rows": 3000, "of": 5760, "epoch": 1, "ce": 0.17879176321320045, "kl": 0.025740427888443017, "lr": 4.6729843538492847e-05, "rows_per_s": 1.0755347038921959, "tokens_per_s": 7108.409624501583, "gpu_gb": 20.68865966796875}
+{"rows": 3200, "of": 5760, "epoch": 1, "ce": 0.16701541836254108, "kl": 0.03204973436368164, "lr": 4.131759111665349e-05, "rows_per_s": 1.0804125320344247, "tokens_per_s": 7160.172056019132, "gpu_gb": 20.99252700805664}
+{"rows": 3400, "of": 5760, "epoch": 1, "ce": 0.20029758037451437, "kl": 0.09656489826738834, "lr": 3.60085492984504e-05, "rows_per_s": 1.0882799183947878, "tokens_per_s": 7210.179663011671, "gpu_gb": 20.99252700805664}
+{"rows": 3600, "of": 5760, "epoch": 1, "ce": 0.16556619326102323, "kl": 0.04844531934871893, "lr": 3.086582838174551e-05, "rows_per_s": 1.0910289527226837, "tokens_per_s": 7234.691782039589, "gpu_gb": 20.99252700805664}
+{"rows": 3800, "of": 5760, "epoch": 1, "ce": 0.15622588924386285, "kl": 0.0237639391407891, "lr": 2.595056155403063e-05, "rows_per_s": 1.0968982378400165, "tokens_per_s": 7259.713606579813, "gpu_gb": 20.99252700805664}
+{"rows": 4000, "of": 5760, "epoch": 2, "ce": 0.09368964977329597, "kl": 0.03326954931581996, "lr": 2.132117818244771e-05, "rows_per_s": 1.0920866362557793, "tokens_per_s": 7229.689978077797, "gpu_gb": 20.99252700805664}
+{"rows": 4200, "of": 5760, "epoch": 2, "ce": 0.07026117825598428, "kl": 0.07036887253086795, "lr": 1.703270924499656e-05, "rows_per_s": 1.0942484435493463, "tokens_per_s": 7240.796187889496, "gpu_gb": 20.99252700805664}
+{"rows": 4400, "of": 5760, "epoch": 2, "ce": 0.0927521621977741, "kl": 0.09895767261776414, "lr": 1.3136133159493802e-05, "rows_per_s": 1.0984046849259557, "tokens_per_s": 7256.831230450025, "gpu_gb": 20.99252700805664}
+{"rows": 4600, "of": 5760, "epoch": 2, "ce": 0.07493404685245718, "kl": 0.007646463126036451, "lr": 9.677769786625867e-06, "rows_per_s": 1.1003894157143876, "tokens_per_s": 7268.2433687982375, "gpu_gb": 20.99252700805664}
+{"rows": 4800, "of": 5760, "epoch": 2, "ce": 0.08476917690464429, "kl": 0.03275518031053556, "lr": 6.698729810778065e-06, "rows_per_s": 1.102253908686706, "tokens_per_s": 7277.659906996618, "gpu_gb": 20.99252700805664}
+{"rows": 5000, "of": 5760, "epoch": 2, "ce": 0.08343447117065941, "kl": 0.022468336584374218, "lr": 4.234426044027645e-06, "rows_per_s": 1.1055304173254539, "tokens_per_s": 7296.812956137849, "gpu_gb": 20.99252700805664}
+{"rows": 5200, "of": 5760, "epoch": 2, "ce": 0.07309539373964072, "kl": 0.03068972138407844, "lr": 2.314152462588659e-06, "rows_per_s": 1.1094303796001321, "tokens_per_s": 7322.74316266363, "gpu_gb": 20.99252700805664}
+{"rows": 5400, "of": 5760, "epoch": 2, "ce": 0.07884265224520977, "kl": 0.04088277054093355, "lr": 9.607359798384785e-07, "rows_per_s": 1.1096436538028849, "tokens_per_s": 7318.137706910081, "gpu_gb": 20.99252700805664}
+{"rows": 5600, "of": 5760, "epoch": 2, "ce": 0.08927944804908651, "kl": 0.015799456091104624, "lr": 1.9026509541272275e-07, "rows_per_s": 1.1078380699685184, "tokens_per_s": 7338.002058995711, "gpu_gb": 20.99252700805664}
+{"rows": 5760, "of": 5760, "epoch": 2, "ce": 0.07990307508614557, "kl": 0.024603099601253318, "lr": 0.0, "rows_per_s": 1.1101904560740863, "tokens_per_s": 7358.410958796954, "gpu_gb": 20.99252700805664}

ablations/specialists/reasoning/training_meta.json

@@ -0,0 +1,25 @@
+{
+ "base_model": "Qwen/Qwen3.5-4B",
+ "families": [
+  "Reasoning base"
+ ],
+ "training_rows": 1282,
+ "rows_trained": 5760,
+ "validation_ce": 0.2812701124697924,
+ "minutes": 86.9,
+ "args": {
+  "adapter": "reasoning",
+  "epochs": 3.0,
+  "lr": 0.0001,
+  "rank": 64,
+  "alpha": 128,
+  "grad_accum": 8,
+  "beta": 0.1,
+  "anchor_ratio": 0.5,
+  "limit": 0,
+  "max_steps": 0,
+  "seed": 0,
+  "ce_on": "all",
+  "exclude_variants": ""
+ }
+}

ablations/specialists/rest/adapter_config.json

@@ -0,0 +1,51 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "kasa_config": null,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.21.0",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "k_proj",
+    "v_proj",
+    "o_proj",
+    "gate_proj",
+    "up_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

ablations/specialists/rest/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4614e69a7ec2bfed880debc1e4b3da5c3b25f7bec0a34fc9e676e3b1dc346b7f
+size 339776632

ablations/specialists/rest/train_log.jsonl

@@ -0,0 +1,188 @@
+{"rows": 100, "of": 18780, "epoch": 0, "ce": 0.0910480237319408, "kl": 3.3721110921005465e-05, "lr": 9.28511534746903e-06, "rows_per_s": 0.4074049894452845, "tokens_per_s": 2438.4084909277058, "gpu_gb": 14.59104061126709}
+{"rows": 200, "of": 18780, "epoch": 0, "ce": 0.081655192854149, "kl": 0.00028173104264259813, "lr": 1.856622982810083e-05, "rows_per_s": 0.5317701222775152, "tokens_per_s": 3189.5731465242047, "gpu_gb": 15.127739429473877}
+{"rows": 300, "of": 18780, "epoch": 0, "ce": 0.05328094694343191, "kl": 0.0004588219059907682, "lr": 2.7126215974693476e-05, "rows_per_s": 0.5864311802323369, "tokens_per_s": 3506.5847899052656, "gpu_gb": 15.428138256072998}
+{"rows": 400, "of": 18780, "epoch": 0, "ce": 0.117563187610358, "kl": 3.314872252253853e-05, "lr": 3.638779270709435e-05, "rows_per_s": 0.6208603442746593, "tokens_per_s": 3691.78461353975, "gpu_gb": 15.428138256072998}
+{"rows": 500, "of": 18780, "epoch": 0, "ce": 0.09492876022245343, "kl": 0.002296019458442692, "lr": 4.492256091923526e-05, "rows_per_s": 0.6459646174349439, "tokens_per_s": 3824.8908604727294, "gpu_gb": 15.428138256072998}
+{"rows": 600, "of": 18780, "epoch": 0, "ce": 0.08485315197245463, "kl": 0.0012316993251564854, "lr": 4.987412446711993e-05, "rows_per_s": 0.6463897345388093, "tokens_per_s": 3883.3242267185988, "gpu_gb": 16.32239532470703}
+{"rows": 700, "of": 18780, "epoch": 0, "ce": 0.10123045904951733, "kl": 0.0059531385705266105, "lr": 4.9830671049356946e-05, "rows_per_s": 0.6598250131827227, "tokens_per_s": 3979.1671175002543, "gpu_gb": 16.32239532470703}
+{"rows": 800, "of": 18780, "epoch": 0, "ce": 0.11340892525683892, "kl": 3.1772448809512056e-05, "lr": 4.977636744164722e-05, "rows_per_s": 0.6700094961730876, "tokens_per_s": 4061.322861907826, "gpu_gb": 16.32239532470703}
+{"rows": 900, "of": 18780, "epoch": 0, "ce": 0.11091390783648665, "kl": 0.003568099932793858, "lr": 4.971958189107542e-05, "rows_per_s": 0.6800100147662589, "tokens_per_s": 4105.08899054105, "gpu_gb": 16.32239532470703}
+{"rows": 1000, "of": 18780, "epoch": 0, "ce": 0.12637326150434092, "kl": 0.017706028998071388, "lr": 4.96508675908065e-05, "rows_per_s": 0.6889540550939247, "tokens_per_s": 4142.450622625368, "gpu_gb": 16.32239532470703}
+{"rows": 1100, "of": 18780, "epoch": 0, "ce": 0.09797620974215014, "kl": 0.03257625946889998, "lr": 4.9580813491872494e-05, "rows_per_s": 0.6911556077829428, "tokens_per_s": 4179.3086320127695, "gpu_gb": 16.32239532470703}
+{"rows": 1200, "of": 18780, "epoch": 0, "ce": 0.10943058164926564, "kl": 0.009250587916499572, "lr": 4.9497765440461933e-05, "rows_per_s": 0.6916372062980546, "tokens_per_s": 4175.512949087592, "gpu_gb": 17.01154851913452}
+{"rows": 1300, "of": 18780, "epoch": 0, "ce": 0.11972605642911635, "kl": 0.05227775017690751, "lr": 4.941452123479513e-05, "rows_per_s": 0.6947511179625755, "tokens_per_s": 4200.5593178146255, "gpu_gb": 17.01154851913452}
+{"rows": 1400, "of": 18780, "epoch": 0, "ce": 0.10469158575240825, "kl": 5.4756470854391625e-05, "lr": 4.931723242358207e-05, "rows_per_s": 0.6979856983686735, "tokens_per_s": 4206.52248893055, "gpu_gb": 17.01154851913452}
+{"rows": 1500, "of": 18780, "epoch": 0, "ce": 0.12136512486513977, "kl": 0.009002412064880295, "lr": 4.922089132216128e-05, "rows_per_s": 0.7022511291368824, "tokens_per_s": 4231.651071328513, "gpu_gb": 17.01154851913452}
+{"rows": 1600, "of": 18780, "epoch": 0, "ce": 0.1392585281850543, "kl": 0.07483413557946979, "lr": 4.910947068828132e-05, "rows_per_s": 0.7067561058844102, "tokens_per_s": 4254.327213822531, "gpu_gb": 17.01154851913452}
+{"rows": 1700, "of": 18780, "epoch": 0, "ce": 0.10987310542872078, "kl": 0.004048554959746135, "lr": 4.9000140567063704e-05, "rows_per_s": 0.7084390440948259, "tokens_per_s": 4275.228767806831, "gpu_gb": 17.01154851913452}
+{"rows": 1800, "of": 18780, "epoch": 0, "ce": 0.10196332538572069, "kl": 0.0025491969266285123, "lr": 4.8874712871468805e-05, "rows_per_s": 0.7132940349472745, "tokens_per_s": 4292.527417615637, "gpu_gb": 17.01154851913452}
+{"rows": 1900, "of": 18780, "epoch": 0, "ce": 0.13307823205278033, "kl": 0.003333085214681248, "lr": 4.8752516150597974e-05, "rows_per_s": 0.7118554461256669, "tokens_per_s": 4291.859659380613, "gpu_gb": 17.01154851913452}
+{"rows": 2000, "of": 18780, "epoch": 0, "ce": 0.12482323893942894, "kl": 0.0010576527204174653, "lr": 4.8613221838357956e-05, "rows_per_s": 0.7148981793249848, "tokens_per_s": 4301.770397517638, "gpu_gb": 17.01154851913452}
+{"rows": 2100, "of": 18780, "epoch": 0, "ce": 0.09829765979788806, "kl": 0.01791716763258838, "lr": 4.8478295345086555e-05, "rows_per_s": 0.7178217123638071, "tokens_per_s": 4312.637298615998, "gpu_gb": 17.01154851913452}
+{"rows": 2200, "of": 18780, "epoch": 0, "ce": 0.1000944222197742, "kl": 0.010426520941804015, "lr": 4.8325290388128625e-05, "rows_per_s": 0.7194699718442936, "tokens_per_s": 4320.648719443195, "gpu_gb": 17.01154851913452}
+{"rows": 2300, "of": 18780, "epoch": 0, "ce": 0.11542780434310033, "kl": 0.0017028986259613422, "lr": 4.8177785203608855e-05, "rows_per_s": 0.7175032932428814, "tokens_per_s": 4322.583616159967, "gpu_gb": 17.01154851913452}
+{"rows": 2400, "of": 18780, "epoch": 0, "ce": 0.10401739749910408, "kl": 0.030036423725461614, "lr": 4.801124092607129e-05, "rows_per_s": 0.7179289956947192, "tokens_per_s": 4336.067379459112, "gpu_gb": 17.01154851913452}
+{"rows": 2500, "of": 18780, "epoch": 0, "ce": 0.11829645834355192, "kl": 0.0014101894112172886, "lr": 4.785132221618489e-05, "rows_per_s": 0.7200960962834495, "tokens_per_s": 4347.4563247827655, "gpu_gb": 17.01154851913452}
+{"rows": 2600, "of": 18780, "epoch": 0, "ce": 0.1029375484213233, "kl": 0.005930502164816313, "lr": 4.767142510258034e-05, "rows_per_s": 0.7204230618247667, "tokens_per_s": 4342.760646294021, "gpu_gb": 17.01154851913452}
+{"rows": 2700, "of": 18780, "epoch": 0, "ce": 0.11433159165522631, "kl": 0.004359957079484132, "lr": 4.7499271932997544e-05, "rows_per_s": 0.720380358588772, "tokens_per_s": 4344.208395187752, "gpu_gb": 17.01154851913452}
+{"rows": 2800, "of": 18780, "epoch": 0, "ce": 0.10737344884389155, "kl": 0.3099275430956702, "lr": 4.730622341940088e-05, "rows_per_s": 0.7195176476226208, "tokens_per_s": 4342.048992872917, "gpu_gb": 17.01154851913452}
+{"rows": 2900, "of": 18780, "epoch": 0, "ce": 0.12129080687698565, "kl": 0.0012167722781679842, "lr": 4.712202855507527e-05, "rows_per_s": 0.7200029235921726, "tokens_per_s": 4349.310736360948, "gpu_gb": 17.01154851913452}
+{"rows": 3000, "of": 18780, "epoch": 0, "ce": 0.10231209232512375, "kl": 0.09819812071782508, "lr": 4.691604480356969e-05, "rows_per_s": 0.721832813502043, "tokens_per_s": 4352.954553977114, "gpu_gb": 17.01154851913452}
+{"rows": 3100, "of": 18780, "epoch": 0, "ce": 0.12612309956568338, "kl": 0.00035574374378743306, "lr": 4.6720014492893695e-05, "rows_per_s": 0.7229424629638356, "tokens_per_s": 4359.1700118953095, "gpu_gb": 17.01154851913452}
+{"rows": 3200, "of": 18780, "epoch": 0, "ce": 0.11671324799189696, "kl": 0.0041842913689196575, "lr": 4.650132614952769e-05, "rows_per_s": 0.7246943517869074, "tokens_per_s": 4364.223072887193, "gpu_gb": 17.01154851913452}
+{"rows": 3300, "of": 18780, "epoch": 0, "ce": 0.1339273740333028, "kl": 0.0016656778947145136, "lr": 4.629367989339015e-05, "rows_per_s": 0.7249325344509515, "tokens_per_s": 4365.732204922588, "gpu_gb": 17.01154851913452}
+{"rows": 3400, "of": 18780, "epoch": 0, "ce": 0.11441916732001119, "kl": 0.002292712276684483, "lr": 4.60625318299164e-05, "rows_per_s": 0.7258642140228421, "tokens_per_s": 4373.606436952104, "gpu_gb": 17.01154851913452}
+{"rows": 3500, "of": 18780, "epoch": 0, "ce": 0.13307112246449734, "kl": 0.0001933547977108357, "lr": 4.584350213592103e-05, "rows_per_s": 0.7262375593887173, "tokens_per_s": 4373.058553532151, "gpu_gb": 17.01154851913452}
+{"rows": 3600, "of": 18780, "epoch": 0, "ce": 0.09808383000781759, "kl": 0.0009913165144028735, "lr": 4.5600153175606256e-05, "rows_per_s": 0.7265335321178316, "tokens_per_s": 4371.903016997109, "gpu_gb": 17.01154851913452}
+{"rows": 3700, "of": 18780, "epoch": 0, "ce": 0.11843852331049931, "kl": 0.0002672453235189306, "lr": 4.5369985297726016e-05, "rows_per_s": 0.7269751604257761, "tokens_per_s": 4373.741525462399, "gpu_gb": 17.01154851913452}
+{"rows": 3800, "of": 18780, "epoch": 0, "ce": 0.10978568904101849, "kl": 0.07527547081013619, "lr": 4.511470792553904e-05, "rows_per_s": 0.7278644559061497, "tokens_per_s": 4374.5921816459095, "gpu_gb": 17.01154851913452}
+{"rows": 3900, "of": 18780, "epoch": 0, "ce": 0.11584214970651197, "kl": 0.0017692871452413951, "lr": 4.487365958949811e-05, "rows_per_s": 0.7282151773519746, "tokens_per_s": 4373.7794837101, "gpu_gb": 17.01154851913452}
+{"rows": 4000, "of": 18780, "epoch": 0, "ce": 0.10899776709266007, "kl": 0.0038453903098876197, "lr": 4.460673964700039e-05, "rows_per_s": 0.7295198654369914, "tokens_per_s": 4378.165688868918, "gpu_gb": 17.01154851913452}
+{"rows": 4100, "of": 18780, "epoch": 0, "ce": 0.10205585328817722, "kl": 0.006608869835135062, "lr": 4.4355080761691095e-05, "rows_per_s": 0.7301732210722791, "tokens_per_s": 4381.006557684242, "gpu_gb": 17.01154851913452}
+{"rows": 4200, "of": 18780, "epoch": 0, "ce": 0.12142072368140627, "kl": 0.00082981012835881, "lr": 4.407681712697158e-05, "rows_per_s": 0.7309608509691683, "tokens_per_s": 4385.87997109159, "gpu_gb": 17.01154851913452}
+{"rows": 4300, "of": 18780, "epoch": 0, "ce": 0.11198186472330879, "kl": 0.018603357048015443, "lr": 4.381482948222949e-05, "rows_per_s": 0.7316643313068589, "tokens_per_s": 4383.9152157452545, "gpu_gb": 17.01154851913452}
+{"rows": 4400, "of": 18780, "epoch": 0, "ce": 0.12138256682083011, "kl": 0.001501147752543175, "lr": 4.352553373524204e-05, "rows_per_s": 0.7328705022805022, "tokens_per_s": 4390.296721190552, "gpu_gb": 17.01154851913452}
+{"rows": 4500, "of": 18780, "epoch": 0, "ce": 0.11611213509080022, "kl": 0.003066904309675219, "lr": 4.325351068631764e-05, "rows_per_s": 0.7340592523485424, "tokens_per_s": 4394.1374092985625, "gpu_gb": 17.01154851913452}
+{"rows": 4600, "of": 18780, "epoch": 0, "ce": 0.11954817306038669, "kl": 0.00019243210784338863, "lr": 4.295350675999582e-05, "rows_per_s": 0.7345756343514847, "tokens_per_s": 4397.195617065545, "gpu_gb": 17.01154851913452}
+{"rows": 4700, "of": 18780, "epoch": 0, "ce": 0.11571193690456096, "kl": 0.00461615146121544, "lr": 4.267175289907599e-05, "rows_per_s": 0.7353349470168749, "tokens_per_s": 4398.623456980661, "gpu_gb": 17.01154851913452}
+{"rows": 4800, "of": 18780, "epoch": 0, "ce": 0.12748073828829962, "kl": 0.0003708805550793152, "lr": 4.236137671661584e-05, "rows_per_s": 0.7361358541451141, "tokens_per_s": 4404.804761687034, "gpu_gb": 17.01154851913452}
+{"rows": 4900, "of": 18780, "epoch": 0, "ce": 0.11573176710616524, "kl": 0.0034143106095483075, "lr": 4.2070207531763105e-05, "rows_per_s": 0.7367829692573316, "tokens_per_s": 4407.226716316282, "gpu_gb": 17.01154851913452}
+{"rows": 5000, "of": 18780, "epoch": 0, "ce": 0.1161545449629044, "kl": 5.791796083491997e-05, "lr": 4.174980663048014e-05, "rows_per_s": 0.7367791507377923, "tokens_per_s": 4409.505332501569, "gpu_gb": 17.01154851913452}
+{"rows": 5100, "of": 18780, "epoch": 0, "ce": 0.11048830882646143, "kl": 0.004604833016539606, "lr": 4.144954815237139e-05, "rows_per_s": 0.7375304218200056, "tokens_per_s": 4412.104236561659, "gpu_gb": 17.01154851913452}
+{"rows": 5200, "of": 18780, "epoch": 0, "ce": 0.10978209982035546, "kl": 0.00016448534502868587, "lr": 4.11194812945528e-05, "rows_per_s": 0.7374927012549916, "tokens_per_s": 4412.149079470578, "gpu_gb": 17.012797355651855}
+{"rows": 5300, "of": 18780, "epoch": 0, "ce": 0.09806595825301927, "kl": 0.0006838897649874727, "lr": 4.081046973141319e-05, "rows_per_s": 0.7382974999873938, "tokens_per_s": 4413.864798399163, "gpu_gb": 17.012797355651855}
+{"rows": 5400, "of": 18780, "epoch": 0, "ce": 0.11744000383704487, "kl": 0.004885029998137286, "lr": 4.047110650260125e-05, "rows_per_s": 0.7382766623912393, "tokens_per_s": 4411.687039155222, "gpu_gb": 17.012797355651855}
+{"rows": 5500, "of": 18780, "epoch": 0, "ce": 0.10762561081178687, "kl": 0.020702875576525455, "lr": 4.015368786374199e-05, "rows_per_s": 0.7384935971589116, "tokens_per_s": 4414.084721641815, "gpu_gb": 17.012797355651855}
+{"rows": 5600, "of": 18780, "epoch": 0, "ce": 0.13221884725822342, "kl": 0.0005580901963810654, "lr": 3.980540825889825e-05, "rows_per_s": 0.7391244260598532, "tokens_per_s": 4415.995497615999, "gpu_gb": 17.012797355651855}
+{"rows": 5700, "of": 18780, "epoch": 0, "ce": 0.10682644525934991, "kl": 0.0006485376363028298, "lr": 3.9479937967279834e-05, "rows_per_s": 0.7391374737003368, "tokens_per_s": 4416.125960849812, "gpu_gb": 17.012797355651855}
+{"rows": 5800, "of": 18780, "epoch": 0, "ce": 0.09356876164453015, "kl": 0.0010612463226045143, "lr": 3.912313196529361e-05, "rows_per_s": 0.7394340484340285, "tokens_per_s": 4416.891998781952, "gpu_gb": 17.012797355651855}
+{"rows": 5900, "of": 18780, "epoch": 0, "ce": 0.11583376341523269, "kl": 0.0005758258491474842, "lr": 3.878997445954831e-05, "rows_per_s": 0.7395300980169639, "tokens_per_s": 4417.337867475228, "gpu_gb": 17.012797355651855}
+{"rows": 6000, "of": 18780, "epoch": 0, "ce": 0.10416724750539288, "kl": 0.004517630048291671, "lr": 3.8425041586565916e-05, "rows_per_s": 0.7405381759404395, "tokens_per_s": 4421.483892644322, "gpu_gb": 17.012797355651855}
+{"rows": 6100, "of": 18780, "epoch": 0, "ce": 0.09808784229680896, "kl": 0.002651057127162737, "lr": 3.808456991292513e-05, "rows_per_s": 0.7416313670902787, "tokens_per_s": 4424.077666728342, "gpu_gb": 17.012797355651855}
+{"rows": 6200, "of": 18780, "epoch": 0, "ce": 0.10196536885840553, "kl": 0.03393591845016941, "lr": 3.7711918794988685e-05, "rows_per_s": 0.7419972409636784, "tokens_per_s": 4425.54512667074, "gpu_gb": 17.012797355651855}
+{"rows": 6300, "of": 18780, "epoch": 0, "ce": 0.12451219269767022, "kl": 0.01339086651269351, "lr": 3.736451418957215e-05, "rows_per_s": 0.7416299109278692, "tokens_per_s": 4423.408283124955, "gpu_gb": 17.012797355651855}
+{"rows": 6400, "of": 18780, "epoch": 0, "ce": 0.10839275729197723, "kl": 0.000517572242994144, "lr": 3.698456209506906e-05, "rows_per_s": 0.7418794984741908, "tokens_per_s": 4423.765548830946, "gpu_gb": 17.012797355651855}
+{"rows": 6500, "of": 18780, "epoch": 0, "ce": 0.11331146366374437, "kl": 0.00014617847010813177, "lr": 3.663061355700358e-05, "rows_per_s": 0.7428272976032775, "tokens_per_s": 4427.115613428493, "gpu_gb": 17.012797355651855}
+{"rows": 6600, "of": 18780, "epoch": 0, "ce": 0.10041742490079159, "kl": 1.3419539625606305, "lr": 3.624378592943888e-05, "rows_per_s": 0.7428166455270205, "tokens_per_s": 4429.304009684841, "gpu_gb": 17.012797355651855}
+{"rows": 6700, "of": 18780, "epoch": 0, "ce": 0.11273798086615495, "kl": 0.0018742687237544417, "lr": 3.588368978528458e-05, "rows_per_s": 0.743044486266143, "tokens_per_s": 4435.156459639386, "gpu_gb": 17.012797355651855}
+{"rows": 6800, "of": 18780, "epoch": 0, "ce": 0.114377216410798, "kl": 0.013738256711511129, "lr": 3.549041976689933e-05, "rows_per_s": 0.7422427618343663, "tokens_per_s": 4435.376410554692, "gpu_gb": 17.012797355651855}
+{"rows": 6900, "of": 18780, "epoch": 0, "ce": 0.10764496655362409, "kl": 0.004147641836856373, "lr": 3.512457922687121e-05, "rows_per_s": 0.7430923952074723, "tokens_per_s": 4437.386361269123, "gpu_gb": 17.012797355651855}
+{"rows": 7000, "of": 18780, "epoch": 0, "ce": 0.11116319360505593, "kl": 0.023048399899650063, "lr": 3.472530717364043e-05, "rows_per_s": 0.743149823277977, "tokens_per_s": 4434.885220935499, "gpu_gb": 17.012797355651855}
+{"rows": 7100, "of": 18780, "epoch": 0, "ce": 0.10018096542036212, "kl": 0.0033633225558482582, "lr": 3.435413188012204e-05, "rows_per_s": 0.7429076225286091, "tokens_per_s": 4434.443850289899, "gpu_gb": 17.012797355651855}
+{"rows": 7200, "of": 18780, "epoch": 0, "ce": 0.11350928687542262, "kl": 0.0005763816857300152, "lr": 3.3949304868675243e-05, "rows_per_s": 0.743710432554893, "tokens_per_s": 4437.358625150332, "gpu_gb": 17.012797355651855}
+{"rows": 7300, "of": 18780, "epoch": 0, "ce": 0.12080218707926052, "kl": 0.0006634149429490017, "lr": 3.357321043753011e-05, "rows_per_s": 0.7439064549438453, "tokens_per_s": 4437.54263263152, "gpu_gb": 17.38079309463501}
+{"rows": 7400, "of": 18780, "epoch": 0, "ce": 0.11814825568834077, "kl": 0.0010331438938560034, "lr": 3.316328176454644e-05, "rows_per_s": 0.7443074304416137, "tokens_per_s": 4438.038665197577, "gpu_gb": 17.38079309463501}
+{"rows": 7500, "of": 18780, "epoch": 0, "ce": 0.10442721686865154, "kl": 0.0004141772053014095, "lr": 3.278268931974075e-05, "rows_per_s": 0.7441176393948568, "tokens_per_s": 4440.020775446911, "gpu_gb": 17.38079309463501}
+{"rows": 7600, "of": 18780, "epoch": 0, "ce": 0.09631381489868675, "kl": 0.007534560414467206, "lr": 3.236811799437944e-05, "rows_per_s": 0.7439344029188822, "tokens_per_s": 4441.644886622495, "gpu_gb": 17.38079309463501}
+{"rows": 7700, "of": 18780, "epoch": 0, "ce": 0.1202871174399936, "kl": 0.0031555618110092254, "lr": 3.198345369643724e-05, "rows_per_s": 0.7445533580260436, "tokens_per_s": 4445.119307534269, "gpu_gb": 17.38079309463501}
+{"rows": 7800, "of": 18780, "epoch": 0, "ce": 0.1000407074028865, "kl": 0.0014069003513015523, "lr": 3.156470392637155e-05, "rows_per_s": 0.744121073929313, "tokens_per_s": 4444.929007003851, "gpu_gb": 17.38079309463501}
+{"rows": 7900, "of": 18780, "epoch": 0, "ce": 0.11384696188953615, "kl": 0.002534153291112188, "lr": 3.1176398495190263e-05, "rows_per_s": 0.7439880422802394, "tokens_per_s": 4444.474190659928, "gpu_gb": 17.38079309463501}
+{"rows": 8000, "of": 18780, "epoch": 0, "ce": 0.0940958141513607, "kl": 0.010087146410261323, "lr": 3.075393916682056e-05, "rows_per_s": 0.74335126329789, "tokens_per_s": 4441.7938205512855, "gpu_gb": 17.38079309463501}
+{"rows": 8100, "of": 18780, "epoch": 0, "ce": 0.10968365783903848, "kl": 0.0014739587520529174, "lr": 3.03624273993813e-05, "rows_per_s": 0.7430620152598858, "tokens_per_s": 4442.521202733053, "gpu_gb": 17.38079309463501}
+{"rows": 8200, "of": 18780, "epoch": 0, "ce": 0.09929899214521835, "kl": 0.0023920476665597, "lr": 2.9936731552809194e-05, "rows_per_s": 0.7425898028844029, "tokens_per_s": 4443.304921983093, "gpu_gb": 17.38079309463501}
+{"rows": 8300, "of": 18780, "epoch": 0, "ce": 0.10387079069568288, "kl": 0.02069884188178328, "lr": 2.9542451836321843e-05, "rows_per_s": 0.7425429103337466, "tokens_per_s": 4443.207332512782, "gpu_gb": 17.38079309463501}
+{"rows": 8400, "of": 18780, "epoch": 0, "ce": 0.11361207558136237, "kl": 0.002974346075834041, "lr": 2.9113996135673294e-05, "rows_per_s": 0.7424879328226426, "tokens_per_s": 4441.0089535037305, "gpu_gb": 17.38079309463501}
+{"rows": 8500, "of": 18780, "epoch": 0, "ce": 0.12906495600032644, "kl": 0.005890546791621829, "lr": 2.8717389956701478e-05, "rows_per_s": 0.7423668042077707, "tokens_per_s": 4440.767828930391, "gpu_gb": 17.38079309463501}
+{"rows": 8600, "of": 18780, "epoch": 0, "ce": 0.11377146764425561, "kl": 0.0031944885926976333, "lr": 2.8286654156392022e-05, "rows_per_s": 0.7422287191125718, "tokens_per_s": 4441.4205335730885, "gpu_gb": 17.38079309463501}
+{"rows": 8700, "of": 18780, "epoch": 0, "ce": 0.0940737800542038, "kl": 0.0004907657721984883, "lr": 2.788816560650763e-05, "rows_per_s": 0.7428295218113558, "tokens_per_s": 4443.067946911679, "gpu_gb": 17.38079309463501}
+{"rows": 8800, "of": 18780, "epoch": 0, "ce": 0.09874170044487393, "kl": 0.0007960516582811588, "lr": 2.745563201404728e-05, "rows_per_s": 0.742326409047665, "tokens_per_s": 4442.786104408728, "gpu_gb": 18.028878211975098}
+{"rows": 8900, "of": 18780, "epoch": 0, "ce": 0.11203517273745754, "kl": 0.0015616318572893768, "lr": 2.705570729256813e-05, "rows_per_s": 0.7427074814443492, "tokens_per_s": 4441.42578815431, "gpu_gb": 18.028878211975098}
+{"rows": 9000, "of": 18780, "epoch": 0, "ce": 0.12159633624056974, "kl": 0.0022928011763563294, "lr": 2.6621860228507494e-05, "rows_per_s": 0.7428631199408089, "tokens_per_s": 4441.9014919622305, "gpu_gb": 18.028878211975098}
+{"rows": 9100, "of": 18780, "epoch": 0, "ce": 0.0956079700051553, "kl": 0.001621419707663237, "lr": 2.6220947142874868e-05, "rows_per_s": 0.7432595035068629, "tokens_per_s": 4442.932511223178, "gpu_gb": 18.028878211975098}
+{"rows": 9200, "of": 18780, "epoch": 0, "ce": 0.12527150819759544, "kl": 0.07407421492340724, "lr": 2.578627239849723e-05, "rows_per_s": 0.7429432242606355, "tokens_per_s": 4445.48589781551, "gpu_gb": 18.028878211975098}
+{"rows": 9300, "of": 18780, "epoch": 0, "ce": 0.08952350041363388, "kl": 0.004356835280539724, "lr": 2.5384819862852755e-05, "rows_per_s": 0.7438021853457468, "tokens_per_s": 4446.249357215398, "gpu_gb": 18.028878211975098}
+{"rows": 9400, "of": 18780, "epoch": 0, "ce": 0.09814780101268775, "kl": 0.006786820355468293, "lr": 2.4949804156219327e-05, "rows_per_s": 0.7439782661630968, "tokens_per_s": 4446.825371487281, "gpu_gb": 18.028878211975098}
+{"rows": 9500, "of": 18780, "epoch": 0, "ce": 0.12738295074640996, "kl": 0.012172163916667755, "lr": 2.454826168874261e-05, "rows_per_s": 0.7439473051021321, "tokens_per_s": 4448.328131753544, "gpu_gb": 18.028878211975098}
+{"rows": 9600, "of": 18780, "epoch": 0, "ce": 0.09565829741768539, "kl": 0.032960128190066884, "lr": 2.411339211970005e-05, "rows_per_s": 0.7435473984301889, "tokens_per_s": 4445.9349388804985, "gpu_gb": 18.028878211975098}
+{"rows": 9700, "of": 18780, "epoch": 0, "ce": 0.10110956263777457, "kl": 0.004817786117392042, "lr": 2.3712209339270036e-05, "rows_per_s": 0.7438909512582911, "tokens_per_s": 4447.156032997547, "gpu_gb": 18.028878211975098}
+{"rows": 9800, "of": 18780, "epoch": 0, "ce": 0.08525047284097244, "kl": 0.00308526413067253, "lr": 2.3277972844030456e-05, "rows_per_s": 0.7442183473617432, "tokens_per_s": 4446.753531263528, "gpu_gb": 18.028878211975098}
+{"rows": 9900, "of": 18780, "epoch": 0, "ce": 0.1264366149295901, "kl": 0.0010555304377720834, "lr": 2.2877598966773926e-05, "rows_per_s": 0.744192118820836, "tokens_per_s": 4446.45259322655, "gpu_gb": 18.028878211975098}
+{"rows": 10000, "of": 18780, "epoch": 0, "ce": 0.09764008736237884, "kl": 0.00028681194371937835, "lr": 2.2444481772678108e-05, "rows_per_s": 0.7441925938952778, "tokens_per_s": 4445.901068389814, "gpu_gb": 18.028878211975098}
+{"rows": 10100, "of": 18780, "epoch": 0, "ce": 0.08046582746319472, "kl": 0.010835173299389322, "lr": 2.2045365108969245e-05, "rows_per_s": 0.7433220460853254, "tokens_per_s": 4442.796716251147, "gpu_gb": 18.028878211975098}
+{"rows": 10200, "of": 18780, "epoch": 0, "ce": 0.09862953060794444, "kl": 0.026102999712659253, "lr": 2.1613852190043695e-05, "rows_per_s": 0.7434629020273364, "tokens_per_s": 4444.717155709833, "gpu_gb": 18.028878211975098}
+{"rows": 10300, "of": 18780, "epoch": 0, "ce": 0.09829839661197606, "kl": 0.012904083560357296, "lr": 2.1216439642517786e-05, "rows_per_s": 0.7436600485370918, "tokens_per_s": 4446.156576591076, "gpu_gb": 18.028878211975098}
+{"rows": 10400, "of": 18780, "epoch": 0, "ce": 0.11019668774679303, "kl": 0.002350079639331876, "lr": 2.078701417643501e-05, "rows_per_s": 0.743661239898344, "tokens_per_s": 4446.704364488258, "gpu_gb": 18.028878211975098}
+{"rows": 10500, "of": 18780, "epoch": 0, "ce": 0.09447295422590263, "kl": 0.002090633684540373, "lr": 2.039175073957849e-05, "rows_per_s": 0.7434549091533021, "tokens_per_s": 4444.836796339916, "gpu_gb": 18.028878211975098}
+{"rows": 10600, "of": 18780, "epoch": 0, "ce": 0.09308563811047112, "kl": 0.0010000416738914497, "lr": 1.9964893566628816e-05, "rows_per_s": 0.7432601395965971, "tokens_per_s": 4443.1420808581315, "gpu_gb": 18.028878211975098}
+{"rows": 10700, "of": 18780, "epoch": 0, "ce": 0.07959520678240352, "kl": 0.0026815836640997318, "lr": 1.9572221828505843e-05, "rows_per_s": 0.7435332468154952, "tokens_per_s": 4443.748130188415, "gpu_gb": 18.028878211975098}
+{"rows": 10800, "of": 18780, "epoch": 0, "ce": 0.09485353049240075, "kl": 0.008445330342945923, "lr": 1.914841091318645e-05, "rows_per_s": 0.7434063444830811, "tokens_per_s": 4445.973031448944, "gpu_gb": 18.028878211975098}
+{"rows": 10900, "of": 18780, "epoch": 0, "ce": 0.09847847279161215, "kl": 0.018562712314031324, "lr": 1.8758770559860113e-05, "rows_per_s": 0.743528607531045, "tokens_per_s": 4445.492741476085, "gpu_gb": 18.028878211975098}
+{"rows": 11000, "of": 18780, "epoch": 0, "ce": 0.1113675397161084, "kl": 0.0003545139748117876, "lr": 1.8338480455684165e-05, "rows_per_s": 0.7438629316889882, "tokens_per_s": 4445.561292937865, "gpu_gb": 18.028878211975098}
+{"rows": 11100, "of": 18780, "epoch": 0, "ce": 0.08999607192818075, "kl": 0.00809617037266741, "lr": 1.795230777888704e-05, "rows_per_s": 0.7435616550637535, "tokens_per_s": 4444.979821878821, "gpu_gb": 18.028878211975098}
+{"rows": 11200, "of": 18780, "epoch": 0, "ce": 0.10207623873765652, "kl": 0.04663487695268748, "lr": 1.7536009097012185e-05, "rows_per_s": 0.7437467843933059, "tokens_per_s": 4445.059367618903, "gpu_gb": 18.028878211975098}
+{"rows": 11300, "of": 18780, "epoch": 0, "ce": 0.09538649690748993, "kl": 0.0003297446312838396, "lr": 1.7153736505617778e-05, "rows_per_s": 0.7440385552161476, "tokens_per_s": 4445.525016824593, "gpu_gb": 18.028878211975098}
+{"rows": 11400, "of": 18780, "epoch": 0, "ce": 0.09714443772099912, "kl": 0.005819700961993524, "lr": 1.6741895387889034e-05, "rows_per_s": 0.7440210267871865, "tokens_per_s": 4444.953652572846, "gpu_gb": 18.028878211975098}
+{"rows": 11500, "of": 18780, "epoch": 0, "ce": 0.10543953458703048, "kl": 0.00545170631626282, "lr": 1.6363950923730775e-05, "rows_per_s": 0.7442434339439111, "tokens_per_s": 4445.906416396845, "gpu_gb": 18.028878211975098}
+{"rows": 11600, "of": 18780, "epoch": 0, "ce": 0.10092131168182407, "kl": 0.0037020385025095712, "lr": 1.5957028520727853e-05, "rows_per_s": 0.7441392377074229, "tokens_per_s": 4446.1051850317035, "gpu_gb": 18.028878211975098}
+{"rows": 11700, "of": 18780, "epoch": 0, "ce": 0.10958679672330618, "kl": 0.01229057863763939, "lr": 1.5583835379308188e-05, "rows_per_s": 0.7435622887050691, "tokens_per_s": 4445.012184262285, "gpu_gb": 18.028878211975098}
+{"rows": 11800, "of": 18780, "epoch": 0, "ce": 0.0897589340252248, "kl": 0.007448011440745173, "lr": 1.518228733398161e-05, "rows_per_s": 0.7434422291604517, "tokens_per_s": 4445.272815312956, "gpu_gb": 18.028878211975098}
+{"rows": 11900, "of": 18780, "epoch": 0, "ce": 0.08785739771433566, "kl": 0.004530809050642681, "lr": 1.481426339060753e-05, "rows_per_s": 0.7428838831438703, "tokens_per_s": 4443.383028291882, "gpu_gb": 18.028878211975098}
+{"rows": 12000, "of": 18780, "epoch": 0, "ce": 0.08030235096812248, "kl": 0.00309750713140867, "lr": 1.4418539328081931e-05, "rows_per_s": 0.7428695416440637, "tokens_per_s": 4443.790130522347, "gpu_gb": 18.028878211975098}
+{"rows": 12100, "of": 18780, "epoch": 0, "ce": 0.09484802588651126, "kl": 1.3642179548985685e-06, "lr": 1.4056096669957758e-05, "rows_per_s": 0.7432009303600363, "tokens_per_s": 4444.9738371379035, "gpu_gb": 18.028878211975098}
+{"rows": 12200, "of": 18780, "epoch": 0, "ce": 0.10147819312241597, "kl": 0.0011475077165831612, "lr": 1.366663969407336e-05, "rows_per_s": 0.7431779264383654, "tokens_per_s": 4444.772226634033, "gpu_gb": 18.028878211975098}
+{"rows": 12300, "of": 18780, "epoch": 0, "ce": 0.10011866240264763, "kl": 0.0002467061328899807, "lr": 1.3310184158874612e-05, "rows_per_s": 0.7433423616509642, "tokens_per_s": 4444.79413686749, "gpu_gb": 18.028878211975098}
+{"rows": 12400, "of": 18780, "epoch": 0, "ce": 0.09764061707097131, "kl": 0.0020653822675113466, "lr": 1.292743035603092e-05, "rows_per_s": 0.7430843279596364, "tokens_per_s": 4442.917017375713, "gpu_gb": 18.13348150253296}
+{"rows": 12500, "of": 18780, "epoch": 0, "ce": 0.09461826607584953, "kl": 0.002342756631962817, "lr": 1.2577361077475958e-05, "rows_per_s": 0.7427712571367572, "tokens_per_s": 4442.483870807247, "gpu_gb": 18.13348150253296}
+{"rows": 12600, "of": 18780, "epoch": 0, "ce": 0.09693271279908143, "kl": 0.007365204187644455, "lr": 1.2201739028333e-05, "rows_per_s": 0.742873886285524, "tokens_per_s": 4442.817768089888, "gpu_gb": 18.13348150253296}
+{"rows": 12700, "of": 18780, "epoch": 0, "ce": 0.09763457066713808, "kl": 0.0004092652009081306, "lr": 1.18584479892613e-05, "rows_per_s": 0.7429992429630173, "tokens_per_s": 4442.903797564345, "gpu_gb": 18.13348150253296}
+{"rows": 12800, "of": 18780, "epoch": 0, "ce": 0.09194803433049292, "kl": 0.0090392841026294, "lr": 1.1490378288845297e-05, "rows_per_s": 0.7432025924891917, "tokens_per_s": 4444.430468360819, "gpu_gb": 18.13348150253296}
+{"rows": 12900, "of": 18780, "epoch": 0, "ce": 0.08642993282335691, "kl": 0.002003251189645364, "lr": 1.1154249882302764e-05, "rows_per_s": 0.7431703642733983, "tokens_per_s": 4444.355344035767, "gpu_gb": 18.13348150253296}
+{"rows": 13000, "of": 18780, "epoch": 0, "ce": 0.08992617547784637, "kl": 0.0006671167881967039, "lr": 1.0794144669053538e-05, "rows_per_s": 0.7432135870761772, "tokens_per_s": 4445.776416855474, "gpu_gb": 18.13348150253296}
+{"rows": 13100, "of": 18780, "epoch": 0, "ce": 0.08935791360335829, "kl": 0.007927287016023521, "lr": 1.0465555267876337e-05, "rows_per_s": 0.7436317737545842, "tokens_per_s": 4447.980321938981, "gpu_gb": 18.13348150253296}
+{"rows": 13200, "of": 18780, "epoch": 0, "ce": 0.07884387121885084, "kl": 0.0014589582010113844, "lr": 1.0113817762163818e-05, "rows_per_s": 0.7436577947096363, "tokens_per_s": 4448.180197697238, "gpu_gb": 18.13348150253296}
+{"rows": 13300, "of": 18780, "epoch": 0, "ce": 0.09934724494814873, "kl": 0.04962210586829476, "lr": 9.793135297542678e-06, "rows_per_s": 0.7436186791651522, "tokens_per_s": 4448.066392672339, "gpu_gb": 18.13348150253296}
+{"rows": 13400, "of": 18780, "epoch": 0, "ce": 0.11076726427390463, "kl": 0.0014534503769914053, "lr": 9.450159350169218e-06, "rows_per_s": 0.7437960303067004, "tokens_per_s": 4447.097738621667, "gpu_gb": 18.13348150253296}
+{"rows": 13500, "of": 18780, "epoch": 0, "ce": 0.10391732421186235, "kl": 0.002207123376702924, "lr": 9.137742899666074e-06, "rows_per_s": 0.7440012494139837, "tokens_per_s": 4449.019673981264, "gpu_gb": 18.13348150253296}
+{"rows": 13600, "of": 18780, "epoch": 0, "ce": 0.07529682985928264, "kl": 0.06412543792661235, "lr": 8.803912550860166e-06, "rows_per_s": 0.7440926617045998, "tokens_per_s": 4448.847955288231, "gpu_gb": 18.13348150253296}
+{"rows": 13700, "of": 18780, "epoch": 0, "ce": 0.09455915731264324, "kl": 0.0015161350155573627, "lr": 8.50011193633848e-06, "rows_per_s": 0.7441321693081605, "tokens_per_s": 4449.3198465544265, "gpu_gb": 18.13348150253296}
+{"rows": 13800, "of": 18780, "epoch": 0, "ce": 0.09182557080029732, "kl": 0.00013400036708591245, "lr": 8.175800985733636e-06, "rows_per_s": 0.7445548033573619, "tokens_per_s": 4450.656619499601, "gpu_gb": 18.13348150253296}
+{"rows": 13900, "of": 18780, "epoch": 0, "ce": 0.10155540985349369, "kl": 0.0010319250013776187, "lr": 7.880956381652615e-06, "rows_per_s": 0.7445261045505032, "tokens_per_s": 4450.094767138177, "gpu_gb": 18.13348150253296}
+{"rows": 14000, "of": 18780, "epoch": 0, "ce": 0.13154433900490403, "kl": 0.001986302742632086, "lr": 7.566527969732956e-06, "rows_per_s": 0.7445103224684056, "tokens_per_s": 4450.917334826324, "gpu_gb": 18.13348150253296}
+{"rows": 14100, "of": 18780, "epoch": 0, "ce": 0.08197282074724098, "kl": 0.0008762738467474116, "lr": 7.280969522244205e-06, "rows_per_s": 0.7443842147366334, "tokens_per_s": 4451.414647705491, "gpu_gb": 18.13348150253296}
+{"rows": 14200, "of": 18780, "epoch": 0, "ce": 0.1030243962236187, "kl": 0.0018166016441480224, "lr": 6.9767757237254125e-06, "rows_per_s": 0.7442978823776477, "tokens_per_s": 4451.3574549403365, "gpu_gb": 18.13348150253296}
+{"rows": 14300, "of": 18780, "epoch": 0, "ce": 0.09324079061694004, "kl": 0.0026047436957208226, "lr": 6.700823180998622e-06, "rows_per_s": 0.7439887402548663, "tokens_per_s": 4450.877884415402, "gpu_gb": 18.13348150253296}
+{"rows": 14400, "of": 18780, "epoch": 0, "ce": 0.09611743705681874, "kl": 0.0013654983283174493, "lr": 6.407204610599552e-06, "rows_per_s": 0.7443891723800529, "tokens_per_s": 4450.505288367501, "gpu_gb": 18.13348150253296}
+{"rows": 14500, "of": 18780, "epoch": 0, "ce": 0.09741969885895164, "kl": 0.0010732944374097775, "lr": 6.1411669647910896e-06, "rows_per_s": 0.7441119778278843, "tokens_per_s": 4449.397967920074, "gpu_gb": 18.13348150253296}
+{"rows": 14600, "of": 18780, "epoch": 0, "ce": 0.09957372822572251, "kl": 0.0019459132374685982, "lr": 5.858452395837472e-06, "rows_per_s": 0.7440790840896595, "tokens_per_s": 4449.136588327469, "gpu_gb": 18.13348150253296}
+{"rows": 14700, "of": 18780, "epoch": 0, "ce": 0.08981905119832266, "kl": 0.031786237844647625, "lr": 5.6026275371028e-06, "rows_per_s": 0.74432544954533, "tokens_per_s": 4449.977143943058, "gpu_gb": 18.13348150253296}
+{"rows": 14800, "of": 18780, "epoch": 0, "ce": 0.08765776465624207, "kl": 0.0011204692754578855, "lr": 5.331133533390112e-06, "rows_per_s": 0.7445287300592967, "tokens_per_s": 4450.040673665125, "gpu_gb": 18.13348150253296}
+{"rows": 14900, "of": 18780, "epoch": 0, "ce": 0.09862447355706723, "kl": 1.3469017396403165, "lr": 5.085807916327443e-06, "rows_per_s": 0.7445465483834134, "tokens_per_s": 4450.432250662811, "gpu_gb": 18.13348150253296}
+{"rows": 15000, "of": 18780, "epoch": 0, "ce": 0.10849789200827133, "kl": 0.0004977649176943608, "lr": 4.825838477655142e-06, "rows_per_s": 0.7448072295554565, "tokens_per_s": 4451.039803240337, "gpu_gb": 18.13348150253296}
+{"rows": 15100, "of": 18780, "epoch": 0, "ce": 0.09368669492791273, "kl": 0.005718123175013773, "lr": 4.591286800553823e-06, "rows_per_s": 0.7449855312010988, "tokens_per_s": 4452.076161444453, "gpu_gb": 18.13348150253296}
+{"rows": 15200, "of": 18780, "epoch": 0, "ce": 0.08310830626541223, "kl": 0.14005196112738313, "lr": 4.343133022327845e-06, "rows_per_s": 0.7449337588708209, "tokens_per_s": 4452.122486537823, "gpu_gb": 18.13348150253296}
+{"rows": 15300, "of": 18780, "epoch": 0, "ce": 0.09753033269599483, "kl": 0.014149606909249747, "lr": 4.119617919580634e-06, "rows_per_s": 0.7447295730768402, "tokens_per_s": 4450.514929287819, "gpu_gb": 18.13348150253296}
+{"rows": 15400, "of": 18780, "epoch": 0, "ce": 0.09270527297630907, "kl": 0.0005650183560794719, "lr": 3.883557666865334e-06, "rows_per_s": 0.7448782663403134, "tokens_per_s": 4451.5720643331315, "gpu_gb": 18.13348150253296}
+{"rows": 15500, "of": 18780, "epoch": 0, "ce": 0.09047702939133159, "kl": 0.0005308696983035988, "lr": 3.6713294148889383e-06, "rows_per_s": 0.7450951008100073, "tokens_per_s": 4451.962936306976, "gpu_gb": 18.13348150253296}
+{"rows": 15600, "of": 18780, "epoch": 0, "ce": 0.08270806632936001, "kl": 0.001646245509193206, "lr": 3.4476270112734384e-06, "rows_per_s": 0.7451325291932668, "tokens_per_s": 4450.219200134232, "gpu_gb": 18.13348150253296}
+{"rows": 15700, "of": 18780, "epoch": 0, "ce": 0.10108385272767573, "kl": 0.03129804809219837, "lr": 3.2469232482667144e-06, "rows_per_s": 0.7454542060015185, "tokens_per_s": 4452.0828018652155, "gpu_gb": 18.13348150253296}
+{"rows": 15800, "of": 18780, "epoch": 0, "ce": 0.08743420488348133, "kl": 0.006183864605099193, "lr": 3.0358291798940038e-06, "rows_per_s": 0.7455404084329964, "tokens_per_s": 4452.508405606235, "gpu_gb": 18.13348150253296}
+{"rows": 15900, "of": 18780, "epoch": 0, "ce": 0.09040465392740216, "kl": 0.00016601806213031867, "lr": 2.8468746397474462e-06, "rows_per_s": 0.7457515541356812, "tokens_per_s": 4452.848853667682, "gpu_gb": 18.13348150253296}
+{"rows": 16000, "of": 18780, "epoch": 0, "ce": 0.09262850571696352, "kl": 0.0022561092634023863, "lr": 2.648625274837732e-06, "rows_per_s": 0.7456750068022387, "tokens_per_s": 4451.253078086723, "gpu_gb": 18.13348150253296}
+{"rows": 16100, "of": 18780, "epoch": 0, "ce": 0.08740413161676104, "kl": 0.00023284176749436034, "lr": 2.4716315354923857e-06, "rows_per_s": 0.7452625347481873, "tokens_per_s": 4449.574132671389, "gpu_gb": 18.13348150253296}
+{"rows": 16200, "of": 18780, "epoch": 0, "ce": 0.08000004879857708, "kl": 0.023543155854416277, "lr": 2.28644885967462e-06, "rows_per_s": 0.7451075658832051, "tokens_per_s": 4448.189509058102, "gpu_gb": 18.13348150253296}
+{"rows": 16300, "of": 18780, "epoch": 0, "ce": 0.10207023006990891, "kl": 0.012281727768920364, "lr": 2.1216141062120444e-06, "rows_per_s": 0.7454027332431731, "tokens_per_s": 4449.071391918901, "gpu_gb": 18.13348150253296}
+{"rows": 16400, "of": 18780, "epoch": 0, "ce": 0.08644120483772784, "kl": 0.00011291276811536349, "lr": 1.949705473960148e-06, "rows_per_s": 0.7457513684337359, "tokens_per_s": 4450.008368298302, "gpu_gb": 18.13348150253296}
+{"rows": 16500, "of": 18780, "epoch": 0, "ce": 0.09663423880733349, "kl": 0.002999017249753424, "lr": 1.7972142766887301e-06, "rows_per_s": 0.7456080262280292, "tokens_per_s": 4449.516214686566, "gpu_gb": 18.13348150253296}
+{"rows": 16600, "of": 18780, "epoch": 0, "ce": 0.08004518784582615, "kl": 0.017983467241802695, "lr": 1.6387721791406823e-06, "rows_per_s": 0.7458943698929698, "tokens_per_s": 4449.6646764407205, "gpu_gb": 18.13348150253296}
+{"rows": 16700, "of": 18780, "epoch": 0, "ce": 0.09577778516159881, "kl": 0.00023894890230677368, "lr": 1.4987952869269183e-06, "rows_per_s": 0.7461033386258649, "tokens_per_s": 4449.536659270457, "gpu_gb": 18.13348150253296}
+{"rows": 16800, "of": 18780, "epoch": 0, "ce": 0.09957051811445701, "kl": 0.002770488519340091, "lr": 1.3539971363467823e-06, "rows_per_s": 0.7460930215059427, "tokens_per_s": 4450.411032635187, "gpu_gb": 18.13348150253296}
+{"rows": 16900, "of": 18780, "epoch": 0, "ce": 0.08984015706098741, "kl": 0.0007928760339474321, "lr": 1.2266912854226903e-06, "rows_per_s": 0.7461742482998738, "tokens_per_s": 4450.874289010411, "gpu_gb": 18.13348150253296}
+{"rows": 17000, "of": 18780, "epoch": 0, "ce": 0.09079431036952883, "kl": 0.00031326957554425605, "lr": 1.095699216546922e-06, "rows_per_s": 0.7461914648889199, "tokens_per_s": 4451.009087807843, "gpu_gb": 18.13348150253296}
+{"rows": 17100, "of": 18780, "epoch": 0, "ce": 0.09596199000526856, "kl": 0.00022895688102727205, "lr": 9.812069550079377e-07, "rows_per_s": 0.7463931962668072, "tokens_per_s": 4451.881948357039, "gpu_gb": 18.13348150253296}
+{"rows": 17200, "of": 18780, "epoch": 0, "ce": 0.09811817868612707, "kl": 0.002966103812506171, "lr": 8.641676434983176e-07, "rows_per_s": 0.7463083720722579, "tokens_per_s": 4451.977283214581, "gpu_gb": 18.13348150253296}
+{"rows": 17300, "of": 18780, "epoch": 0, "ce": 0.08688222382886288, "kl": 0.006310779778723286, "lr": 7.626171716879904e-07, "rows_per_s": 0.7463324933028247, "tokens_per_s": 4451.592045796648, "gpu_gb": 18.13348150253296}
+{"rows": 17400, "of": 18780, "epoch": 0, "ce": 0.08330899256099833, "kl": 0.0004821702990336691, "lr": 6.596616698946372e-07, "rows_per_s": 0.7464066793393954, "tokens_per_s": 4452.066010506593, "gpu_gb": 18.13348150253296}
+{"rows": 17500, "of": 18780, "epoch": 0, "ce": 0.07492589116914243, "kl": 0.0005444354214527899, "lr": 5.711666968549384e-07, "rows_per_s": 0.746400463629219, "tokens_per_s": 4451.644026594021, "gpu_gb": 18.13348150253296}
+{"rows": 17600, "of": 18780, "epoch": 0, "ce": 0.09728635881641289, "kl": 0.00016518396643325526, "lr": 4.824102870731124e-07, "rows_per_s": 0.7466727521636336, "tokens_per_s": 4452.762424027144, "gpu_gb": 18.13348150253296}
+{"rows": 17700, "of": 18780, "epoch": 0, "ce": 0.1006327414112847, "kl": 0.0013748409273280544, "lr": 4.0706990322110506e-07, "rows_per_s": 0.7468530292565182, "tokens_per_s": 4452.5867864082265, "gpu_gb": 18.13348150253296}
+{"rows": 17800, "of": 18780, "epoch": 0, "ce": 0.08093963654408622, "kl": 4.2338073451870475e-07, "lr": 3.326119686062529e-07, "rows_per_s": 0.7468712730610579, "tokens_per_s": 4452.76571683992, "gpu_gb": 18.13348150253296}
+{"rows": 17900, "of": 18780, "epoch": 0, "ce": 0.09346023195972428, "kl": 0.004674899184927316, "lr": 2.705105347796305e-07, "rows_per_s": 0.7468099831892758, "tokens_per_s": 4453.157087112319, "gpu_gb": 18.13348150253296}
+{"rows": 18000, "of": 18780, "epoch": 0, "ce": 0.09495253317161567, "kl": 0.0004746615678783428, "lr": 2.1043444806514988e-07, "rows_per_s": 0.7472601809968665, "tokens_per_s": 4454.48037666189, "gpu_gb": 18.13348150253296}
+{"rows": 18100, "of": 18780, "epoch": 0, "ce": 0.08039041068404913, "kl": 0.0013096659812608842, "lr": 1.616415010609329e-07, "rows_per_s": 0.7470629462177906, "tokens_per_s": 4454.423820085151, "gpu_gb": 18.13348150253296}
+{"rows": 18200, "of": 18780, "epoch": 0, "ce": 0.08964484409453014, "kl": 0.0002100807072715306, "lr": 1.1601453120327899e-07, "rows_per_s": 0.7470904792678738, "tokens_per_s": 4454.466230222098, "gpu_gb": 18.13348150253296}
+{"rows": 18300, "of": 18780, "epoch": 0, "ce": 0.07054481006976542, "kl": 0.037409276434284405, "lr": 8.05847059154291e-08, "rows_per_s": 0.7472548680025138, "tokens_per_s": 4454.425305083959, "gpu_gb": 18.13348150253296}
+{"rows": 18400, "of": 18780, "epoch": 0, "ce": 0.0810119897415418, "kl": 8.406061307267287e-05, "lr": 4.945794277108617e-08, "rows_per_s": 0.7475730601561082, "tokens_per_s": 4455.413714134306, "gpu_gb": 18.13348150253296}
+{"rows": 18500, "of": 18780, "epoch": 0, "ce": 0.07415697733151981, "kl": 0.011315914836464528, "lr": 2.7430911014214577e-08, "rows_per_s": 0.7475174241072993, "tokens_per_s": 4454.0082238644045, "gpu_gb": 18.13348150253296}
+{"rows": 18600, "of": 18780, "epoch": 0, "ce": 0.1005731946193115, "kl": 3.114230567004471e-05, "lr": 1.0839208132887302e-08, "rows_per_s": 0.7476193061374252, "tokens_per_s": 4454.732563438071, "gpu_gb": 18.13348150253296}
+{"rows": 18700, "of": 18780, "epoch": 0, "ce": 0.09240605033940086, "kl": 0.002143554769276849, "lr": 2.2396342205494024e-09, "rows_per_s": 0.7473653818351138, "tokens_per_s": 4454.661686640901, "gpu_gb": 18.13348150253296}
+{"rows": 18780, "of": 18780, "epoch": 0, "ce": 0.10464811540829638, "kl": 0.002295198764175943, "lr": 0.0, "rows_per_s": 0.7472687694701062, "tokens_per_s": 4454.647317428433, "gpu_gb": 18.13348150253296}

ablations/specialists/rest/training_meta.json

@@ -0,0 +1,29 @@
+{
+ "base_model": "Qwen/Qwen3.5-4B",
+ "families": [
+  "Form transaction",
+  "Navigation",
+  "Reasoning base",
+  "Text entry"
+ ],
+ "training_rows": 15024,
+ "rows_trained": 18780,
+ "validation_ce": 0.12546929783071392,
+ "minutes": 421.4,
+ "args": {
+  "adapter": "rest",
+  "epochs": 1.0,
+  "lr": 5e-05,
+  "rank": 64,
+  "alpha": 128,
+  "grad_accum": 8,
+  "beta": 0.1,
+  "anchor_ratio": 0.25,
+  "limit": 0,
+  "max_steps": 0,
+  "seed": 0,
+  "ce_on": "all",
+  "exclude_variants": "",
+  "init_adapter": "the main executor (repository root)"
+ }
+}

ablations/specialists/select/adapter_config.json

@@ -0,0 +1,51 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "kasa_config": null,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.21.0",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "k_proj",
+    "v_proj",
+    "o_proj",
+    "gate_proj",
+    "up_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

ablations/specialists/select/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a99139f35f79abb0fb07f8cbad49673c326be3f539820c99d23e9f1deebd1d71
+size 339776632

ablations/specialists/select/train_log.jsonl

@@ -0,0 +1,158 @@
+{"rows": 50, "of": 7881, "epoch": 0, "ce": 0.08112904304502214, "kl": 4.023217946581682e-05, "lr": 1.2067860608049161e-05, "rows_per_s": 0.19979615581206606, "tokens_per_s": 1328.904171152795, "gpu_gb": 15.868685722351074}
+{"rows": 100, "of": 7881, "epoch": 0, "ce": 0.0837922141960773, "kl": 0.00032846825701510207, "lr": 2.240558595802162e-05, "rows_per_s": 0.24933344147778858, "tokens_per_s": 1634.4430422472735, "gpu_gb": 17.829620361328125}
+{"rows": 150, "of": 7881, "epoch": 0, "ce": 0.07664935879542359, "kl": 0.0008350254756953243, "lr": 3.273163592559035e-05, "rows_per_s": 0.26866195817680777, "tokens_per_s": 1771.9617362348567, "gpu_gb": 17.829620361328125}
+{"rows": 200, "of": 7881, "epoch": 0, "ce": 0.10014406370464712, "kl": 0.003094735879189986, "lr": 4.475637267001127e-05, "rows_per_s": 0.2877967541476866, "tokens_per_s": 1901.8357785639912, "gpu_gb": 17.829620361328125}
+{"rows": 250, "of": 7881, "epoch": 0, "ce": 0.10223189959415814, "kl": 0.0002233092915241302, "lr": 4.987790247528478e-05, "rows_per_s": 0.29961804822217813, "tokens_per_s": 1982.4264114786201, "gpu_gb": 17.829620361328125}
+{"rows": 300, "of": 7881, "epoch": 0, "ce": 0.09316687498155028, "kl": 0.001758820125996636, "lr": 4.982612518406752e-05, "rows_per_s": 0.3077253544197876, "tokens_per_s": 2051.25002800796, "gpu_gb": 17.829620361328125}
+{"rows": 350, "of": 7881, "epoch": 0, "ce": 0.1081337986048311, "kl": 0.0002527106371076199, "lr": 4.976525660366432e-05, "rows_per_s": 0.31578841235808497, "tokens_per_s": 2096.5905476012585, "gpu_gb": 17.829620361328125}
+{"rows": 400, "of": 7881, "epoch": 0, "ce": 0.10570818589767442, "kl": 0.001324782029938798, "lr": 4.9682782802340184e-05, "rows_per_s": 0.32020394043816325, "tokens_per_s": 2134.2160992197855, "gpu_gb": 17.829620361328125}
+{"rows": 450, "of": 7881, "epoch": 0, "ce": 0.10937565394005803, "kl": 0.0028952434345785277, "lr": 4.960229732584409e-05, "rows_per_s": 0.3239136346601181, "tokens_per_s": 2169.084055461096, "gpu_gb": 17.829620361328125}
+{"rows": 500, "of": 7881, "epoch": 0, "ce": 0.11144169761127858, "kl": 0.0013210151540508721, "lr": 4.951280252558245e-05, "rows_per_s": 0.32708831267684735, "tokens_per_s": 2197.716839701743, "gpu_gb": 17.829620361328125}
+{"rows": 550, "of": 7881, "epoch": 0, "ce": 0.11779358610510826, "kl": 0.0011205926247696686, "lr": 4.941433117441464e-05, "rows_per_s": 0.3296859248848861, "tokens_per_s": 2218.6651898265072, "gpu_gb": 17.829620361328125}
+{"rows": 600, "of": 7881, "epoch": 0, "ce": 0.11089769536412733, "kl": 0.00012700890559222015, "lr": 4.928815085846087e-05, "rows_per_s": 0.3312180628076961, "tokens_per_s": 2239.881470790709, "gpu_gb": 17.829620361328125}
+{"rows": 650, "of": 7881, "epoch": 0, "ce": 0.11248048525303603, "kl": 0.003862331001892233, "lr": 4.917035838675963e-05, "rows_per_s": 0.33259200048694154, "tokens_per_s": 2255.310448741957, "gpu_gb": 17.879604816436768}
+{"rows": 700, "of": 7881, "epoch": 0, "ce": 0.1114975800498938, "kl": 0.002897128037127479, "lr": 4.904371476667369e-05, "rows_per_s": 0.33431817481488013, "tokens_per_s": 2266.1852999305165, "gpu_gb": 17.879604816436768}
+{"rows": 750, "of": 7881, "epoch": 0, "ce": 0.09807620361083891, "kl": 0.009082421921421006, "lr": 4.8908266374903594e-05, "rows_per_s": 0.33531974425999084, "tokens_per_s": 2278.850418617599, "gpu_gb": 17.879604816436768}
+{"rows": 800, "of": 7881, "epoch": 0, "ce": 0.10299079625734262, "kl": 0.003760476317451397, "lr": 4.873918134940002e-05, "rows_per_s": 0.33615507979470843, "tokens_per_s": 2289.621580466967, "gpu_gb": 17.879604816436768}
+{"rows": 850, "of": 7881, "epoch": 0, "ce": 0.13046966404773486, "kl": 0.0006150131550363122, "lr": 4.858483038753156e-05, "rows_per_s": 0.3374617653959088, "tokens_per_s": 2296.7381753571653, "gpu_gb": 17.879604816436768}
+{"rows": 900, "of": 7881, "epoch": 0, "ce": 0.12523869971911877, "kl": 0.0001525189401817158, "lr": 4.842184269673884e-05, "rows_per_s": 0.33890299816046116, "tokens_per_s": 2303.644553899332, "gpu_gb": 17.879604816436768}
+{"rows": 950, "of": 7881, "epoch": 0, "ce": 0.11561032039092177, "kl": 0.003183731898943517, "lr": 4.825027796286572e-05, "rows_per_s": 0.340366836428032, "tokens_per_s": 2312.5866420222183, "gpu_gb": 17.879604816436768}
+{"rows": 1000, "of": 7881, "epoch": 0, "ce": 0.08595053996204545, "kl": 0.004521621390713067, "lr": 4.803936266823792e-05, "rows_per_s": 0.3409823596466644, "tokens_per_s": 2317.184837950267, "gpu_gb": 17.879604816436768}
+{"rows": 1050, "of": 7881, "epoch": 0, "ce": 0.12116317149429094, "kl": 0.004317330755962967, "lr": 4.7849434029923976e-05, "rows_per_s": 0.3412533427182491, "tokens_per_s": 2324.4117185783475, "gpu_gb": 17.879604816436768}
+{"rows": 1100, "of": 7881, "epoch": 0, "ce": 0.1212392614417991, "kl": 0.0018501514580582642, "lr": 4.7651137963710567e-05, "rows_per_s": 0.342086439053439, "tokens_per_s": 2331.016180156789, "gpu_gb": 17.879604816436768}
+{"rows": 1150, "of": 7881, "epoch": 0, "ce": 0.09878259303513914, "kl": 0.0022178078633755677, "lr": 4.744454708531476e-05, "rows_per_s": 0.3423104281611625, "tokens_per_s": 2335.7200825311506, "gpu_gb": 17.879604816436768}
+{"rows": 1200, "of": 7881, "epoch": 0, "ce": 0.12648315269034355, "kl": 0.00019712260292380213, "lr": 4.7193141769868265e-05, "rows_per_s": 0.3418628543799443, "tokens_per_s": 2340.356065942541, "gpu_gb": 17.879604816436768}
+{"rows": 1250, "of": 7881, "epoch": 0, "ce": 0.1220479605196998, "kl": 0.0016524095801212656, "lr": 4.696884234499382e-05, "rows_per_s": 0.342858155448323, "tokens_per_s": 2344.985211351914, "gpu_gb": 17.879604816436768}
+{"rows": 1300, "of": 7881, "epoch": 0, "ce": 0.1189333973103203, "kl": 0.0003812499474370734, "lr": 4.673649796355214e-05, "rows_per_s": 0.3436620106010812, "tokens_per_s": 2348.0984447481283, "gpu_gb": 17.879604816436768}
+{"rows": 1350, "of": 7881, "epoch": 0, "ce": 0.1113097622918968, "kl": 0.34708880339413883, "lr": 4.649619370970142e-05, "rows_per_s": 0.34475158728045996, "tokens_per_s": 2350.46984445678, "gpu_gb": 17.879604816436768}
+{"rows": 1400, "of": 7881, "epoch": 0, "ce": 0.12136290367278788, "kl": 0.013578459140143051, "lr": 4.620589591309603e-05, "rows_per_s": 0.3449168650795269, "tokens_per_s": 2351.5276389624128, "gpu_gb": 17.879604816436768}
+{"rows": 1450, "of": 7881, "epoch": 0, "ce": 0.13677624828721346, "kl": 0.0048611547827111246, "lr": 4.594865099860148e-05, "rows_per_s": 0.3453105518218504, "tokens_per_s": 2353.240209446157, "gpu_gb": 17.879604816436768}
+{"rows": 1500, "of": 7881, "epoch": 0, "ce": 0.13925005014784433, "kl": 0.0002106315870359045, "lr": 4.56837347200551e-05, "rows_per_s": 0.3451715674738576, "tokens_per_s": 2354.8559307736577, "gpu_gb": 18.01590633392334}
+{"rows": 1550, "of": 7881, "epoch": 0, "ce": 0.09757101010984263, "kl": 0.0008377450640034167, "lr": 4.541124408939413e-05, "rows_per_s": 0.34531983318767456, "tokens_per_s": 2356.660822093038, "gpu_gb": 18.01590633392334}
+{"rows": 1600, "of": 7881, "epoch": 0, "ce": 0.10095553258685647, "kl": 0.00035042277662108133, "lr": 4.5083898491176136e-05, "rows_per_s": 0.34514860074861325, "tokens_per_s": 2357.4279327326653, "gpu_gb": 18.01590633392334}
+{"rows": 1650, "of": 7881, "epoch": 0, "ce": 0.13497693555126355, "kl": 0.004645874111277116, "lr": 4.479534273409534e-05, "rows_per_s": 0.34542321286578037, "tokens_per_s": 2360.972265580447, "gpu_gb": 18.01590633392334}
+{"rows": 1700, "of": 7881, "epoch": 0, "ce": 0.10531843452833506, "kl": 0.0014682905136677502, "lr": 4.449953795269053e-05, "rows_per_s": 0.3457247683964078, "tokens_per_s": 2363.9703835645505, "gpu_gb": 18.01590633392334}
+{"rows": 1750, "of": 7881, "epoch": 0, "ce": 0.1191714326492869, "kl": 0.0018370602015290525, "lr": 4.41965924702216e-05, "rows_per_s": 0.3456484219925588, "tokens_per_s": 2366.0383061115995, "gpu_gb": 18.14210271835327}
+{"rows": 1800, "of": 7881, "epoch": 0, "ce": 0.12367003023820489, "kl": 0.0016225997033231937, "lr": 4.38342791679207e-05, "rows_per_s": 0.34594930135561597, "tokens_per_s": 2367.8127074815898, "gpu_gb": 18.14210271835327}
+{"rows": 1850, "of": 7881, "epoch": 0, "ce": 0.11905794182703608, "kl": 1.6381730206133227, "lr": 4.35162461781166e-05, "rows_per_s": 0.34632148936709567, "tokens_per_s": 2370.144900087056, "gpu_gb": 18.14210271835327}
+{"rows": 1900, "of": 7881, "epoch": 0, "ce": 0.12516607127250418, "kl": 0.004563029444159354, "lr": 4.319143256719685e-05, "rows_per_s": 0.3466548952856714, "tokens_per_s": 2370.7996490006053, "gpu_gb": 18.14210271835327}
+{"rows": 1950, "of": 7881, "epoch": 0, "ce": 0.1272408339457634, "kl": 0.005762273290107881, "lr": 4.2859957281410456e-05, "rows_per_s": 0.3471816952285003, "tokens_per_s": 2372.0336505810287, "gpu_gb": 18.14210271835327}
+{"rows": 2000, "of": 7881, "epoch": 0, "ce": 0.09646510239690542, "kl": 0.002349670455144367, "lr": 4.246497857268759e-05, "rows_per_s": 0.3474575131271955, "tokens_per_s": 2372.4584886094435, "gpu_gb": 18.14210271835327}
+{"rows": 2050, "of": 7881, "epoch": 0, "ce": 0.11164875448931282, "kl": 0.0001557140411630341, "lr": 4.211948927129083e-05, "rows_per_s": 0.3483314047594994, "tokens_per_s": 2373.5057238740164, "gpu_gb": 18.14210271835327}
+{"rows": 2100, "of": 7881, "epoch": 0, "ce": 0.12234918293252121, "kl": 6.982944525835144e-06, "lr": 4.176773083902782e-05, "rows_per_s": 0.3491371026150422, "tokens_per_s": 2374.836058427701, "gpu_gb": 18.14210271835327}
+{"rows": 2150, "of": 7881, "epoch": 0, "ce": 0.12124350821463073, "kl": 0.009231240116690643, "lr": 4.1409832089300073e-05, "rows_per_s": 0.3489263150692104, "tokens_per_s": 2376.164673381469, "gpu_gb": 18.14210271835327}
+{"rows": 2200, "of": 7881, "epoch": 0, "ce": 0.13119616450333013, "kl": 7.741211165319179e-05, "lr": 4.098469784213812e-05, "rows_per_s": 0.3491915680038746, "tokens_per_s": 2378.0467981181464, "gpu_gb": 18.14210271835327}
+{"rows": 2250, "of": 7881, "epoch": 0, "ce": 0.11777349795341831, "kl": 0.0011427605857715119, "lr": 4.0613947619727606e-05, "rows_per_s": 0.3493453768995677, "tokens_per_s": 2380.0346233502864, "gpu_gb": 18.14210271835327}
+{"rows": 2300, "of": 7881, "epoch": 0, "ce": 0.12106773099647118, "kl": 0.0007932578767766429, "lr": 4.0237479593507887e-05, "rows_per_s": 0.3493335337690823, "tokens_per_s": 2381.2046937908285, "gpu_gb": 18.14210271835327}
+{"rows": 2350, "of": 7881, "epoch": 0, "ce": 0.12626307624343194, "kl": 0.0023483902874269283, "lr": 3.985543162549613e-05, "rows_per_s": 0.34898801486440695, "tokens_per_s": 2382.152538485732, "gpu_gb": 18.14210271835327}
+{"rows": 2400, "of": 7881, "epoch": 0, "ce": 0.10973528102417854, "kl": 0.007160376068695103, "lr": 3.940284332939771e-05, "rows_per_s": 0.3491693489140407, "tokens_per_s": 2382.606226998262, "gpu_gb": 18.14210271835327}
+{"rows": 2450, "of": 7881, "epoch": 0, "ce": 0.12549861390808864, "kl": 0.0002632642681896745, "lr": 3.900918809552583e-05, "rows_per_s": 0.34886942404784405, "tokens_per_s": 2384.430952942662, "gpu_gb": 18.14210271835327}
+{"rows": 2500, "of": 7881, "epoch": 0, "ce": 0.11909317640321596, "kl": 0.00011588107850546425, "lr": 3.861040271833547e-05, "rows_per_s": 0.349246677488605, "tokens_per_s": 2384.1419433855895, "gpu_gb": 18.14210271835327}
+{"rows": 2550, "of": 7881, "epoch": 0, "ce": 0.11558077179301435, "kl": 0.0002803067502299795, "lr": 3.820663323242368e-05, "rows_per_s": 0.34928072068730537, "tokens_per_s": 2385.1412017816374, "gpu_gb": 18.14210271835327}
+{"rows": 2600, "of": 7881, "epoch": 0, "ce": 0.11283887832470965, "kl": 0.0046764551552816135, "lr": 3.772946683196179e-05, "rows_per_s": 0.3493172765933783, "tokens_per_s": 2386.3068308697916, "gpu_gb": 18.14210271835327}
+{"rows": 2650, "of": 7881, "epoch": 0, "ce": 0.11523845332807728, "kl": 1.9208870107000064e-05, "lr": 3.731540804467163e-05, "rows_per_s": 0.3491995893486278, "tokens_per_s": 2387.478251243718, "gpu_gb": 18.14210271835327}
+{"rows": 2700, "of": 7881, "epoch": 0, "ce": 0.12425444895602963, "kl": 0.011715986646047966, "lr": 3.689683937373484e-05, "rows_per_s": 0.349518752100196, "tokens_per_s": 2387.5444724018853, "gpu_gb": 18.14210271835327}
+{"rows": 2750, "of": 7881, "epoch": 0, "ce": 0.11785337779567473, "kl": 0.0006625117789127444, "lr": 3.647391409836045e-05, "rows_per_s": 0.3499168163441494, "tokens_per_s": 2389.7649134985245, "gpu_gb": 18.14210271835327}
+{"rows": 2800, "of": 7881, "epoch": 0, "ce": 0.11555427557877487, "kl": 0.0017233899385138765, "lr": 3.597520171816503e-05, "rows_per_s": 0.34972243071269005, "tokens_per_s": 2391.027153708171, "gpu_gb": 18.14210271835327}
+{"rows": 2850, "of": 7881, "epoch": 0, "ce": 0.11343527100980282, "kl": 0.0014408573547337877, "lr": 3.5543370488628656e-05, "rows_per_s": 0.34988801691872107, "tokens_per_s": 2391.6190262985906, "gpu_gb": 18.14210271835327}
+{"rows": 2900, "of": 7881, "epoch": 0, "ce": 0.12260413981783085, "kl": 0.000579910366572577, "lr": 3.510767829289656e-05, "rows_per_s": 0.34992913695689, "tokens_per_s": 2391.9947943524403, "gpu_gb": 18.14210271835327}
+{"rows": 2950, "of": 7881, "epoch": 0, "ce": 0.11427439829438096, "kl": 0.002844052278886261, "lr": 3.4668284680786446e-05, "rows_per_s": 0.3501040508195231, "tokens_per_s": 2391.9062467047506, "gpu_gb": 18.14210271835327}
+{"rows": 3000, "of": 7881, "epoch": 0, "ce": 0.111582133964431, "kl": 0.0017957321312955798, "lr": 3.4151195358094365e-05, "rows_per_s": 0.3504710230898656, "tokens_per_s": 2392.5341311771795, "gpu_gb": 18.14210271835327}
+{"rows": 3050, "of": 7881, "epoch": 0, "ce": 0.1025324217043817, "kl": 0.0024058066544616528, "lr": 3.370433573137822e-05, "rows_per_s": 0.3504001040912947, "tokens_per_s": 2394.1137661560924, "gpu_gb": 18.14210271835327}
+{"rows": 3100, "of": 7881, "epoch": 0, "ce": 0.11070433322207204, "kl": 0.0012162467746677959, "lr": 3.325428859019016e-05, "rows_per_s": 0.3506068875902027, "tokens_per_s": 2395.234740406212, "gpu_gb": 18.14210271835327}
+{"rows": 3150, "of": 7881, "epoch": 0, "ce": 0.11355097132424513, "kl": 0.0004539966578631649, "lr": 3.280121874110706e-05, "rows_per_s": 0.3508131695553405, "tokens_per_s": 2396.853245243261, "gpu_gb": 18.14210271835327}
+{"rows": 3200, "of": 7881, "epoch": 0, "ce": 0.108288824153868, "kl": 0.008527442859412732, "lr": 3.226903828830935e-05, "rows_per_s": 0.3507120956916215, "tokens_per_s": 2397.1940019206954, "gpu_gb": 18.14210271835327}
+{"rows": 3250, "of": 7881, "epoch": 0, "ce": 0.10958007871439414, "kl": 0.0014140021693786846, "lr": 3.180998980650813e-05, "rows_per_s": 0.350860917677685, "tokens_per_s": 2396.7995894389255, "gpu_gb": 18.14210271835327}
+{"rows": 3300, "of": 7881, "epoch": 0, "ce": 0.09785355463926051, "kl": 7.356050523752946e-05, "lr": 3.134844751682408e-05, "rows_per_s": 0.35076139887632746, "tokens_per_s": 2396.760291497921, "gpu_gb": 18.14210271835327}
+{"rows": 3350, "of": 7881, "epoch": 0, "ce": 0.10941918043638854, "kl": 0.015381404429639523, "lr": 3.088458043533989e-05, "rows_per_s": 0.35098194884508077, "tokens_per_s": 2395.9897985713537, "gpu_gb": 18.14210271835327}
+{"rows": 3400, "of": 7881, "epoch": 0, "ce": 0.12302674842732293, "kl": 0.0035206266907325903, "lr": 3.034069056048837e-05, "rows_per_s": 0.35104842534672653, "tokens_per_s": 2395.432620112559, "gpu_gb": 18.14210271835327}
+{"rows": 3450, "of": 7881, "epoch": 0, "ce": 0.11572272940115495, "kl": 0.004864113947760466, "lr": 2.9872370219028968e-05, "rows_per_s": 0.35110152424696506, "tokens_per_s": 2396.6856629151116, "gpu_gb": 18.14210271835327}
+{"rows": 3500, "of": 7881, "epoch": 0, "ce": 0.11079833977969318, "kl": 0.005617409808760366, "lr": 2.9402265623025054e-05, "rows_per_s": 0.3511640492737864, "tokens_per_s": 2397.345092445433, "gpu_gb": 18.14210271835327}
+{"rows": 3550, "of": 7881, "epoch": 0, "ce": 0.11192968271465766, "kl": 0.0002326338827762786, "lr": 2.893054892406285e-05, "rows_per_s": 0.35127164475102046, "tokens_per_s": 2398.8457380762907, "gpu_gb": 18.14210271835327}
+{"rows": 3600, "of": 7881, "epoch": 0, "ce": 0.11361729757996594, "kl": 0.0026719365472047426, "lr": 2.8378405742013713e-05, "rows_per_s": 0.3516791403803417, "tokens_per_s": 2399.508240144823, "gpu_gb": 18.14210271835327}
+{"rows": 3650, "of": 7881, "epoch": 0, "ce": 0.10134117697000604, "kl": 0.003282329639780607, "lr": 2.790378945378661e-05, "rows_per_s": 0.35186085898107367, "tokens_per_s": 2399.4028215472636, "gpu_gb": 18.14210271835327}
+{"rows": 3700, "of": 7881, "epoch": 0, "ce": 0.10886479691011695, "kl": 0.002696012913522191, "lr": 2.742810980228942e-05, "rows_per_s": 0.3520560044507519, "tokens_per_s": 2399.9929001681458, "gpu_gb": 18.14210271835327}
+{"rows": 3750, "of": 7881, "epoch": 0, "ce": 0.10554261264415124, "kl": 0.004270280490981306, "lr": 2.695154098068515e-05, "rows_per_s": 0.35214443922262223, "tokens_per_s": 2400.144941991343, "gpu_gb": 18.14210271835327}
+{"rows": 3800, "of": 7881, "epoch": 0, "ce": 0.1053063746963051, "kl": 3.0245330449804597e-05, "lr": 2.6394653051429064e-05, "rows_per_s": 0.3524335517186525, "tokens_per_s": 2400.8901330440103, "gpu_gb": 18.14210271835327}
+{"rows": 3850, "of": 7881, "epoch": 0, "ce": 0.09815928067725438, "kl": 0.00010466489848133657, "lr": 2.59167567365324e-05, "rows_per_s": 0.3522769411467296, "tokens_per_s": 2401.39934789743, "gpu_gb": 18.14210271835327}
+{"rows": 3900, "of": 7881, "epoch": 0, "ce": 0.10582923782183914, "kl": 0.0013470960252889852, "lr": 2.5438524706716728e-05, "rows_per_s": 0.35222433142551585, "tokens_per_s": 2401.25722773397, "gpu_gb": 18.14210271835327}
+{"rows": 3950, "of": 7881, "epoch": 0, "ce": 0.10648514779804405, "kl": 8.589219479106831e-05, "lr": 2.4960132089822155e-05, "rows_per_s": 0.35227282059189363, "tokens_per_s": 2402.078979247915, "gpu_gb": 18.14210271835327}
+{"rows": 4000, "of": 7881, "epoch": 0, "ce": 0.09706557054861067, "kl": 0.014215183616298107, "lr": 2.4402038123554933e-05, "rows_per_s": 0.3523804363546986, "tokens_per_s": 2402.9342597121613, "gpu_gb": 18.14210271835327}
+{"rows": 4050, "of": 7881, "epoch": 0, "ce": 0.10776543771711791, "kl": 0.00592954908566838, "lr": 2.3923898544815143e-05, "rows_per_s": 0.35240577413650087, "tokens_per_s": 2403.441663036864, "gpu_gb": 18.14210271835327}
+{"rows": 4100, "of": 7881, "epoch": 0, "ce": 0.12846701116122436, "kl": 0.0007149715039334037, "lr": 2.3446153032787338e-05, "rows_per_s": 0.3525940728524637, "tokens_per_s": 2404.130694282304, "gpu_gb": 18.14210271835327}
+{"rows": 4150, "of": 7881, "epoch": 0, "ce": 0.114844310508911, "kl": 1.6003082717121396e-06, "lr": 2.296897653714952e-05, "rows_per_s": 0.35265046029606256, "tokens_per_s": 2405.0568496638484, "gpu_gb": 18.14210271835327}
+{"rows": 4200, "of": 7881, "epoch": 0, "ce": 0.10019052027103802, "kl": 0.0008372855555545122, "lr": 2.2413222907755195e-05, "rows_per_s": 0.35300414798589275, "tokens_per_s": 2405.963899684736, "gpu_gb": 18.14210271835327}
+{"rows": 4250, "of": 7881, "epoch": 0, "ce": 0.10425313653993523, "kl": 0.0006422162651921272, "lr": 2.1937878373803932e-05, "rows_per_s": 0.35306276984732304, "tokens_per_s": 2405.8742203195343, "gpu_gb": 18.14210271835327}
+{"rows": 4300, "of": 7881, "epoch": 0, "ce": 0.10495267823598413, "kl": 0.00831119328665005, "lr": 2.1463655184122865e-05, "rows_per_s": 0.35307499846252643, "tokens_per_s": 2406.5503215903864, "gpu_gb": 18.14210271835327}
+{"rows": 4350, "of": 7881, "epoch": 0, "ce": 0.10782617076516285, "kl": 0.00039285099983008, "lr": 2.0990726998520924e-05, "rows_per_s": 0.353409119911525, "tokens_per_s": 2407.016057510293, "gpu_gb": 18.14210271835327}
+{"rows": 4400, "of": 7881, "epoch": 0, "ce": 0.11211016473201353, "kl": 0.002145166808824101, "lr": 2.0440845208356402e-05, "rows_per_s": 0.3533396402845207, "tokens_per_s": 2407.3939542158128, "gpu_gb": 18.14210271835327}
+{"rows": 4450, "of": 7881, "epoch": 0, "ce": 0.12113836288917809, "kl": 0.005113993874913458, "lr": 1.9971316266884595e-05, "rows_per_s": 0.35344436463114887, "tokens_per_s": 2407.577946942775, "gpu_gb": 18.14210271835327}
+{"rows": 4500, "of": 7881, "epoch": 0, "ce": 0.09754754564809529, "kl": 0.0017750637890201944, "lr": 1.950362882172778e-05, "rows_per_s": 0.353663367563879, "tokens_per_s": 2408.2754169378013, "gpu_gb": 18.14210271835327}
+{"rows": 4550, "of": 7881, "epoch": 0, "ce": 0.11038847780990892, "kl": 0.002171902272943745, "lr": 1.9037954139315226e-05, "rows_per_s": 0.3536919449527754, "tokens_per_s": 2408.6936830975224, "gpu_gb": 18.14210271835327}
+{"rows": 4600, "of": 7881, "epoch": 0, "ce": 0.09919550201218379, "kl": 0.0015605589092663325, "lr": 1.849743837849513e-05, "rows_per_s": 0.35355483844101715, "tokens_per_s": 2408.8399021260857, "gpu_gb": 18.14210271835327}
+{"rows": 4650, "of": 7881, "epoch": 0, "ce": 0.11199251218172519, "kl": 0.0006752163792678179, "lr": 1.8036708622367915e-05, "rows_per_s": 0.35377122043014136, "tokens_per_s": 2409.9226482971953, "gpu_gb": 18.14210271835327}
+{"rows": 4700, "of": 7881, "epoch": 0, "ce": 0.1119166434957431, "kl": 0.0011205657926364894, "lr": 1.7578528812925103e-05, "rows_per_s": 0.3539852915686041, "tokens_per_s": 2409.8915709286334, "gpu_gb": 18.14210271835327}
+{"rows": 4750, "of": 7881, "epoch": 0, "ce": 0.09582457252301622, "kl": 0.0005217474993762199, "lr": 1.712306673491427e-05, "rows_per_s": 0.354026995399871, "tokens_per_s": 2410.5926930014243, "gpu_gb": 18.14210271835327}
+{"rows": 4800, "of": 7881, "epoch": 0, "ce": 0.09214032678643153, "kl": 0.0008081881126124901, "lr": 1.6595351677694083e-05, "rows_per_s": 0.35368195366056704, "tokens_per_s": 2411.3314973596875, "gpu_gb": 18.14210271835327}
+{"rows": 4850, "of": 7881, "epoch": 0, "ce": 0.10614188416481116, "kl": 0.0003972421627815535, "lr": 1.614634878589294e-05, "rows_per_s": 0.35345733927192236, "tokens_per_s": 2411.2895395253477, "gpu_gb": 18.14210271835327}
+{"rows": 4900, "of": 7881, "epoch": 0, "ce": 0.10370026875494255, "kl": 0.0035379835823328176, "lr": 1.5700588087665796e-05, "rows_per_s": 0.35333459728973593, "tokens_per_s": 2410.994849154348, "gpu_gb": 18.14210271835327}
+{"rows": 4950, "of": 7881, "epoch": 0, "ce": 0.09188798344176677, "kl": 0.009863047313862339, "lr": 1.5258232819900598e-05, "rows_per_s": 0.35318504105135945, "tokens_per_s": 2411.9991803986613, "gpu_gb": 18.14210271835327}
+{"rows": 5000, "of": 7881, "epoch": 0, "ce": 0.10940351203466073, "kl": 0.021478232028962648, "lr": 1.4746671799249784e-05, "rows_per_s": 0.3531439057229424, "tokens_per_s": 2411.913759797879, "gpu_gb": 18.14210271835327}
+{"rows": 5050, "of": 7881, "epoch": 0, "ce": 0.1133196825782458, "kl": 0.010227535617603352, "lr": 1.4312248933115987e-05, "rows_per_s": 0.3531588843391178, "tokens_per_s": 2412.036297592671, "gpu_gb": 18.14210271835327}
+{"rows": 5100, "of": 7881, "epoch": 0, "ce": 0.12415518498400578, "kl": 0.01082342541506375, "lr": 1.3881739905125512e-05, "rows_per_s": 0.35318651890898395, "tokens_per_s": 2411.89986502485, "gpu_gb": 18.14210271835327}
+{"rows": 5150, "of": 7881, "epoch": 0, "ce": 0.10389299005711157, "kl": 0.0019936111658529093, "lr": 1.3455302367027933e-05, "rows_per_s": 0.35334298288083166, "tokens_per_s": 2411.884758780012, "gpu_gb": 18.14210271835327}
+{"rows": 5200, "of": 7881, "epoch": 0, "ce": 0.09081864366307854, "kl": 0.0013251463428219923, "lr": 1.2963146066081217e-05, "rows_per_s": 0.3534737701588645, "tokens_per_s": 2412.0156194856763, "gpu_gb": 18.14210271835327}
+{"rows": 5250, "of": 7881, "epoch": 0, "ce": 0.10161151361038409, "kl": 0.007390453495280839, "lr": 1.2546063739077025e-05, "rows_per_s": 0.35345362342331577, "tokens_per_s": 2412.438259142607, "gpu_gb": 18.14210271835327}
+{"rows": 5300, "of": 7881, "epoch": 0, "ce": 0.09811767693175825, "kl": 0.0002757578011564859, "lr": 1.213354202454101e-05, "rows_per_s": 0.3536789063729187, "tokens_per_s": 2412.883383191618, "gpu_gb": 18.14210271835327}
+{"rows": 5350, "of": 7881, "epoch": 0, "ce": 0.0939898087971789, "kl": 0.0005090700198054701, "lr": 1.1725731987296001e-05, "rows_per_s": 0.35364863680573116, "tokens_per_s": 2412.3733906976017, "gpu_gb": 18.14210271835327}
+{"rows": 5400, "of": 7881, "epoch": 0, "ce": 0.08981802885551961, "kl": 0.0022342575490095925, "lr": 1.125610778308672e-05, "rows_per_s": 0.3536141918820328, "tokens_per_s": 2412.378543552933, "gpu_gb": 18.14210271835327}
+{"rows": 5450, "of": 7881, "epoch": 0, "ce": 0.10204237564984295, "kl": 0.003696178624798243, "lr": 1.0859016319281876e-05, "rows_per_s": 0.35385703869747154, "tokens_per_s": 2412.140911651287, "gpu_gb": 18.14210271835327}
+{"rows": 5500, "of": 7881, "epoch": 0, "ce": 0.11183636465242931, "kl": 0.0009385639101668897, "lr": 1.0467103262137832e-05, "rows_per_s": 0.35398024110870496, "tokens_per_s": 2411.5364743444406, "gpu_gb": 18.14210271835327}
+{"rows": 5550, "of": 7881, "epoch": 0, "ce": 0.07514825033453795, "kl": 0.00019623727030041146, "lr": 1.0080512129618501e-05, "rows_per_s": 0.3541118924863004, "tokens_per_s": 2411.475700606534, "gpu_gb": 18.14210271835327}
+{"rows": 5600, "of": 7881, "epoch": 0, "ce": 0.09555623648422105, "kl": 0.01650010496503926, "lr": 9.636404220353013e-06, "rows_per_s": 0.35423236746036674, "tokens_per_s": 2410.9903189361066, "gpu_gb": 18.14210271835327}
+{"rows": 5650, "of": 7881, "epoch": 0, "ce": 0.10276433913303273, "kl": 0.0010513066600173575, "lr": 9.261826913103009e-06, "rows_per_s": 0.3544302138014995, "tokens_per_s": 2410.514386120209, "gpu_gb": 18.14210271835327}
+{"rows": 5700, "of": 7881, "epoch": 0, "ce": 0.08586389429341906, "kl": 0.0002948080244487031, "lr": 8.893012900838857e-06, "rows_per_s": 0.3542102976004182, "tokens_per_s": 2410.572401099001, "gpu_gb": 18.14210271835327}
+{"rows": 5750, "of": 7881, "epoch": 0, "ce": 0.09609906260783856, "kl": 0.0019267380242358391, "lr": 8.530097242689436e-06, "rows_per_s": 0.3543317453345797, "tokens_per_s": 2410.7407676180933, "gpu_gb": 18.14210271835327}
+{"rows": 5800, "of": 7881, "epoch": 0, "ce": 0.10997136807205474, "kl": 0.00028150575860390174, "lr": 8.11432768484267e-06, "rows_per_s": 0.3544341588484257, "tokens_per_s": 2410.955012429749, "gpu_gb": 18.14210271835327}
+{"rows": 5850, "of": 7881, "epoch": 0, "ce": 0.08865141784545125, "kl": 0.0004392856746452308, "lr": 7.764644762675815e-06, "rows_per_s": 0.354363578113408, "tokens_per_s": 2411.2014251544765, "gpu_gb": 18.14210271835327}
+{"rows": 5900, "of": 7881, "epoch": 0, "ce": 0.10368858785590246, "kl": 0.0013282307687131206, "lr": 7.42127340131529e-06, "rows_per_s": 0.35445395975718136, "tokens_per_s": 2411.7249881174275, "gpu_gb": 18.14210271835327}
+{"rows": 5950, "of": 7881, "epoch": 0, "ce": 0.09626056697232195, "kl": 0.006887896287601823, "lr": 7.084339342830038e-06, "rows_per_s": 0.35444099409804697, "tokens_per_s": 2411.545278224802, "gpu_gb": 18.14210271835327}
+{"rows": 6000, "of": 7881, "epoch": 0, "ce": 0.1048011263133958, "kl": 0.0037830519571404597, "lr": 6.699550118560632e-06, "rows_per_s": 0.3546018222551607, "tokens_per_s": 2411.6284947622867, "gpu_gb": 18.14210271835327}
+{"rows": 6050, "of": 7881, "epoch": 0, "ce": 0.10633669481095341, "kl": 8.697074627139756e-07, "lr": 6.376983620161852e-06, "rows_per_s": 0.35466830940091426, "tokens_per_s": 2412.441060761308, "gpu_gb": 18.14210271835327}
+{"rows": 6100, "of": 7881, "epoch": 0, "ce": 0.09774804168846458, "kl": 0.0004226139824040076, "lr": 6.061236841963802e-06, "rows_per_s": 0.35481096569651555, "tokens_per_s": 2412.684844047212, "gpu_gb": 18.14210271835327}
+{"rows": 6150, "of": 7881, "epoch": 0, "ce": 0.09988635304643838, "kl": 0.00045288701645091703, "lr": 5.752425409955181e-06, "rows_per_s": 0.3548562863349174, "tokens_per_s": 2413.173229222928, "gpu_gb": 18.14210271835327}
+{"rows": 6200, "of": 7881, "epoch": 0, "ce": 0.11084061023956608, "kl": 0.000553094805603397, "lr": 5.4010616387919095e-06, "rows_per_s": 0.354781857507353, "tokens_per_s": 2413.579889387798, "gpu_gb": 18.14210271835327}
+{"rows": 6250, "of": 7881, "epoch": 0, "ce": 0.09823018362602362, "kl": 0.0003262754329489616, "lr": 5.107661293194194e-06, "rows_per_s": 0.35477954657285393, "tokens_per_s": 2413.5107611967715, "gpu_gb": 18.14210271835327}
+{"rows": 6300, "of": 7881, "epoch": 0, "ce": 0.09288779341815305, "kl": 0.00545118033123352, "lr": 4.821545491697788e-06, "rows_per_s": 0.3547910631021548, "tokens_per_s": 2413.808551970281, "gpu_gb": 18.14210271835327}
+{"rows": 6350, "of": 7881, "epoch": 0, "ce": 0.10894227874066148, "kl": 0.0010907609828259741, "lr": 4.542819009472913e-06, "rows_per_s": 0.3547809589766109, "tokens_per_s": 2413.388757559718, "gpu_gb": 18.14210271835327}
+{"rows": 6400, "of": 7881, "epoch": 0, "ce": 0.11336540494386743, "kl": 0.009394696126029759, "lr": 4.227113410952585e-06, "rows_per_s": 0.354763979832929, "tokens_per_s": 2413.6881775704082, "gpu_gb": 18.14210271835327}
+{"rows": 6450, "of": 7881, "epoch": 0, "ce": 0.08434413905834427, "kl": 0.0011595013994856697, "lr": 3.9647436126595165e-06, "rows_per_s": 0.35484849955045533, "tokens_per_s": 2413.7015000506963, "gpu_gb": 18.14210271835327}
+{"rows": 6500, "of": 7881, "epoch": 0, "ce": 0.08780445242527782, "kl": 0.00028799445168845337, "lr": 3.7100768931795733e-06, "rows_per_s": 0.35478514375742276, "tokens_per_s": 2413.989011724176, "gpu_gb": 18.14210271835327}
+{"rows": 6550, "of": 7881, "epoch": 0, "ce": 0.08942597520512503, "kl": 0.001966686043677356, "lr": 3.4632065110770996e-06, "rows_per_s": 0.3549211981137051, "tokens_per_s": 2413.678454553124, "gpu_gb": 18.14210271835327}
+{"rows": 6600, "of": 7881, "epoch": 0, "ce": 0.09034750055579935, "kl": 0.000788496940761052, "lr": 3.1851652170651584e-06, "rows_per_s": 0.3547799936794328, "tokens_per_s": 2413.8867389227808, "gpu_gb": 18.14210271835327}
+{"rows": 6650, "of": 7881, "epoch": 0, "ce": 0.07828214398193124, "kl": 0.0019378861205344294, "lr": 2.9554931788704e-06, "rows_per_s": 0.3548670608583431, "tokens_per_s": 2413.780080130439, "gpu_gb": 18.14210271835327}
+{"rows": 6700, "of": 7881, "epoch": 0, "ce": 0.07726879052719787, "kl": 0.004422470554846514, "lr": 2.7338938054586595e-06, "rows_per_s": 0.3547471185114929, "tokens_per_s": 2413.8411343576086, "gpu_gb": 18.14210271835327}
+{"rows": 6750, "of": 7881, "epoch": 0, "ce": 0.08772913748637223, "kl": 0.0015729774672785196, "lr": 2.5204482461825605e-06, "rows_per_s": 0.3547941194138462, "tokens_per_s": 2413.8357467915494, "gpu_gb": 18.14210271835327}
+{"rows": 6800, "of": 7881, "epoch": 0, "ce": 0.09148549247765914, "kl": 0.000170326991907499, "lr": 2.2818380530311655e-06, "rows_per_s": 0.35481366365370665, "tokens_per_s": 2414.0400359464093, "gpu_gb": 18.14210271835327}
+{"rows": 6850, "of": 7881, "epoch": 0, "ce": 0.08052904867471601, "kl": 0.0014583624086916824, "lr": 2.0863232118170097e-06, "rows_per_s": 0.3549552905591309, "tokens_per_s": 2414.1273630712735, "gpu_gb": 18.14210271835327}
+{"rows": 6900, "of": 7881, "epoch": 0, "ce": 0.09236458982000263, "kl": 0.00011193361166209985, "lr": 1.8991993241024298e-06, "rows_per_s": 0.3549028390152498, "tokens_per_s": 2414.382256733657, "gpu_gb": 18.14210271835327}
+{"rows": 6950, "of": 7881, "epoch": 0, "ce": 0.06854723644367344, "kl": 0.0016074941756578172, "lr": 1.7205349143697785e-06, "rows_per_s": 0.35501259532909313, "tokens_per_s": 2414.2303094853114, "gpu_gb": 18.14210271835327}
+{"rows": 7000, "of": 7881, "epoch": 0, "ce": 0.09787144471192732, "kl": 0.0007308541197012658, "lr": 1.5228720559187642e-06, "rows_per_s": 0.355110762551146, "tokens_per_s": 2414.361345986372, "gpu_gb": 18.14210271835327}
+{"rows": 7050, "of": 7881, "epoch": 0, "ce": 0.09466787906629699, "kl": 0.0004142965550808439, "lr": 1.3627567987539253e-06, "rows_per_s": 0.35518071734905576, "tokens_per_s": 2414.6297057364964, "gpu_gb": 18.14210271835327}
+{"rows": 7100, "of": 7881, "epoch": 0, "ce": 0.07888386834705205, "kl": 0.0030428846056490544, "lr": 1.211297464005795e-06, "rows_per_s": 0.35524718140611017, "tokens_per_s": 2415.0977778707143, "gpu_gb": 18.14210271835327}
+{"rows": 7150, "of": 7881, "epoch": 0, "ce": 0.08980546247912571, "kl": 0.0005657210504075721, "lr": 1.0685495158517966e-06, "rows_per_s": 0.3552413859257695, "tokens_per_s": 2415.2295430239983, "gpu_gb": 18.14210271835327}
+{"rows": 7200, "of": 7881, "epoch": 0, "ce": 0.08522592262597754, "kl": 0.0018233578388974908, "lr": 9.130900286137001e-07, "rows_per_s": 0.35534220000512295, "tokens_per_s": 2415.349670278655, "gpu_gb": 18.14210271835327}
+{"rows": 7250, "of": 7881, "epoch": 0, "ce": 0.08452724412696935, "kl": 0.0003058793615789362, "lr": 7.893917980809184e-07, "rows_per_s": 0.3553408823053975, "tokens_per_s": 2415.5479147190777, "gpu_gb": 18.14210271835327}
+{"rows": 7300, "of": 7881, "epoch": 0, "ce": 0.12089039026819265, "kl": 0.0060507724590082934, "lr": 6.745594553537876e-07, "rows_per_s": 0.35541466882220013, "tokens_per_s": 2415.5635762279085, "gpu_gb": 18.14210271835327}
+{"rows": 7350, "of": 7881, "epoch": 0, "ce": 0.08610098976218565, "kl": 0.0012619211272942483, "lr": 5.686350518611078e-07, "rows_per_s": 0.3553772118283921, "tokens_per_s": 2416.115427821802, "gpu_gb": 18.14210271835327}
+{"rows": 7400, "of": 7881, "epoch": 0, "ce": 0.09638716007161531, "kl": 9.438584983034737e-08, "lr": 4.56366793614163e-07, "rows_per_s": 0.3553742283917881, "tokens_per_s": 2416.51809999703, "gpu_gb": 18.14210271835327}
+{"rows": 7450, "of": 7881, "epoch": 0, "ce": 0.08936755587671731, "kl": 0.0006797864884015649, "lr": 3.6987162253368234e-07, "rows_per_s": 0.3554316702720424, "tokens_per_s": 2416.9078774952472, "gpu_gb": 18.14210271835327}
+{"rows": 7500, "of": 7881, "epoch": 0, "ce": 0.09322166013444075, "kl": 0.002659181861074167, "lr": 2.9239596690742166e-07, "rows_per_s": 0.355397959917548, "tokens_per_s": 2417.0786792741096, "gpu_gb": 18.14210271835327}
+{"rows": 7550, "of": 7881, "epoch": 0, "ce": 0.08632906711397645, "kl": 0.0012073768523527086, "lr": 2.2396819820233173e-07, "rows_per_s": 0.35521756051859116, "tokens_per_s": 2417.1022617668127, "gpu_gb": 18.14210271835327}
+{"rows": 7600, "of": 7881, "epoch": 0, "ce": 0.1036922133005247, "kl": 0.0006465281106154346, "lr": 1.5560457070883105e-07, "rows_per_s": 0.3553433645089684, "tokens_per_s": 2417.2480175963615, "gpu_gb": 18.14210271835327}
+{"rows": 7650, "of": 7881, "epoch": 0, "ce": 0.09213094261940569, "kl": 0.0012346084673397345, "lr": 1.068620873410392e-07, "rows_per_s": 0.35537506368851973, "tokens_per_s": 2416.5884326639075, "gpu_gb": 18.14210271835327}
+{"rows": 7700, "of": 7881, "epoch": 0, "ce": 0.07674118545079647, "kl": 0.0004819635549952501, "lr": 6.723543311820124e-08, "rows_per_s": 0.3554247432997654, "tokens_per_s": 2416.9686173576365, "gpu_gb": 18.14210271835327}
+{"rows": 7750, "of": 7881, "epoch": 0, "ce": 0.08984228905378464, "kl": 3.306349218979517e-05, "lr": 3.673911926067031e-08, "rows_per_s": 0.3554192137104483, "tokens_per_s": 2416.9028425297843, "gpu_gb": 18.14210271835327}
+{"rows": 7800, "of": 7881, "epoch": 0, "ce": 0.08743503488316808, "kl": 0.000615512762973669, "lr": 1.2714534999183625e-08, "rows_per_s": 0.35546135850478183, "tokens_per_s": 2417.5215918193917, "gpu_gb": 18.14210271835327}
+{"rows": 7850, "of": 7881, "epoch": 0, "ce": 0.10119197343308957, "kl": 0.22764251472950622, "lr": 2.034470465553384e-09, "rows_per_s": 0.3554906573588969, "tokens_per_s": 2417.591381748814, "gpu_gb": 18.14210271835327}
+{"rows": 7881, "of": 7881, "epoch": 0, "ce": 0.10267136649539073, "kl": 0.01944962856763155, "lr": 0.0, "rows_per_s": 0.3554265725391139, "tokens_per_s": 2417.902661564681, "gpu_gb": 18.14210271835327}

ablations/specialists/select/training_meta.json

@@ -0,0 +1,26 @@
+{
+ "base_model": "Qwen/Qwen3.5-4B",
+ "families": [
+  "Discrete selection"
+ ],
+ "training_rows": 6305,
+ "rows_trained": 7881,
+ "validation_ce": 0.1338922423461918,
+ "minutes": 372.3,
+ "args": {
+  "adapter": "select",
+  "epochs": 1.0,
+  "lr": 5e-05,
+  "rank": 64,
+  "alpha": 128,
+  "grad_accum": 8,
+  "beta": 0.1,
+  "anchor_ratio": 0.25,
+  "limit": 0,
+  "max_steps": 0,
+  "seed": 0,
+  "ce_on": "all",
+  "exclude_variants": "",
+  "init_adapter": "the main executor (repository root)"
+ }
+}

ablations/task_centric/adapter_config.json

@@ -0,0 +1,51 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "kasa_config": null,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.21.0",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "k_proj",
+    "v_proj",
+    "o_proj",
+    "gate_proj",
+    "up_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

ablations/task_centric/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:08599b99a76d350f624ace6a960d9ce44bf409893aeaa362230dffb75dc9c39b
+size 339776632

ablations/task_centric/train_log.jsonl

@@ -0,0 +1,131 @@
+{"rows": 200, "of": 26016, "epoch": 0, "ce": 0.101214769189246, "kl": 0.0, "lr": 2.680021530956854e-05, "rows_per_s": 1.2794773014261045, "tokens_per_s": 8426.02335808764, "gpu_gb": 16.234378337860107}
+{"rows": 400, "of": 26016, "epoch": 0, "ce": 0.0878394184820354, "kl": 0.0, "lr": 5.254665813753962e-05, "rows_per_s": 1.3279027324950752, "tokens_per_s": 8745.886092868364, "gpu_gb": 16.234378337860107}
+{"rows": 600, "of": 26016, "epoch": 0, "ce": 0.07611608442850411, "kl": 0.0, "lr": 7.824773440931213e-05, "rows_per_s": 1.318830753495827, "tokens_per_s": 8871.06231015954, "gpu_gb": 17.2341570854187}
+{"rows": 800, "of": 26016, "epoch": 0, "ce": 0.1016095300600864, "kl": 0.0, "lr": 9.976686853167967e-05, "rows_per_s": 1.3205928901807025, "tokens_per_s": 8900.063150763885, "gpu_gb": 17.2341570854187}
+{"rows": 1000, "of": 26016, "epoch": 0, "ce": 0.09223198872059583, "kl": 0.0, "lr": 9.963589143088399e-05, "rows_per_s": 1.3266889225577472, "tokens_per_s": 8850.7928766164, "gpu_gb": 17.2341570854187}
+{"rows": 1200, "of": 26016, "epoch": 0, "ce": 0.08975416766479612, "kl": 0.0, "lr": 9.947596400765343e-05, "rows_per_s": 1.305099512140423, "tokens_per_s": 8649.547016710654, "gpu_gb": 17.2341570854187}
+{"rows": 1400, "of": 26016, "epoch": 0, "ce": 0.10392994111287407, "kl": 0.0, "lr": 9.928717954026577e-05, "rows_per_s": 1.3140018083875524, "tokens_per_s": 8681.099364456728, "gpu_gb": 17.2341570854187}
+{"rows": 1600, "of": 26016, "epoch": 0, "ce": 0.09641619171947241, "kl": 0.0, "lr": 9.906964813797955e-05, "rows_per_s": 1.320135726775449, "tokens_per_s": 8674.516150801282, "gpu_gb": 17.2341570854187}
+{"rows": 1800, "of": 26016, "epoch": 0, "ce": 0.09574287496507168, "kl": 0.0, "lr": 9.882349667681238e-05, "rows_per_s": 1.3056610053980582, "tokens_per_s": 8548.200321436798, "gpu_gb": 17.2341570854187}
+{"rows": 2000, "of": 26016, "epoch": 0, "ce": 0.10773482281714677, "kl": 0.0, "lr": 9.854886872554006e-05, "rows_per_s": 1.2984604798793056, "tokens_per_s": 8487.664797273776, "gpu_gb": 17.2341570854187}
+{"rows": 2200, "of": 26016, "epoch": 0, "ce": 0.12273376471828669, "kl": 0.0, "lr": 9.824592446195956e-05, "rows_per_s": 1.2950568208589086, "tokens_per_s": 8460.832256952708, "gpu_gb": 17.2341570854187}
+{"rows": 2400, "of": 26016, "epoch": 0, "ce": 0.09668787938077003, "kl": 0.0, "lr": 9.791484057946465e-05, "rows_per_s": 1.2955942760768149, "tokens_per_s": 8428.736885253968, "gpu_gb": 17.2341570854187}
+{"rows": 2600, "of": 26016, "epoch": 0, "ce": 0.1276158706843853, "kl": 0.0, "lr": 9.755581018398885e-05, "rows_per_s": 1.3016420591646591, "tokens_per_s": 8493.851239395268, "gpu_gb": 17.2341570854187}
+{"rows": 2800, "of": 26016, "epoch": 0, "ce": 0.09540158527903259, "kl": 0.0, "lr": 9.716904268137568e-05, "rows_per_s": 1.304476296873418, "tokens_per_s": 8522.667301530357, "gpu_gb": 17.2341570854187}
+{"rows": 3000, "of": 26016, "epoch": 0, "ce": 0.0937056295759976, "kl": 0.0, "lr": 9.675476365524183e-05, "rows_per_s": 1.3048507867340704, "tokens_per_s": 8539.020099633912, "gpu_gb": 17.2341570854187}
+{"rows": 3200, "of": 26016, "epoch": 0, "ce": 0.10784881011117249, "kl": 0.0, "lr": 9.631321473540476e-05, "rows_per_s": 1.3045918588695378, "tokens_per_s": 8530.444047158428, "gpu_gb": 17.2341570854187}
+{"rows": 3400, "of": 26016, "epoch": 0, "ce": 0.11549179875757545, "kl": 0.0, "lr": 9.584465345695108e-05, "rows_per_s": 1.2951517585161703, "tokens_per_s": 8463.743603921515, "gpu_gb": 17.2341570854187}
+{"rows": 3600, "of": 26016, "epoch": 0, "ce": 0.11224200371652841, "kl": 0.0, "lr": 9.534935311002834e-05, "rows_per_s": 1.296103516839737, "tokens_per_s": 8467.2440995264, "gpu_gb": 17.2341570854187}
+{"rows": 3800, "of": 26016, "epoch": 0, "ce": 0.09777469113003462, "kl": 0.0, "lr": 9.482760258044745e-05, "rows_per_s": 1.299644373045316, "tokens_per_s": 8492.374302479977, "gpu_gb": 17.2341570854187}
+{"rows": 4000, "of": 26016, "epoch": 0, "ce": 0.11347274864092469, "kl": 0.0, "lr": 9.427970618118888e-05, "rows_per_s": 1.3004826201184663, "tokens_per_s": 8502.19643713138, "gpu_gb": 17.2341570854187}
+{"rows": 4200, "of": 26016, "epoch": 0, "ce": 0.10587982336059212, "kl": 0.0, "lr": 9.37059834749111e-05, "rows_per_s": 1.302841578740315, "tokens_per_s": 8524.978843887277, "gpu_gb": 17.2341570854187}
+{"rows": 4400, "of": 26016, "epoch": 0, "ce": 0.10273228042759001, "kl": 0.0, "lr": 9.31067690875643e-05, "rows_per_s": 1.3006484219242838, "tokens_per_s": 8532.24182374674, "gpu_gb": 17.2341570854187}
+{"rows": 4600, "of": 26016, "epoch": 0, "ce": 0.10445060102734714, "kl": 0.0, "lr": 9.248241251321865e-05, "rows_per_s": 1.3012208068272395, "tokens_per_s": 8527.791566265665, "gpu_gb": 17.2341570854187}
+{"rows": 4800, "of": 26016, "epoch": 0, "ce": 0.12074538596905768, "kl": 0.0, "lr": 9.183327791022047e-05, "rows_per_s": 1.3046135046568805, "tokens_per_s": 8557.416591771109, "gpu_gb": 17.2341570854187}
+{"rows": 5000, "of": 26016, "epoch": 0, "ce": 0.10229002986103296, "kl": 0.0, "lr": 9.11597438887955e-05, "rows_per_s": 1.3050383527278648, "tokens_per_s": 8550.795036473033, "gpu_gb": 17.2341570854187}
+{"rows": 5200, "of": 26016, "epoch": 0, "ce": 0.0994475709926337, "kl": 0.0, "lr": 9.04622032902231e-05, "rows_per_s": 1.3015868942956539, "tokens_per_s": 8525.11581828529, "gpu_gb": 17.673460960388184}
+{"rows": 5400, "of": 26016, "epoch": 0, "ce": 0.10379408159293234, "kl": 0.0, "lr": 8.974106295771002e-05, "rows_per_s": 1.3044628067835886, "tokens_per_s": 8534.53681010624, "gpu_gb": 17.673460960388184}
+{"rows": 5600, "of": 26016, "epoch": 0, "ce": 0.09124722224194556, "kl": 0.0, "lr": 8.899674349909759e-05, "rows_per_s": 1.3060469557566272, "tokens_per_s": 8532.387037035074, "gpu_gb": 17.673460960388184}
+{"rows": 5800, "of": 26016, "epoch": 0, "ce": 0.10192924229893834, "kl": 0.0, "lr": 8.822967904154058e-05, "rows_per_s": 1.3072147954395692, "tokens_per_s": 8527.778894517598, "gpu_gb": 17.673460960388184}
+{"rows": 6000, "of": 26016, "epoch": 0, "ce": 0.11276082823053002, "kl": 0.0, "lr": 8.744031697830088e-05, "rows_per_s": 1.3068346311044512, "tokens_per_s": 8515.141915974287, "gpu_gb": 17.673460960388184}
+{"rows": 6200, "of": 26016, "epoch": 0, "ce": 0.09716706731356681, "kl": 0.0, "lr": 8.662911770780372e-05, "rows_per_s": 1.3081142869112217, "tokens_per_s": 8509.345044320688, "gpu_gb": 17.673460960388184}
+{"rows": 6400, "of": 26016, "epoch": 0, "ce": 0.10494738984387368, "kl": 0.0, "lr": 8.579655436510847e-05, "rows_per_s": 1.3055376694958754, "tokens_per_s": 8513.906431135967, "gpu_gb": 23.384910106658936}
+{"rows": 6600, "of": 26016, "epoch": 0, "ce": 0.09862258568871766, "kl": 0.0, "lr": 8.494311254595087e-05, "rows_per_s": 1.3074388581891607, "tokens_per_s": 8529.023509068289, "gpu_gb": 23.384910106658936}
+{"rows": 6800, "of": 26016, "epoch": 0, "ce": 0.12068125354126095, "kl": 0.0, "lr": 8.406929002351746e-05, "rows_per_s": 1.3074842636014143, "tokens_per_s": 8541.703170209588, "gpu_gb": 23.384910106658936}
+{"rows": 7000, "of": 26016, "epoch": 0, "ce": 0.0868050510622561, "kl": 0.0, "lr": 8.317559645811748e-05, "rows_per_s": 1.3094364864977739, "tokens_per_s": 8545.675824657448, "gpu_gb": 23.384910106658936}
+{"rows": 7200, "of": 26016, "epoch": 0, "ce": 0.10754081818275153, "kl": 0.0, "lr": 8.22625530999215e-05, "rows_per_s": 1.310537050108543, "tokens_per_s": 8553.730438906809, "gpu_gb": 23.384910106658936}
+{"rows": 7400, "of": 26016, "epoch": 0, "ce": 0.10953794126398861, "kl": 0.0, "lr": 8.133069248494038e-05, "rows_per_s": 1.3108894748692441, "tokens_per_s": 8560.485240193802, "gpu_gb": 23.384910106658936}
+{"rows": 7600, "of": 26016, "epoch": 0, "ce": 0.09635531350504607, "kl": 0.0, "lr": 8.038055812442152e-05, "rows_per_s": 1.3117631021525535, "tokens_per_s": 8556.39183637619, "gpu_gb": 23.384910106658936}
+{"rows": 7800, "of": 26016, "epoch": 0, "ce": 0.10271831860765815, "kl": 0.0, "lr": 7.94127041878439e-05, "rows_per_s": 1.3101083897026946, "tokens_per_s": 8567.50621879636, "gpu_gb": 23.384910106658936}
+{"rows": 8000, "of": 26016, "epoch": 0, "ce": 0.10019719297997653, "kl": 0.0, "lr": 7.842769517969666e-05, "rows_per_s": 1.3098656988372233, "tokens_per_s": 8571.389786265168, "gpu_gb": 23.384910106658936}
+{"rows": 8200, "of": 26016, "epoch": 0, "ce": 0.09967302082572133, "kl": 0.0, "lr": 7.74261056102296e-05, "rows_per_s": 1.3105294596546293, "tokens_per_s": 8577.603901825207, "gpu_gb": 23.384910106658936}
+{"rows": 8400, "of": 26016, "epoch": 0, "ce": 0.10891576926223934, "kl": 0.0, "lr": 7.640851966036806e-05, "rows_per_s": 1.3102121879309558, "tokens_per_s": 8575.980150069627, "gpu_gb": 23.384910106658936}
+{"rows": 8600, "of": 26016, "epoch": 0, "ce": 0.08241698800120503, "kl": 0.0, "lr": 7.537553084098697e-05, "rows_per_s": 1.3106354642983018, "tokens_per_s": 8579.82330310011, "gpu_gb": 23.384910106658936}
+{"rows": 8800, "of": 26016, "epoch": 0, "ce": 0.10798336253035813, "kl": 0.0, "lr": 7.432774164674359e-05, "rows_per_s": 1.310737179035337, "tokens_per_s": 8597.908024862436, "gpu_gb": 23.384910106658936}
+{"rows": 9000, "of": 26016, "epoch": 0, "ce": 0.08983938575722278, "kl": 0.0, "lr": 7.326576320466999e-05, "rows_per_s": 1.3112016656067162, "tokens_per_s": 8592.485751928813, "gpu_gb": 23.384910106658936}
+{"rows": 9200, "of": 26016, "epoch": 0, "ce": 0.09661569789517671, "kl": 0.0, "lr": 7.219021491773108e-05, "rows_per_s": 1.311911174304482, "tokens_per_s": 8602.130270394146, "gpu_gb": 23.384910106658936}
+{"rows": 9400, "of": 26016, "epoch": 0, "ce": 0.09280427047982812, "kl": 0.0, "lr": 7.110172410355538e-05, "rows_per_s": 1.312068598149493, "tokens_per_s": 8603.260597765251, "gpu_gb": 23.384910106658936}
+{"rows": 9600, "of": 26016, "epoch": 0, "ce": 0.09468722312478349, "kl": 0.0, "lr": 7.000092562854959e-05, "rows_per_s": 1.3108765215583538, "tokens_per_s": 8603.19248822662, "gpu_gb": 23.384910106658936}
+{"rows": 9800, "of": 26016, "epoch": 0, "ce": 0.08775657075922937, "kl": 0.0, "lr": 6.888846153761047e-05, "rows_per_s": 1.3109299217666923, "tokens_per_s": 8607.622584104472, "gpu_gb": 23.384910106658936}
+{"rows": 10000, "of": 26016, "epoch": 0, "ce": 0.11158812100999058, "kl": 0.0, "lr": 6.776498067964953e-05, "rows_per_s": 1.3094506603913174, "tokens_per_s": 8592.528809744239, "gpu_gb": 23.384910106658936}
+{"rows": 10200, "of": 26016, "epoch": 0, "ce": 0.10190904300659895, "kl": 0.0, "lr": 6.663113832914968e-05, "rows_per_s": 1.3097147425336169, "tokens_per_s": 8597.29217380136, "gpu_gb": 23.384910106658936}
+{"rows": 10400, "of": 26016, "epoch": 0, "ce": 0.09363205465022474, "kl": 0.0, "lr": 6.548759580397363e-05, "rows_per_s": 1.3100739908236592, "tokens_per_s": 8592.343485270021, "gpu_gb": 23.384910106658936}
+{"rows": 10600, "of": 26016, "epoch": 0, "ce": 0.11129171031992882, "kl": 0.0, "lr": 6.433502007964793e-05, "rows_per_s": 1.3082054165289165, "tokens_per_s": 8576.425878215481, "gpu_gb": 23.384910106658936}
+{"rows": 10800, "of": 26016, "epoch": 0, "ce": 0.11577033136971294, "kl": 0.0, "lr": 6.317408340034685e-05, "rows_per_s": 1.3087308791738759, "tokens_per_s": 8571.718054332945, "gpu_gb": 23.384910106658936}
+{"rows": 11000, "of": 26016, "epoch": 0, "ce": 0.08696674837265164, "kl": 0.0, "lr": 6.200546288680347e-05, "rows_per_s": 1.30658349040858, "tokens_per_s": 8557.450040701495, "gpu_gb": 23.384910106658936}
+{"rows": 11200, "of": 26016, "epoch": 0, "ce": 0.09200721811503172, "kl": 0.0, "lr": 6.082984014137638e-05, "rows_per_s": 1.3071103825478951, "tokens_per_s": 8556.87301021318, "gpu_gb": 23.384910106658936}
+{"rows": 11400, "of": 26016, "epoch": 0, "ce": 0.10855140430852771, "kl": 0.0, "lr": 5.96479008505026e-05, "rows_per_s": 1.3072240331470955, "tokens_per_s": 8560.427675704153, "gpu_gb": 23.384910106658936}
+{"rows": 11600, "of": 26016, "epoch": 0, "ce": 0.09824764555087313, "kl": 0.0, "lr": 5.846033438476825e-05, "rows_per_s": 1.30805507856246, "tokens_per_s": 8560.740117240035, "gpu_gb": 23.384910106658936}
+{"rows": 11800, "of": 26016, "epoch": 0, "ce": 0.10935585226397962, "kl": 0.0, "lr": 5.726783339683062e-05, "rows_per_s": 1.3088283633645923, "tokens_per_s": 8567.103759901702, "gpu_gb": 23.384910106658936}
+{"rows": 12000, "of": 26016, "epoch": 0, "ce": 0.10163947968278081, "kl": 0.0, "lr": 5.607109341742579e-05, "rows_per_s": 1.3085864754306364, "tokens_per_s": 8581.242504306892, "gpu_gb": 23.385355949401855}
+{"rows": 12200, "of": 26016, "epoch": 0, "ce": 0.09023875411134213, "kl": 0.0, "lr": 5.487081244969783e-05, "rows_per_s": 1.307749839235083, "tokens_per_s": 8578.820079482824, "gpu_gb": 23.385355949401855}
+{"rows": 12400, "of": 26016, "epoch": 0, "ce": 0.08826861077919602, "kl": 0.0, "lr": 5.366769056208558e-05, "rows_per_s": 1.3079064485241472, "tokens_per_s": 8580.679735748326, "gpu_gb": 23.385355949401855}
+{"rows": 12600, "of": 26016, "epoch": 0, "ce": 0.11737227831268683, "kl": 0.0, "lr": 5.2462429480005345e-05, "rows_per_s": 1.3081410538990117, "tokens_per_s": 8576.76037462899, "gpu_gb": 23.385355949401855}
+{"rows": 12800, "of": 26016, "epoch": 0, "ce": 0.07100480829365552, "kl": 0.0, "lr": 5.125573217656664e-05, "rows_per_s": 1.3084947086653822, "tokens_per_s": 8579.956006274759, "gpu_gb": 23.385355949401855}
+{"rows": 13000, "of": 26016, "epoch": 0, "ce": 0.09240765829570591, "kl": 0.0, "lr": 5.004830246256059e-05, "rows_per_s": 1.3076357099842366, "tokens_per_s": 8569.89399686641, "gpu_gb": 23.385355949401855}
+{"rows": 13200, "of": 26016, "epoch": 0, "ce": 0.08983383147511631, "kl": 0.0, "lr": 4.8840844575959563e-05, "rows_per_s": 1.3077877287897872, "tokens_per_s": 8561.54729332275, "gpu_gb": 23.385355949401855}
+{"rows": 13400, "of": 26016, "epoch": 0, "ce": 0.08263354171183891, "kl": 0.0, "lr": 4.763406277116772e-05, "rows_per_s": 1.3079254334758708, "tokens_per_s": 8562.833594893025, "gpu_gb": 23.385355949401855}
+{"rows": 13600, "of": 26016, "epoch": 0, "ce": 0.10486351697938517, "kl": 0.0, "lr": 4.6428660908261864e-05, "rows_per_s": 1.3086644731643267, "tokens_per_s": 8566.621179787588, "gpu_gb": 23.385355949401855}
+{"rows": 13800, "of": 26016, "epoch": 0, "ce": 0.07979557300452143, "kl": 0.0, "lr": 4.5225342042462565e-05, "rows_per_s": 1.3074161829611266, "tokens_per_s": 8569.698874704975, "gpu_gb": 23.385355949401855}
+{"rows": 14000, "of": 26016, "epoch": 0, "ce": 0.08564892544585746, "kl": 0.0, "lr": 4.4024808014074267e-05, "rows_per_s": 1.3071645163378502, "tokens_per_s": 8566.744296517836, "gpu_gb": 23.385355949401855}
+{"rows": 14200, "of": 26016, "epoch": 0, "ce": 0.08424756520777009, "kl": 0.0, "lr": 4.2827759039134546e-05, "rows_per_s": 1.306448586529592, "tokens_per_s": 8565.35551964857, "gpu_gb": 23.385355949401855}
+{"rows": 14400, "of": 26016, "epoch": 0, "ce": 0.08440441401675343, "kl": 0.0, "lr": 4.163489330101017e-05, "rows_per_s": 1.3076476867261468, "tokens_per_s": 8570.360715291894, "gpu_gb": 23.385355949401855}
+{"rows": 14600, "of": 26016, "epoch": 0, "ce": 0.0684824161347933, "kl": 0.0, "lr": 4.044690654317925e-05, "rows_per_s": 1.3077759017477195, "tokens_per_s": 8568.825028401183, "gpu_gb": 23.385355949401855}
+{"rows": 14800, "of": 26016, "epoch": 0, "ce": 0.06843636301346123, "kl": 0.0, "lr": 3.926449166343597e-05, "rows_per_s": 1.3072240477254713, "tokens_per_s": 8571.885332719798, "gpu_gb": 23.385355949401855}
+{"rows": 15000, "of": 26016, "epoch": 0, "ce": 0.08945686204358935, "kl": 0.0, "lr": 3.8088338309755514e-05, "rows_per_s": 1.30739116267601, "tokens_per_s": 8572.48680474741, "gpu_gb": 23.385355949401855}
+{"rows": 15200, "of": 26016, "epoch": 0, "ce": 0.07946186201646924, "kl": 0.0, "lr": 3.691913247805415e-05, "rows_per_s": 1.3059071540104414, "tokens_per_s": 8567.96989153883, "gpu_gb": 23.385355949401855}
+{"rows": 15400, "of": 26016, "epoch": 0, "ce": 0.09912534289062024, "kl": 0.0, "lr": 3.5757556112079526e-05, "rows_per_s": 1.3068355072183697, "tokens_per_s": 8569.287692524049, "gpu_gb": 23.385355949401855}
+{"rows": 15600, "of": 26016, "epoch": 0, "ce": 0.07420192262565252, "kl": 0.0, "lr": 3.46042867056644e-05, "rows_per_s": 1.3063950101574686, "tokens_per_s": 8561.820130100541, "gpu_gb": 23.385355949401855}
+{"rows": 15800, "of": 26016, "epoch": 0, "ce": 0.0816245180554688, "kl": 0.0, "lr": 3.3459996907575756e-05, "rows_per_s": 1.307095572525077, "tokens_per_s": 8564.78775003668, "gpu_gb": 23.385355949401855}
+{"rows": 16000, "of": 26016, "epoch": 0, "ce": 0.08653278279001825, "kl": 0.0, "lr": 3.232535412918992e-05, "rows_per_s": 1.3068080712129095, "tokens_per_s": 8569.930371691888, "gpu_gb": 23.385355949401855}
+{"rows": 16200, "of": 26016, "epoch": 0, "ce": 0.08726967943599448, "kl": 0.0, "lr": 3.1201020155222247e-05, "rows_per_s": 1.3071227538290526, "tokens_per_s": 8574.128829847208, "gpu_gb": 23.385355949401855}
+{"rows": 16400, "of": 26016, "epoch": 0, "ce": 0.07609634039690719, "kl": 0.0, "lr": 3.0087650757738707e-05, "rows_per_s": 1.3073424312057687, "tokens_per_s": 8570.20135971946, "gpu_gb": 23.385355949401855}
+{"rows": 16600, "of": 26016, "epoch": 0, "ce": 0.07342546508647502, "kl": 0.0, "lr": 2.8985895313674392e-05, "rows_per_s": 1.3067347464949945, "tokens_per_s": 8568.88224230836, "gpu_gb": 23.385355949401855}
+{"rows": 16800, "of": 26016, "epoch": 0, "ce": 0.07961538979783654, "kl": 0.0, "lr": 2.7896396426081844e-05, "rows_per_s": 1.3071731590473403, "tokens_per_s": 8570.8866634366, "gpu_gb": 23.385355949401855}
+{"rows": 17000, "of": 26016, "epoch": 0, "ce": 0.06130263954284601, "kl": 0.0, "lr": 2.6819789549330455e-05, "rows_per_s": 1.3073216039606868, "tokens_per_s": 8572.306777659109, "gpu_gb": 23.385355949401855}
+{"rows": 17200, "of": 26016, "epoch": 0, "ce": 0.07921179708093405, "kl": 0.0, "lr": 2.5756702618475244e-05, "rows_per_s": 1.3072297441317502, "tokens_per_s": 8562.71324821839, "gpu_gb": 23.385355949401855}
+{"rows": 17400, "of": 26016, "epoch": 0, "ce": 0.06909668329637497, "kl": 0.0, "lr": 2.4707755683011425e-05, "rows_per_s": 1.3076917321531434, "tokens_per_s": 8566.15974888539, "gpu_gb": 23.385355949401855}
+{"rows": 17600, "of": 26016, "epoch": 0, "ce": 0.09042671537259593, "kl": 0.0, "lr": 2.3673560545228078e-05, "rows_per_s": 1.3075969850227251, "tokens_per_s": 8563.829777771552, "gpu_gb": 23.385355949401855}
+{"rows": 17800, "of": 26016, "epoch": 0, "ce": 0.05831700397888198, "kl": 0.0, "lr": 2.265472040337221e-05, "rows_per_s": 1.307557964849351, "tokens_per_s": 8560.672496509864, "gpu_gb": 23.385355949401855}
+{"rows": 18000, "of": 26016, "epoch": 0, "ce": 0.07406770232249983, "kl": 0.0, "lr": 2.1651829499831043e-05, "rows_per_s": 1.307657083576258, "tokens_per_s": 8561.44857843056, "gpu_gb": 23.385355949401855}
+{"rows": 18200, "of": 26016, "epoch": 0, "ce": 0.08319879051763564, "kl": 0.0, "lr": 2.06654727745379e-05, "rows_per_s": 1.3077549111451154, "tokens_per_s": 8556.875492087835, "gpu_gb": 23.385355949401855}
+{"rows": 18400, "of": 26016, "epoch": 0, "ce": 0.07702640515984967, "kl": 0.0, "lr": 1.96962255238038e-05, "rows_per_s": 1.3076082453946543, "tokens_per_s": 8554.655414195235, "gpu_gb": 23.385355949401855}
+{"rows": 18600, "of": 26016, "epoch": 0, "ce": 0.05874788155313581, "kl": 0.0, "lr": 1.874465306477363e-05, "rows_per_s": 1.3071526929816477, "tokens_per_s": 8553.2549775802, "gpu_gb": 23.385355949401855}
+{"rows": 18800, "of": 26016, "epoch": 0, "ce": 0.07376383375376463, "kl": 0.0, "lr": 1.7811310405702796e-05, "rows_per_s": 1.305511265424238, "tokens_per_s": 8550.396312360619, "gpu_gb": 23.846881866455078}
+{"rows": 19000, "of": 26016, "epoch": 0, "ce": 0.06260637430474163, "kl": 0.0, "lr": 1.6896741922246555e-05, "rows_per_s": 1.305902020122018, "tokens_per_s": 8548.632296045565, "gpu_gb": 23.846881866455078}
+{"rows": 19200, "of": 26016, "epoch": 0, "ce": 0.09029084144392982, "kl": 0.0, "lr": 1.6001481039950872e-05, "rows_per_s": 1.305595289461014, "tokens_per_s": 8546.461036688146, "gpu_gb": 23.846881866455078}
+{"rows": 19400, "of": 26016, "epoch": 0, "ce": 0.0628396339341998, "kl": 0.0, "lr": 1.5126049923129892e-05, "rows_per_s": 1.3053009512169194, "tokens_per_s": 8544.559774462072, "gpu_gb": 23.846881866455078}
+{"rows": 19600, "of": 26016, "epoch": 0, "ce": 0.07810155213694088, "kl": 0.0, "lr": 1.4270959170311587e-05, "rows_per_s": 1.3052513909112773, "tokens_per_s": 8544.451838719991, "gpu_gb": 23.846881866455078}
+{"rows": 19800, "of": 26016, "epoch": 0, "ce": 0.07273113187635318, "kl": 0.0, "lr": 1.3436707516429365e-05, "rows_per_s": 1.3052968748951663, "tokens_per_s": 8544.164290955197, "gpu_gb": 23.846881866455078}
+{"rows": 20000, "of": 26016, "epoch": 0, "ce": 0.07944329025456681, "kl": 0.0, "lr": 1.262378154193285e-05, "rows_per_s": 1.3054900069657516, "tokens_per_s": 8545.325834049614, "gpu_gb": 23.846881866455078}
+{"rows": 20200, "of": 26016, "epoch": 0, "ce": 0.07112028512172401, "kl": 0.0, "lr": 1.1832655388988156e-05, "rows_per_s": 1.3050242073713345, "tokens_per_s": 8543.314873072293, "gpu_gb": 23.846881866455078}
+{"rows": 20400, "of": 26016, "epoch": 0, "ce": 0.06595994144910947, "kl": 0.0, "lr": 1.1063790484932463e-05, "rows_per_s": 1.3057887345306485, "tokens_per_s": 8545.364143624038, "gpu_gb": 23.846881866455078}
+{"rows": 20600, "of": 26016, "epoch": 0, "ce": 0.09845127704553307, "kl": 0.0, "lr": 1.0317635273144971e-05, "rows_per_s": 1.305842724688772, "tokens_per_s": 8544.167489016141, "gpu_gb": 23.846881866455078}
+{"rows": 20800, "of": 26016, "epoch": 0, "ce": 0.06711255298694596, "kl": 0.0, "lr": 9.594624951490455e-06, "rows_per_s": 1.3060593490916153, "tokens_per_s": 8545.957729801403, "gpu_gb": 23.846881866455078}
+{"rows": 21000, "of": 26016, "epoch": 0, "ce": 0.06078801407827996, "kl": 0.0, "lr": 8.895181218488551e-06, "rows_per_s": 1.3066171226829946, "tokens_per_s": 8552.382237338017, "gpu_gb": 23.846881866455078}
+{"rows": 21200, "of": 26016, "epoch": 0, "ce": 0.06715412786579691, "kl": 0.0, "lr": 8.219712027356374e-06, "rows_per_s": 1.3055530747177304, "tokens_per_s": 8549.018702574342, "gpu_gb": 23.846881866455078}
+{"rows": 21400, "of": 26016, "epoch": 0, "ce": 0.07682103851111606, "kl": 0.0, "lr": 7.5686113480683606e-06, "rows_per_s": 1.3052135492208126, "tokens_per_s": 8545.075749227304, "gpu_gb": 23.846881866455078}
+{"rows": 21600, "of": 26016, "epoch": 0, "ce": 0.05676541410852223, "kl": 0.0, "lr": 6.942258937571772e-06, "rows_per_s": 1.30511493984266, "tokens_per_s": 8545.787491830695, "gpu_gb": 23.846881866455078}
+{"rows": 21800, "of": 26016, "epoch": 0, "ce": 0.06049736668355763, "kl": 0.0, "lr": 6.341020118291935e-06, "rows_per_s": 1.3051961130501009, "tokens_per_s": 8546.921320202646, "gpu_gb": 23.846881866455078}
+{"rows": 22000, "of": 26016, "epoch": 0, "ce": 0.06279707045527175, "kl": 0.0, "lr": 5.765245565056615e-06, "rows_per_s": 1.3051496288922062, "tokens_per_s": 8542.343616159882, "gpu_gb": 23.846881866455078}
+{"rows": 22200, "of": 26016, "epoch": 0, "ce": 0.07164741162559948, "kl": 0.0, "lr": 5.215271100563562e-06, "rows_per_s": 1.3051304775797257, "tokens_per_s": 8544.747085741037, "gpu_gb": 23.846881866455078}
+{"rows": 22400, "of": 26016, "epoch": 0, "ce": 0.0739423557522241, "kl": 0.0, "lr": 4.691417499510686e-06, "rows_per_s": 1.3048593476990187, "tokens_per_s": 8540.07281815586, "gpu_gb": 23.846881866455078}
+{"rows": 22600, "of": 26016, "epoch": 0, "ce": 0.07274793753866106, "kl": 0.0, "lr": 4.193990301502926e-06, "rows_per_s": 1.3058473511544784, "tokens_per_s": 8542.883879544695, "gpu_gb": 23.846881866455078}
+{"rows": 22800, "of": 26016, "epoch": 0, "ce": 0.05820969991385937, "kl": 0.0, "lr": 3.7232796328451557e-06, "rows_per_s": 1.3065139659667628, "tokens_per_s": 8547.938678360255, "gpu_gb": 23.846881866455078}
+{"rows": 23000, "of": 26016, "epoch": 0, "ce": 0.07114249044330791, "kl": 0.0, "lr": 3.2795600373249135e-06, "rows_per_s": 1.3066047018004479, "tokens_per_s": 8550.407079974912, "gpu_gb": 23.846881866455078}
+{"rows": 23200, "of": 26016, "epoch": 0, "ce": 0.07257074234541505, "kl": 0.0, "lr": 2.8630903160836773e-06, "rows_per_s": 1.3074743831630808, "tokens_per_s": 8553.265225236553, "gpu_gb": 23.846881866455078}
+{"rows": 23400, "of": 26016, "epoch": 0, "ce": 0.06967641732655466, "kl": 0.0, "lr": 2.474113376670151e-06, "rows_per_s": 1.3078010790904342, "tokens_per_s": 8556.394301814193, "gpu_gb": 23.846881866455078}
+{"rows": 23600, "of": 26016, "epoch": 0, "ce": 0.06633248233934864, "kl": 0.0, "lr": 2.112856091363458e-06, "rows_per_s": 1.3073185840938508, "tokens_per_s": 8560.570479177513, "gpu_gb": 23.846881866455078}
+{"rows": 23800, "of": 26016, "epoch": 0, "ce": 0.0742102218058426, "kl": 0.0, "lr": 1.7795291648490586e-06, "rows_per_s": 1.3070669754590991, "tokens_per_s": 8559.571269153883, "gpu_gb": 23.846881866455078}
+{"rows": 24000, "of": 26016, "epoch": 0, "ce": 0.05395677152322605, "kl": 0.0, "lr": 1.4743270113244279e-06, "rows_per_s": 1.3069762102795994, "tokens_per_s": 8557.421508900836, "gpu_gb": 23.846881866455078}
+{"rows": 24200, "of": 26016, "epoch": 0, "ce": 0.078603034846019, "kl": 0.0, "lr": 1.1974276411062691e-06, "rows_per_s": 1.3069928332800842, "tokens_per_s": 8555.00670679286, "gpu_gb": 23.846881866455078}
+{"rows": 24400, "of": 26016, "epoch": 0, "ce": 0.0735311275080312, "kl": 0.0, "lr": 9.489925568052504e-07, "rows_per_s": 1.3067669638762671, "tokens_per_s": 8554.571266122888, "gpu_gb": 23.846881866455078}
+{"rows": 24600, "of": 26016, "epoch": 0, "ce": 0.09146337467245758, "kl": 0.0, "lr": 7.291666591290547e-07, "rows_per_s": 1.3066637797989777, "tokens_per_s": 8555.286763482874, "gpu_gb": 23.846881866455078}
+{"rows": 24800, "of": 26016, "epoch": 0, "ce": 0.06396220365422778, "kl": 0.0, "lr": 5.380781623684661e-07, "rows_per_s": 1.3071194551634064, "tokens_per_s": 8560.109004677892, "gpu_gb": 23.846881866455078}
+{"rows": 25000, "of": 26016, "epoch": 0, "ce": 0.0658585247740848, "kl": 0.0, "lr": 3.758385196159009e-07, "rows_per_s": 1.3070951187219786, "tokens_per_s": 8558.29793673417, "gpu_gb": 23.846881866455078}
+{"rows": 25200, "of": 26016, "epoch": 0, "ce": 0.061776661277981475, "kl": 0.0, "lr": 2.4254235775997837e-07, "rows_per_s": 1.3069872051089269, "tokens_per_s": 8557.248738215436, "gpu_gb": 23.846881866455078}
+{"rows": 25400, "of": 26016, "epoch": 0, "ce": 0.08016695802682079, "kl": 0.0, "lr": 1.3826742229398616e-07, "rows_per_s": 1.3062104007877502, "tokens_per_s": 8549.60887510067, "gpu_gb": 23.846881866455078}
+{"rows": 25600, "of": 26016, "epoch": 0, "ce": 0.07184232336177956, "kl": 0.0, "lr": 6.307453197059165e-08, "rows_per_s": 1.3056618208096638, "tokens_per_s": 8552.999093587523, "gpu_gb": 23.846881866455078}
+{"rows": 25800, "of": 26016, "epoch": 0, "ce": 0.06300472088158131, "kl": 0.0, "lr": 1.7007543329017328e-08, "rows_per_s": 1.305771417453511, "tokens_per_s": 8553.838088976134, "gpu_gb": 23.846881866455078}
+{"rows": 26000, "of": 26016, "epoch": 0, "ce": 0.06399069912498817, "kl": 0.0, "lr": 9.332511557436085e-11, "rows_per_s": 1.3060319093310573, "tokens_per_s": 8554.406934701512, "gpu_gb": 23.846881866455078}
+{"rows": 26016, "of": 26016, "epoch": 0, "ce": 0.07345758628798649, "kl": 0.0, "lr": 0.0, "rows_per_s": 1.3060358806146826, "tokens_per_s": 8554.117745184854, "gpu_gb": 23.846881866455078}

ablations/task_centric/training_meta.json

@@ -0,0 +1,36 @@
+{
+ "base_model": "Qwen/Qwen3.5-4B",
+ "families": [
+  "Discrete selection",
+  "Drag & gesture",
+  "Form transaction",
+  "Navigation",
+  "Out-of-page I/O",
+  "Reasoning base",
+  "Text entry"
+ ],
+ "training_rows": 26019,
+ "rows_trained": 26016,
+ "validation_ce": 0.06781610395206371,
+ "minutes": 334.7,
+ "args": {
+  "adapter": "pooled",
+  "epochs": 1.0,
+  "lr": 0.0001,
+  "rank": 64,
+  "alpha": 128,
+  "grad_accum": 8,
+  "beta": 0.0,
+  "anchor_ratio": 0.5,
+  "limit": 0,
+  "max_steps": 0,
+  "seed": 0,
+  "ce_on": "all",
+  "exclude_variants": "",
+  "frac": 1.0,
+  "episode_ids": null,
+  "init_adapter": null,
+  "drop_done": false
+ },
+ "gpus": 4
+}

adapter_config.json

@@ -0,0 +1,51 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "kasa_config": null,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.21.0",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "k_proj",
+    "v_proj",
+    "o_proj",
+    "gate_proj",
+    "up_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f26988bda0a20270dc1c606d5755b152353a544a7c90e67167c20858d15060ef
+size 339776632

train_log.jsonl

@@ -0,0 +1,139 @@
+{"rows": 200, "of": 27644, "epoch": 0, "ce": 0.36085896074771884, "kl": 0.0, "lr": 2.5239457519507743e-05, "rows_per_s": 0.8489428435116587, "tokens_per_s": 5514.206366888759, "gpu_gb": 15.715003967285156}
+{"rows": 400, "of": 27644, "epoch": 0, "ce": 0.2631891492754221, "kl": 0.0, "lr": 4.948898067637296e-05, "rows_per_s": 1.062190341226625, "tokens_per_s": 6748.328919687819, "gpu_gb": 15.993857860565186}
+{"rows": 600, "of": 27644, "epoch": 0, "ce": 0.22552833627909422, "kl": 0.0, "lr": 7.370064985053864e-05, "rows_per_s": 1.1429982542525752, "tokens_per_s": 7415.1173545916035, "gpu_gb": 15.993857860565186}
+{"rows": 800, "of": 27644, "epoch": 0, "ce": 0.2049107265472412, "kl": 0.0, "lr": 9.78557041525851e-05, "rows_per_s": 1.2064230405647018, "tokens_per_s": 7787.563272803598, "gpu_gb": 15.993857860565186}
+{"rows": 1000, "of": 27644, "epoch": 0, "ce": 0.23842138186097145, "kl": 0.0, "lr": 9.96773764410274e-05, "rows_per_s": 1.249190184561534, "tokens_per_s": 7966.86030486333, "gpu_gb": 15.993857860565186}
+{"rows": 1200, "of": 27644, "epoch": 0, "ce": 0.22905686378479004, "kl": 0.0, "lr": 9.953564214713239e-05, "rows_per_s": 1.2771554248411945, "tokens_per_s": 8107.3570937834065, "gpu_gb": 15.993857860565186}
+{"rows": 1400, "of": 27644, "epoch": 0, "ce": 0.21381109148263933, "kl": 0.0, "lr": 9.936831122537051e-05, "rows_per_s": 1.295142654664457, "tokens_per_s": 8187.470016324035, "gpu_gb": 15.993857860565186}
+{"rows": 1600, "of": 27644, "epoch": 0, "ce": 0.23067699164152144, "kl": 0.0, "lr": 9.917547014090404e-05, "rows_per_s": 1.311033297524685, "tokens_per_s": 8243.289014931892, "gpu_gb": 15.993857860565186}
+{"rows": 1800, "of": 27644, "epoch": 0, "ce": 0.20589447677135467, "kl": 0.0, "lr": 9.895721854080047e-05, "rows_per_s": 1.321934010968016, "tokens_per_s": 8300.335525934108, "gpu_gb": 15.993857860565186}
+{"rows": 2000, "of": 27644, "epoch": 0, "ce": 0.21627589035779238, "kl": 0.0, "lr": 9.871366920254174e-05, "rows_per_s": 1.326208613010953, "tokens_per_s": 8331.836648393435, "gpu_gb": 15.993857860565186}
+{"rows": 2200, "of": 27644, "epoch": 0, "ce": 0.2108935915026814, "kl": 0.0, "lr": 9.844494797574866e-05, "rows_per_s": 1.3296080092868248, "tokens_per_s": 8361.317325411646, "gpu_gb": 15.993857860565186}
+{"rows": 2400, "of": 27644, "epoch": 0, "ce": 0.20943921141326427, "kl": 0.0, "lr": 9.815119371715029e-05, "rows_per_s": 1.3243657112627003, "tokens_per_s": 8312.304132073848, "gpu_gb": 15.993857860565186}
+{"rows": 2600, "of": 27644, "epoch": 0, "ce": 0.20467571575194599, "kl": 0.0, "lr": 9.783255821883243e-05, "rows_per_s": 1.330490790751763, "tokens_per_s": 8331.359344430286, "gpu_gb": 15.993857860565186}
+{"rows": 2800, "of": 27644, "epoch": 0, "ce": 0.22089658852666616, "kl": 0.0, "lr": 9.748920612980175e-05, "rows_per_s": 1.336821439197119, "tokens_per_s": 8378.76326879226, "gpu_gb": 15.993857860565186}
+{"rows": 3000, "of": 27644, "epoch": 0, "ce": 0.21258714515715837, "kl": 0.0, "lr": 9.712131487090656e-05, "rows_per_s": 1.3408848988054898, "tokens_per_s": 8358.13962757079, "gpu_gb": 15.993857860565186}
+{"rows": 3200, "of": 27644, "epoch": 0, "ce": 0.2240411265939474, "kl": 0.0, "lr": 9.672907454315778e-05, "rows_per_s": 1.3441680074704951, "tokens_per_s": 8405.793279546955, "gpu_gb": 15.993857860565186}
+{"rows": 3400, "of": 27644, "epoch": 0, "ce": 0.1996938092261553, "kl": 0.0, "lr": 9.631268782949782e-05, "rows_per_s": 1.3456492743578568, "tokens_per_s": 8434.763952960464, "gpu_gb": 16.013582706451416}
+{"rows": 3600, "of": 27644, "epoch": 0, "ce": 0.2080060131382197, "kl": 0.0, "lr": 9.587236989006785e-05, "rows_per_s": 1.3506900085815385, "tokens_per_s": 8432.206146140248, "gpu_gb": 16.013582706451416}
+{"rows": 3800, "of": 27644, "epoch": 0, "ce": 0.19808816716074942, "kl": 0.0, "lr": 9.540834825102782e-05, "rows_per_s": 1.3512586139565306, "tokens_per_s": 8429.909361062142, "gpu_gb": 16.013582706451416}
+{"rows": 4000, "of": 27644, "epoch": 0, "ce": 0.19630078106187285, "kl": 0.0, "lr": 9.492086268698655e-05, "rows_per_s": 1.351504155708512, "tokens_per_s": 8454.586977311714, "gpu_gb": 16.97444248199463}
+{"rows": 4200, "of": 27644, "epoch": 0, "ce": 0.21574503377079965, "kl": 0.0, "lr": 9.441016509710252e-05, "rows_per_s": 1.3542319030073284, "tokens_per_s": 8454.908283662391, "gpu_gb": 16.97444248199463}
+{"rows": 4400, "of": 27644, "epoch": 0, "ce": 0.21556146416813135, "kl": 0.0, "lr": 9.387651937491975e-05, "rows_per_s": 1.3587678332678104, "tokens_per_s": 8456.888232945372, "gpu_gb": 16.97444248199463}
+{"rows": 4600, "of": 27644, "epoch": 0, "ce": 0.19279830690473318, "kl": 0.0, "lr": 9.332020127200561e-05, "rows_per_s": 1.3585312932222424, "tokens_per_s": 8462.561950408433, "gpu_gb": 16.97444248199463}
+{"rows": 4800, "of": 27644, "epoch": 0, "ce": 0.20191135670989752, "kl": 0.0, "lr": 9.274149825546134e-05, "rows_per_s": 1.3597932214188169, "tokens_per_s": 8477.707893884828, "gpu_gb": 16.97444248199463}
+{"rows": 5000, "of": 27644, "epoch": 0, "ce": 0.22213074111379683, "kl": 0.0, "lr": 9.214070935937871e-05, "rows_per_s": 1.3602687364940376, "tokens_per_s": 8478.214423275718, "gpu_gb": 16.97444248199463}
+{"rows": 5200, "of": 27644, "epoch": 0, "ce": 0.21097509548068047, "kl": 0.0, "lr": 9.151814503031972e-05, "rows_per_s": 1.3620004759472215, "tokens_per_s": 8477.15277616326, "gpu_gb": 16.97444248199463}
+{"rows": 5400, "of": 27644, "epoch": 0, "ce": 0.22293307147920133, "kl": 0.0, "lr": 9.087412696689899e-05, "rows_per_s": 1.361504985480816, "tokens_per_s": 8479.771742593346, "gpu_gb": 16.97444248199463}
+{"rows": 5600, "of": 27644, "epoch": 0, "ce": 0.20907054984942078, "kl": 0.0, "lr": 9.020898795355199e-05, "rows_per_s": 1.363832628852235, "tokens_per_s": 8481.185113251826, "gpu_gb": 16.97444248199463}
+{"rows": 5800, "of": 27644, "epoch": 0, "ce": 0.19664805002510546, "kl": 0.0, "lr": 8.95230716885749e-05, "rows_per_s": 1.364152364223438, "tokens_per_s": 8491.163568724589, "gpu_gb": 16.97444248199463}
+{"rows": 6000, "of": 27644, "epoch": 0, "ce": 0.1967082131654024, "kl": 0.0, "lr": 8.881673260652477e-05, "rows_per_s": 1.3650491455350569, "tokens_per_s": 8494.323169067728, "gpu_gb": 16.97444248199463}
+{"rows": 6200, "of": 27644, "epoch": 0, "ce": 0.21727254492230713, "kl": 0.0, "lr": 8.809033569507202e-05, "rows_per_s": 1.368689144193678, "tokens_per_s": 8505.452895990062, "gpu_gb": 16.97444248199463}
+{"rows": 6400, "of": 27644, "epoch": 0, "ce": 0.21077738636173307, "kl": 0.0, "lr": 8.734425630639985e-05, "rows_per_s": 1.3707163994998264, "tokens_per_s": 8516.597672308048, "gpu_gb": 16.97444248199463}
+{"rows": 6600, "of": 27644, "epoch": 0, "ce": 0.20477758564054965, "kl": 0.0, "lr": 8.657887996324792e-05, "rows_per_s": 1.3714119390632868, "tokens_per_s": 8528.412723992877, "gpu_gb": 16.97444248199463}
+{"rows": 6800, "of": 27644, "epoch": 0, "ce": 0.17484037813730538, "kl": 0.0, "lr": 8.579460215970059e-05, "rows_per_s": 1.3713591705164032, "tokens_per_s": 8534.653797708836, "gpu_gb": 16.97444248199463}
+{"rows": 7000, "of": 27644, "epoch": 0, "ce": 0.1956051228940487, "kl": 0.0, "lr": 8.499182815682263e-05, "rows_per_s": 1.373425384853003, "tokens_per_s": 8538.708048693998, "gpu_gb": 16.97444248199463}
+{"rows": 7200, "of": 27644, "epoch": 0, "ce": 0.19462666084989905, "kl": 0.0, "lr": 8.41709727732481e-05, "rows_per_s": 1.374072629079629, "tokens_per_s": 8536.931561535222, "gpu_gb": 16.97444248199463}
+{"rows": 7400, "of": 27644, "epoch": 0, "ce": 0.20932421209290625, "kl": 0.0, "lr": 8.33324601708305e-05, "rows_per_s": 1.3759249117869576, "tokens_per_s": 8544.588157582843, "gpu_gb": 16.97444248199463}
+{"rows": 7600, "of": 27644, "epoch": 0, "ce": 0.194134057238698, "kl": 0.0, "lr": 8.247672363546486e-05, "rows_per_s": 1.377438387691729, "tokens_per_s": 8539.878764389807, "gpu_gb": 16.97444248199463}
+{"rows": 7800, "of": 27644, "epoch": 0, "ce": 0.20587014392018319, "kl": 0.0, "lr": 8.160420535319541e-05, "rows_per_s": 1.378449497235504, "tokens_per_s": 8536.254925606854, "gpu_gb": 16.97444248199463}
+{"rows": 8000, "of": 27644, "epoch": 0, "ce": 0.18950007118284704, "kl": 0.0, "lr": 8.071535618172398e-05, "rows_per_s": 1.3812814879919957, "tokens_per_s": 8554.047653048166, "gpu_gb": 16.97444248199463}
+{"rows": 8200, "of": 27644, "epoch": 0, "ce": 0.18335368227213622, "kl": 0.0, "lr": 7.981063541743774e-05, "rows_per_s": 1.3821792661806545, "tokens_per_s": 8560.614261248584, "gpu_gb": 16.97444248199463}
+{"rows": 8400, "of": 27644, "epoch": 0, "ce": 0.19024737037718295, "kl": 0.0, "lr": 7.88905105580761e-05, "rows_per_s": 1.3817939197203468, "tokens_per_s": 8577.068586499718, "gpu_gb": 17.73898410797119}
+{"rows": 8600, "of": 27644, "epoch": 0, "ce": 0.18951277220621704, "kl": 0.0, "lr": 7.795545706116006e-05, "rows_per_s": 1.3820581367835514, "tokens_per_s": 8568.092560428197, "gpu_gb": 17.73898410797119}
+{"rows": 8800, "of": 27644, "epoch": 0, "ce": 0.19110010009258985, "kl": 0.0, "lr": 7.700595809830829e-05, "rows_per_s": 1.3822114940299595, "tokens_per_s": 8573.056843079254, "gpu_gb": 17.73898410797119}
+{"rows": 9000, "of": 27644, "epoch": 0, "ce": 0.18395390592515468, "kl": 0.0, "lr": 7.604250430556701e-05, "rows_per_s": 1.3828019239884972, "tokens_per_s": 8581.750479230788, "gpu_gb": 17.73898410797119}
+{"rows": 9200, "of": 27644, "epoch": 0, "ce": 0.17939317617565392, "kl": 0.0, "lr": 7.506559352988301e-05, "rows_per_s": 1.3849389244758508, "tokens_per_s": 8587.738315100232, "gpu_gb": 17.73898410797119}
+{"rows": 9400, "of": 27644, "epoch": 0, "ce": 0.18097828153520823, "kl": 0.0, "lr": 7.40757305718504e-05, "rows_per_s": 1.3850563088760282, "tokens_per_s": 8584.457588966972, "gpu_gb": 17.73898410797119}
+{"rows": 9600, "of": 27644, "epoch": 0, "ce": 0.17336546540260314, "kl": 0.0, "lr": 7.307342692486418e-05, "rows_per_s": 1.3859083109491912, "tokens_per_s": 8589.049532362978, "gpu_gb": 17.73898410797119}
+{"rows": 9800, "of": 27644, "epoch": 0, "ce": 0.18869875993579627, "kl": 0.0, "lr": 7.20592005108155e-05, "rows_per_s": 1.384989464478349, "tokens_per_s": 8582.74240145206, "gpu_gb": 17.73898410797119}
+{"rows": 10000, "of": 27644, "epoch": 0, "ce": 0.17679173666983844, "kl": 0.0, "lr": 7.103357541246516e-05, "rows_per_s": 1.3831154696570647, "tokens_per_s": 8571.355222414892, "gpu_gb": 17.73898410797119}
+{"rows": 10200, "of": 27644, "epoch": 0, "ce": 0.165219550691545, "kl": 0.0, "lr": 6.999708160263342e-05, "rows_per_s": 1.3838206555499692, "tokens_per_s": 8581.440361232073, "gpu_gb": 17.73898410797119}
+{"rows": 10400, "of": 27644, "epoch": 0, "ce": 0.1411630467325449, "kl": 0.0, "lr": 6.895025467034648e-05, "rows_per_s": 1.3843135720634494, "tokens_per_s": 8586.086731641622, "gpu_gb": 17.73898410797119}
+{"rows": 10600, "of": 27644, "epoch": 0, "ce": 0.1758753677457571, "kl": 0.0, "lr": 6.789363554408052e-05, "rows_per_s": 1.3847466950750846, "tokens_per_s": 8583.965335790204, "gpu_gb": 17.73898410797119}
+{"rows": 10800, "of": 27644, "epoch": 0, "ce": 0.17210819371044636, "kl": 0.0, "lr": 6.682777021224713e-05, "rows_per_s": 1.3839285684470897, "tokens_per_s": 8589.704280436476, "gpu_gb": 17.73898410797119}
+{"rows": 11000, "of": 27644, "epoch": 0, "ce": 0.20207075713202358, "kl": 0.0, "lr": 6.575320944106378e-05, "rows_per_s": 1.3847603845265353, "tokens_per_s": 8594.501730734712, "gpu_gb": 17.73898410797119}
+{"rows": 11200, "of": 27644, "epoch": 0, "ce": 0.17303466711193324, "kl": 0.0, "lr": 6.467050848995558e-05, "rows_per_s": 1.3845181334457055, "tokens_per_s": 8596.034989469303, "gpu_gb": 17.73898410797119}
+{"rows": 11400, "of": 27644, "epoch": 0, "ce": 0.16000845842063427, "kl": 0.0, "lr": 6.358022682463518e-05, "rows_per_s": 1.384289650816315, "tokens_per_s": 8596.053559045422, "gpu_gb": 17.73898410797119}
+{"rows": 11600, "of": 27644, "epoch": 0, "ce": 0.1615778366662562, "kl": 0.0, "lr": 6.248292782800926e-05, "rows_per_s": 1.3859979597833614, "tokens_per_s": 8602.154757456638, "gpu_gb": 17.73898410797119}
+{"rows": 11800, "of": 27644, "epoch": 0, "ce": 0.18867135945707558, "kl": 0.0, "lr": 6.137917850906081e-05, "rows_per_s": 1.386882605654892, "tokens_per_s": 8604.528090564072, "gpu_gb": 17.73898410797119}
+{"rows": 12000, "of": 27644, "epoch": 0, "ce": 0.17251531437039375, "kl": 0.0, "lr": 6.0269549209857736e-05, "rows_per_s": 1.387015366360134, "tokens_per_s": 8604.806153270623, "gpu_gb": 17.73898410797119}
+{"rows": 12200, "of": 27644, "epoch": 0, "ce": 0.19103072963654996, "kl": 0.0, "lr": 5.9154613310839015e-05, "rows_per_s": 1.387877198423959, "tokens_per_s": 8609.05004120278, "gpu_gb": 17.73898410797119}
+{"rows": 12400, "of": 27644, "epoch": 0, "ce": 0.1621233719959855, "kl": 0.0, "lr": 5.80349469345311e-05, "rows_per_s": 1.3874336392370512, "tokens_per_s": 8613.08688399572, "gpu_gb": 17.73898410797119}
+{"rows": 12600, "of": 27644, "epoch": 0, "ce": 0.16437096677720547, "kl": 0.0, "lr": 5.691112864784718e-05, "rows_per_s": 1.3872882365065526, "tokens_per_s": 8605.539187722581, "gpu_gb": 17.73898410797119}
+{"rows": 12800, "of": 27644, "epoch": 0, "ce": 0.15417302498593927, "kl": 0.0, "lr": 5.578373916312339e-05, "rows_per_s": 1.3880066329819136, "tokens_per_s": 8616.327474305606, "gpu_gb": 17.73898410797119}
+{"rows": 13000, "of": 27644, "epoch": 0, "ce": 0.18225131146609783, "kl": 0.0, "lr": 5.465336103804653e-05, "rows_per_s": 1.3881629371489126, "tokens_per_s": 8620.444855718413, "gpu_gb": 17.73898410797119}
+{"rows": 13200, "of": 27644, "epoch": 0, "ce": 0.17639901410788297, "kl": 0.0, "lr": 5.352057837462802e-05, "rows_per_s": 1.3881602016356724, "tokens_per_s": 8623.242759468667, "gpu_gb": 17.73898410797119}
+{"rows": 13400, "of": 27644, "epoch": 0, "ce": 0.17552863938733934, "kl": 0.0, "lr": 5.2385976517380054e-05, "rows_per_s": 1.3874683032878061, "tokens_per_s": 8624.716684772673, "gpu_gb": 17.73898410797119}
+{"rows": 13600, "of": 27644, "epoch": 0, "ce": 0.14498583190143108, "kl": 0.0, "lr": 5.125014175084966e-05, "rows_per_s": 1.3871155287524546, "tokens_per_s": 8619.03041445007, "gpu_gb": 17.73898410797119}
+{"rows": 13800, "of": 27644, "epoch": 0, "ce": 0.1834407548420131, "kl": 0.0, "lr": 5.011366099666695e-05, "rows_per_s": 1.3868432671065682, "tokens_per_s": 8618.510952519093, "gpu_gb": 17.73898410797119}
+{"rows": 14000, "of": 27644, "epoch": 0, "ce": 0.17281464175321162, "kl": 0.0, "lr": 4.897712151026418e-05, "rows_per_s": 1.386796731694343, "tokens_per_s": 8614.161993487647, "gpu_gb": 17.73898410797119}
+{"rows": 14200, "of": 27644, "epoch": 0, "ce": 0.17652987971901893, "kl": 0.0, "lr": 4.784111057742244e-05, "rows_per_s": 1.387004261753605, "tokens_per_s": 8620.749952898752, "gpu_gb": 17.73898410797119}
+{"rows": 14400, "of": 27644, "epoch": 0, "ce": 0.16347156874835492, "kl": 0.0, "lr": 4.670621521080252e-05, "rows_per_s": 1.3865712227531535, "tokens_per_s": 8615.184133808187, "gpu_gb": 17.73898410797119}
+{"rows": 14600, "of": 27644, "epoch": 0, "ce": 0.1692851831763983, "kl": 0.0, "lr": 4.5573021846616976e-05, "rows_per_s": 1.3865260294195854, "tokens_per_s": 8611.38837975103, "gpu_gb": 17.73898410797119}
+{"rows": 14800, "of": 27644, "epoch": 0, "ce": 0.18402940277010202, "kl": 0.0, "lr": 4.444211604160004e-05, "rows_per_s": 1.3857593967275548, "tokens_per_s": 8611.78828789899, "gpu_gb": 17.73898410797119}
+{"rows": 15000, "of": 27644, "epoch": 0, "ce": 0.15683865858241916, "kl": 0.0, "lr": 4.331408217043202e-05, "rows_per_s": 1.3855295070187765, "tokens_per_s": 8610.309571748132, "gpu_gb": 17.73898410797119}
+{"rows": 15200, "of": 27644, "epoch": 0, "ce": 0.1643610978499055, "kl": 0.0, "lr": 4.218950312377447e-05, "rows_per_s": 1.3855353426585857, "tokens_per_s": 8606.788035103547, "gpu_gb": 17.73898410797119}
+{"rows": 15400, "of": 27644, "epoch": 0, "ce": 0.1720756763778627, "kl": 0.0, "lr": 4.1068960007072224e-05, "rows_per_s": 1.3856927047965208, "tokens_per_s": 8607.232395364686, "gpu_gb": 17.73898410797119}
+{"rows": 15600, "of": 27644, "epoch": 0, "ce": 0.1853026487864554, "kl": 0.0, "lr": 3.995303184027782e-05, "rows_per_s": 1.3861855552335367, "tokens_per_s": 8603.907658702972, "gpu_gb": 17.73898410797119}
+{"rows": 15800, "of": 27644, "epoch": 0, "ce": 0.1507757867872715, "kl": 0.0, "lr": 3.884229525865376e-05, "rows_per_s": 1.3871914408711918, "tokens_per_s": 8603.858949263313, "gpu_gb": 17.73898410797119}
+{"rows": 16000, "of": 27644, "epoch": 0, "ce": 0.1430858861654997, "kl": 0.0, "lr": 3.773732421480681e-05, "rows_per_s": 1.3878025114962296, "tokens_per_s": 8605.341134873996, "gpu_gb": 17.73898410797119}
+{"rows": 16200, "of": 27644, "epoch": 0, "ce": 0.14958408737555146, "kl": 0.0, "lr": 3.663868968210861e-05, "rows_per_s": 1.3881411665420982, "tokens_per_s": 8609.450653170083, "gpu_gb": 17.73898410797119}
+{"rows": 16400, "of": 27644, "epoch": 0, "ce": 0.1495011910609901, "kl": 0.0, "lr": 3.554695935965571e-05, "rows_per_s": 1.3882318376807967, "tokens_per_s": 8611.437411360126, "gpu_gb": 17.73898410797119}
+{"rows": 16600, "of": 27644, "epoch": 0, "ce": 0.1550422583054751, "kl": 0.0, "lr": 3.446269737892163e-05, "rows_per_s": 1.3884849531757826, "tokens_per_s": 8613.619642232848, "gpu_gb": 17.73898410797119}
+{"rows": 16800, "of": 27644, "epoch": 0, "ce": 0.1474022769089788, "kl": 0.0, "lr": 3.3386464012252234e-05, "rows_per_s": 1.3885795716933338, "tokens_per_s": 8618.602954781994, "gpu_gb": 17.73898410797119}
+{"rows": 17000, "of": 27644, "epoch": 0, "ce": 0.14810169523581862, "kl": 0.0, "lr": 3.231881538335546e-05, "rows_per_s": 1.3891911004308037, "tokens_per_s": 8620.743046381273, "gpu_gb": 17.73898410797119}
+{"rows": 17200, "of": 27644, "epoch": 0, "ce": 0.1417401478579268, "kl": 0.0, "lr": 3.1260303179934556e-05, "rows_per_s": 1.3888488566753114, "tokens_per_s": 8615.818194912306, "gpu_gb": 17.73898410797119}
+{"rows": 17400, "of": 27644, "epoch": 0, "ce": 0.15734178267419338, "kl": 0.0, "lr": 3.0211474368613752e-05, "rows_per_s": 1.3894888891336963, "tokens_per_s": 8613.462386217438, "gpu_gb": 17.73898410797119}
+{"rows": 17600, "of": 27644, "epoch": 0, "ce": 0.1380095674470067, "kl": 0.0, "lr": 2.9172870912303308e-05, "rows_per_s": 1.3894239842239264, "tokens_per_s": 8612.703606000658, "gpu_gb": 17.73898410797119}
+{"rows": 17800, "of": 27644, "epoch": 0, "ce": 0.14660378344357014, "kl": 0.0, "lr": 2.8145029490150175e-05, "rows_per_s": 1.3890173477087984, "tokens_per_s": 8612.930434412177, "gpu_gb": 17.73898410797119}
+{"rows": 18000, "of": 27644, "epoch": 0, "ce": 0.14610900938510896, "kl": 0.0, "lr": 2.7128481220219036e-05, "rows_per_s": 1.389123803646175, "tokens_per_s": 8616.41422076567, "gpu_gb": 17.73898410797119}
+{"rows": 18200, "of": 27644, "epoch": 0, "ce": 0.1300594929791987, "kl": 0.0, "lr": 2.612375138504679e-05, "rows_per_s": 1.388792493319898, "tokens_per_s": 8617.069780068316, "gpu_gb": 17.73898410797119}
+{"rows": 18400, "of": 27644, "epoch": 0, "ce": 0.1290073513239622, "kl": 0.0, "lr": 2.513135916021253e-05, "rows_per_s": 1.3893520603384668, "tokens_per_s": 8618.71774632674, "gpu_gb": 17.73898410797119}
+{"rows": 18600, "of": 27644, "epoch": 0, "ce": 0.14802748183021322, "kl": 0.0, "lr": 2.4151817346063155e-05, "rows_per_s": 1.3886449896001374, "tokens_per_s": 8619.465780780289, "gpu_gb": 17.73898410797119}
+{"rows": 18800, "of": 27644, "epoch": 0, "ce": 0.13398919036611914, "kl": 0.0, "lr": 2.3185632102733266e-05, "rows_per_s": 1.3883950621247052, "tokens_per_s": 8617.943915514845, "gpu_gb": 17.73898410797119}
+{"rows": 19000, "of": 27644, "epoch": 0, "ce": 0.1269960441440344, "kl": 0.0, "lr": 2.223330268859628e-05, "rows_per_s": 1.388467678952904, "tokens_per_s": 8615.201085299093, "gpu_gb": 17.73898410797119}
+{"rows": 19200, "of": 27644, "epoch": 0, "ce": 0.12839976143091916, "kl": 0.0, "lr": 2.129532120228187e-05, "rows_per_s": 1.3880198829826724, "tokens_per_s": 8609.854947010914, "gpu_gb": 17.73898410797119}
+{"rows": 19400, "of": 27644, "epoch": 0, "ce": 0.1504750985093415, "kl": 0.0, "lr": 2.0372172328393062e-05, "rows_per_s": 1.3879455546332673, "tokens_per_s": 8610.14171124306, "gpu_gb": 17.73898410797119}
+{"rows": 19600, "of": 27644, "epoch": 0, "ce": 0.15901072138920427, "kl": 0.0, "lr": 1.9464333087054505e-05, "rows_per_s": 1.3881209571341546, "tokens_per_s": 8609.270370751574, "gpu_gb": 17.73898410797119}
+{"rows": 19800, "of": 27644, "epoch": 0, "ce": 0.13902726129628717, "kl": 0.0, "lr": 1.8572272587421057e-05, "rows_per_s": 1.388038294445861, "tokens_per_s": 8610.15296280707, "gpu_gb": 17.73898410797119}
+{"rows": 20000, "of": 27644, "epoch": 0, "ce": 0.12834056928753854, "kl": 0.0, "lr": 1.7696451785274347e-05, "rows_per_s": 1.387991368244487, "tokens_per_s": 8614.263901703216, "gpu_gb": 17.73898410797119}
+{"rows": 20200, "of": 27644, "epoch": 0, "ce": 0.14513432241976262, "kl": 0.0, "lr": 1.6837323244832354e-05, "rows_per_s": 1.3879818708169038, "tokens_per_s": 8608.339145977621, "gpu_gb": 17.73898410797119}
+{"rows": 20400, "of": 27644, "epoch": 0, "ce": 0.15517583202570676, "kl": 0.0, "lr": 1.5995330904895223e-05, "rows_per_s": 1.3877304057195157, "tokens_per_s": 8607.27974833881, "gpu_gb": 17.73898410797119}
+{"rows": 20600, "of": 27644, "epoch": 0, "ce": 0.13032510291785002, "kl": 0.0, "lr": 1.5170909849448101e-05, "rows_per_s": 1.3883809086936458, "tokens_per_s": 8607.126179058654, "gpu_gb": 17.73898410797119}
+{"rows": 20800, "of": 27644, "epoch": 0, "ce": 0.13141531277447938, "kl": 0.0, "lr": 1.4364486082839534e-05, "rows_per_s": 1.388411101679877, "tokens_per_s": 8606.403971463415, "gpu_gb": 17.73898410797119}
+{"rows": 21000, "of": 27644, "epoch": 0, "ce": 0.12583129175007343, "kl": 0.0, "lr": 1.357647630965153e-05, "rows_per_s": 1.3887657668708695, "tokens_per_s": 8612.72108916327, "gpu_gb": 17.73898410797119}
+{"rows": 21200, "of": 27644, "epoch": 0, "ce": 0.1305518364533782, "kl": 0.0, "lr": 1.2807287719375261e-05, "rows_per_s": 1.3884317735385276, "tokens_per_s": 8615.646163068992, "gpu_gb": 17.73898410797119}
+{"rows": 21400, "of": 27644, "epoch": 0, "ce": 0.13091811086982488, "kl": 0.0, "lr": 1.2057317776003335e-05, "rows_per_s": 1.3885387762824761, "tokens_per_s": 8614.331057882826, "gpu_gb": 17.73898410797119}
+{"rows": 21600, "of": 27644, "epoch": 0, "ce": 0.14964440142270177, "kl": 0.0, "lr": 1.1326954012647661e-05, "rows_per_s": 1.3882424774394952, "tokens_per_s": 8616.317691027905, "gpu_gb": 17.73898410797119}
+{"rows": 21800, "of": 27644, "epoch": 0, "ce": 0.13317173598334192, "kl": 0.0, "lr": 1.0616573831288884e-05, "rows_per_s": 1.387940967634077, "tokens_per_s": 8612.280919425846, "gpu_gb": 17.73898410797119}
+{"rows": 22000, "of": 27644, "epoch": 0, "ce": 0.13332346592098474, "kl": 0.0, "lr": 9.926544307760832e-06, "rows_per_s": 1.3880913256631104, "tokens_per_s": 8612.543867802104, "gpu_gb": 17.73898410797119}
+{"rows": 22200, "of": 27644, "epoch": 0, "ce": 0.16741748777218163, "kl": 0.0, "lr": 9.257222002070925e-06, "rows_per_s": 1.3882116351045863, "tokens_per_s": 8610.521988156153, "gpu_gb": 17.73898410797119}
+{"rows": 22400, "of": 27644, "epoch": 0, "ce": 0.12876728530973197, "kl": 0.0, "lr": 8.608952774154345e-06, "rows_per_s": 1.3879162849625273, "tokens_per_s": 8612.388841692386, "gpu_gb": 17.73898410797119}
+{"rows": 22600, "of": 27644, "epoch": 0, "ce": 0.1317625662405044, "kl": 0.0, "lr": 7.98207160515726e-06, "rows_per_s": 1.388563497865564, "tokens_per_s": 8611.766118532125, "gpu_gb": 17.73898410797119}
+{"rows": 22800, "of": 27644, "epoch": 0, "ce": 0.12762531377607955, "kl": 0.0, "lr": 7.376902424341625e-06, "rows_per_s": 1.3888253642786386, "tokens_per_s": 8611.489396699384, "gpu_gb": 17.73898410797119}
+{"rows": 23000, "of": 27644, "epoch": 0, "ce": 0.12915205211378633, "kl": 0.0, "lr": 6.793757941700601e-06, "rows_per_s": 1.3892586162278142, "tokens_per_s": 8612.280899650536, "gpu_gb": 17.73898410797119}
+{"rows": 23200, "of": 27644, "epoch": 0, "ce": 0.1360259204916656, "kl": 0.0, "lr": 6.232939486371514e-06, "rows_per_s": 1.3894697288754587, "tokens_per_s": 8614.614577019329, "gpu_gb": 17.73898410797119}
+{"rows": 23400, "of": 27644, "epoch": 0, "ce": 0.14520388450473548, "kl": 0.0, "lr": 5.6947368509295606e-06, "rows_per_s": 1.3899298478280597, "tokens_per_s": 8616.939944494716, "gpu_gb": 17.73898410797119}
+{"rows": 23600, "of": 27644, "epoch": 0, "ce": 0.12198533607646823, "kl": 0.0, "lr": 5.179428141642861e-06, "rows_per_s": 1.3896197981593994, "tokens_per_s": 8614.062114950066, "gpu_gb": 17.73898410797119}
+{"rows": 23800, "of": 27644, "epoch": 0, "ce": 0.13558543171733617, "kl": 0.0, "lr": 4.687279634766173e-06, "rows_per_s": 1.3898143389765283, "tokens_per_s": 8615.418911169902, "gpu_gb": 17.73898410797119}
+{"rows": 24000, "of": 27644, "epoch": 0, "ce": 0.14841709772124886, "kl": 0.0, "lr": 4.218545638947585e-06, "rows_per_s": 1.3893672931842282, "tokens_per_s": 8610.66979445568, "gpu_gb": 17.73898410797119}
+{"rows": 24200, "of": 27644, "epoch": 0, "ce": 0.12530215360224248, "kl": 0.0, "lr": 3.773468363819149e-06, "rows_per_s": 1.3891257696404458, "tokens_per_s": 8606.369116738862, "gpu_gb": 17.73898410797119}
+{"rows": 24400, "of": 27644, "epoch": 0, "ce": 0.1442721876129508, "kl": 0.0, "lr": 3.352277794839631e-06, "rows_per_s": 1.3893584685190943, "tokens_per_s": 8607.446283988622, "gpu_gb": 17.73898410797119}
+{"rows": 24600, "of": 27644, "epoch": 0, "ce": 0.11735559469088912, "kl": 0.0, "lr": 2.9551915744536973e-06, "rows_per_s": 1.3894268472423805, "tokens_per_s": 8609.720820160683, "gpu_gb": 17.73898410797119}
+{"rows": 24800, "of": 27644, "epoch": 0, "ce": 0.11297125708311796, "kl": 0.0, "lr": 2.5824148896292254e-06, "rows_per_s": 1.3894417139158168, "tokens_per_s": 8610.572890702346, "gpu_gb": 17.73898410797119}
+{"rows": 25000, "of": 27644, "epoch": 0, "ce": 0.12124939039815218, "kl": 0.0, "lr": 2.2341403658307446e-06, "rows_per_s": 1.3895282687682156, "tokens_per_s": 8614.521455976137, "gpu_gb": 17.73898410797119}
+{"rows": 25200, "of": 27644, "epoch": 0, "ce": 0.12851434271782636, "kl": 0.0, "lr": 1.9105479674837556e-06, "rows_per_s": 1.3893133146404193, "tokens_per_s": 8618.376682515656, "gpu_gb": 17.73898410797119}
+{"rows": 25400, "of": 27644, "epoch": 0, "ce": 0.1329329369869083, "kl": 0.0, "lr": 1.6118049049814676e-06, "rows_per_s": 1.3895325725682355, "tokens_per_s": 8621.480670315243, "gpu_gb": 17.73898410797119}
+{"rows": 25600, "of": 27644, "epoch": 0, "ce": 0.12408415727317333, "kl": 0.0, "lr": 1.3380655482818916e-06, "rows_per_s": 1.3896937485640746, "tokens_per_s": 8621.783134272055, "gpu_gb": 17.73898410797119}
+{"rows": 25800, "of": 27644, "epoch": 0, "ce": 0.12144216819666326, "kl": 0.0, "lr": 1.0894713471400474e-06, "rows_per_s": 1.3901335760719598, "tokens_per_s": 8623.928129911359, "gpu_gb": 17.73898410797119}
+{"rows": 26000, "of": 27644, "epoch": 0, "ce": 0.12853420542553068, "kl": 0.0, "lr": 8.661507580163819e-07, "rows_per_s": 1.3905432379055662, "tokens_per_s": 8626.490622373225, "gpu_gb": 17.73898410797119}
+{"rows": 26200, "of": 27644, "epoch": 0, "ce": 0.1427439607679844, "kl": 0.0, "lr": 6.682191776992863e-07, "rows_per_s": 1.3908375972577987, "tokens_per_s": 8627.166439352533, "gpu_gb": 17.73898410797119}
+{"rows": 26400, "of": 27644, "epoch": 0, "ce": 0.13167895659804343, "kl": 0.0, "lr": 4.957788836759348e-07, "rows_per_s": 1.3908509383437202, "tokens_per_s": 8627.127631890315, "gpu_gb": 17.73898410797119}
+{"rows": 26600, "of": 27644, "epoch": 0, "ce": 0.13655862426385285, "kl": 0.0, "lr": 3.4891898128229796e-07, "rows_per_s": 1.3912108475774747, "tokens_per_s": 8629.01122347449, "gpu_gb": 17.73898410797119}
+{"rows": 26800, "of": 27644, "epoch": 0, "ce": 0.14189616140909492, "kl": 0.0, "lr": 2.2771535765956364e-07, "rows_per_s": 1.391232673620704, "tokens_per_s": 8626.295002278292, "gpu_gb": 17.73898410797119}
+{"rows": 27000, "of": 27644, "epoch": 0, "ce": 0.12253829274326562, "kl": 0.0, "lr": 1.3223064254090366e-07, "rows_per_s": 1.3916149015463086, "tokens_per_s": 8623.8843443762, "gpu_gb": 17.73898410797119}
+{"rows": 27200, "of": 27644, "epoch": 0, "ce": 0.13998983905650675, "kl": 0.0, "lr": 6.251417588870046e-08, "rows_per_s": 1.3919402766863644, "tokens_per_s": 8626.37055010457, "gpu_gb": 17.73898410797119}
+{"rows": 27400, "of": 27644, "epoch": 0, "ce": 0.13824107202701272, "kl": 0.0, "lr": 1.8601982399019114e-08, "rows_per_s": 1.392383328794809, "tokens_per_s": 8627.699012899599, "gpu_gb": 17.73898410797119}
+{"rows": 27600, "of": 27644, "epoch": 0, "ce": 0.14036576688289643, "kl": 0.0, "lr": 5.16752886531302e-10, "rows_per_s": 1.3925534374220379, "tokens_per_s": 8624.390606988267, "gpu_gb": 17.73898410797119}
+{"rows": 27644, "of": 27644, "epoch": 0, "ce": 0.14875189418142493, "kl": 0.0, "lr": 0.0, "rows_per_s": 1.3924991021120716, "tokens_per_s": 8623.441681654736, "gpu_gb": 17.73898410797119}

training_meta.json

@@ -0,0 +1,32 @@
+{
+ "base_model": "Qwen/Qwen3.5-4B",
+ "families": [
+  "Discrete selection",
+  "Drag & gesture",
+  "Form transaction",
+  "Navigation",
+  "Out-of-page I/O",
+  "Reasoning base",
+  "Text entry"
+ ],
+ "training_rows": 27646,
+ "rows_trained": 27644,
+ "validation_ce": 0.11895407103933393,
+ "minutes": 333.5,
+ "gpus": 4,
+ "args": {
+  "adapter": "pooled",
+  "epochs": 1.0,
+  "lr": 0.0001,
+  "rank": 64,
+  "alpha": 128,
+  "grad_accum": 8,
+  "beta": 0.0,
+  "anchor_ratio": 0.5,
+  "limit": 0,
+  "max_steps": 0,
+  "seed": 0,
+  "ce_on": "all",
+  "exclude_variants": ""
+ }
+}