Adds notebooks/train_whispers_grpo_kaggle_t4.ipynb -- the single-T4 sibling
of the existing 2xT4 notebook -- and folds in the latest stability fixes
(numpy/transformers ABI safety net, Unsloth compiled-cache wipe, and the
defensive _get_train_sampler signature patch) into both Kaggle notebooks.
Made-with: Cursor
Adds scripts/train_grpo_a6000.py, a Python sibling of the T4 notebooks tuned
for an RTX A6000 (48 GB Ampere). Five concrete optimisations target the
"terminal reward stuck near 0" failure mode the T4 notebooks hit on an
untrained 1.5B base:
* Default model upgraded to Qwen/Qwen2.5-3B-Instruct (overridable to 7B);
the 1.5B model rarely emits valid WhispersAction JSON, so _coerce_action
falls back to wait and the editor never publishes.
* Dense multi-component reward (max ~2.25) replaces the [0, 1] terminal-
only signal: format / legal-tool / neighbour-validity / per-step shaping
/ 1.5x terminal_score. Restores variance across the GRPO group so
group-relative advantages are non-zero on the first few hundred steps.
* Three-stage curriculum (40% t1 -> 30% t1+t2 -> 30% full mix) instead of
uniform task sampling, so easier tasks bootstrap JSON / publish skills
before the policy is exposed to t4 cascades and t5 coalitions.
* num_generations=8, temperature=0.9, top_p=0.95 for di…
feat(kaggle nb): integrate Liger-Kernel for fused Triton kernels on 2xT4
- pip install liger-kernel in the install cell
- patch Qwen2 / Llama / Mistral / Gemma / Phi3 model families before model load
via a small dispatcher, with a graceful no-op fallback if liger import fails
- use rope+rms_norm+swiglu+fused_linear_cross_entropy=True (CE left to HF for
PEFT compat); composes cleanly with bnb 4-bit + device_map='auto' multi-GPU
- log liger_kernel flag to wandb config for run provenance
- update markdown cells to call out the speedup vs pure HF stack
Made-with: Cursor
feat(notebooks): add Kaggle 2x T4 GRPO training notebook
Adds notebooks/train_whispers_grpo_kaggle.ipynb, a Kaggle-tuned variant
of the Phase-1 GRPO training notebook. Highlights:
- Detects Kaggle vs Colab vs local; clones to /kaggle/working/whispers
on Kaggle and reads creds from kaggle_secrets.UserSecretsClient.
- Drops Unsloth (single-GPU only) in favour of the vanilla HF stack
(transformers + bitsandbytes + peft + trl) so device_map='auto' can
shard the model across both T4 GPUs.
- Defaults to Qwen/Qwen2.5-3B-Instruct with 4-bit nf4 quant and
per-GPU max_memory caps that force accelerate to spread the model
across both devices rather than keeping everything on GPU 0.
- Picks the input device from hf_device_map so token ids land where
the embedding layer lives even under multi-GPU sharding.
- Tunes GRPO_STEPS=80, GENERATIONS=4, MAX_STEPS_PER_EP=12 to stay
well under Kaggle's 9-hour notebook quota (~2h end-to-end).
- All knobs surfaced via env vars (WHISPERS_MODEL, WHISPERS_GRPO_STEPS,
WHISPERS_GEN_PER_STEP, WHISPERS_MAX_S…
chore(noise): silence ToolError WARNING spam and Qwen max_length warnings
- whispers/env.py: demote ToolError log from WARNING to INFO. The error is
still surfaced programmatically via info["tool_error"] and the agent is
already penalised via shaping (-0.05). Avoids spamming Colab during
long GRPO runs (~21k generations).
- notebooks/train_whispers_grpo.ipynb:
* cell 6 (imports): explicitly set whispers logger to ERROR so the noise
is silenced even when the notebook clones an older env.py from HF.
* cell 10 (Unsloth load): clear Qwen's generation_config.max_length and
set pad_token_id once; also dial transformers logging to error level.
* cell 13 (rollout_one): pass pad_token_id=eos to .generate() so the
"max_new_tokens vs max_length" warning never fires.
Made-with: Cursor
