Add kaggle/KAGGLE_SETUP.md
Browse files- kaggle/KAGGLE_SETUP.md +52 -21
kaggle/KAGGLE_SETUP.md
CHANGED
|
@@ -1,47 +1,78 @@
|
|
| 1 |
-
# Kaggle GRPO dispatch
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2 |
|
| 3 |
-
## 1. Notebook settings
|
| 4 |
- **Accelerator:** GPU T4 x2
|
| 5 |
- **Internet:** On
|
| 6 |
- **Language:** Python
|
| 7 |
|
| 8 |
-
##
|
| 9 |
-
|
| 10 |
-
Kaggle **cannot** create/attach User Secrets via API. We use a **private dataset** instead:
|
| 11 |
|
| 12 |
```bash
|
| 13 |
-
|
| 14 |
-
|
|
|
|
|
|
|
| 15 |
```
|
| 16 |
|
| 17 |
-
Notebook loads `/kaggle/input/.../secrets.json` first, then
|
|
|
|
|
|
|
| 18 |
|
| 19 |
**Do not** make `deepagents-rl-secrets` public.
|
| 20 |
|
| 21 |
-
|
| 22 |
|
| 23 |
-
## 3. Checkpoint / observability contract
|
| 24 |
Every `save_steps` (default 5):
|
| 25 |
-
|
| 26 |
-
|
|
|
|
| 27 |
3. W&B `eval/*` metrics
|
| 28 |
4. **Then** push LoRA to `vanishingradient/deepagents-rl-coding` revision `grpo-step-{n}`
|
| 29 |
|
| 30 |
Baseline eval at step 0 runs before training starts.
|
| 31 |
|
| 32 |
-
##
|
| 33 |
-
1. Open [Kaggle Notebooks](https://www.kaggle.com/code) → New Notebook
|
| 34 |
-
2. Apply settings + secrets above
|
| 35 |
-
3. Upload / paste cells from `notebooks/kaggle_grpo.ipynb`
|
| 36 |
-
4. Run All
|
| 37 |
-
|
| 38 |
-
Or use CLI after `pip install kaggle` (with `~/.kaggle/kaggle.json`):
|
| 39 |
|
| 40 |
```bash
|
|
|
|
| 41 |
kaggle kernels push -p notebooks/
|
|
|
|
| 42 |
```
|
| 43 |
|
| 44 |
-
##
|
|
|
|
| 45 |
- Langfuse: filter `kaggle-ckpt-eval`
|
| 46 |
-
- W&B: project `deepagents-rl-coding`, run `kaggle-grpo-
|
| 47 |
- HF: https://huggingface.co/vanishingradient/deepagents-rl-coding/tree/main
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Kaggle agent GRPO dispatch (deepagents + SWE Docker reward)
|
| 2 |
+
|
| 3 |
+
## Architecture
|
| 4 |
+
|
| 5 |
+
1. **Kaggle T4×2** — Qwen QLoRA + TRL `GRPOTrainer` + `DeepAgentSWEEnv` (git checkout + deepagents tools)
|
| 6 |
+
2. **This machine** — `scripts/swe_grader_server.py` runs official Docker `evaluate_model_patch`
|
| 7 |
+
3. Tunnel (`cloudflared` / ngrok) exposes the grader; Kaggle calls `SWE_GRADER_URL`
|
| 8 |
+
|
| 9 |
+
## 1. Local grader (keep running for the whole kernel)
|
| 10 |
+
|
| 11 |
+
```bash
|
| 12 |
+
# Docker Desktop must be up
|
| 13 |
+
python scripts/swe_grader_server.py --host 127.0.0.1 --port 8787
|
| 14 |
+
|
| 15 |
+
# separate terminal — public HTTPS URL
|
| 16 |
+
cloudflared tunnel --url http://127.0.0.1:8787
|
| 17 |
+
```
|
| 18 |
+
|
| 19 |
+
Optional auth: set `SWE_GRADER_TOKEN` in `.env` (same value in Kaggle secrets).
|
| 20 |
+
|
| 21 |
+
Smoke:
|
| 22 |
+
|
| 23 |
+
```bash
|
| 24 |
+
curl -s http://127.0.0.1:8787/health
|
| 25 |
+
# POST /grade with gold patch should FULL-resolve (see verify script / notebook smoke)
|
| 26 |
+
```
|
| 27 |
+
|
| 28 |
+
## 2. Notebook settings
|
| 29 |
|
|
|
|
| 30 |
- **Accelerator:** GPU T4 x2
|
| 31 |
- **Internet:** On
|
| 32 |
- **Language:** Python
|
| 33 |
|
| 34 |
+
## 3. Secrets (API path — recommended)
|
|
|
|
|
|
|
| 35 |
|
| 36 |
```bash
|
| 37 |
+
# .env must include SWE_GRADER_URL=https://....trycloudflare.com
|
| 38 |
+
# and optional SWE_GRADER_TOKEN
|
| 39 |
+
python scripts/push_kaggle_secrets_dataset.py
|
| 40 |
+
kaggle kernels push -p notebooks/
|
| 41 |
```
|
| 42 |
|
| 43 |
+
Notebook loads `/kaggle/input/.../secrets.json` first, then Add-ons → Secrets.
|
| 44 |
+
|
| 45 |
+
**Required keys:** `HF_TOKEN`, `WANDB_API_KEY`, `LANGFUSE_*`, **`SWE_GRADER_URL`**
|
| 46 |
|
| 47 |
**Do not** make `deepagents-rl-secrets` public.
|
| 48 |
|
| 49 |
+
## 4. Checkpoint / observability contract
|
| 50 |
|
|
|
|
| 51 |
Every `save_steps` (default 5):
|
| 52 |
+
|
| 53 |
+
1. Held-out **agent** eval on SWE Lite (`n_eval=2` smoke)
|
| 54 |
+
2. Langfuse trace `kaggle-ckpt-eval` + scores
|
| 55 |
3. W&B `eval/*` metrics
|
| 56 |
4. **Then** push LoRA to `vanishingradient/deepagents-rl-coding` revision `grpo-step-{n}`
|
| 57 |
|
| 58 |
Baseline eval at step 0 runs before training starts.
|
| 59 |
|
| 60 |
+
## 5. Upload / run
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 61 |
|
| 62 |
```bash
|
| 63 |
+
python scripts/push_trainer_to_hf.py
|
| 64 |
kaggle kernels push -p notebooks/
|
| 65 |
+
# start the kernel in the Kaggle UI (or kernels status / pull)
|
| 66 |
```
|
| 67 |
|
| 68 |
+
## 6. Watch
|
| 69 |
+
|
| 70 |
- Langfuse: filter `kaggle-ckpt-eval`
|
| 71 |
+
- W&B: project `deepagents-rl-coding`, run `kaggle-grpo-swe`
|
| 72 |
- HF: https://huggingface.co/vanishingradient/deepagents-rl-coding/tree/main
|
| 73 |
+
- Local grader logs: each `/grade` call
|
| 74 |
+
|
| 75 |
+
## Notes
|
| 76 |
+
|
| 77 |
+
- First launch is a **smoke** GRPO (`max_steps=20`, `num_generations=2`, short tool budget).
|
| 78 |
+
- Keep Docker Desktop + grader + tunnel alive until the kernel finishes.
|