vanishingradient commited on
Commit
29c1d2c
·
verified ·
1 Parent(s): aa44576

Add kaggle/KAGGLE_SETUP.md

Browse files
Files changed (1) hide show
  1. kaggle/KAGGLE_SETUP.md +52 -21
kaggle/KAGGLE_SETUP.md CHANGED
@@ -1,47 +1,78 @@
1
- # Kaggle GRPO dispatch
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2
 
3
- ## 1. Notebook settings
4
  - **Accelerator:** GPU T4 x2
5
  - **Internet:** On
6
  - **Language:** Python
7
 
8
- ## 2. Secrets (API path — recommended)
9
-
10
- Kaggle **cannot** create/attach User Secrets via API. We use a **private dataset** instead:
11
 
12
  ```bash
13
- python scripts/push_kaggle_secrets_dataset.py # uploads saqlain22/deepagents-rl-secrets (private)
14
- kaggle kernels push -p notebooks/ # attaches it via dataset_sources + enable_gpu
 
 
15
  ```
16
 
17
- Notebook loads `/kaggle/input/.../secrets.json` first, then falls back to Add-ons → Secrets if present.
 
 
18
 
19
  **Do not** make `deepagents-rl-secrets` public.
20
 
21
- **OpenTelemetry / google-adk warnings:** Notebook uninstalls unused `google-adk`.
22
 
23
- ## 3. Checkpoint / observability contract
24
  Every `save_steps` (default 5):
25
- 1. Held-out MBPP eval (`n_eval=16`)
26
- 2. Langfuse trace `kaggle-ckpt-eval` + scores (`mean_pass_frac`, …)
 
27
  3. W&B `eval/*` metrics
28
  4. **Then** push LoRA to `vanishingradient/deepagents-rl-coding` revision `grpo-step-{n}`
29
 
30
  Baseline eval at step 0 runs before training starts.
31
 
32
- ## 4. Upload notebook
33
- 1. Open [Kaggle Notebooks](https://www.kaggle.com/code) → New Notebook
34
- 2. Apply settings + secrets above
35
- 3. Upload / paste cells from `notebooks/kaggle_grpo.ipynb`
36
- 4. Run All
37
-
38
- Or use CLI after `pip install kaggle` (with `~/.kaggle/kaggle.json`):
39
 
40
  ```bash
 
41
  kaggle kernels push -p notebooks/
 
42
  ```
43
 
44
- ## 5. Watch
 
45
  - Langfuse: filter `kaggle-ckpt-eval`
46
- - W&B: project `deepagents-rl-coding`, run `kaggle-grpo-mbpp`
47
  - HF: https://huggingface.co/vanishingradient/deepagents-rl-coding/tree/main
 
 
 
 
 
 
 
1
+ # Kaggle agent GRPO dispatch (deepagents + SWE Docker reward)
2
+
3
+ ## Architecture
4
+
5
+ 1. **Kaggle T4×2** — Qwen QLoRA + TRL `GRPOTrainer` + `DeepAgentSWEEnv` (git checkout + deepagents tools)
6
+ 2. **This machine** — `scripts/swe_grader_server.py` runs official Docker `evaluate_model_patch`
7
+ 3. Tunnel (`cloudflared` / ngrok) exposes the grader; Kaggle calls `SWE_GRADER_URL`
8
+
9
+ ## 1. Local grader (keep running for the whole kernel)
10
+
11
+ ```bash
12
+ # Docker Desktop must be up
13
+ python scripts/swe_grader_server.py --host 127.0.0.1 --port 8787
14
+
15
+ # separate terminal — public HTTPS URL
16
+ cloudflared tunnel --url http://127.0.0.1:8787
17
+ ```
18
+
19
+ Optional auth: set `SWE_GRADER_TOKEN` in `.env` (same value in Kaggle secrets).
20
+
21
+ Smoke:
22
+
23
+ ```bash
24
+ curl -s http://127.0.0.1:8787/health
25
+ # POST /grade with gold patch should FULL-resolve (see verify script / notebook smoke)
26
+ ```
27
+
28
+ ## 2. Notebook settings
29
 
 
30
  - **Accelerator:** GPU T4 x2
31
  - **Internet:** On
32
  - **Language:** Python
33
 
34
+ ## 3. Secrets (API path — recommended)
 
 
35
 
36
  ```bash
37
+ # .env must include SWE_GRADER_URL=https://....trycloudflare.com
38
+ # and optional SWE_GRADER_TOKEN
39
+ python scripts/push_kaggle_secrets_dataset.py
40
+ kaggle kernels push -p notebooks/
41
  ```
42
 
43
+ Notebook loads `/kaggle/input/.../secrets.json` first, then Add-ons → Secrets.
44
+
45
+ **Required keys:** `HF_TOKEN`, `WANDB_API_KEY`, `LANGFUSE_*`, **`SWE_GRADER_URL`**
46
 
47
  **Do not** make `deepagents-rl-secrets` public.
48
 
49
+ ## 4. Checkpoint / observability contract
50
 
 
51
  Every `save_steps` (default 5):
52
+
53
+ 1. Held-out **agent** eval on SWE Lite (`n_eval=2` smoke)
54
+ 2. Langfuse trace `kaggle-ckpt-eval` + scores
55
  3. W&B `eval/*` metrics
56
  4. **Then** push LoRA to `vanishingradient/deepagents-rl-coding` revision `grpo-step-{n}`
57
 
58
  Baseline eval at step 0 runs before training starts.
59
 
60
+ ## 5. Upload / run
 
 
 
 
 
 
61
 
62
  ```bash
63
+ python scripts/push_trainer_to_hf.py
64
  kaggle kernels push -p notebooks/
65
+ # start the kernel in the Kaggle UI (or kernels status / pull)
66
  ```
67
 
68
+ ## 6. Watch
69
+
70
  - Langfuse: filter `kaggle-ckpt-eval`
71
+ - W&B: project `deepagents-rl-coding`, run `kaggle-grpo-swe`
72
  - HF: https://huggingface.co/vanishingradient/deepagents-rl-coding/tree/main
73
+ - Local grader logs: each `/grade` call
74
+
75
+ ## Notes
76
+
77
+ - First launch is a **smoke** GRPO (`max_steps=20`, `num_generations=2`, short tool budget).
78
+ - Keep Docker Desktop + grader + tunnel alive until the kernel finishes.