| --- |
| license: mit |
| base_model: zai-org/GLM-4.7-Flash |
| tags: |
| - lora |
| - grpo |
| - reinforcement-learning |
| - code-generation |
| - cpp |
| extra_gated_prompt: "Access requests are reviewed manually." |
| --- |
| |
| # execution-midband-RL-v2 |
|
|
| Canonical name **execution-midband-RL-v2** in the execution-grounded RL series. |
| Historical run identity: `execution-bank-RL-v2-think-r2` (W&B project |
| `execution-bank-RL-v2-think`). This is the complete preserved run tree. |
|
|
| ## Lineage |
|
|
| | Series name | Run identity | Adapter SHA-256 | Notes | |
| |---|---|---|---| |
| | Synth-v1 ep50 | `iter_0000649` | `4acb7f23c295f45380155c5d9ee6bc59422262f0cb51f0c02f7e550d405b575a` | SFT starting point ([checkpoints](https://huggingface.co/TokenBender/glm47-synth-v1-100ep)) | |
| | execution-midband-RL-v1 | `issue111-bank-official-grpo20-20260817T151213Z` | `186b0fc5b200fb8bb55bf85ee4416f2682a470580f0231c6f3f2a4d414bd898e` | bank drill, thinking off ([archive](https://huggingface.co/TokenBender/glm47-bank-account-official-grpo20)) | |
| | **execution-midband-RL-v2 (this repo)** | `execution-bank-RL-v2-think-r2` | `f5c298f5554c6f5fa4a27f3329e4a08c6c2fef76834c0f10147a270ede41148d` | bank drill, thinking on — lineage anchor | |
|
|
| ## Identity |
|
|
| - Base model: GLM-4.7-Flash, revision `7dd20894a642a0aa287e9827cb1a1f7f91386b67` |
| - Cumulative LoRA rank 16 / alpha 32; serve as **base + this adapter only** — |
| the adapter already contains the Synth-v1 SFT state (do not stack a second |
| Synth-v1 adapter) |
| - Final checkpoint: `checkpoints/grpo_lora_r16/iter_0000019/adapter` |
| - Training: 20 GRPO updates, 8 exact-task bank-account drill prompts × 32 |
| samples, thinking enabled, response budget 8192, LR 3e-5 (0.1 warmup), |
| KL 0.1 against the frozen starting policy, strict binary execution reward |
| (official GCC 13 oracle in Docker) |
| - Source commit: `7c70b963425d81ce1a3cd344524b87eb055c5852` |
| (tokenbender/browser-is-all-you-need-upstream, branch `client/5-aug-release`) |
|
|
| ## Results |
|
|
| - Training environment pass rate: 33.59% → 55.08% over 20 updates |
| (5,120 executed rollouts, zero infrastructure-invalid samples) |
| - Official bank-account, 64 independent one-shot samples: **60/64 = 93.75%** |
| - Official fixed26, four trials × 26 tasks, `fixed26-contract-v2`, tries=2, |
| thinking enabled, temperature 0.7: |
| - Pass@1: 11, 9, 10, 12 → **mean 10.5/26 (40.38%)** |
| - By turn 2: 14, 17, 12, 15 → **mean 14.5/26 (55.77%)** |
| - bank-account: 4/4 Pass@1 |
| - Fixed26 evidence: `execution-bank-RL-v2-think-r2/fixed26-mt2-4x-20260818/` |
| in this repo (all 104 task receipts; `summary.json` SHA-256 |
| `d324cccff759f33b64bcbb4176a584f53776bcb1c8a1165d935608220c15fcf7`) |
|
|
| ## Contents |
|
|
| - `execution-bank-RL-v2-think-r2/` — full run tree: checkpoints at updates |
| 5/10/15/20 (native TP4 shards + HF adapters + finiteness receipts), rollout |
| dumps, drill dataset, run logs, online-W&B local records, eval64 evidence, |
| fixed26 four-trial evidence, `run_status.txt` |
| - `launch-configs/` — exact SkyPilot training and eval64 launch configurations |
| - `SHA256SUMS` — digest of every file in this repo |
|
|
| ## External evidence |
|
|
| - W&B run: https://wandb.ai/ahm-rimer/execution-bank-RL-v2-think/runs/execution-bank-RL-v2-think-r2 |
| - W&B artifact (GCS-reference bundle): https://wandb.ai/ahm-rimer/execution-bank-RL-v2-think/artifacts/run-bundle/execution-bank-RL-v2-think-r2/v0 |
| - Issue ledger: https://github.com/tokenbender/browser-is-all-you-need/issues/111 |
|
|
| ## Caveats |
|
|
| - The saved `training_state_rank*.pt` files hold optimizer metadata and |
| scheduler state only — no Adam moment tensors — so exact in-place resume is |
| not possible; any continuation must warm-start from the adapter. |
| - The fixed26 evaluation wrapper exited 141 after the completion marker and |
| summary were written (post-completion log-pipe artifact); no trial was |
| affected. |
|
|
