Add README.md

README.md

@@ -1,32 +1,68 @@
 # deepagents-rl
 
-Post-train a small coding LM with **GRPO** + verifiable unit-test rewards, using:
+Train / evaluate a coding **agent** in the **deepagents harness** on long-horizon SWE-bench tasks, with **official Docker evaluation** (not proxy rewards).
 
-- Local **deepagents-style** harness smoke (`scripts/smoke_e2e.py`)
-- **Langfuse** traces (every tool + checkpoint eval)
-- **Weights & Biases** metrics
-- **Hugging Face** adapters (`vanishingradient/deepagents-rl-coding`)
-- **Kaggle T4×2** training (`notebooks/kaggle_grpo.ipynb`)
+## Protocol
 
-## Checkpoint rule
-
-Before every HF checkpoint push: run held-out eval → write **Langfuse** `kaggle-ckpt-eval` → log W&B → then push revision `grpo-step-{n}`.
+1. Load [SWE-bench/SWE-bench_Lite](https://huggingface.co/datasets/SWE-bench/SWE-bench_Lite) (`image`, `eval_script`, `FAIL_TO_PASS`, `PASS_TO_PASS`)
+2. Full `git checkout` of `repo` @ `base_commit` as the agent sandbox
+3. Agent edits via deepagents tools (`ls` / `read_file` / `edit_file` / `execute` / …)
+4. Export `git diff` as `model_patch`
+5. Grade inside the instance Docker image:
+   - apply patch → run official `eval_script` (applies `test_patch`, runs pytest)
+   - parse `>>>>> Start/End Test Output`
+   - **resolved = FULL** iff all FAIL_TO_PASS pass and all PASS_TO_PASS maintained
+6. Record trajectory (messages, tools, patch, Langfuse, W&B, JSONL)
 
 ## Quick start
 
 ```bash
-# local smoke (model on :8792)
-python scripts/smoke_e2e.py
+# Requires Docker Desktop running
+python scripts/verify_swe_docker_grader.py   # gold patch must FULL-resolve
+
+python scripts/prepare_swe_lite_and_push.py  # push metadata to HF
+
+python scripts/run_deepagent_rollouts.py \
+  --split dev --limit 1 --mode tool_loop --log-level DEBUG
+```
+
+Env: `MODEL_BASE_URL`, `MODEL_NAME`, Langfuse + W&B keys, Docker engine up.
+
+## Logging (every step)
 
-# build + push MBPP dataset
-python scripts/prepare_mbpp_and_push.py
+Every stage is logged: repo clone/fetch, each LLM turn, every tool call (args + result), every `docker …` command, patch apply, eval_script, F2P/P2P grade.
 
-# upload trainer to HF for Kaggle download
-python scripts/push_trainer_to_hf.py
+Under `--artifacts` (default `artifacts/swe_lite/`):
+
+| File | Contents |
+|------|----------|
+| `run.log` | Human-readable step log |
+| `events.jsonl` | One JSON event per line (full replay) |
+| `trajectories.jsonl` | Full episode trajectories |
+| `evals/<episode_id>/` | model.patch, eval.sh, test_output.txt, report.json |
+
+Env: `DARL_LOG_LEVEL`, `DARL_LOG_FILE`, `DARL_EVENTS_JSONL`.
+
+## Langfuse trace shape
+
+Each episode is one agent trace with nested children:
+
+```
+deepagent-swe-episode          (agent)
+├── agent-loop                 (span)
+│   ├── turn-0                 (span)
+│   │   ├── llm-turn-0         (generation via CallbackHandler)
+│   │   ├── ls / read_file …   (tool)
+│   │   └── …
+│   ├── turn-1 …
+│   └── …
+└── reward_grader              (evaluator)
+    ├── git_diff_model_patch   (span)
+    └── docker_swe_eval        (span)
 ```
 
-Then open the Kaggle notebook with **GPU T4 x2** and attach secrets — see [`kaggle/KAGGLE_SETUP.md`](kaggle/KAGGLE_SETUP.md).
+Scores on the trace: `pass_frac`, `reward_total`, `resolved`, `n_tool_calls`.
 
-## Dataset
+## Grader
 
-`vanishingradient/deepagents-rl-mbpp` — filtered MBPP sanitized (reference must pass sandbox).
+`src/deepagents_rl/harness/swe_docker_eval.py` — same protocol as the SWE-bench harness (instance image + eval script + F2P/P2P). No file-overlap shortcuts.