| ---
|
| {}
|
| ---
|
| # deepagents-rl
|
|
|
| Train / evaluate a coding **agent** in the **deepagents harness** on long-horizon SWE-bench tasks, with **official Docker evaluation** (not proxy rewards).
|
|
|
| ## Protocol
|
|
|
| 1. Load [SWE-bench/SWE-bench_Lite](https://huggingface.co/datasets/SWE-bench/SWE-bench_Lite) (`image`, `eval_script`, `FAIL_TO_PASS`, `PASS_TO_PASS`)
|
| 2. Full `git checkout` of `repo` @ `base_commit` as the agent sandbox
|
| 3. Agent edits via deepagents tools (`ls` / `read_file` / `edit_file` / `execute` / …)
|
| 4. Export `git diff` as `model_patch`
|
| 5. Grade inside the instance Docker image:
|
| - apply patch → run official `eval_script` (applies `test_patch`, runs pytest)
|
| - parse `>>>>> Start/End Test Output`
|
| - **resolved = FULL** iff all FAIL_TO_PASS pass and all PASS_TO_PASS maintained
|
| 6. Record trajectory (messages, tools, patch, Langfuse, W&B, JSONL)
|
|
|
| ## Quick start
|
|
|
| ```bash
|
| # Requires Docker Desktop running
|
| python scripts/verify_swe_docker_grader.py # gold patch must FULL-resolve
|
|
|
| python scripts/prepare_swe_lite_and_push.py # push metadata to HF
|
|
|
| python scripts/run_deepagent_rollouts.py \
|
| --split dev --limit 1 --mode tool_loop --log-level DEBUG
|
| ```
|
|
|
| Env: `MODEL_BASE_URL`, `MODEL_NAME`, Langfuse + W&B keys, Docker engine up.
|
|
|
| ## Logging (every step)
|
|
|
| Every stage is logged: repo clone/fetch, each LLM turn, every tool call (args + result), every `docker …` command, patch apply, eval_script, F2P/P2P grade.
|
|
|
| Under `--artifacts` (default `artifacts/swe_lite/`):
|
|
|
| | File | Contents |
|
| |------|----------|
|
| | `run.log` | Human-readable step log |
|
| | `events.jsonl` | One JSON event per line (full replay) |
|
| | `trajectories.jsonl` | Full episode trajectories |
|
| | `evals/<episode_id>/` | model.patch, eval.sh, test_output.txt, report.json |
|
|
|
| Env: `DARL_LOG_LEVEL`, `DARL_LOG_FILE`, `DARL_EVENTS_JSONL`.
|
|
|
| ## Langfuse trace shape
|
|
|
| Each episode is one agent trace with nested children:
|
|
|
| ```
|
| deepagent-swe-episode (agent)
|
| ├── agent-loop (span)
|
| │ ├── turn-0 (span)
|
| │ │ ├── llm-turn-0 (generation via CallbackHandler)
|
| │ │ ├── ls / read_file … (tool)
|
| │ │ └── …
|
| │ ├── turn-1 …
|
| │ └── …
|
| └── reward_grader (evaluator)
|
| ├── git_diff_model_patch (span)
|
| └── docker_swe_eval (span)
|
| ```
|
|
|
| Scores on the trace: `pass_frac`, `reward_total`, `resolved`, `n_tool_calls`.
|
|
|
| ## Grader
|
|
|
| `src/deepagents_rl/harness/swe_docker_eval.py` — same protocol as the SWE-bench harness (instance image + eval script + F2P/P2P). No file-overlap shortcuts.
|
|
|
