File size: 2,696 Bytes
48be033 af71b36 a0141b0 af71b36 a0141b0 af71b36 a0141b0 af71b36 a0141b0 af71b36 a0141b0 af71b36 a0141b0 af71b36 a0141b0 af71b36 a0141b0 af71b36 a0141b0 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 | ---
{}
---
# deepagents-rl
Train / evaluate a coding **agent** in the **deepagents harness** on long-horizon SWE-bench tasks, with **official Docker evaluation** (not proxy rewards).
## Protocol
1. Load [SWE-bench/SWE-bench_Lite](https://huggingface.co/datasets/SWE-bench/SWE-bench_Lite) (`image`, `eval_script`, `FAIL_TO_PASS`, `PASS_TO_PASS`)
2. Full `git checkout` of `repo` @ `base_commit` as the agent sandbox
3. Agent edits via deepagents tools (`ls` / `read_file` / `edit_file` / `execute` / …)
4. Export `git diff` as `model_patch`
5. Grade inside the instance Docker image:
- apply patch → run official `eval_script` (applies `test_patch`, runs pytest)
- parse `>>>>> Start/End Test Output`
- **resolved = FULL** iff all FAIL_TO_PASS pass and all PASS_TO_PASS maintained
6. Record trajectory (messages, tools, patch, Langfuse, W&B, JSONL)
## Quick start
```bash
# Requires Docker Desktop running
python scripts/verify_swe_docker_grader.py # gold patch must FULL-resolve
python scripts/prepare_swe_lite_and_push.py # push metadata to HF
python scripts/run_deepagent_rollouts.py \
--split dev --limit 1 --mode tool_loop --log-level DEBUG
```
Env: `MODEL_BASE_URL`, `MODEL_NAME`, Langfuse + W&B keys, Docker engine up.
## Logging (every step)
Every stage is logged: repo clone/fetch, each LLM turn, every tool call (args + result), every `docker …` command, patch apply, eval_script, F2P/P2P grade.
Under `--artifacts` (default `artifacts/swe_lite/`):
| File | Contents |
|------|----------|
| `run.log` | Human-readable step log |
| `events.jsonl` | One JSON event per line (full replay) |
| `trajectories.jsonl` | Full episode trajectories |
| `evals/<episode_id>/` | model.patch, eval.sh, test_output.txt, report.json |
Env: `DARL_LOG_LEVEL`, `DARL_LOG_FILE`, `DARL_EVENTS_JSONL`.
## Langfuse trace shape
Each episode is one agent trace with nested children:
```
deepagent-swe-episode (agent)
├── agent-loop (span)
│ ├── turn-0 (span)
│ │ ├── llm-turn-0 (generation via CallbackHandler)
│ │ ├── ls / read_file … (tool)
│ │ └── …
│ ├── turn-1 …
│ └── …
└── reward_grader (evaluator)
├── git_diff_model_patch (span)
└── docker_swe_eval (span)
```
Scores on the trace: `pass_frac`, `reward_total`, `resolved`, `n_tool_calls`.
## Grader
`src/deepagents_rl/harness/swe_docker_eval.py` — same protocol as the SWE-bench harness (instance image + eval script + F2P/P2P). No file-overlap shortcuts.
|