---
{}
---
# deepagents-rl

Train / evaluate a coding **agent** in the **deepagents harness** on long-horizon SWE-bench tasks, with **official Docker evaluation** (not proxy rewards).

## Protocol

1. Load [SWE-bench/SWE-bench_Lite](https://huggingface.co/datasets/SWE-bench/SWE-bench_Lite) (`image`, `eval_script`, `FAIL_TO_PASS`, `PASS_TO_PASS`)
2. Full `git checkout` of `repo` @ `base_commit` as the agent sandbox
3. Agent edits via deepagents tools (`ls` / `read_file` / `edit_file` / `execute` / …)
4. Export `git diff` as `model_patch`
5. Grade inside the instance Docker image:
   - apply patch → run official `eval_script` (applies `test_patch`, runs pytest)
   - parse `>>>>> Start/End Test Output`
   - **resolved = FULL** iff all FAIL_TO_PASS pass and all PASS_TO_PASS maintained
6. Record trajectory (messages, tools, patch, Langfuse, W&B, JSONL)

## Quick start

```bash
# Requires Docker Desktop running
python scripts/verify_swe_docker_grader.py   # gold patch must FULL-resolve

python scripts/prepare_swe_lite_and_push.py  # push metadata to HF

python scripts/run_deepagent_rollouts.py \
  --split dev --limit 1 --mode tool_loop --log-level DEBUG
```

Env: `MODEL_BASE_URL`, `MODEL_NAME`, Langfuse + W&B keys, Docker engine up.

## Logging (every step)

Every stage is logged: repo clone/fetch, each LLM turn, every tool call (args + result), every `docker …` command, patch apply, eval_script, F2P/P2P grade.

Under `--artifacts` (default `artifacts/swe_lite/`):

| File | Contents |
|------|----------|
| `run.log` | Human-readable step log |
| `events.jsonl` | One JSON event per line (full replay) |
| `trajectories.jsonl` | Full episode trajectories |
| `evals/<episode_id>/` | model.patch, eval.sh, test_output.txt, report.json |

Env: `DARL_LOG_LEVEL`, `DARL_LOG_FILE`, `DARL_EVENTS_JSONL`.

## Langfuse trace shape

Each episode is one agent trace with nested children:

```
deepagent-swe-episode          (agent)
├── agent-loop                 (span)
│   ├── turn-0                 (span)
│   │   ├── llm-turn-0         (generation via CallbackHandler)
│   │   ├── ls / read_file …   (tool)
│   │   └── …
│   ├── turn-1 …
│   └── …
└── reward_grader              (evaluator)
    ├── git_diff_model_patch   (span)
    └── docker_swe_eval        (span)
```

Scores on the trace: `pass_frac`, `reward_total`, `resolved`, `n_tool_calls`.

## Grader

`src/deepagents_rl/harness/swe_docker_eval.py` — same protocol as the SWE-bench harness (instance image + eval script + F2P/P2P). No file-overlap shortcuts.