README.md

---
title: DarwinX
emoji: 🧬
colorFrom: blue
colorTo: indigo
sdk: static
app_file: index.html
pinned: false
license: cc-by-4.0
short_description: Evolving Agent Harnesses Through Natural Selection
---

# DarwinX: Evolving Agent Harnesses Through Natural Selection

Project page for [arXiv:2608.07545](https://arxiv.org/abs/2608.07545).
Paper page: [huggingface.co/papers/2608.07545](https://huggingface.co/papers/2608.07545).
Official DarwinX implementation (in Beagle): [github.com/SalesforceAIResearch/Beagle](https://github.com/SalesforceAIResearch/Beagle).

DarwinX treats agent self-evolution as selection over a *population* of harnesses with the base
model frozen. Across four benchmarks, one loop adds ~17 points on average:

| Benchmark | Result | Gain | What it isolates |
| --- | --- | --- | --- |
| Terminal-Bench 2.1 (avg@5) | 84.7% | +7.7 on matched base | in-domain evolution |
| TerminalWorld (held-out) | 68.3% | +7.3 | disjoint held-out task split |
| WebArena-Infinity (audit-clean pass@1) | 93.0% | +49.5 pp | synthetic → real intent shift |
| SWE-bench Verified (zero-shot transfer) | 84.2% | +3.4 | cross-benchmark transfer |

## Page sections

`Demo` · `Abstract` · `Method` · `Results at a glance` · `Benchmark detail` · `Ablation: what
evolution changes` · `Limitations` · `BibTeX`

The demo is a 17-second animated walkthrough of the loop (`assets/demo.mp4`, 1080p H.264, 2.9 MB,
faststart so it streams rather than buffering). It sits under the teaser, autoplays muted on loop,
and carries narration for anyone who unmutes. It was re-encoded from a 2560×1440 master, which is
kept out of the Space: at 9.99 MiB the master is within kilobytes of Hugging Face's 10 MiB limit for
files not tracked with LFS. To rebuild it from a new master:

```
ffmpeg -i master.mp4 -vf "scale=1920:-2:flags=lanczos" \
  -c:v libx264 -profile:v high -level 4.0 -pix_fmt yuv420p -crf 23 -preset slow \
  -c:a aac -b:a 128k -ac 2 -movflags +faststart assets/demo.mp4
ffmpeg -i master.mp4 -vf "thumbnail=48,scale=1920:-2" -frames:v 1 -q:v 3 assets/demo_poster.jpg
```

The ablation section reproduces the paper's framing: it is an *exploratory attribution, not a
per-skill causal ablation*, since the skills were co-selected rather than independently randomized.
The limitations section is carried over from the paper's Discussion.

## Interactive figures

Eight figures are interactive, and each is driven by real run artifacts or the paper's own plotting
scripts rather than illustrative numbers. `tools/build_data.py` regenerates `assets/data.js` from the
original files, so nothing is hand-transcribed.

**1. TerminalWorld merge explorer.** Toggle any subset of the four evolved specialists and watch
which of the 41 held-out tasks the selection covers. Built from the `per_task_results` arrays of the
five evaluation runs, joined on `task_id` (the run files list the tasks in different orders, so
position-based joining would silently misalign them).

It surfaces something the paper states only in aggregate. The four specialists solve 24/25/26/27
tasks and their **union is 29**, but the harness recombination actually produced solves **28**:

| | tasks |
| --- | --- |
| best single specialist (D) | 27/41 |
| union of all four | 29/41 |
| realized merge | 28/41 |
| solved by merge, by no specialist | `tw_448247` |
| solved by some specialist, not by merge | `tw_449421`, `tw_498533` |
| solved by all four | 21/41 |
| solved by none | 12/41 |

So recombination is not a free set union: it adds a capability no parent had and loses two. These are
single attempts and one task is worth 2.4 points, so individual flips sit inside the noise band —
the page says so next to the widget.

**2. TB2.1 cluster explorer.** Per-cluster base vs. evolved avg@5, sortable by gain, base rate,
cluster size, or name, with exact rates on hover. Numbers from `notes/TB21_RESULTS.md` (paired
protocol, 88 tasks). Deltas are carried over as reported rather than recomputed, because the source
rounds them from unrounded rates.

**3. WebArena-Infinity evolution curve.** All 37 evaluated variants with a best-so-far envelope,
from `notes/tw_dynamics.json` → `wai_adaptive_scores`. Toggle either series.

**4. Headline four-benchmark panels.** Replaces the right half of the teaser, and sits directly under
it. Clicking the schematic expands it in place to the paper's full figure, so while expanded the
paper's static bars and this interactive version of the same chart are both on screen. From
`scripts/gen_summary_figure.py`. The reason to make this one interactive is that the paper's figure
has to *state* its truncation in the caption: on a shared axis WebArena-Infinity's +49.5 flattens the
two terminal benchmarks into slivers, so each panel is scaled to its own range. Here the reader can
switch between per-panel and shared 0–100 axes and see both framings. The best-prior-agent bar can be
hidden, since those systems use different models and effort settings and are context rather than a
controlled comparison. Per-panel labels carry the exceptions: on SWE-bench Verified the grey bar is
the fix-skill reference, not an unevolved Monet, and there is no prior-agent bar.

**5. TerminalWorld specialist bars.** From `notes/tw_dynamics.json` → `tw_heldout`. Same axis
question in miniature: the six arms span 58.5–68.3, so a 0–100 track renders them near-identical.
Defaults to a 55–70 axis with the full axis one click away and the truncation named in both notes.
Spec. D and the Claude Code reference are both 27/41, so D's bar landing exactly on the dashed
reference line is a built-in check that the axis transform is right.

**6. TB2.1 compute.** From `scripts/gen_tb21_compute.py` (medians over clean attempts). Switch
between turns and tokens; the bars share one scale across both task groups so the newly-solved vs.
already-solved contrast is not rescaled away.

**7. WAI invalid-trajectory composition.** From `scripts/gen_wai_invalid_composition.py`. Break the
293 → 17 collapse down by application or by mechanism. Both rows sit on a shared absolute scale by
default, which makes the "after" row a near-invisible sliver — that *is* the finding; normalizing
each row to its own total then shows what the remainder consists of. `build_data.py` asserts both
decompositions still total 293 and 17.

**8. WAI audit dumbbells.** From `scripts/gen_wai_audit_by_app.py`. One line per application from raw
pass@1 to audited pass@1, so line length reads directly as how much a harness was leaning on
trajectories the audit rejects. Sortable by audit loss. `build_data.py` cross-checks the audited
DarwinX column against the per-application table rendered elsewhere on the page.

Three figures stay static because they are conceptual diagrams with no underlying data: the teaser's
selection schematic, the method overview, and the per-generation operators. The archive lineage tree
also stays static for a different reason — its node/edge data lives in a `state.db` on the cluster,
not on this machine, so there is nothing truthful to make interactive yet.

Tables are click-to-sort. Numeric columns open descending, text columns A–Z, and `Overall` rows stay
pinned to the bottom.

## Local preview

```bash
python3 -m http.server 8000
# open http://localhost:8000
```

## Regenerate and test

```bash
python3 tools/build_data.py          # rebuild assets/data.js from the run artifacts
# then, with the server running, open:
#   http://localhost:8000/tools/interaction_test.html
# it drives every widget with real click events and writes PASS/FAIL into the page title
```

Headless run:

```bash
python3 -m http.server 8000 &
"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" --headless --disable-gpu \
  --virtual-time-budget=6000 --dump-dom http://localhost:8000/tools/interaction_test.html \
  | grep -o '<title>.*</title>'
```

## Layout

```
index.html                          the page: inline CSS + vanilla JS, no build step, no external
                                    requests, no web fonts, no chart library
assets/data.js                      generated — real numbers behind the interactive figures
assets/*.png                        web-sized figures (max 2000px wide, 256-color) — 736 KB total
assets/full/*.png                   full-resolution originals for the three dense diagrams,
                                    linked from the page via click-to-zoom
tools/build_data.py                 regenerates assets/data.js from the original run artifacts
tools/interaction_test.html         click-driven test suite for the interactive figures (49 checks)
```

| Figure | Content |
| --- | --- |
| `teaser.png` | four-benchmark teaser — the no-JavaScript fallback for the headline panels |
| `teaser_schematic.png` | teaser's left half — the selection schematic shown on the page |
| `full/teaser.png` | the paper's full teaser, swapped in place when the schematic is clicked |
| `demo.mp4`, `demo_poster.jpg` | 17-second walkthrough of the loop, and its poster frame |
| `overview.png` | method overview |
| `population.png` | per-generation operators (mutation, classification, merge) |
| `tb21_category_gain.png` | per-cluster TB2.1 gain |
| `tb21_compute.png` | per-task compute: newly-solved vs already-solved |
| `tw_heldout_specialists.png` | held-out generalization vs archive diversity |
| `wai_evolution_curve.png` | best-so-far screening score |
| `wai_tree.png` | archive lineage tree |
| `wai_invalid_composition.png` | invalid trajectories, 293 → 17 |
| `wai_audit_by_app.png` | raw vs audited pass@1 per application |

Result figures are rendered from the paper's PDF sources at 200 DPI, then palette-reduced without
dithering (RMSE vs. truecolor < 0.5%, which is imperceptible for line art but 4.7x smaller).

## Hosting

The frontmatter above configures this directory as a Hugging Face **static Space**. The same files
work unchanged as a GitHub Pages site or behind any static file server.

One thing to change when hosting: `og:image` and `twitter:image` are relative paths, which most
crawlers resolve but X/Twitter wants absolute. Replace them with the deployed absolute URL to get a
large social card.