| # AgentArena |
|
|
| Benchmark platform for comparing AI agents on real coding tasks. |
| Agents solve tasks in isolated Docker sandboxes. Results land in a leaderboard |
| with pass@k, cost, and speed metrics. |
|
|
| ## Status |
|
|
| MVP — core evaluation pipeline implemented, E2E Docker test pending. |
|
|
| ## Run |
|
|
| ```bash |
| cd ~/Projekte/AgentArena |
| python3 -m venv .venv && source .venv/bin/activate |
| pip install -r requirements.txt |
| uvicorn main:app --reload --port 8420 |
| # → http://localhost:8420 |
| ``` |
|
|
| ## Usage |
|
|
| ```bash |
| # Import a task (YAML) |
| curl -X POST http://localhost:8420/api/tasks/import \ |
| --data-binary @tasks/example_task.yaml |
| |
| # Start a run |
| curl -X POST http://localhost:8420/api/runs \ |
| -H "Content-Type: application/json" \ |
| -d '{"task_slug": "example", "agent_name": "claude-code", "k": 1}' |
| ``` |
|
|
| ## What works |
|
|
| - Leaderboard: pass@1, pass@3, avg_cost, avg_duration, composite_score |
| - Task detail: best run per agent, run list |
| - Run detail: trajectory, diff viewer, metrics |
| - SSE streaming for live run updates |
| - Task import via YAML |
| - Adapters: `claude_code`, `manual` |
|
|
| ## Environment |
|
|
| Copy `.env.example` to `.env` and fill in your values. |
| Docker must be running for sandbox execution. |
|
|
