YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
AgentArena
Benchmark platform for comparing AI agents on real coding tasks. Agents solve tasks in isolated Docker sandboxes. Results land in a leaderboard with pass@k, cost, and speed metrics.
Status
MVP — core evaluation pipeline implemented, E2E Docker test pending.
Run
cd ~/Projekte/AgentArena
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
uvicorn main:app --reload --port 8420
# → http://localhost:8420
Usage
# Import a task (YAML)
curl -X POST http://localhost:8420/api/tasks/import \
--data-binary @tasks/example_task.yaml
# Start a run
curl -X POST http://localhost:8420/api/runs \
-H "Content-Type: application/json" \
-d '{"task_slug": "example", "agent_name": "claude-code", "k": 1}'
What works
- Leaderboard: pass@1, pass@3, avg_cost, avg_duration, composite_score
- Task detail: best run per agent, run list
- Run detail: trajectory, diff viewer, metrics
- SSE streaming for live run updates
- Task import via YAML
- Adapters:
claude_code,manual
Environment
Copy .env.example to .env and fill in your values.
Docker must be running for sandbox execution.
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support