|
Download README.md from benchgen/README: direct link, hf CLI and curl.
- Browser
- Download file 3.68 kB
-
https://huggingface.co/spaces/benchgen/README/resolve/main/README.md
- Command line
-
hf download hf://spaces/benchgen/README/README.md
-
curl -L -o README.md https://huggingface.co/spaces/benchgen/README/resolve/main/README.md
3.68 kB
| title: Benchgen | |
| emoji: 📊 | |
| colorFrom: yellow | |
| colorTo: red | |
| sdk: static | |
| pinned: false | |
| <p align="center"> | |
| <a href="https://benchgen.com/?utm_source=huggingface&utm_medium=org_card&utm_campaign=org&utm_content=banner"><img src="https://huggingface.co/spaces/benchgen/README/resolve/main/banner.png" alt="Benchgen: the benchmarking infrastructure for AI agents" width="100%"></a> | |
| </p> | |
| <p align="center"> | |
| <a href="https://benchgen.com/?utm_source=huggingface&utm_medium=org_card&utm_campaign=org&utm_content=badge-website"><img src="https://img.shields.io/badge/WEBSITE-benchgen.com-F97316?style=for-the-badge&labelColor=111827" alt="Website: benchgen.com"></a> | |
| <a href="https://benchgen.com/benchmarks/platform/router-fidelity-benchmark?utm_source=huggingface&utm_medium=org_card&utm_campaign=org&utm_content=badge-benchmark"><img src="https://img.shields.io/badge/RUN%20THE%20BENCHMARK-ROUTER%20FIDELITY-F97316?style=for-the-badge&labelColor=111827" alt="Run the benchmark on Benchgen"></a> | |
| <a href="https://benchgen.com/docs?utm_source=huggingface&utm_medium=org_card&utm_campaign=org&utm_content=badge-docs"><img src="https://img.shields.io/badge/DOCS-benchgen.com%2Fdocs-F97316?style=for-the-badge&labelColor=111827" alt="Docs"></a> | |
| </p> | |
| <p align="center"> | |
| <a href="https://github.com/benchgen-ai"><img src="https://img.shields.io/badge/GitHub-benchgen--ai-181717?style=flat&logo=github&logoColor=white" alt="GitHub"></a> | |
| <a href="https://x.com/BenchGenAI"><img src="https://img.shields.io/badge/X-%40BenchGenAI-000000?style=flat&logo=x&logoColor=white" alt="X"></a> | |
| <a href="https://www.linkedin.com/company/benchgen"><img src="https://img.shields.io/badge/LinkedIn-benchgen-0A66C2?style=flat" alt="LinkedIn"></a> | |
| <a href="https://huggingface.co/benchgen"><img src="https://img.shields.io/badge/Hugging%20Face-benchgen-FFD21E?style=flat&logo=huggingface&logoColor=black" alt="Hugging Face"></a> | |
| </p> | |
| **Nobody can prove an AI agent actually works.** Benchgen helps you discover benchmarks, run evaluations on any model or agent, find where and why it fails, and check the fix with the same tests before and after. | |
| ## What we publish here | |
| | Release | What it is | Link | | |
| |---|---|---| | |
| | **Router Lite** (model) | A 10,245-parameter linear router head on a frozen Qwen3-1.7B. Picks which of 5 models should answer a query | [benchgen/benchgen-router-lite](https://huggingface.co/benchgen/benchgen-router-lite) | | |
| | **Router Pilot rewards** (dataset) | Per-question accuracy of 5 agents over 3 attempts (46 tasks) | [benchgen/router-pilot](https://huggingface.co/datasets/benchgen/router-pilot) | | |
| | **Router Pilot tasks** (dataset) | 1,110 normalized questions from public benchmarks, with per-source licences | [benchgen/router-pilot-tasks](https://huggingface.co/datasets/benchgen/router-pilot-tasks) | | |
| Every card states its method, licence, limits and a link to the relevant benchmark on Benchgen. | |
| ## Run it yourself | |
| 1. Open the [Router Fidelity Benchmark](https://benchgen.com/benchmarks/platform/router-fidelity-benchmark?utm_source=huggingface&utm_medium=org_card&utm_campaign=org&utm_content=router-fidelity) (sign-in required to run). | |
| 2. Register your own model or router head, pick it on the Evaluate tab and run. | |
| 3. Compare with our published run. The [Router Lite guide](https://benchgen.com/docs/guides/router-head/benchgen-router-lite?utm_source=huggingface&utm_medium=org_card&utm_campaign=org&utm_content=guide) has the steps. | |
| ## Honest limits | |
| Our releases are pilots. Router Lite was trained on 46 labelled tasks, and its results are directional, not a large-sample benchmark. Each card says exactly what the numbers do and do not show. | |