Spaces:
Running
Running
Update README.md
Browse files
README.md
CHANGED
|
@@ -1,10 +1,42 @@
|
|
| 1 |
---
|
| 2 |
title: README
|
| 3 |
-
emoji:
|
| 4 |
colorFrom: blue
|
| 5 |
-
colorTo:
|
| 6 |
sdk: static
|
| 7 |
pinned: false
|
| 8 |
---
|
| 9 |
|
| 10 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
title: README
|
| 3 |
+
emoji: 馃搳
|
| 4 |
colorFrom: blue
|
| 5 |
+
colorTo: indigo
|
| 6 |
sdk: static
|
| 7 |
pinned: false
|
| 8 |
---
|
| 9 |
|
| 10 |
+
# Quantiles
|
| 11 |
+
|
| 12 |
+
**Open-source, local-first evaluation infrastructure for applied AI systems, designed for developer and coding-agent workflows.**
|
| 13 |
+
|
| 14 |
+
Quantiles gives developers a CLI and Python SDK for creating, running, analyzing, and comparing AI evaluations from their existing development environment. It records metrics, sample-level outputs, execution traces, and run history locally, making it easier to measure system behavior, detect regressions, validate changes, and ship higher-quality, more reliable AI systems.
|
| 15 |
+
|
| 16 |
+
With Quantiles, developers can:
|
| 17 |
+
|
| 18 |
+
- Run built-in benchmarks or create custom evaluations from configuration or Python.
|
| 19 |
+
- Debug individual samples with step-level inputs, outputs, metrics, and errors.
|
| 20 |
+
- Resume interrupted runs and reuse completed steps.
|
| 21 |
+
- Compare models, prompts, datasets, and configurations from the CLI.
|
| 22 |
+
- Run evaluation workflows with coding agents using the Quantiles agent skill.
|
| 23 |
+
|
| 24 |
+
## Quickstart
|
| 25 |
+
|
| 26 |
+
Install the CLI and run SimpleQA Verified:
|
| 27 |
+
|
| 28 |
+
```bash
|
| 29 |
+
curl -fsSL https://cli.quantiles.io/install.sh | bash
|
| 30 |
+
qt run simpleqa-verified
|
| 31 |
+
qt show 1 --json
|
| 32 |
+
```
|
| 33 |
+
|
| 34 |
+
This first run uses the built-in demo model, so it requires no model API key and incurs no inference cost. Use it to verify the workflow, then configure a provider-backed model for a real evaluation.
|
| 35 |
+
|
| 36 |
+
## Quantiles on Hugging Face
|
| 37 |
+
|
| 38 |
+
Follow this organization for technical articles on benchmarking, evaluating, and improving AI systems.
|
| 39 |
+
|
| 40 |
+
[Documentation](https://quantiles.io/documentation) 路 [Quickstart](https://quantiles.io/documentation/quickstart) 路 [GitHub](https://github.com/quantiles-evals/quantiles) 路 [Benchmark Hub](https://quantiles.io/benchmark-hub) 路 [Agent skill](https://github.com/quantiles-evals/skill)
|
| 41 |
+
|
| 42 |
+
Quantiles is available under the [Apache License 2.0](https://github.com/quantiles-evals/quantiles/blob/main/LICENSE).
|