| --- |
| title: Segmentation Eval Dashboard |
| emoji: 📊 |
| colorFrom: blue |
| colorTo: green |
| sdk: docker |
| app_port: 7860 |
| pinned: false |
| --- |
| |
| # Segmentation Evaluation Metrics Dashboard |
|
|
| Streamlit dashboard for tracking segmentation evaluation metrics stored in Hugging Face **dataset cards** under a Markdown section titled exactly: |
|
|
| `### Performance Metrics` |
|
|
| The app discovers dataset repos (typically with `eval` in the repo id), parses the metrics table, and builds a historical view from commit revisions when possible. |
|
|
| ## Features |
|
|
| - Dataset discovery from Hugging Face Hub (`huggingface_hub` API) |
| - Allowlist/prefix-based repo filtering via environment variables |
| - Robust Markdown section/table parsing for `Performance Metrics` |
| - Historical trend extraction using dataset commit history |
| - Filters by repo, class, and metric |
| - Trend chart with optional standard-deviation bands |
| - Latest snapshot table |
| - Repo-to-repo comparison chart on latest runs |
| - Optional per-class small multiples |
| - CSV export for filtered data |
| - Health/status panel (parsed vs failed repos) |
| - Auto-refresh and cached Hub calls |
|
|
| ## Project Structure |
|
|
| - `app.py`: Streamlit entrypoint and UI |
| - `config.py`: env-based configuration |
| - `hf_client.py`: Hugging Face Hub API access and README retrieval |
| - `parsing.py`: Markdown section/table extraction |
| - `transforms.py`: data normalization, history shaping, latest snapshots |
| - `charts.py`: Altair chart builders |
| - `requirements.txt`: Python dependencies |
|
|
| ## Required Environment Variables |
|
|
| - `HF_TOKEN` (optional, recommended for private repos or higher rate limits) |
| - `HF_OWNER` (optional namespace/organization filter, e.g. `my-org`) |
|
|
| ## Optional Environment Variables |
|
|
| - `EVAL_REPO_ALLOWLIST` (comma-separated dataset ids) |
| - `EVAL_REPO_PREFIXES` (comma-separated prefixes, e.g. `my-org/`) |
| - `EVAL_REPO_ID_CONTAINS` (default: `eval`) |
| - `EVAL_DISCOVERY_ENABLED` (`true`/`false`, default: `true`) |
| - `EVAL_MAX_REPOS` (default: `200`) |
| - `EVAL_INCLUDE_HISTORY` (`true`/`false`, default: `true`) |
| - `EVAL_MAX_COMMITS_PER_REPO` (default: `20`) |
| - `EVAL_METRICS_SECTION_HEADING` (default: `Performance Metrics`) |
| - `EVAL_CACHE_TTL_SECONDS` (default: `600`) |
| - `EVAL_AUTO_REFRESH_SECONDS` (default: `300`) |
| - `EVAL_MAX_LOAD_SECONDS` (default: `90`) |
|
|
| ## Local Run |
|
|
| ```bash |
| pip install -r requirements.txt |
| streamlit run app.py |
| ``` |
|
|
| ## Deploy to Hugging Face Space |
|
|
| 1. Create a new **Space** with SDK = **Streamlit**. |
| 2. Copy all files in this folder to the Space repository root. |
| 3. In Space settings, set secrets/variables as needed: |
| - `HF_TOKEN` (as a secret) |
| - any optional `EVAL_*` vars for discovery behavior |
| 4. Push to the Space. |
| 5. The app will start from `app.py`. |
|
|
| ## Notes / Limitations |
|
|
| - History quality depends on README/table availability across commits. |
| - If commit history is unavailable or inaccessible, the app still supports latest snapshot parsing. |
| - Table parsing is tolerant to minor Markdown inconsistencies, but severely malformed tables are skipped. |
| - Missing numeric values (`nan`, `NA`, empty) are treated as `NaN` and handled safely in charts/tables. |
|
|
