Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -6,29 +6,58 @@ colorTo: blue
|
|
| 6 |
sdk: static
|
| 7 |
app_file: index.html
|
| 8 |
pinned: false
|
| 9 |
-
short_description: Can AI solve riddles from your culture?
|
| 10 |
---
|
| 11 |
|
| 12 |
-
# Cultural Riddles
|
| 13 |
|
| 14 |
-
|
| 15 |
-
|
| 16 |
-
|
| 17 |
-
|
| 18 |
-
|
|
|
|
|
|
|
|
|
|
| 19 |
|
| 20 |
-
|
| 21 |
|
| 22 |
-
|
| 23 |
-
- **Cultures:** the hardest and easiest communities, every model in every community, and results by country and region.
|
| 24 |
-
- **Answers:** how answers were scored, the kinds of mistakes models make, topics, and how many models solve each riddle.
|
| 25 |
-
- **What helps, what hurts:** thinking, web search, translating into English, language resources and model size.
|
| 26 |
-
- **How the benchmark is built:** writing, checking, answering and scoring.
|
| 27 |
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
|
| 32 |
|
| 33 |
-
|
| 34 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 6 |
sdk: static
|
| 7 |
app_file: index.html
|
| 8 |
pinned: false
|
|
|
|
| 9 |
---
|
| 10 |
|
| 11 |
+
# Cultural Riddles: results dashboard
|
| 12 |
|
| 13 |
+
The dashboard is one static HTML page with Plotly loaded from a CDN. It shows
|
| 14 |
+
which model solved which riddle, the leaderboard with each model's spread across
|
| 15 |
+
communities, a model-by-community grid, a page for each community, geography
|
| 16 |
+
and language resources, how answers were scored, the controlled comparisons,
|
| 17 |
+
and how the benchmark is built. Published pages need an internet connection for Plotly, but no
|
| 18 |
+
Python runtime or application server. The front matter at the top of this file
|
| 19 |
+
configures a static Hugging Face Space when you upload this README and the built
|
| 20 |
+
page together.
|
| 21 |
|
| 22 |
+
## Data and build contract
|
| 23 |
|
| 24 |
+
Run commands from the repository root, using the root uv environment:
|
|
|
|
|
|
|
|
|
|
|
|
|
| 25 |
|
| 26 |
+
```bash
|
| 27 |
+
uv run python -m analysis.export_dashboard
|
| 28 |
+
uv run python -m dashboard.build_static
|
| 29 |
+
```
|
| 30 |
|
| 31 |
+
`analysis.export_dashboard` uses the shared analysis loaders and table functions,
|
| 32 |
+
canonical inputs under `data/canonical/`, and topic summaries under
|
| 33 |
+
`data/derived/`. It writes aggregates only to
|
| 34 |
+
`data/derived/dashboard/dashboard_data.json`: no answers, riddle text, individual
|
| 35 |
+
survey responses, or contributor identities.
|
| 36 |
+
The export also carries fitted local-term odds ratios and confidence intervals
|
| 37 |
+
from the released riddle-feature summary. Translation comparison model labels
|
| 38 |
+
come from the shared English-condition configuration map; the high-resource
|
| 39 |
+
comparison is recomputed for those same configurations. These headline values
|
| 40 |
+
are rendered from JSON, not fixed in the page template.
|
| 41 |
+
|
| 42 |
+
The riddle matrix records, for riddles that every model has a score for, which
|
| 43 |
+
models solved each one, as one bit per riddle and model. It carries each
|
| 44 |
+
column's community and solve count, not riddle IDs or text.
|
| 45 |
+
|
| 46 |
+
`dashboard.compute` loads that JSON and validates relationships between the
|
| 47 |
+
leaderboard, score distributions, model-community grid, per-community topics,
|
| 48 |
+
riddle matrix, coverage, error types, topics, and paired comparisons. It does not pin historical numerical results.
|
| 49 |
+
`dashboard.build_static` uses only that JSON and Python's standard library; it
|
| 50 |
+
does not import the analysis pipeline or read its source tables. It embeds the
|
| 51 |
+
validated aggregates in `template.html` (layout, styles and chart code) and
|
| 52 |
+
writes the ignored build artifact `dashboard/index.html`.
|
| 53 |
+
You can rebuild from an existing JSON export without regenerating analysis.
|
| 54 |
+
|
| 55 |
+
To run the data consistency checks without writing HTML:
|
| 56 |
+
|
| 57 |
+
```bash
|
| 58 |
+
uv run python -m dashboard.compute
|
| 59 |
+
```
|
| 60 |
+
|
| 61 |
+
For prerequisites, deployment, and browser smoke checks, see
|
| 62 |
+
[Rebuild and publish the dashboard](DEPLOY.md). There is no nested Python project
|
| 63 |
+
or lockfile.
|