Spaces:
Runtime error
Runtime error
Commit History
-
Feature: quality benchmark suite + CI regression test
Adds a full testing pipeline for measuring and tracking extraction quality across all models: - tests/create_fixture.py — deterministic sampler: 50 posts from saved_posts.json (30 food candidates + 20 non-food), diversified by creator and hashtag set, with guaranteed edge-case coverage (thin captions, no hashtags) - tests/fixtures/sampled_posts.json — the 50-post fixture (checked in, seed=42) - tests/generate_ground_truth.py — runs sampled posts through Claude Opus 4.8 to produce the authoritative oracle; checkpointed so it's resumable - tests/fixtures/test_posts.json — 50 Opus 4.8–labelled ground truth records (30 food / 20 not-food); this is the permanent benchmark oracle - tests/benchmark.py — CLI runner that evaluates any model/provider against the oracle. Metrics: is_food_accuracy (40%), name_recall fuzzy (25%), city_recall (15%), field_completeness (20%) → composite_score. Writes tests/benchmark_results.json. First result: claude-haiku-4-5 scores 89.1 at $0.055…a42a323