Spaces:
Runtime error
Runtime error
Commit History
-
Feat: capture inbox (Phase B) — phone captures merge into the library
/capture with a token enqueues place(s) into an ephemeral, TTL'd in-memory inbox. New POST /capture-inbox/{token} drains a token's captures and merges them into the caller's library (reuses the living-library _merge_rows) → returns a fresh job like /import. Client _pullCaptureInbox() drains on load and shows "📲 N places captured from your phone" with ✨ New badges; a #capture-setup card in the import card surfaces the token (localStorage). Fixes an empty-multipart-body 400 by always sending merge_into (even ""). tests/test_capture.py extended (enqueue, drain+merge, idempotency, status preserved); visual review of the setup card + full round-trip at mobile/desktop. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>eda6c91 -
Feat: /capture endpoint core (Phase A) — single-post extraction
Refactors _run_url_pipeline's extract core into shared _places_from_caption() (analyze_batch → split_multi_venue → location-tag coords). New POST /capture accepts embed_html (the iOS Shortcut fetches /embed/captioned/ on-device so the server never touches Instagram), or a ready caption, or a url (local fetch), plus optional lat/lng for a free location-tag pin. Returns {places, count}. BYOK api_key. tests/test_capture.py added to CI; e2e still green (refactor safe). Inbox delivery (Phase B) and the Shortcut itself (Phase C) are next. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>8221e29 -
Feat: living-library merge polish (Phase 3)
- /import returns added_urls + kept; merge_rows now returns the added rows. - Client tags freshly-merged places (row._isNew) → a "✨ New" card badge, floats them to the top of Browse (stable, transient — resets on reload). - Merge banner reports "N new · M already in your library". - Import card gains a Merge/Replace radio (#import-mode), shown only when a cached library exists; a zero-overlap merge still offers Replace inline. Completes the living-library feature (Phases 1–3). Visual review at mobile + desktop; tests/test_merge.py extended (added_urls/kept) and green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>f981d71 -
Feat: living-library cost-skip on re-extract (Phase 2)
POST /upload now accepts merge_into (the cached library CSV). _run_pipeline seeds it as the job's places_full.csv and runs extract.run(resume=True), so _warm_start_from_csv skips the LLM on posts already in the library (keyed on instagram_url). new_urls excludes known posts, so the cost estimate and progress message reflect only genuinely new ones ("X already in your library, skipped"). After extraction the result is folded back with _merge_rows (keep-existing-wins) so known posts keep status/coords and any place the new export omitted is re-added. Client sends the cached library on the Extract upload. tests/test_merge.py gains a warm-start section that patches extract.analyze_batch to prove known posts never reach the LLM and keep their visited status. A 600-post re-export where 550 are already mapped now only pays to extract the 50 new ones. Phase 3 (polish) is next. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>1a26356 -
Feat: living-library merge on import (Phase 1)
POST /import now accepts an optional merge_into field (the caller's existing library CSV). The uploaded file is folded into it server-side via _merge_rows, keyed on instagram_url (else normalized name|city), keep-existing-wins — so user-owned status and manual edits are never overwritten and only genuinely new posts are added. Response gains added/merged/no_overlap. Client (importPlacesFile) sends the cached localStorage library as merge_into (unless restoring, which forceReplaces), shows an "N new places added" banner, and offers Replace when nothing in the file matched the library. Also fixes the restore card lingering after a library loads (showSubTabBar now hides it). This is the linchpin of the hybrid plan — it's what lets a capture/Shortcut or a re-import grow one living map instead of replacing it. Pinned by tests/test_merge.py (added to CI guardrails). Phase 2 (server-side warm-start cost-skip for bulk re-export) is next. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.c…d1b556d -
Feat: client-side library persistence (localStorage restore card)
Cache the canonical CSV in the browser after load and write-through on every edit (status, fields, coords). A returning visitor with no live job sees a #restore-card that replays the cached CSV through /import — a pure reconstruction-cache, so status/coords/edits round-trip losslessly with no new data shape and no server change. Stored only on-device; Download stays the durable backup. Works in both deployment modes (no HOSTED branch). - app.js: importPlacesFile() refactored out of handleImportFile; cacheLibrary/ cacheLibrarySoon/maybeShowRestore + restore/forget handlers; quota note - index.html + app.css: #restore-card above the import card - tests/test_persistence.py: cache/restore round-trip is lossless (CI guardrail) - docs: README, CLAUDE.md, TODO (item marked done; IndexedDB deferred) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>1c43cdb
-
Test: export.build_kml coordinate invariants (regression for coord-less KML)
Five new checks in test_pipeline.py guard against places_map.kml being generated without <Point><coordinates> elements when geocoded rows exist: build_kml writes coords, omits Point for ungeocoded rows, correct pinned count in mixed batches, full export→import round-trip, and export.run() end-to-end via a temp CSV. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>1fec3cb
-
Feat: multi-venue extraction for transcription path + tests
Harshith Belagur +1: ClaudeA single reel can name several places ("my top 3 spots in Rome"). The transcription path previously kept only the first venue because transcribe.run() used analyze_post() (returns results[0]). - extract.py: add analyze_post_all() — returns every venue found (empty list for non-places). analyze_post() keeps its single-result contract for the benchmark suite. - transcribe.py: run() now extracts all venues; the per-URL checkpoint stores a list and the resume path stays back-compatible with the legacy single-dict format. - tests: add the four TODO test cases — multi-venue parse, person-handle exclusion, analyze_post_all, and the full transcription path (test_pipeline.py); plus /extract-url route tests for 2-venue, 5-venue, and person-handle (test_e2e.py). Closes the "Test cases needed" item under the single-URL feature in TODO. Co-Authored-By: Claude <noreply@anthropic.com>291da19 -
Fix: URL fetch works on hosted via Instagram's embed/captioned endpoint
Uses /embed/captioned/ (statically served for website embeds — works from any server IP, no proxy needed) as the primary caption source. The proxy path (CAPTION_PROXY_URL) is kept as an override but is no longer needed or set by default. yt-dlp remains as a local fallback that also captures geotag data. Adds 3 new unit tests for _parse_embed_caption (basic, empty page, creator fallback) bringing pipeline checks to 101. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>495ba69 -
Test: add pipeline tests for multi-venue extraction and URL fetch helpers
17 new checks in test_pipeline.py covering: - _parse_batch_response: None guard, empty string, single place, not-place, multi-venue array for single post (must not trim), batch trim, padding, markdown fences - split_multi_venue: single row passthrough, two separate venue rows - _parse_og_tags: description/title extraction, content-first ordering, empty page resilience - _fetch_via_proxy: empty contents raises ValueError cleanly TODO.md: added test cases for multi-venue URL posts (multiple @handles, multi-place video transcription, person vs venue handle disambiguation). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>82a7965
-
Fix KML round-trip: embed City/State in description CDATA; read them back on import
Previously the KML export stored city only in the folder hierarchy (country → city → placemark), but the actual export uses flat country-only folders — so city and state were silently dropped on re-import, leaving all rows with city=UNKNOWN. - export.py: write "City: {city}" and "State: {state}" lines into the Placemark description CDATA so they survive re-import regardless of folder structure - web/app.py _parse_import_kml: parse "City:" and "State:" from CDATA; CDATA values take priority, folder-derived city used as fallback - test_pipeline.py: update KML fixture to flat country folder + CDATA city, add explicit city/state CDATA checks, add folder-city fallback test Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>8a1e3cd -
UX: drop Exports tab, auto-spin Roulette on first visit
- Remove the Exports sub-tab entirely — it was a one-shot action poorly placed as a dedicated tab. Download buttons (KML / GeoJSON / CSV) and the Google My Maps instructions now live in a #download-bar at the top of Browse, revealed when showSubTabBar() fires. - Roulette now auto-spins on first visit: showSubTab('roulette') triggers spin-btn.click() if #pick-card is still hidden. Subsequent tab switches don't re-spin (card is already visible). - Clean up orphaned success-block / success-ring CSS (no longer used). - Update test_routes.py: assert no exports tab, assert download-bar is inside the explore panel. - CLAUDE.md + README.md updated to reflect two-sub-tab nav. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>36362ca
-
Tests: add test_pipeline.py + test_routes.py; consolidate TASKS into TODO
- tests/test_pipeline.py — unit tests for geocoder helpers (_country_code, _norm_city, _city_matches, public_bulk_blocked), prefilter skip/override logic, chatbot _extract_json_array tolerant parsing, parse_response/ parse_responses, _parse_import_csv/_parse_import_kml edge cases, and a JS↔Python FOOD_CATEGORIES sync invariant (87 checks, no network needed) - tests/test_routes.py — route-level tests via TestClient covering /roulette food_only + city + status filters, PATCH field edits, 413/422 error paths, CSV download, and HTML nav structure (two-tier top-nav + sub-tab-bar) - Remove TASKS.md — open items already captured in TODO.md; adds clutter - CLAUDE.md + README.md updated to document new test files and quick commands Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>24e6b37 -
Web UI: tab-bar SPA, KML/CSV import, refactor index.html
Three-tab SPA (Exports | Roulette | Browse) replaces the post-extraction long scroll. Roulette is the default tab after extraction (with nudge toast); Browse is the default after import. Tab badges show unvisited/total counts. KML/CSV import card (always visible before extraction) lets returning users skip re-extraction entirely — POST /import parses places_map.kml or places_full.csv and returns a job immediately at step "done". Refactored index.html: CSS → web/static/app.css, JS → web/static/app.js. Template drops from 2553 → 465 lines; Jinja vars passed via body data-* attributes. FastAPI serves static/ via StaticFiles mount. All 24 e2e tests pass; deployment-mode and prefilter guardrails unchanged. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>74aafdd -
Fix: KML invalid-XML chars rejected by Google My Maps
Caption/name data from Instagram can contain control characters that xml.sax escape() leaves intact, so the KML failed to parse on import ("The file contains invalid or unsupported data"). Add _xml_safe() to strip characters not permitted in XML 1.0 and apply it to the final KML. Guards: test_deployment_modes asserts build_kml stays valid XML with a control char + '&' in the URL; the e2e now ET.fromstring()-validates the downloaded KML. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>7178492 -
Fix: chatbot prompt no longer hardcodes a (wrong) post count
The prompt said "Include ALL {post_count} posts", where post_count was the post-PREFILTER export size — confusing (it's not the user's total saved count) and outright wrong when a large library is split into multiple chunk files (the shared prompt named the total while each file held <=350). Reworded to "include every post from the attached file, in the same order"; get_prompt now ignores its count arg. Updated the regression guard accordingly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>2d58180 -
Add end-to-end pipeline test through the real routes (CI)
tests/test_e2e.py drives the actual FastAPI routes — GET /, upload → extract → geocode → export → results → KML/GeoJSON download, and the chatbot prepare → process flow — with ONLY the two third-party boundaries mocked (the LLM analyze_batch and the Nominatim geocode_row). Everything else runs for real. This covers the integration gaps that let two 500s reach production (the TemplateResponse signature and the chatbot get_prompt KeyError). Wired into the guardrails CI job (installs httpx for Starlette's TestClient). CLAUDE.md lists it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>925baf1 -
Fix: chatbot get_prompt KeyError (500 on Prepare extraction file)
get_prompt did _PROMPT.format(post_count=...), but _PROMPT contains literal JSON braces ({"post_number": ...}) as examples, so str.format treated them as fields and raised KeyError: '"post_number"' — 500ing the /chatbot-prepare endpoint (the no-key hosted provider). Use .replace("{post_count}", ...) so the JSON examples are left intact. Add a guardrail-test regression check (get_prompt substitutes the count, keeps the JSON example, no leftover token). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>c90b7f4 -
#19: public-URL hardening
For a public Space: MAX_UPLOAD_MB (25) rejects oversized uploads on /upload and /chatbot-prepare (raw + post-unzip, guarding zip-bombs); MAX_POSTS (5000) caps library size with a "use Docker/CLI locally" message; safe_err() redacts sk-ant-… keys from error messages/SSE. Guardrail test covers safe_err; CLAUDE.md env list updated. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>8ec40f3 -
#18: extend guardrail test for client-geocode + set-coords
Extract valid_latlng() from /set-coords (pure) and add a client-geocode section to tests/test_deployment_modes.py: client-geocode ON when the server is bulk-blocked (hosted) / OFF for local Docker, and valid_latlng accepts valid coords while rejecting non-numeric and out-of-range. Runs in the guardrails CI job. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>50e43d1 -
Demarcate the two deployment modes + add guardrails, CLAUDE.md
The app ships in two web shapes over one codebase (local Docker vs hosted free web), differentiated only by env config. Make each mode's contract explicit and guard against a change to one breaking the other. Guardrails: - Extract pure helpers encoding each contract: web.app.resolve_provider() (hosted hides Ollama -> Claude; local honors it) and pipeline.geocode.public_bulk_blocked() (HOSTED=1 + public OSM refused; unset/self-hosted allowed). Behavior unchanged; now unit-testable. - tests/test_deployment_modes.py pins both columns (hosted vs local). - CI: add a fast `guardrails` job (no Ollama) running the deployment-mode + prefilter tests on every push. CI fix: benchmark.yml set working-directory/artifact path to a nonexistent nested `your_instagram_activity/saved` folder (repo root IS that folder), so the existing benchmark job could never run. Fixed both. Docs: - DEPLOY.md — the two options side-by-side + the guardrail that protects them. - deploy/hosted.env.example — hos…c68fdea -
Benchmark: regenerate oracle with is_place labels (fair model comparison)
The oracle used a food-only is_food flag while extraction is generalized to is_place, which penalized more-capable models for correctly flagging non-food places. Regenerate the oracle and align the scorer: - generate_ground_truth.py: emit is_place (+ category) instead of is_food. - benchmark.py: score is_place_accuracy; rename weight/metric/print header. - test_prefilter.py: validate "zero false negatives" against is_place (the correct invariant — no real *place* should be skipped, incl. hikes/shops). - Oracle re-labelled by Opus 4.8: now 33 places / 17 non-places (was 30/20); three hotel/outdoor/shop posts previously mislabelled non-food are now places. Re-benchmarked on the fair oracle: claude-opus-4-8 92 (is_place 100, name 94.2, city 87.9, complete 73.9) claude-haiku-4-5 81 (is_place 96, name 73.4, city 66.7, complete 71.2) Opus is now correctly on top (was artificially below Haiku). README quality table + caveat updated; the caveat now notes Opus labels the oracle…125cd4e -
Benchmark: restore analyze_post, run Opus, persist results as CI baseline
- Fix broken benchmark: tests/benchmark.py imported analyze_post, which was lost when extract.py moved to batched extraction. Re-add it as a thin single-post wrapper over analyze_batch. - Run claude-opus-4-8 against the 50-post oracle: composite 80.1 (is_food 76, name 83.2, city 80.0, completeness 84.6), cost ~$0.28/100. - README quality table: add Opus row + a caveat explaining why it scores below Haiku (food-only oracle vs generalized is_place extraction — an artifact, not lower quality; the oracle should be regenerated with is_place labels). - Track tests/benchmark_results.json instead of gitignoring it, so the "stored baseline" the README/CI refer to actually exists in the repo. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>826360c
-
Chore: remove temp task-planning files from tests/
TASKS.md and TASKS_RESULTS_BROWSER.md were session-scoped planning documents, not part of the test suite. Work they tracked is complete and in the commit history. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>9c2ec1f -
Feature: file upload validation + keyword pre-filter before LLM calls
File validation (web UI): - Run button stays disabled until FileReader confirms the JSON is valid - Structural checks: must be non-empty array with label_values/timestamp fields - Specific error messages per failure case (wrong format, empty, wrong file) - Drop zone turns red/green to reflect validity; preview text colour matches Pre-filter (pipeline/prefilter.py): - Keyword/regex gate screens obvious non-food posts before any LLM call - Catches fitness, fashion, travel scenery, product ads, pet posts - Conservative: food signals (brewery, cafe, ramen…) always override skip - Zero false negatives verified against the 50-post Opus 4.8 oracle - 30% of non-food oracle posts caught upfront; rest reach the model Integration: - estimate_cost() subtracts pre-filtered posts from token count - Web SSE extraction message shows "(N pre-filtered, no LLM call)" - CLI print distinguishes LLM-skipped vs pre-filtered at end of run - tests/test_prefilter.py: runnable validation script, exits 1 on fal…96b9134 -
Feature: in-app results browser (R2–R4 backend + R6–R11 frontend)
Backend: R2 — PATCH /results/{job_id}: inline field editing. Accepts {url, updates} with a whitelist of editable fields (name/city/state/country/address/cuisine/ price_range/highlight/occasion). Writes changes back to the CSV immediately. Returns the updated row dict. Non-whitelisted fields return 422. R3 — POST /regeocode/{job_id}: re-geocode a single restaurant row. Reads the current CSV row by instagram_url, calls geocode_mod.geocode_one(), updates lat/lng in-place, writes back, returns {url, lat, lng, geocoded}. R4 — POST /export/{job_id}: re-generate the KML from the current CSV state using asyncio.to_thread so the event loop isn't blocked. Lets the UI rebuild the download after inline edits without re-running AI. Shared: added UpdateRequest/RegeocodeRequest Pydantic models, _EDITABLE_FIELDS whitelist, _read_csv/_write_csv/_row_to_result helpers. Refactored GET /results to use the same helpers. Added asyncio, JSONResp…72a7372 -
Results browser #R5 + #R1: geocode_one helper + GET /results endpoint
R5 — pipeline/geocode.py: add geocode_one(row: dict) -> tuple[str, str] Thin wrapper over the existing geocode_row() function. Accepts a restaurant row dict (name/city/state/country/address keys), returns (lat, lng) as formatted strings or ("", "") on no confident match. Caller owns rate-limit sleeps when invoking in bulk. geocode_row() is unchanged; run() delegates to it as before. R1 — web/app.py: GET /results/{job_id} Returns all rows from jobs/{job_id}/restaurants_full.csv as JSON, sorted by saved_at descending (newest saves first). Each row includes all FIELDNAMES plus a derived "geocoded" boolean (true iff lat and lng are non-empty). Returns 404 JSON if the job dir or CSV doesn't exist. Response shape: {"rows": [...], "total": N} Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>08acf65 -
Feature: quality benchmark suite + CI regression test
Adds a full testing pipeline for measuring and tracking extraction quality across all models: - tests/create_fixture.py — deterministic sampler: 50 posts from saved_posts.json (30 food candidates + 20 non-food), diversified by creator and hashtag set, with guaranteed edge-case coverage (thin captions, no hashtags) - tests/fixtures/sampled_posts.json — the 50-post fixture (checked in, seed=42) - tests/generate_ground_truth.py — runs sampled posts through Claude Opus 4.8 to produce the authoritative oracle; checkpointed so it's resumable - tests/fixtures/test_posts.json — 50 Opus 4.8–labelled ground truth records (30 food / 20 not-food); this is the permanent benchmark oracle - tests/benchmark.py — CLI runner that evaluates any model/provider against the oracle. Metrics: is_food_accuracy (40%), name_recall fuzzy (25%), city_recall (15%), field_completeness (20%) → composite_score. Writes tests/benchmark_results.json. First result: claude-haiku-4-5 scores 89.1 at $0.055…a42a323