Verified against the installed acl_anthology==1.2.0 package and a real
local checkout, three fields were wrong:
- Anthology.from_repo(anthology_path) misused the local checkout path
as a repo_url, causing a redundant re-clone into the library's
default data directory instead of reading our already-cloned copy.
Now constructs Anthology(datadir=anthology_path/"data") directly.
- Paper.id is only unique within its parent Volume (e.g. "1") -- using
it as our SQLite primary key would silently collide/overwrite papers
across different volumes. Paper.full_id (e.g. "2023.acl-long.1") is
the actual globally unique identifier.
- Paper.venue_acronym and Paper.url don't exist; the real attributes
are Paper.parent.venue_acronym and Paper.web_url.
Confirmed against a real local acl-anthology data checkout, not just
mocks, since the original implementation's mocks encoded exactly
these wrong assumptions.
fix: don't stub missing snapshot files with empty placeholders
download_snapshot wrote a 0-byte placeholder for any missing file,
including index.faiss. A 0-byte file is not a valid FAISS index and
crashes faiss.read_index instead of being treated as absent, breaking
the very first sync against a brand-new dataset repo. Missing files
are now left absent so load_index's existing FileNotFoundError
fallback (and sqlite3.connect's create-on-open behavior) apply as
intended.
fix: catch huggingface_hub's real EntryNotFoundError on missing files
download_snapshot only caught the built-in FileNotFoundError, but
hf_hub_download actually raises huggingface_hub.errors.EntryNotFoundError
(HTTP 404) for a missing file on a real dataset repo. This meant the
very first sync against a brand-new, empty dataset repo crashed instead
of falling back to an empty snapshot. The prior test suite didn't catch
this because it mocked hf_hub_download to raise FileNotFoundError
directly rather than the real exception type.
refactor: use openai package for the embedding API client
Swaps the hand-rolled httpx POST call for the official openai SDK
pointed at the university's OpenAI-compatible endpoint via base_url,
removing the httpx dependency in favor of openai.
fix: correct FAISS id mapping across syncs, atomic snapshot upload, startup logging, health timestamp, real anthology field access
Also fix common/db.py init_db to pass check_same_thread=False, required
because FastAPI's TestClient/threadpool executes sync route handlers on
a worker thread different from the one that created the sqlite connection.
feat: add search service combining keyword and similarity search
