The full HF dataset (20 GB, float64 embeds, citation graphs) exceeds this
machine's disk and RAM. build_corpus.py converts the downloaded shards to
text-only parquet (~1.3 GB) plus a float16 IndexScalarQuantizer (~3.8 GB),
deleting source shards as consumed. Corpus.load() now mmaps the index.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>