| --- |
| license: mit |
| language: |
| - en |
| library_name: cite |
| tags: |
| - cite |
| - paragraph-citation |
| - retrieval |
| - question-answering |
| - news |
| - encoder |
| - modernbert |
| - on-device |
| pipeline_tag: text-ranking |
| base_model: jhu-clsp/ettin-encoder-32m |
| --- |
| |
| # Cite |
|
|
| Which paragraphs answer the question. |
|
|
| A small encoder that, given a question and ordered paragraph texts, returns a |
| calibrated probability per paragraph and two thresholds (`needed`, |
| `contributing`). |
|
|
| | | | |
| | --- | --- | |
| | **Task** | Paragraph citation / needed-blocks | |
| | **Size** | ~42 MB weights (trimmed vocab, int8 embeddings, fp16 rest); a larger 68M build is under [`68m/`](https://huggingface.co/localangle/cite/tree/main/68m) | |
| | **Runtime** | CPU (PyTorch); browser (ONNX Runtime WASM) | |
| | **Weights** | [v0.3.0](https://huggingface.co/localangle/cite/tree/v0.3.0) · [`web/`](https://huggingface.co/localangle/cite/tree/main/web) for WASM | |
| | **Demo** | [localangle/cite-demo](https://huggingface.co/spaces/localangle/cite-demo) (in-browser) | |
|
|
| Cite does **not** write an answer. It ranks the paragraphs you already have and |
| flags which ones are needed to answer the question. |
|
|
| > Question: *Who voted against the library renovation?* |
| > → Cite highlights the paragraph naming Paulsen and Chen. |
|
|
| ## Try it |
|
|
| - **Live demo:** [localangle/cite-demo](https://huggingface.co/spaces/localangle/cite-demo) — paste an article, ask a question, see which paragraphs Cite cites. |
| - **Local playground:** from the [cite](https://github.com/localangle/cite) repo, `uv run python deploy/cite_playground.py`. |
|
|
| ## Install |
|
|
| ```bash |
| pip install "cite @ git+https://github.com/localangle/cite.git" |
| # or, from a checkout: |
| uv sync |
| ``` |
|
|
| Python ≥ 3.12. Needs `torch`, `transformers`, `safetensors`, `huggingface_hub`. |
|
|
| ## Usage |
|
|
| ```python |
| from cite import CitePackage |
| |
| cite = CitePackage.from_pretrained("localangle/cite") |
| |
| paragraphs = [ |
| "The council approved a $4.2 million library renovation.", |
| "Paulsen and Chen voted against the measure.", |
| "Construction starts in March.", |
| ] |
| scores = cite.predict("Who voted against the renovation?", paragraphs) |
| |
| for score in scores: |
| if score.needed: |
| print(score.index, score.probability, paragraphs[score.index]) |
| ``` |
|
|
| `score.needed` uses the packaged threshold (0.76). `score.contributing` is a |
| looser relatedness cut (0.17). Logits are Platt-scaled to probabilities at load |
| time; you do not fit thresholds yourself for the published package. |
|
|
| Load a local folder the same way: |
|
|
| ```python |
| cite = CitePackage.load("path/to/package") # or from_pretrained("./path") |
| ``` |
|
|
| ## Files |
|
|
| | File | Contents | |
| | --- | --- | |
| | `cite.json` | Format, Platt scaling, thresholds, source metadata | |
| | `weights.safetensors` | Trimmed encoder + head (int8 token embeddings, fp16 elsewhere) | |
| | `encoder/` | ModernBERT/Ettin config for the trimmed vocabulary | |
| | `tokenizer/` | Tokenizer with `[MASK]` as the block marker | |
| | `web/` | Browser package: `model.onnx` (~36 MB int8), `vocab_map.bin`, `cite-web.json`, `tokenizer/` | |
| | `68m/` | The 68M build as a full Python package (same layout as the repo root; ~107 MB weights) | |
| | `web-68m/` | Browser package for the 68M build (`model.onnx` ~88 MB int8) | |
|
|
| PyTorch weights expand to fp32 at load; only storage is compressed. The `web/` |
| ONNX graph is what the static demo runs in WASM (soft cap 2048 tokens). |
|
|
| ## Accuracy |
|
|
| On a held-out real-news needed-blocks eval (v59, 1,800 questions over local |
| news outlets; threshold chosen on a separate calibration split): |
|
|
| | Metric | Cite-32M package | |
| | --- | --- | |
| | Coverage | 0.692 | |
| | Exact set match | 0.725 | |
| | Empty-set accuracy | 0.897 | |
| | Any-extra rate | 0.066 | |
|
|
| The browser ONNX build on the same eval: exact 0.722, coverage 0.687, empty 0.902. |
|
|
| ### Broad and abstract questions |
|
|
| The eval above uses specific fact-lookup questions. Later versions add training so that broad |
| questions ("How will the library change as a result of the renovations?") score the paragraph |
| that answers them instead of washing out. |
|
|
| - **v0.2.0** fine-tuned on LLM-written broad rewrites of real training questions. |
| - **v0.3.0** adds synthetic short articles where nearly every paragraph shares the topic word, |
| with hard negatives, then averages the weights 0.75/0.25 with v0.2.0 to keep the specific-question |
| operating point. |
|
|
| On a 117-item hand-written probe (specific question vs. an abstract rewrite of it; 92 items after |
| removing 25 where several paragraphs plausibly answer the rewrite), abstract-question results: |
|
|
| | Abstract question | v0.1.0 | v0.2.0 | v0.3.0 | |
| | --- | --- | --- | --- | |
| | Top-1 (all 117) | 0.650 | 0.744 | 0.752 | |
| | Needed recall (all 117) | 0.402 | 0.752 | 0.761 | |
| | Top-1 (92 audited) | 0.826 | 0.848 | 0.859 | |
| | Top-3 (92 audited) | n/a | 0.957 | 0.978 | |
|
|
| Packaged-model numbers for v0.2.0 and v0.3.0; v0.1.0 from the fp32 student. Specific-question probe |
| top-1 is 0.949 (v0.2.0) and 0.966 (v0.3.0). The probe is small and written by one author; |
| treat it as a regression check, not a benchmark. |
|
|
| On a short, topic-saturated example article, v0.3.0 improves but does not solve the hardest |
| case: for "What specific improvements will be made...?" the improvements paragraph moves from |
| rank 4 (18%) to rank 2 (60%), still behind a funding paragraph that repeats the word |
| "renovation". See `experiments/107-cite-abstract-probe`, `experiments/108-cite-broad-consistency` |
| and `experiments/109-cite-hard-negatives`. |
|
|
| ## Cite 68M |
|
|
| A larger build, distilled from the same teacher with the same data and training steps, on |
| [Ettin-68M](https://huggingface.co/jhu-clsp/ettin-encoder-68m). It is more accurate, especially on broad |
| and abstract questions, and about 3x slower. The 32M build above stays the default. |
|
|
| ```python |
| cite = CitePackage.from_pretrained("localangle/cite", subfolder="68m") |
| ``` |
|
|
| The [demo](https://huggingface.co/spaces/localangle/cite-demo) has a model switch; the 68M build downloads |
| about 88 MB on first use and is cached after that. Packaged thresholds: needed 0.79, contributing 0.17. |
|
|
| | | Cite 32M (v0.3.0) | Cite 68M | |
| | --- | --- | --- | |
| | Weights (package) | 41.8 MB | 107.0 MB | |
| | Browser `model.onnx` | 36.3 MB | 88.3 MB | |
| | v59 eval: coverage / exact / empty-set / any-extra rate | 0.692 / 0.725 / 0.897 / 0.066 | 0.729 / 0.763 / 0.923 / 0.042 | |
| | Abstract probe, 92 audited items: top-1 / needed recall | 0.859 / 0.859 | 0.902 / 0.891 | |
| | Abstract probe, all 117: top-1 | 0.752 | 0.786 | |
| | Browser (WASM, 1 thread): 7 paragraphs / 44 paragraphs | 185 ms / 1.4 s | 547 ms / 4.2 s | |
|
|
| On the short example article, for "What specific improvements will be made to the library as a result of the |
| renovation?", the improvements paragraph scores 93.5% on 68M (60% on 32M, behind a funding paragraph). |
| The same caveats apply: one run per size, a small hand-written probe, and browser timings from a single |
| machine. See `experiments/110-cite-68m`. |
|
|
| ## Latency |
|
|
| Batch-1, CPU, AMD EPYC (HF `cpu-upgrade`, 8 vCPU) for the related ONNX int8 |
| form: about 44 ms p50 at 5–12 paragraphs and 268 ms at 33–60. The portable |
| PyTorch package this repo ships is the accuracy-matched serving artifact; long |
| documents are dominated by attention cost. |
|
|
| ## How it works |
|
|
| Cite is a fine-tuned [Ettin](https://huggingface.co/jhu-clsp/ettin-encoder-32m) |
| encoder (ModernBERT family). The question and each paragraph are packed into |
| one sequence with a `[MASK]` marker before every paragraph; the head reads the |
| marker states. A 400M teacher labeled real news; the 32M student was distilled |
| then fine-tuned on LLM-written broad rewrites of its training questions |
| (consistency distillation) and on synthetic hard-negative articles, and compressed (trimmed vocabulary, int8 |
| embeddings, fp16 weights). |
|
|
| ## Limitations |
|
|
| - Cite ranks paragraphs; it does not generate answers or quote spans inside a |
| paragraph. |
| - Lexical overlap still matters: "big 10" and "Big Ten" can score differently. |
| Scores for a paragraph also drift down in very long articles. |
| - Eval is English news-style text. Other domains are unmeasured. |
| - Empty-set accuracy drops a few points from calibration to eval; treat the |
| packaged threshold as a starting point for a new domain. |
| - Documents longer than `max_tokens` (7999) drop trailing paragraphs. |
| - Training used private teacher labels and news text that is not redistributed |
| here. |
|
|
| ## Built on |
|
|
| - [Ettin encoder 32M](https://huggingface.co/jhu-clsp/ettin-encoder-32m) (MIT) |
| by JHU CLSP. Fine-tuned and compressed for Cite; see |
| `THIRD_PARTY_NOTICES.md`. |
|
|
| ## License |
|
|
| MIT for the Cite code and this package's weights, subject to the Ettin MIT |
| notice above. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @software{cite_2026, |
| title = {Cite: Which paragraphs answer the question}, |
| author = {Local Angle}, |
| year = {2026}, |
| url = {https://huggingface.co/localangle/cite}, |
| } |
| ``` |
|
|
