Blame for README.md

---
license: mit
language:
- en
library_name: cite
tags:
- cite
- paragraph-citation
- retrieval
- question-answering
- news
- encoder
- modernbert
- on-device
pipeline_tag: text-ranking
base_model: jhu-clsp/ettin-encoder-32m
---

# Cite

Which paragraphs answer the question.

A small encoder that, given a question and ordered paragraph texts, returns a
calibrated probability per paragraph and two thresholds (`needed`,
`contributing`).

| | |
| --- | --- |
| **Task** | Paragraph citation / needed-blocks |
| **Size** | ~42 MB weights (trimmed vocab, int8 embeddings, fp16 rest); a larger 68M build is under [`68m/`](https://huggingface.co/localangle/cite/tree/main/68m) |
| **Runtime** | CPU (PyTorch); browser (ONNX Runtime WASM) |
| **Weights** | [v0.3.0](https://huggingface.co/localangle/cite/tree/v0.3.0) · [`web/`](https://huggingface.co/localangle/cite/tree/main/web) for WASM |
| **Demo** | [localangle/cite-demo](https://huggingface.co/spaces/localangle/cite-demo) (in-browser) |

Cite does **not** write an answer. It ranks the paragraphs you already have and
flags which ones are needed to answer the question.

> Question: *Who voted against the library renovation?*
> → Cite highlights the paragraph naming Paulsen and Chen.

## Try it

- **Live demo:** [localangle/cite-demo](https://huggingface.co/spaces/localangle/cite-demo) — paste an article, ask a question, see which paragraphs Cite cites.
- **Local playground:** from the [cite](https://github.com/localangle/cite) repo, `uv run python deploy/cite_playground.py`.

## Install

```bash
pip install "cite @ git+https://github.com/localangle/cite.git"
# or, from a checkout:
uv sync
```

Python ≥ 3.12. Needs `torch`, `transformers`, `safetensors`, `huggingface_hub`.

## Usage

```python
from cite import CitePackage

cite = CitePackage.from_pretrained("localangle/cite")

paragraphs = [
    "The council approved a $4.2 million library renovation.",
    "Paulsen and Chen voted against the measure.",
    "Construction starts in March.",
]
scores = cite.predict("Who voted against the renovation?", paragraphs)

for score in scores:
    if score.needed:
        print(score.index, score.probability, paragraphs[score.index])
```

`score.needed` uses the packaged threshold (0.76). `score.contributing` is a
looser relatedness cut (0.17). Logits are Platt-scaled to probabilities at load
time; you do not fit thresholds yourself for the published package.

Load a local folder the same way:

```python
cite = CitePackage.load("path/to/package")  # or from_pretrained("./path")
```

## Files

| File | Contents |
| --- | --- |
| `cite.json` | Format, Platt scaling, thresholds, source metadata |
| `weights.safetensors` | Trimmed encoder + head (int8 token embeddings, fp16 elsewhere) |
| `encoder/` | ModernBERT/Ettin config for the trimmed vocabulary |
| `tokenizer/` | Tokenizer with `[MASK]` as the block marker |
| `web/` | Browser package: `model.onnx` (~36 MB int8), `vocab_map.bin`, `cite-web.json`, `tokenizer/` |
| `68m/` | The 68M build as a full Python package (same layout as the repo root; ~107 MB weights) |
| `web-68m/` | Browser package for the 68M build (`model.onnx` ~88 MB int8) |

PyTorch weights expand to fp32 at load; only storage is compressed. The `web/`
ONNX graph is what the static demo runs in WASM (soft cap 2048 tokens).

## Accuracy

On a held-out real-news needed-blocks eval (v59, 1,800 questions over local
news outlets; threshold chosen on a separate calibration split):

| Metric | Cite-32M package |
| --- | --- |
| Coverage | 0.692 |
| Exact set match | 0.725 |
| Empty-set accuracy | 0.897 |
| Any-extra rate | 0.066 |

The browser ONNX build on the same eval: exact 0.722, coverage 0.687, empty 0.902.

### Broad and abstract questions

The eval above uses specific fact-lookup questions. Later versions add training so that broad
questions ("How will the library change as a result of the renovations?") score the paragraph
that answers them instead of washing out.

- **v0.2.0** fine-tuned on LLM-written broad rewrites of real training questions.
- **v0.3.0** adds synthetic short articles where nearly every paragraph shares the topic word,
  with hard negatives, then averages the weights 0.75/0.25 with v0.2.0 to keep the specific-question
  operating point.

On a 117-item hand-written probe (specific question vs. an abstract rewrite of it; 92 items after
removing 25 where several paragraphs plausibly answer the rewrite), abstract-question results:

| Abstract question | v0.1.0 | v0.2.0 | v0.3.0 |
| --- | --- | --- | --- |
| Top-1 (all 117) | 0.650 | 0.744 | 0.752 |
| Needed recall (all 117) | 0.402 | 0.752 | 0.761 |
| Top-1 (92 audited) | 0.826 | 0.848 | 0.859 |
| Top-3 (92 audited) | n/a | 0.957 | 0.978 |

Packaged-model numbers for v0.2.0 and v0.3.0; v0.1.0 from the fp32 student. Specific-question probe
top-1 is 0.949 (v0.2.0) and 0.966 (v0.3.0). The probe is small and written by one author;
treat it as a regression check, not a benchmark.

On a short, topic-saturated example article, v0.3.0 improves but does not solve the hardest
case: for "What specific improvements will be made...?" the improvements paragraph moves from
rank 4 (18%) to rank 2 (60%), still behind a funding paragraph that repeats the word
"renovation". See `experiments/107-cite-abstract-probe`, `experiments/108-cite-broad-consistency`
and `experiments/109-cite-hard-negatives`.

## Cite 68M

A larger build, distilled from the same teacher with the same data and training steps, on
[Ettin-68M](https://huggingface.co/jhu-clsp/ettin-encoder-68m). It is more accurate, especially on broad
and abstract questions, and about 3x slower. The 32M build above stays the default.

```python
cite = CitePackage.from_pretrained("localangle/cite", subfolder="68m")
```

The [demo](https://huggingface.co/spaces/localangle/cite-demo) has a model switch; the 68M build downloads
about 88 MB on first use and is cached after that. Packaged thresholds: needed 0.79, contributing 0.17.

| | Cite 32M (v0.3.0) | Cite 68M |
| --- | --- | --- |
| Weights (package) | 41.8 MB | 107.0 MB |
| Browser `model.onnx` | 36.3 MB | 88.3 MB |
| v59 eval: coverage / exact / empty-set / any-extra rate | 0.692 / 0.725 / 0.897 / 0.066 | 0.729 / 0.763 / 0.923 / 0.042 |
| Abstract probe, 92 audited items: top-1 / needed recall | 0.859 / 0.859 | 0.902 / 0.891 |
| Abstract probe, all 117: top-1 | 0.752 | 0.786 |
| Browser (WASM, 1 thread): 7 paragraphs / 44 paragraphs | 185 ms / 1.4 s | 547 ms / 4.2 s |

On the short example article, for "What specific improvements will be made to the library as a result of the
renovation?", the improvements paragraph scores 93.5% on 68M (60% on 32M, behind a funding paragraph).
The same caveats apply: one run per size, a small hand-written probe, and browser timings from a single
machine. See `experiments/110-cite-68m`.

## Latency

Batch-1, CPU, AMD EPYC (HF `cpu-upgrade`, 8 vCPU) for the related ONNX int8
form: about 44 ms p50 at 5–12 paragraphs and 268 ms at 33–60. The portable
PyTorch package this repo ships is the accuracy-matched serving artifact; long
documents are dominated by attention cost.

## How it works

Cite is a fine-tuned [Ettin](https://huggingface.co/jhu-clsp/ettin-encoder-32m)
encoder (ModernBERT family). The question and each paragraph are packed into
one sequence with a `[MASK]` marker before every paragraph; the head reads the
marker states. A 400M teacher labeled real news; the 32M student was distilled
then fine-tuned on LLM-written broad rewrites of its training questions
(consistency distillation) and on synthetic hard-negative articles, and compressed (trimmed vocabulary, int8
embeddings, fp16 weights).

## Limitations

- Cite ranks paragraphs; it does not generate answers or quote spans inside a
  paragraph.
- Lexical overlap still matters: "big 10" and "Big Ten" can score differently.
  Scores for a paragraph also drift down in very long articles.
- Eval is English news-style text. Other domains are unmeasured.
- Empty-set accuracy drops a few points from calibration to eval; treat the
  packaged threshold as a starting point for a new domain.
- Documents longer than `max_tokens` (7999) drop trailing paragraphs.
- Training used private teacher labels and news text that is not redistributed
  here.

## Built on

- [Ettin encoder 32M](https://huggingface.co/jhu-clsp/ettin-encoder-32m) (MIT)
  by JHU CLSP. Fine-tuned and compressed for Cite; see
  `THIRD_PARTY_NOTICES.md`.

## License

MIT for the Cite code and this package's weights, subject to the Ettin MIT
notice above.

## Citation

```bibtex
@software{cite_2026,
  title  = {Cite: Which paragraphs answer the question},
  author = {Local Angle},
  year   = {2026},
  url    = {https://huggingface.co/localangle/cite},
}
```