FluidInference/citenet-coreml

CiteNet: a legal-citation checker whose weights were written by hand

A 397,250-parameter network (13,871 nonzero) that flags legal citations that cannot exist: an invented reporter series (F.5th, S.W.4th), a court the reporter never prints (88 F. Supp. 3d 412 (5th Cir. 2015)), a state the regional reporter doesn't cover, or a year that doesn't fit the volume (912 F.3d 455 (9th Cir. 1998); vol 912 is from 2018).

No training. Claude (Anthropic) wrote citenet/spec.py: the reporters and their real series, which courts each reporter prints, the West regional map, and approximate volume→year timelines. It also wrote citenet/net.py, which compiles each entry into specific weights:

• a conv filter that fires only when the characters spell <digit> F.3d<space>
• digit decoders with weight k·10^i on digit k
• a shared ReLU "volume thermometer" that every volume→year curve reads from
• AND gates like relu(8·[federal appellate reporter] + [state court] − 8)

Every nonzero weight has a name, and there was no gradient step.

Results (LePhantomCite, held-out eval split, citation level)

ai-law-society-lab/Legal_Phantom_Citation (Liu, Stammbach & Henderson 2026, CC BY 4.0). The network was designed using only the aux_train split. eval was scored once with frozen weights.

	phantom citations caught	real citations flagged
v1.0, first and only blind run	31 / 31	3 / 983
v1.5 (this release; later fixes were not blind)	31 / 31	1 / 983

• The remaining flag is 652 A.2d 634 (D.C. Cir. 1995). A.2d never prints D.C. Circuit opinions, so it is likely an error in the source brief.
• Out of scope: other hallucination types (case-name mismatch, wrong pincite, misquote) need the case text. They are flagged 2 / 276 times.
• Hand-written vs trained: trained from scratch, the same architecture (v1.1) ties on this benchmark, with 0 / 983 false flags. It caught 1 of 5 planted fakes of kinds the training data never shows, against 5 / 5 for the hand-written net. That 5-fake set is tiny and I wrote it, so treat it as illustrative. A naive fine-tune starting from the hand-written weights made it worse. See results/.

Limits. A string-only network can tell that a citation could exist, not that it does. LePhantomCite's non-existent citations are structurally impossible, so it catches all of them. Real LLM fakes, such as the six citations ChatGPT invented in Mata v. Avianca (e.g. 925 F.3d 1339 (11th Cir. 2019)), copy valid formats and pass. Use this as a fast first filter in front of a case-law lookup.

Real case law. On 3.30 GB of US court opinions (291,339 decisions, COLD Cases shard 0), 0.134% of citations are flagged. Most flags look like real errors in the opinions, e.g. 387 U.S. 1 (1887) for In re Gault (1967). They are unverified, and some are the model's own mistakes.

Speed (Core ML, Apple M5 Pro, native Swift)

1 citation (CiteNet_b1, CPU)	~50 µs
32 citations (CiteNet_b32, Neural Engine)	170 µs
bulk, Neural Engine (CiteNet_lean_b512)	~430k citations/s
bulk, GPU (CiteNet_lean_b8192)	~1.38M citations/s
Neural Engine + GPU together	~1.82M citations/s
raw text, end to end: native scanner + ANE + GPU	3.30 GB of opinions in 2.3 s (1.44 GB/s, 1.68M citations/s)

Every build is fp16, and the labels match the fp32 PyTorch model on all 4,383 LePhantomCite citations. Fixed batch shapes matter: with EnumeratedShapes, Core ML ran everything on the CPU.

Files

• models/: CiteNet_b{1,32,512}.mlpackage output p_fake, the 7 evidence values and the named concepts. CiteNet_lean_b{512,8192}.mlpackage output only p_fake and evidence, for bulk work. meta.json names every neuron; weights.npz holds the raw weights.
• citenet/: the spec, the weight compiler and the tokenizer (text.py: citation extraction and normalization to 64 char codes).
• bench/:
  • CiteScan.swift: a byte-level Swift port of the tokenizer, byte-identical to Python (scripts/scan_parity.py).
  • stream/main.swift: the multithreaded raw-text pipeline.
  • firehose.swift, bench.swift: benchmarks.
• demo/: a local web demo (python demo/server.py, localhost:8025).
  • a live checker
  • /wall: a scrolling text wall that highlights citations as they arrive, with a planted-fakes mode that reports planted, caught and missed counts
  • /live: the full-speed scan
  • /inspect: the neuron and filter view
• results/: the v1.0 blind eval log, v1.5 eval logs, the training comparison and benchmark logs.

Use

import numpy as np, coremltools as ct
from citenet import text

m = ct.models.MLModel("models/CiteNet_b32.mlpackage")
cites = [c for _, _, c in text.extract("See Henley v. Northwind, 412 F.5th 1187 (2d Cir. 2023); "
                                       "Bell Atl. Corp. v. Twombly, 550 U.S. 544, 570 (2007).")]
x = np.zeros((32, 1, 1, 64), np.float16)
x[:len(cites)] = text.encode(cites)
p = m.predict({"codes": x})["p_fake"][:len(cites), 0]
print(list(zip(cites, p.round(3))))   # F.5th -> ~0.99, Twombly -> ~0.02

To rebuild every model from the spec: python scripts/convert.py (torch + coremltools). The demo server expects the models in build/.

Data: LePhantomCite (CC BY 4.0) and COLD Cases (CC0) are not included; download them from the Hub.