DeepQuill fiction NER (roberta-base)
The named-entity model DeepQuill runs inside the app to find a manuscript's
characters and places. It is FacebookAI/roberta-base distilled from
Jean-Baptiste/roberta-large-ner-english, the model it replaces, and is about
a third of its size.
Files
| File | What it is |
|---|---|
model.onnx, model.onnx.data |
the fp32 ONNX graph and its weights |
tokenizer.json |
the roberta-base tokenizer |
config.json |
labels: O, B-/I- Γ PER, ORG, LOC, MISC |
ner_config.json |
how DeepQuill windows text: 510 tokens with special tokens, 50-token overlap |
Labels are aggregated per word with the "simple" strategy.
Training
- Teacher labels: 79 public-domain Project Gutenberg novels, labelled by
Jean-Baptiste/roberta-large-ner-english@8f3abc1e. - Gold labels: LitBank @
3e50db0f, its proper-name mentions (80 training documents). - 3 epochs, seed 1. No CoNLL-2003, OntoNotes or other research-only dataset was trained on. (The teacher's own card lists CoNLL-2003; its weights are MIT.)
Evaluation
Against the teacher, in DeepQuill's full pipeline, on the cast lists of 7 books (pooled cast F1, 95% bootstrap CI):
| Ξ vs teacher | |
|---|---|
| Overall | β0.007 [β0.021, +0.001] |
| Characters | β0.008 [β0.021, +0.002] |
| Places | β0.008 [β0.022, +0.003] |
- LitBank test (10 held-out documents, exact span F1): PER 0.851, LOC 0.732 (teacher: 0.718, 0.562).
- Pronoun resolution and POV detection downstream: same as the teacher or better.
- Speed: a 69-scene novel in 31 s on an Apple M5 Pro CPU (teacher: 80 s), peak memory 0.52 GB (teacher: 2.49 GB).
Licence and attribution
The weights are MIT, like roberta-base and the teacher.
They are trained on LitBank (CC BY 4.0):
- David Bamman, Sejal Popat and Sheng Shen. 2019. An Annotated Dataset of Literary Entities. NAACL.
- David Bamman, Olivia Lewke and Anya Mansoor. 2020. An Annotated Dataset of Coreference in English Literature. LREC.
- Downloads last month
- 24
Model tree for deepquillapp/fiction-ner
Base model
FacebookAI/roberta-base