DeepQuill fiction NER (roberta-base)

The named-entity model DeepQuill runs inside the app to find a manuscript's characters and places. It is FacebookAI/roberta-base distilled from Jean-Baptiste/roberta-large-ner-english, the model it replaces, and is about a third of its size.

Files

File What it is
model.onnx, model.onnx.data the fp32 ONNX graph and its weights
tokenizer.json the roberta-base tokenizer
config.json labels: O, B-/I- Γ— PER, ORG, LOC, MISC
ner_config.json how DeepQuill windows text: 510 tokens with special tokens, 50-token overlap

Labels are aggregated per word with the "simple" strategy.

Training

  • Teacher labels: 79 public-domain Project Gutenberg novels, labelled by Jean-Baptiste/roberta-large-ner-english @ 8f3abc1e.
  • Gold labels: LitBank @ 3e50db0f, its proper-name mentions (80 training documents).
  • 3 epochs, seed 1. No CoNLL-2003, OntoNotes or other research-only dataset was trained on. (The teacher's own card lists CoNLL-2003; its weights are MIT.)

Evaluation

Against the teacher, in DeepQuill's full pipeline, on the cast lists of 7 books (pooled cast F1, 95% bootstrap CI):

Ξ” vs teacher
Overall βˆ’0.007 [βˆ’0.021, +0.001]
Characters βˆ’0.008 [βˆ’0.021, +0.002]
Places βˆ’0.008 [βˆ’0.022, +0.003]
  • LitBank test (10 held-out documents, exact span F1): PER 0.851, LOC 0.732 (teacher: 0.718, 0.562).
  • Pronoun resolution and POV detection downstream: same as the teacher or better.
  • Speed: a 69-scene novel in 31 s on an Apple M5 Pro CPU (teacher: 80 s), peak memory 0.52 GB (teacher: 2.49 GB).

Licence and attribution

The weights are MIT, like roberta-base and the teacher.

They are trained on LitBank (CC BY 4.0):

  • David Bamman, Sejal Popat and Sheng Shen. 2019. An Annotated Dataset of Literary Entities. NAACL.
  • David Bamman, Olivia Lewke and Anya Mansoor. 2020. An Annotated Dataset of Coreference in English Literature. LREC.
Downloads last month
24
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for deepquillapp/fiction-ner

Quantized
(20)
this model