deepquillapp/trait-verifier

DeepQuill trait verifier

A fine-tune of Laya (ModernBERT-large encoder plus a typed-decision head). DeepQuill uses it to check character-trait claims that its local extractor pulls from fiction.

For each claim (a target character, field = value, the quote, and the passage), the model answers four yes/no questions:

• aboutTarget: is it about that character?
• lasting: a lasting attribute, not a momentary state?
• literal: literal, not figurative?
• asserted: asserted by the narration?

The claim is kept when all four pass.

Files

• model.w8.onnx: weight-only 8-bit (MatMulNBits). Activations run in fp32, so a claim's result doesn't depend on what it's batched with.
• tokenizer.json
• verifier_config.json: the exact question texts, sequence limits and calibration temperature.

Training data

Everything the model was trained on is public:

• 3,760 labelled claims from real extractor output over US-public-domain novels from 1894–1930 (Project Gutenberg).
• Synthetic contemporary passages written for this purpose.

The model was never trained on user manuscripts.

Evaluation

The model was scored on held-out sets that share no books or characters with the training data. Metric: keep-F1.

Test set	This model	Qwen3-4B LLM verifier	No verifier
1920s public-domain claims (524)	0.75	0.32	0.45
Contemporary manuscript claims (157)	0.88	0.24	0.45