| --- |
| license: apache-2.0 |
| base_model: convaiinnovations/laya |
| library_name: onnxruntime |
| pipeline_tag: text-classification |
| tags: [typed-decisions, laya, modernbert, fiction, character-traits] |
| --- |
| |
| # DeepQuill trait verifier |
|
|
| A fine-tune of [Laya](https://huggingface.co/convaiinnovations/laya) (ModernBERT-large encoder plus a typed-decision head). [DeepQuill](https://deepquill.app) uses it to check character-trait claims that its local extractor pulls from fiction. |
|
|
| For each claim (a target character, `field = value`, the quote, and the passage), the model answers four yes/no questions: |
|
|
| - **aboutTarget**: is it about that character? |
| - **lasting**: a lasting attribute, not a momentary state? |
| - **literal**: literal, not figurative? |
| - **asserted**: asserted by the narration? |
|
|
| The claim is kept when all four pass. |
|
|
| ## Files |
|
|
| - `model.w8.onnx`: weight-only 8-bit (MatMulNBits). Activations run in fp32, so a claim's result doesn't depend on what it's batched with. |
| - `tokenizer.json` |
| - `verifier_config.json`: the exact question texts, sequence limits and calibration temperature. |
|
|
| ## Training data |
|
|
| Everything the model was trained on is public: |
|
|
| - 3,760 labelled claims from real extractor output over US-public-domain novels from 1894–1930 (Project Gutenberg). |
| - Synthetic contemporary passages written for this purpose. |
|
|
| The model was never trained on user manuscripts. |
|
|
| ## Evaluation |
|
|
| The model was scored on held-out sets that share no books or characters with the training data. Metric: keep-F1. |
|
|
| | Test set | This model | Qwen3-4B LLM verifier | No verifier | |
| |---|---|---|---| |
| | 1920s public-domain claims (524) | 0.75 | 0.32 | 0.45 | |
| | Contemporary manuscript claims (157) | 0.88 | 0.24 | 0.45 | |
|
|
