vadimbelsky commited on
Commit
84eeb99
·
verified ·
1 Parent(s): 3b2ac5d

Document-type head V3a s7001: supportive monitoring, evaluation and reproduction only

Browse files
MANIFEST.sha256 ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ 3abd2ea8f76e3e1196228d41ccd7c7040d5cc03f12a99555e13d481af267ea46 README.md
2
+ 7bf1257edaa508b110646407f151352c117e935001da42bd4f96810c53fb75e0 THIRD_PARTY_NOTICES.md
3
+ ddaff31d9e705ed3eb610467b3b783acd4116892887641c25db538a7215687f1 doc_labels.json
4
+ 0c9c1780673d719310afd1bce2538d8e078ea2adf0bc3a6e397c564ac6486b6e onnx/model_quantized.onnx
README.md ADDED
@@ -0,0 +1,227 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: other
3
+ license_name: proprietary-sciencesoft
4
+ license_link: https://huggingface.co/ScienceSoft/scnsoft-pii-encoder/blob/main/README.md#licence
5
+ language: [en, de, fr, it, es, ru, ar]
6
+ pipeline_tag: token-classification
7
+ tags: [pii, ner, privacy, dlp, onnx, xlm-roberta, document-classification]
8
+ library_name: onnxruntime
9
+ base_model: FacebookAI/xlm-roberta-base
10
+ base_model_relation: finetune
11
+ ---
12
+
13
+ **Release candidate for supportive monitoring. Not an enforcement tool.** This is a document-type head, V3a, seed 7001. Its
14
+ scores support a person looking at documents; nothing here claims that a document should be blocked, redacted or refused
15
+ because of a score. The weights are published for evaluation and reproduction only (see Licence). The training data is not
16
+ published.
17
+
18
+ ## Document-type head V3a (seed 7001)
19
+
20
+ This model adds a second output, `doc_logits`, to the same int8 graph as
21
+ [`ScienceSoft/scnsoft-pii-encoder`](https://huggingface.co/ScienceSoft/scnsoft-pii-encoder), so that a window of text is
22
+ also scored for 17 document types (non-disclosure and master agreements, statements of work, other contracts, requests
23
+ for proposals, proposals, price lists, source code, design and security documents, financial and personal-finance
24
+ documents, personnel files, strategy documents, published documents and patient records). The PII output (`logits`) is the
25
+ encoder's own.
26
+
27
+ ### What this release says, and does not say
28
+
29
+ - **Four types have been evaluated, on one measure only.** `legal.nda`, `tech.design`, `tech.security` and
30
+ `finance.personal` met a false-record line on held-out benign documents on one machine (table below). That is the only
31
+ evaluation of them. Their recall on documents of the type, their behaviour on other machines and their cost on customer
32
+ hardware are not evaluated. Meeting the line does not make a type ready for any decision.
33
+ - **`legal.contract_other` is not yet monitoring-grade.** It missed the same line (8 false records on 18,075 benign
34
+ documents). In `doc_labels.json` it is set to `rules`: the model adds nothing for it.
35
+ - **Every other type with a calibration value is fused at that value as an unvalidated suggestion, never enforcement.**
36
+ That is eleven labels: `legal.master_agreement`, `legal.sow`, `sales.rfp`, `sales.proposal`, `sales.pricing`,
37
+ `tech.source_code`, `finance.report`, `finance.billing`, `hr.personnel`, `corporate.strategy` and
38
+ `health.patient_record`. They are not validated: no recall, no per-language result and no line was evaluated for them,
39
+ and no accuracy is claimed. What was measured is false records on benign documents, in numbers: `health.patient_record`
40
+ **54 of 18,075**, `corporate.strategy` **12 of 18,075**, `legal.master_agreement` **5 of 18,075** (all German),
41
+ `finance.report` **5 of 18,075**, `sales.proposal` **4 of 18,075**, and 1 or 0 for the others (table below). A finding of
42
+ any of them is a suggestion for a person to look at; the counts say how often a benign document reached the value, and
43
+ nothing about how often a document of the type does.
44
+ `public.published` has no calibration value and stays `rules`.
45
+ - **One seed.** The head is seed 7001, the highest of three seeds on the development split. No three-seed repeat was done.
46
+
47
+ ### Files
48
+
49
+ | file | what it is |
50
+ |---|---|
51
+ | `onnx/model_quantized.onnx` | Two outputs: `logits` (PII, unchanged) and `doc_logits` `[batch, 17]`. 279,041,365 bytes, sha256 `0c9c1780673d719310afd1bce2538d8e078ea2adf0bc3a6e397c564ac6486b6e`. |
52
+ | `doc_labels.json` | Version 2: the 17 labels in output order and, per label, a mode, `t_add`, `t_confirm`, `delta` and a `status` that says whether the type is evaluated or an unvalidated suggestion. |
53
+ | `THIRD_PARTY_NOTICES.md` | Attributions for the training sources that ask for them. |
54
+ | `MANIFEST.sha256` | The sha256 of every file above. |
55
+
56
+ The encoder is the one at revision `b1a36210f99ef293a4d42697ef6b2ba548dd9208` (blob sha256
57
+ `c996fc5ad1118e0571263ee637f541b3d0981bec648e57211ad4676a3dc1ab47`). The tokenizer and `config.json` are the encoder's own.
58
+ No separate PII-only graph is included, so `pii_only_graph` in `doc_labels.json` is `null`.
59
+
60
+ ### The head
61
+
62
+ Two layers over the mean-pooled encoder states of a 450-character window: 768 to 256, exact GELU (written with `Erf`, as the
63
+ released graph already does), 256 to 17. The encoder is frozen. The graph is the released graph with nodes added; none of
64
+ its nodes is changed.
65
+
66
+ **The PII output is unchanged.** On this graph `logits` is byte-identical to the released encoder's on all 6,825 rows of the
67
+ PII release check, at batch size 1:
68
+
69
+ | host | onnxruntime | threads |
70
+ |---|---|---|
71
+ | aarch64 Linux (NVIDIA GB10 host) | 1.24.2 | 1 and 4 |
72
+ | Apple Mac, arm64 | 1.24.2 | 1 and 4 |
73
+ | Apple Mac, arm64 | 1.19.2 | 1 and 4 |
74
+
75
+ Not checked: the fp32 graph, Windows, and any batch size above 1.
76
+
77
+ ### Modes and thresholds
78
+
79
+ | mode | meaning |
80
+ |---|---|
81
+ | `rules` | A rule-based classifier decides. The head's score is information only. |
82
+ | `add` | The rule-based classifier's decision stands, unchanged. When the rules find nothing for the type and the document's score reaches `t_add`, the head's finding is reported as a model-added type: a suggestion, softer than a rule decision. |
83
+
84
+ False records are counted at `t_add` on 18,075 benign documents (see "What was measured").
85
+
86
+ | label | mode | `t_add` | false records | status |
87
+ |---|---|---|---|---|
88
+ | `legal.nda` | `add` | 0.84 | 1 of 18,075 | false-record line met; nothing else evaluated |
89
+ | `legal.master_agreement` | `add` | 0.82 | 5 of 18,075 | unvalidated suggestion |
90
+ | `legal.sow` | `add` | 0.87 | 1 of 18,075 | unvalidated suggestion |
91
+ | `legal.contract_other` | `rules` | | 8 of 18,075 | **not yet monitoring-grade**: line missed |
92
+ | `sales.rfp` | `add` | 0.70 | 1 of 18,075 | unvalidated suggestion |
93
+ | `sales.proposal` | `add` | 0.72 | 4 of 18,075 | unvalidated suggestion |
94
+ | `sales.pricing` | `add` | 0.90 | 1 of 18,075 | unvalidated suggestion |
95
+ | `tech.source_code` | `add` | 0.99 | 0 of 18,075 | unvalidated suggestion |
96
+ | `tech.design` | `add` | 0.97 | 0 of 18,075 | false-record line met; nothing else evaluated |
97
+ | `tech.security` | `add` | 0.97 | 2 of 18,075 | false-record line met; nothing else evaluated |
98
+ | `finance.report` | `add` | 0.98 | 5 of 18,075 | unvalidated suggestion |
99
+ | `finance.billing` | `add` | 0.69 | 1 of 18,075 | unvalidated suggestion |
100
+ | `finance.personal` | `add` | 0.81 | 0 of 18,075 | false-record line met; nothing else evaluated |
101
+ | `hr.personnel` | `add` | 0.93 | 0 of 18,075 | unvalidated suggestion |
102
+ | `corporate.strategy` | `add` | 0.66 | 12 of 18,075 | unvalidated suggestion |
103
+ | `public.published` | `rules` | | no reporting value | the rules decide |
104
+ | `health.patient_record` | `add` | 0.95 | 54 of 18,075 | unvalidated suggestion |
105
+
106
+ **How `t_add` was set.** Each is the *reporting value* of the type: the smallest value from 0.05 to 0.99 at which a
107
+ document's score (rule `lead2`, below) stays at or under a false-positive rate of 2.78 x 10^-4 in every one of seven
108
+ languages, with the upper 95% Wilson bound over all languages under it too, read on 80,739 ordinary documents of a
109
+ calibration set. **Setting `t_add` equal to the reporting value is a choice made for this release; it is not a registered
110
+ rule.** The reporting value says nothing about recall.
111
+
112
+ ### How a document is scored
113
+
114
+ 1. **Windows.** The text is cut into 450-character windows overlapping by 50 characters.
115
+ 2. **Scores.** Each window is scored on its own, at **batch size 1**, with `sigmoid(doc_logits)`.
116
+ 3. **Evidence rule, `lead2`.** A document reaches a type when both of its first two windows reach the threshold (the only
117
+ window, for a one-window text), or when at least `max(3, ceil(n / 5))` of its `n` windows do.
118
+
119
+ **The thresholds are valid at batch size 1 only.** The features and thresholds were produced one window per call on one
120
+ thread. On the same 200 windows, scores computed in padded batches of 64 differ from batch-1 scores by up to 0.53 (per
121
+ type the largest difference is between 0.14 and 0.53; 197 of the 200 windows moved by more than 0.001 on some type).
122
+ Thread count made no difference. Do not score in batches.
123
+
124
+ A file name or title is not used.
125
+
126
+ ### What was measured
127
+
128
+ **False records (held-out benign documents).** A false record is a benign document whose gold label lacks the type and whose
129
+ score is at or above the type's calibration value. Population: 18,075 benign documents from two held-out sets (English
130
+ 3,139; German 2,509; French 2,509; Italian 2,510; Spanish 2,509; Russian 2,450; Arabic 2,449), read once for this
131
+ head. Machine: aarch64 NVIDIA GB10 host, onnxruntime 1.24.2, batch 1, 1 thread. The line, fixed before the read: met when the
132
+ pooled count is at most 5 and no language has more than 2.
133
+
134
+ | type | value | pooled | by language | line |
135
+ |---|---|---|---|---|
136
+ | `legal.nda` | 0.84 | 1 | es 1 | met |
137
+ | `legal.contract_other` | 0.86 | 8 | en 3, de 3, es 1, ru 1 | **missed** |
138
+ | `tech.design` | 0.97 | 0 | none | met |
139
+ | `tech.security` | 0.97 | 2 | ru 2 | met |
140
+ | `finance.personal` | 0.81 | 0 | none | met |
141
+
142
+ Reported only, with no line and no verdict (these eleven types are fused as unvalidated suggestions; the numbers describe
143
+ the head's score on benign documents, not an evaluation of the type):
144
+
145
+ | type | value | pooled | by language |
146
+ |---|---|---|---|
147
+ | `health.patient_record` | 0.95 | 54 | en 33, es 6, it 6, fr 4, de 3, ru 2 |
148
+ | `corporate.strategy` | 0.66 | 12 | ru 3, en 2, de 2, it 2, es 2, fr 1 |
149
+ | `legal.master_agreement` | 0.82 | 5 | de 5 |
150
+ | `finance.report` | 0.98 | 5 | en 3, de 1, it 1 |
151
+ | `sales.proposal` | 0.72 | 4 | de 3, ar 1 |
152
+ | `legal.sow` | 0.87 | 1 | es 1 |
153
+ | `sales.rfp` | 0.70 | 1 | de 1 |
154
+ | `sales.pricing` | 0.90 | 1 | fr 1 |
155
+ | `finance.billing` | 0.69 | 1 | en 1 |
156
+ | `tech.source_code` | 0.99 | 0 | none |
157
+ | `hr.personnel` | 0.93 | 0 | none |
158
+
159
+ `public.published` has no calibration value. Meeting the false-record line does not make a type ready: recall,
160
+ agreement between machines and cost on customer hardware are further conditions, and none of them is met.
161
+
162
+ **`legal.contract_other` against the earlier preview (`doc-head-v2`), recall.** 484 held-out contract documents from 430
163
+ filers, each model at its own calibration value (V3a 0.86, v2 0.85), batch 1 for both. V3a found 448 (recall 0.9256, 95%
164
+ interval 0.9017 to 0.9481), v2 found 442 (0.9132, 0.8866 to 0.9380). Paired difference +0.0124 (interval -0.0063 to
165
+ +0.0327). The difference is inside the interval around zero: V3a is **not shown to be better** than v2 on this type. Absolute
166
+ recall is optimistic, because the set keeps only documents on which the rules and a model agreed; only the difference is a
167
+ comparison, and it is inside noise.
168
+
169
+ **Cost.** p50 time per 450-character window at batch size 1 (aarch64 NVIDIA GB10 host, onnxruntime 1.24.2, 200 windows, one
170
+ pass, no repeat): 41.6 ms on 1 thread and 14.1 ms on 4 threads for this graph; 41.5 and 13.7 ms for the released encoder
171
+ alone. The head's cost is **not distinguishable from zero** at this resolution (about plus or minus 0.5 ms); it is not
172
+ claimed to be zero. Peak memory 502 MB, against 501 MB for the encoder alone. Cost on a Mac, on Windows and on customer
173
+ machines was not evaluated, nor was engine time per file.
174
+
175
+ **PII output.** Byte-identical to the encoder's on 6,825 rows at batch 1 (table above).
176
+
177
+ ### Limitations
178
+
179
+ - **Machines disagree.** Two files of `doc_logits` for the same 6,825 rows, computed on the aarch64 Linux host and on an Apple
180
+ Mac, differ in bytes on every row. Against the Mac with onnxruntime 1.24.2, 611 rows fall on opposite sides of a
181
+ threshold for some evaluated type (`legal.nda` 466, `legal.contract_other` 167, `tech.security` 7); against the Mac with
182
+ 1.19.2, 138 (103, 35 and 2). Those files were produced without the engine's graph-arithmetic setting, so they are not
183
+ a measurement of an engine and not a test of agreement between machines; the engine-level cells on a Mac and on Windows are
184
+ pending, and the agreement bar between machines is **not met**. A document near a threshold may reach it on one machine
185
+ and not on another. Thresholds were read on the aarch64 Linux host only.
186
+ - **Batch composition changes scores** (above). Only batch size 1 is described here.
187
+ - **Not evaluated.** Recall of the four types on documents of the type; the cross-machine bar; cost on a Mac, on Windows and
188
+ on customer machines; engine time per file; a three-seed repeat; the fp32 graph; the PII output at batch sizes above 1;
189
+ `hr.personnel`; and, beyond the false records above, every type other than the four.
190
+ - **Training data.** Six types have no real training document, only generated text: `corporate.strategy`,
191
+ `finance.billing`, `health.patient_record`, `sales.pricing`, `sales.proposal` and `sales.rfp`. For the ten other types
192
+ that have real training documents, all the real documents are in English and their other-language training documents
193
+ are machine translations and generated text; only `public.published`, the examples of no type, has real documents in
194
+ other languages.
195
+
196
+ ### Training data
197
+
198
+ 47,824 training documents and 6,185 development documents. The training documents are 20,561 real, 6,680 machine
199
+ translations and 20,583 generated. The data is **not published**. Provenance, as it is:
200
+
201
+ - **Real public documents**, under open licences or in the public domain: agreements from CUAD, ContractNLI, the KL3M
202
+ collection of EDGAR agreements, Common Paper and Bonterms (CC BY 4.0), SEC EDGAR filings, official legislation (EU, French
203
+ and Italian texts), United Nations documents, European Union and UK government pages (the GOV.UK pages are used as
204
+ examples of documents that are none of the types, under the Open Government Licence v3.0), software and
205
+ security documentation and advisories, Python Enhancement Proposals, US drug labels, payslip examples, and Wikinews.
206
+ - **Machine translations** of registered training documents, made with OPUS-MT (CC BY 4.0).
207
+ - **Generated text** written by an open-weight language model released under a permissive open licence, from prompts
208
+ written for each type, and documents based on templates written by ScienceSoft.
209
+
210
+ Attributions that the sources ask for are in `THIRD_PARTY_NOTICES.md`. Text was cut into 450-character windows, pooled by the
211
+ frozen encoder and used only to fit the head; no source text is redistributed with the model.
212
+
213
+ ## Licence
214
+
215
+ Proprietary to ScienceSoft. The weights are published so that the results on this page can be
216
+ evaluated and reproduced; no licence to use them in production, to modify them or to
217
+ redistribute them is granted. For licensing, contact ScienceSoft (https://www.scnsoft.com).
218
+
219
+ **Basis for these terms.** The model is derived from
220
+ [`FacebookAI/xlm-roberta-base`](https://huggingface.co/FacebookAI/xlm-roberta-base) (MIT
221
+ licence). Its head was fitted on documents from the sources listed in
222
+ [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md) and described above. The licences those sources carry that this release has
223
+ checked (CC BY 4.0, the Open Government Licence v3.0, the Licence Ouverte 2.0, the European Commission's reuse policy for
224
+ EUR-Lex, the United Nations corpus terms, and public-domain and permissive software and documentation licences) permit a
225
+ derivative work to be distributed under proprietary terms, provided notices are retained and attribution is given; none
226
+ is share-alike and none restricts commercial use. The SEC EDGAR filings used for training come through a dataset whose card
227
+ says only that EDGAR data is publicly available; this release has not confirmed a licence statement for them beyond that.
THIRD_PARTY_NOTICES.md ADDED
@@ -0,0 +1,24 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+ ## Document-type head, V3a: training-data sources
3
+
4
+ The document-type head was trained on documents from the following sources, which ask for a notice or an attribution; the model card describes the other sources. Text was cut into 450-character windows, pooled by the frozen encoder and used only to fit a small two-layer head; no source text is redistributed with the model.
5
+
6
+ - Bonterms Standard Agreements, Bonterms (https://bonterms.com/legal-center), CC BY 4.0.
7
+ - Common Paper standard agreements, Common Paper (https://commonpaper.com/standards/), CC BY 4.0.
8
+ - ContractNLI dataset, Stanford NLP Group (https://stanfordnlp.github.io/contract-nli/), CC BY 4.0.
9
+ - Contract Understanding Atticus Dataset (CUAD), The Atticus Project (https://zenodo.org/records/4595826), CC BY 4.0.
10
+ - European Commission topic pages (https://commission.europa.eu/), (c) European Union, CC BY 4.0.
11
+ - GitHub Advisory Database, GitHub (https://github.com/github/advisory-database), CC BY 4.0.
12
+ - Kubernetes documentation, The Kubernetes Authors (https://kubernetes.io/docs/), CC BY 4.0.
13
+ - KL3M Data Project, EDGAR agreements, ALEA Institute (https://huggingface.co/datasets/alea-institute/kl3m-data-edgar-agreements), CC BY 4.0; text decoded with the kl3m-004-128k-cased tokenizer, ALEA Institute, CC BY 4.0.
14
+ - OPUS-MT translation model opus-mt-en-de, Helsinki-NLP (https://huggingface.co/Helsinki-NLP/opus-mt-en-de), CC BY 4.0, used to machine-translate registered training documents.
15
+ - Wikinews, Wikinews contributors (https://www.wikinews.org/). The English, Spanish and Italian editions are CC BY 4.0 today (earlier articles CC BY 2.5); the German edition is CC BY 2.5. The Arabic edition has been CC BY-SA 4.0 for text written since 2024-08-21; this head was trained only on Arabic text written before that date, which is CC BY 2.5. No French Wikinews text was used.
16
+ - Your Europe (https://europa.eu/youreurope/), (c) European Union, CC BY 4.0.
17
+ - Contains text from the United Nations Parallel Corpus (https://www.un.org/dgacm/en/content/uncorpus); the United Nations is acknowledged as the source of the information.
18
+ - Service Public fiches (https://entreprendre.service-public.gouv.fr/), Direction de l'information legale et administrative, Licence Ouverte / Open Licence 2.0 (etalab-2.0): source identified, texts as of the crawl date recorded in the curation.
19
+ - Contains public sector information published on GOV.UK (https://www.gov.uk/), licensed under the Open Government Licence v3.0 (https://www.nationalarchives.gov.uk/doc/open-government-licence/version/3/); used as examples of documents that are none of the types.
20
+ - EUR-Lex legal documents (https://eur-lex.europa.eu/), (c) European Union, re-used under the European Commission's reuse policy (Decision 2011/833/EU); editorial and consolidated content CC BY 4.0.
21
+ - Normattiva, Istituto Poligrafico e Zecca dello Stato (IPZS) (https://dati.normattiva.it/), CC BY 4.0.
22
+ - Legifrance codes in force (LEGI), Direction de l'information legale et administrative (DILA) (https://echanges.dila.gouv.fr/), Licence Ouverte / Open Licence 2.0 (etalab-2.0): source identified, download URL, file name and date as recorded in the curation.
23
+ - Python documentation: Copyright © 2001 Python Software Foundation; All Rights Reserved. Used under the Python Software Foundation License Version 2 (https://docs.python.org/3/license.html).
24
+ - "The Rust Programming Language" and its translations, The Rust Project Developers, under the MIT or Apache-2.0 licence.
doc_labels.json ADDED
@@ -0,0 +1,150 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "version": 2,
3
+ "encoder_revision": "b1a36210f99ef293a4d42697ef6b2ba548dd9208",
4
+ "encoder_onnx_sha256": "c996fc5ad1118e0571263ee637f541b3d0981bec648e57211ad4676a3dc1ab47",
5
+ "labels": [
6
+ "legal.nda",
7
+ "legal.master_agreement",
8
+ "legal.sow",
9
+ "legal.contract_other",
10
+ "sales.rfp",
11
+ "sales.proposal",
12
+ "sales.pricing",
13
+ "tech.source_code",
14
+ "tech.design",
15
+ "tech.security",
16
+ "finance.report",
17
+ "finance.billing",
18
+ "finance.personal",
19
+ "hr.personnel",
20
+ "corporate.strategy",
21
+ "public.published",
22
+ "health.patient_record"
23
+ ],
24
+ "types": {
25
+ "legal.nda": {
26
+ "mode": "add",
27
+ "t_add": 0.84,
28
+ "t_confirm": null,
29
+ "delta": 0,
30
+ "status": "false-record line met on one host; recall and other machines not evaluated"
31
+ },
32
+ "legal.master_agreement": {
33
+ "mode": "add",
34
+ "t_add": 0.82,
35
+ "t_confirm": null,
36
+ "delta": 0,
37
+ "status": "unvalidated suggestion: not evaluated, no accuracy claimed"
38
+ },
39
+ "legal.sow": {
40
+ "mode": "add",
41
+ "t_add": 0.87,
42
+ "t_confirm": null,
43
+ "delta": 0,
44
+ "status": "unvalidated suggestion: not evaluated, no accuracy claimed"
45
+ },
46
+ "legal.contract_other": {
47
+ "mode": "rules",
48
+ "t_add": null,
49
+ "t_confirm": null,
50
+ "delta": 0,
51
+ "status": "not yet monitoring-grade: the false-record line was missed"
52
+ },
53
+ "sales.rfp": {
54
+ "mode": "add",
55
+ "t_add": 0.7,
56
+ "t_confirm": null,
57
+ "delta": 0,
58
+ "status": "unvalidated suggestion: not evaluated, no accuracy claimed"
59
+ },
60
+ "sales.proposal": {
61
+ "mode": "add",
62
+ "t_add": 0.72,
63
+ "t_confirm": null,
64
+ "delta": 0,
65
+ "status": "unvalidated suggestion: not evaluated, no accuracy claimed"
66
+ },
67
+ "sales.pricing": {
68
+ "mode": "add",
69
+ "t_add": 0.9,
70
+ "t_confirm": null,
71
+ "delta": 0,
72
+ "status": "unvalidated suggestion: not evaluated, no accuracy claimed"
73
+ },
74
+ "tech.source_code": {
75
+ "mode": "add",
76
+ "t_add": 0.99,
77
+ "t_confirm": null,
78
+ "delta": 0,
79
+ "status": "unvalidated suggestion: not evaluated, no accuracy claimed"
80
+ },
81
+ "tech.design": {
82
+ "mode": "add",
83
+ "t_add": 0.97,
84
+ "t_confirm": null,
85
+ "delta": 0,
86
+ "status": "false-record line met on one host; recall and other machines not evaluated"
87
+ },
88
+ "tech.security": {
89
+ "mode": "add",
90
+ "t_add": 0.97,
91
+ "t_confirm": null,
92
+ "delta": 0,
93
+ "status": "false-record line met on one host; recall and other machines not evaluated"
94
+ },
95
+ "finance.report": {
96
+ "mode": "add",
97
+ "t_add": 0.98,
98
+ "t_confirm": null,
99
+ "delta": 0,
100
+ "status": "unvalidated suggestion: not evaluated, no accuracy claimed"
101
+ },
102
+ "finance.billing": {
103
+ "mode": "add",
104
+ "t_add": 0.69,
105
+ "t_confirm": null,
106
+ "delta": 0,
107
+ "status": "unvalidated suggestion: not evaluated, no accuracy claimed"
108
+ },
109
+ "finance.personal": {
110
+ "mode": "add",
111
+ "t_add": 0.81,
112
+ "t_confirm": null,
113
+ "delta": 0,
114
+ "status": "false-record line met on one host; recall and other machines not evaluated"
115
+ },
116
+ "hr.personnel": {
117
+ "mode": "add",
118
+ "t_add": 0.93,
119
+ "t_confirm": null,
120
+ "delta": 0,
121
+ "status": "unvalidated suggestion: not evaluated, no accuracy claimed"
122
+ },
123
+ "corporate.strategy": {
124
+ "mode": "add",
125
+ "t_add": 0.66,
126
+ "t_confirm": null,
127
+ "delta": 0,
128
+ "status": "unvalidated suggestion: not evaluated, no accuracy claimed"
129
+ },
130
+ "public.published": {
131
+ "mode": "rules",
132
+ "t_add": null,
133
+ "t_confirm": null,
134
+ "delta": 0,
135
+ "status": "no reporting value: the rules decide"
136
+ },
137
+ "health.patient_record": {
138
+ "mode": "add",
139
+ "t_add": 0.95,
140
+ "t_confirm": null,
141
+ "delta": 0,
142
+ "status": "unvalidated suggestion: not evaluated, no accuracy claimed"
143
+ }
144
+ },
145
+ "pooling": "mean",
146
+ "window_chars": 450,
147
+ "pii_only_graph": null,
148
+ "evidence": "lead2",
149
+ "thresholds_kind": "t_add is set equal to the type's reporting value, a mapping that is not a registered rule; every finding is a suggestion for supportive monitoring, never enforcement"
150
+ }
onnx/model_quantized.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0c9c1780673d719310afd1bce2538d8e078ea2adf0bc3a6e397c564ac6486b6e
3
+ size 279041365