Document-type head V3a s7001: supportive monitoring, evaluation and reproduction only
Browse files- MANIFEST.sha256 +4 -0
- README.md +227 -0
- THIRD_PARTY_NOTICES.md +24 -0
- doc_labels.json +150 -0
- onnx/model_quantized.onnx +3 -0
MANIFEST.sha256
ADDED
|
@@ -0,0 +1,4 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
3abd2ea8f76e3e1196228d41ccd7c7040d5cc03f12a99555e13d481af267ea46 README.md
|
| 2 |
+
7bf1257edaa508b110646407f151352c117e935001da42bd4f96810c53fb75e0 THIRD_PARTY_NOTICES.md
|
| 3 |
+
ddaff31d9e705ed3eb610467b3b783acd4116892887641c25db538a7215687f1 doc_labels.json
|
| 4 |
+
0c9c1780673d719310afd1bce2538d8e078ea2adf0bc3a6e397c564ac6486b6e onnx/model_quantized.onnx
|
README.md
ADDED
|
@@ -0,0 +1,227 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: other
|
| 3 |
+
license_name: proprietary-sciencesoft
|
| 4 |
+
license_link: https://huggingface.co/ScienceSoft/scnsoft-pii-encoder/blob/main/README.md#licence
|
| 5 |
+
language: [en, de, fr, it, es, ru, ar]
|
| 6 |
+
pipeline_tag: token-classification
|
| 7 |
+
tags: [pii, ner, privacy, dlp, onnx, xlm-roberta, document-classification]
|
| 8 |
+
library_name: onnxruntime
|
| 9 |
+
base_model: FacebookAI/xlm-roberta-base
|
| 10 |
+
base_model_relation: finetune
|
| 11 |
+
---
|
| 12 |
+
|
| 13 |
+
**Release candidate for supportive monitoring. Not an enforcement tool.** This is a document-type head, V3a, seed 7001. Its
|
| 14 |
+
scores support a person looking at documents; nothing here claims that a document should be blocked, redacted or refused
|
| 15 |
+
because of a score. The weights are published for evaluation and reproduction only (see Licence). The training data is not
|
| 16 |
+
published.
|
| 17 |
+
|
| 18 |
+
## Document-type head V3a (seed 7001)
|
| 19 |
+
|
| 20 |
+
This model adds a second output, `doc_logits`, to the same int8 graph as
|
| 21 |
+
[`ScienceSoft/scnsoft-pii-encoder`](https://huggingface.co/ScienceSoft/scnsoft-pii-encoder), so that a window of text is
|
| 22 |
+
also scored for 17 document types (non-disclosure and master agreements, statements of work, other contracts, requests
|
| 23 |
+
for proposals, proposals, price lists, source code, design and security documents, financial and personal-finance
|
| 24 |
+
documents, personnel files, strategy documents, published documents and patient records). The PII output (`logits`) is the
|
| 25 |
+
encoder's own.
|
| 26 |
+
|
| 27 |
+
### What this release says, and does not say
|
| 28 |
+
|
| 29 |
+
- **Four types have been evaluated, on one measure only.** `legal.nda`, `tech.design`, `tech.security` and
|
| 30 |
+
`finance.personal` met a false-record line on held-out benign documents on one machine (table below). That is the only
|
| 31 |
+
evaluation of them. Their recall on documents of the type, their behaviour on other machines and their cost on customer
|
| 32 |
+
hardware are not evaluated. Meeting the line does not make a type ready for any decision.
|
| 33 |
+
- **`legal.contract_other` is not yet monitoring-grade.** It missed the same line (8 false records on 18,075 benign
|
| 34 |
+
documents). In `doc_labels.json` it is set to `rules`: the model adds nothing for it.
|
| 35 |
+
- **Every other type with a calibration value is fused at that value as an unvalidated suggestion, never enforcement.**
|
| 36 |
+
That is eleven labels: `legal.master_agreement`, `legal.sow`, `sales.rfp`, `sales.proposal`, `sales.pricing`,
|
| 37 |
+
`tech.source_code`, `finance.report`, `finance.billing`, `hr.personnel`, `corporate.strategy` and
|
| 38 |
+
`health.patient_record`. They are not validated: no recall, no per-language result and no line was evaluated for them,
|
| 39 |
+
and no accuracy is claimed. What was measured is false records on benign documents, in numbers: `health.patient_record`
|
| 40 |
+
**54 of 18,075**, `corporate.strategy` **12 of 18,075**, `legal.master_agreement` **5 of 18,075** (all German),
|
| 41 |
+
`finance.report` **5 of 18,075**, `sales.proposal` **4 of 18,075**, and 1 or 0 for the others (table below). A finding of
|
| 42 |
+
any of them is a suggestion for a person to look at; the counts say how often a benign document reached the value, and
|
| 43 |
+
nothing about how often a document of the type does.
|
| 44 |
+
`public.published` has no calibration value and stays `rules`.
|
| 45 |
+
- **One seed.** The head is seed 7001, the highest of three seeds on the development split. No three-seed repeat was done.
|
| 46 |
+
|
| 47 |
+
### Files
|
| 48 |
+
|
| 49 |
+
| file | what it is |
|
| 50 |
+
|---|---|
|
| 51 |
+
| `onnx/model_quantized.onnx` | Two outputs: `logits` (PII, unchanged) and `doc_logits` `[batch, 17]`. 279,041,365 bytes, sha256 `0c9c1780673d719310afd1bce2538d8e078ea2adf0bc3a6e397c564ac6486b6e`. |
|
| 52 |
+
| `doc_labels.json` | Version 2: the 17 labels in output order and, per label, a mode, `t_add`, `t_confirm`, `delta` and a `status` that says whether the type is evaluated or an unvalidated suggestion. |
|
| 53 |
+
| `THIRD_PARTY_NOTICES.md` | Attributions for the training sources that ask for them. |
|
| 54 |
+
| `MANIFEST.sha256` | The sha256 of every file above. |
|
| 55 |
+
|
| 56 |
+
The encoder is the one at revision `b1a36210f99ef293a4d42697ef6b2ba548dd9208` (blob sha256
|
| 57 |
+
`c996fc5ad1118e0571263ee637f541b3d0981bec648e57211ad4676a3dc1ab47`). The tokenizer and `config.json` are the encoder's own.
|
| 58 |
+
No separate PII-only graph is included, so `pii_only_graph` in `doc_labels.json` is `null`.
|
| 59 |
+
|
| 60 |
+
### The head
|
| 61 |
+
|
| 62 |
+
Two layers over the mean-pooled encoder states of a 450-character window: 768 to 256, exact GELU (written with `Erf`, as the
|
| 63 |
+
released graph already does), 256 to 17. The encoder is frozen. The graph is the released graph with nodes added; none of
|
| 64 |
+
its nodes is changed.
|
| 65 |
+
|
| 66 |
+
**The PII output is unchanged.** On this graph `logits` is byte-identical to the released encoder's on all 6,825 rows of the
|
| 67 |
+
PII release check, at batch size 1:
|
| 68 |
+
|
| 69 |
+
| host | onnxruntime | threads |
|
| 70 |
+
|---|---|---|
|
| 71 |
+
| aarch64 Linux (NVIDIA GB10 host) | 1.24.2 | 1 and 4 |
|
| 72 |
+
| Apple Mac, arm64 | 1.24.2 | 1 and 4 |
|
| 73 |
+
| Apple Mac, arm64 | 1.19.2 | 1 and 4 |
|
| 74 |
+
|
| 75 |
+
Not checked: the fp32 graph, Windows, and any batch size above 1.
|
| 76 |
+
|
| 77 |
+
### Modes and thresholds
|
| 78 |
+
|
| 79 |
+
| mode | meaning |
|
| 80 |
+
|---|---|
|
| 81 |
+
| `rules` | A rule-based classifier decides. The head's score is information only. |
|
| 82 |
+
| `add` | The rule-based classifier's decision stands, unchanged. When the rules find nothing for the type and the document's score reaches `t_add`, the head's finding is reported as a model-added type: a suggestion, softer than a rule decision. |
|
| 83 |
+
|
| 84 |
+
False records are counted at `t_add` on 18,075 benign documents (see "What was measured").
|
| 85 |
+
|
| 86 |
+
| label | mode | `t_add` | false records | status |
|
| 87 |
+
|---|---|---|---|---|
|
| 88 |
+
| `legal.nda` | `add` | 0.84 | 1 of 18,075 | false-record line met; nothing else evaluated |
|
| 89 |
+
| `legal.master_agreement` | `add` | 0.82 | 5 of 18,075 | unvalidated suggestion |
|
| 90 |
+
| `legal.sow` | `add` | 0.87 | 1 of 18,075 | unvalidated suggestion |
|
| 91 |
+
| `legal.contract_other` | `rules` | | 8 of 18,075 | **not yet monitoring-grade**: line missed |
|
| 92 |
+
| `sales.rfp` | `add` | 0.70 | 1 of 18,075 | unvalidated suggestion |
|
| 93 |
+
| `sales.proposal` | `add` | 0.72 | 4 of 18,075 | unvalidated suggestion |
|
| 94 |
+
| `sales.pricing` | `add` | 0.90 | 1 of 18,075 | unvalidated suggestion |
|
| 95 |
+
| `tech.source_code` | `add` | 0.99 | 0 of 18,075 | unvalidated suggestion |
|
| 96 |
+
| `tech.design` | `add` | 0.97 | 0 of 18,075 | false-record line met; nothing else evaluated |
|
| 97 |
+
| `tech.security` | `add` | 0.97 | 2 of 18,075 | false-record line met; nothing else evaluated |
|
| 98 |
+
| `finance.report` | `add` | 0.98 | 5 of 18,075 | unvalidated suggestion |
|
| 99 |
+
| `finance.billing` | `add` | 0.69 | 1 of 18,075 | unvalidated suggestion |
|
| 100 |
+
| `finance.personal` | `add` | 0.81 | 0 of 18,075 | false-record line met; nothing else evaluated |
|
| 101 |
+
| `hr.personnel` | `add` | 0.93 | 0 of 18,075 | unvalidated suggestion |
|
| 102 |
+
| `corporate.strategy` | `add` | 0.66 | 12 of 18,075 | unvalidated suggestion |
|
| 103 |
+
| `public.published` | `rules` | | no reporting value | the rules decide |
|
| 104 |
+
| `health.patient_record` | `add` | 0.95 | 54 of 18,075 | unvalidated suggestion |
|
| 105 |
+
|
| 106 |
+
**How `t_add` was set.** Each is the *reporting value* of the type: the smallest value from 0.05 to 0.99 at which a
|
| 107 |
+
document's score (rule `lead2`, below) stays at or under a false-positive rate of 2.78 x 10^-4 in every one of seven
|
| 108 |
+
languages, with the upper 95% Wilson bound over all languages under it too, read on 80,739 ordinary documents of a
|
| 109 |
+
calibration set. **Setting `t_add` equal to the reporting value is a choice made for this release; it is not a registered
|
| 110 |
+
rule.** The reporting value says nothing about recall.
|
| 111 |
+
|
| 112 |
+
### How a document is scored
|
| 113 |
+
|
| 114 |
+
1. **Windows.** The text is cut into 450-character windows overlapping by 50 characters.
|
| 115 |
+
2. **Scores.** Each window is scored on its own, at **batch size 1**, with `sigmoid(doc_logits)`.
|
| 116 |
+
3. **Evidence rule, `lead2`.** A document reaches a type when both of its first two windows reach the threshold (the only
|
| 117 |
+
window, for a one-window text), or when at least `max(3, ceil(n / 5))` of its `n` windows do.
|
| 118 |
+
|
| 119 |
+
**The thresholds are valid at batch size 1 only.** The features and thresholds were produced one window per call on one
|
| 120 |
+
thread. On the same 200 windows, scores computed in padded batches of 64 differ from batch-1 scores by up to 0.53 (per
|
| 121 |
+
type the largest difference is between 0.14 and 0.53; 197 of the 200 windows moved by more than 0.001 on some type).
|
| 122 |
+
Thread count made no difference. Do not score in batches.
|
| 123 |
+
|
| 124 |
+
A file name or title is not used.
|
| 125 |
+
|
| 126 |
+
### What was measured
|
| 127 |
+
|
| 128 |
+
**False records (held-out benign documents).** A false record is a benign document whose gold label lacks the type and whose
|
| 129 |
+
score is at or above the type's calibration value. Population: 18,075 benign documents from two held-out sets (English
|
| 130 |
+
3,139; German 2,509; French 2,509; Italian 2,510; Spanish 2,509; Russian 2,450; Arabic 2,449), read once for this
|
| 131 |
+
head. Machine: aarch64 NVIDIA GB10 host, onnxruntime 1.24.2, batch 1, 1 thread. The line, fixed before the read: met when the
|
| 132 |
+
pooled count is at most 5 and no language has more than 2.
|
| 133 |
+
|
| 134 |
+
| type | value | pooled | by language | line |
|
| 135 |
+
|---|---|---|---|---|
|
| 136 |
+
| `legal.nda` | 0.84 | 1 | es 1 | met |
|
| 137 |
+
| `legal.contract_other` | 0.86 | 8 | en 3, de 3, es 1, ru 1 | **missed** |
|
| 138 |
+
| `tech.design` | 0.97 | 0 | none | met |
|
| 139 |
+
| `tech.security` | 0.97 | 2 | ru 2 | met |
|
| 140 |
+
| `finance.personal` | 0.81 | 0 | none | met |
|
| 141 |
+
|
| 142 |
+
Reported only, with no line and no verdict (these eleven types are fused as unvalidated suggestions; the numbers describe
|
| 143 |
+
the head's score on benign documents, not an evaluation of the type):
|
| 144 |
+
|
| 145 |
+
| type | value | pooled | by language |
|
| 146 |
+
|---|---|---|---|
|
| 147 |
+
| `health.patient_record` | 0.95 | 54 | en 33, es 6, it 6, fr 4, de 3, ru 2 |
|
| 148 |
+
| `corporate.strategy` | 0.66 | 12 | ru 3, en 2, de 2, it 2, es 2, fr 1 |
|
| 149 |
+
| `legal.master_agreement` | 0.82 | 5 | de 5 |
|
| 150 |
+
| `finance.report` | 0.98 | 5 | en 3, de 1, it 1 |
|
| 151 |
+
| `sales.proposal` | 0.72 | 4 | de 3, ar 1 |
|
| 152 |
+
| `legal.sow` | 0.87 | 1 | es 1 |
|
| 153 |
+
| `sales.rfp` | 0.70 | 1 | de 1 |
|
| 154 |
+
| `sales.pricing` | 0.90 | 1 | fr 1 |
|
| 155 |
+
| `finance.billing` | 0.69 | 1 | en 1 |
|
| 156 |
+
| `tech.source_code` | 0.99 | 0 | none |
|
| 157 |
+
| `hr.personnel` | 0.93 | 0 | none |
|
| 158 |
+
|
| 159 |
+
`public.published` has no calibration value. Meeting the false-record line does not make a type ready: recall,
|
| 160 |
+
agreement between machines and cost on customer hardware are further conditions, and none of them is met.
|
| 161 |
+
|
| 162 |
+
**`legal.contract_other` against the earlier preview (`doc-head-v2`), recall.** 484 held-out contract documents from 430
|
| 163 |
+
filers, each model at its own calibration value (V3a 0.86, v2 0.85), batch 1 for both. V3a found 448 (recall 0.9256, 95%
|
| 164 |
+
interval 0.9017 to 0.9481), v2 found 442 (0.9132, 0.8866 to 0.9380). Paired difference +0.0124 (interval -0.0063 to
|
| 165 |
+
+0.0327). The difference is inside the interval around zero: V3a is **not shown to be better** than v2 on this type. Absolute
|
| 166 |
+
recall is optimistic, because the set keeps only documents on which the rules and a model agreed; only the difference is a
|
| 167 |
+
comparison, and it is inside noise.
|
| 168 |
+
|
| 169 |
+
**Cost.** p50 time per 450-character window at batch size 1 (aarch64 NVIDIA GB10 host, onnxruntime 1.24.2, 200 windows, one
|
| 170 |
+
pass, no repeat): 41.6 ms on 1 thread and 14.1 ms on 4 threads for this graph; 41.5 and 13.7 ms for the released encoder
|
| 171 |
+
alone. The head's cost is **not distinguishable from zero** at this resolution (about plus or minus 0.5 ms); it is not
|
| 172 |
+
claimed to be zero. Peak memory 502 MB, against 501 MB for the encoder alone. Cost on a Mac, on Windows and on customer
|
| 173 |
+
machines was not evaluated, nor was engine time per file.
|
| 174 |
+
|
| 175 |
+
**PII output.** Byte-identical to the encoder's on 6,825 rows at batch 1 (table above).
|
| 176 |
+
|
| 177 |
+
### Limitations
|
| 178 |
+
|
| 179 |
+
- **Machines disagree.** Two files of `doc_logits` for the same 6,825 rows, computed on the aarch64 Linux host and on an Apple
|
| 180 |
+
Mac, differ in bytes on every row. Against the Mac with onnxruntime 1.24.2, 611 rows fall on opposite sides of a
|
| 181 |
+
threshold for some evaluated type (`legal.nda` 466, `legal.contract_other` 167, `tech.security` 7); against the Mac with
|
| 182 |
+
1.19.2, 138 (103, 35 and 2). Those files were produced without the engine's graph-arithmetic setting, so they are not
|
| 183 |
+
a measurement of an engine and not a test of agreement between machines; the engine-level cells on a Mac and on Windows are
|
| 184 |
+
pending, and the agreement bar between machines is **not met**. A document near a threshold may reach it on one machine
|
| 185 |
+
and not on another. Thresholds were read on the aarch64 Linux host only.
|
| 186 |
+
- **Batch composition changes scores** (above). Only batch size 1 is described here.
|
| 187 |
+
- **Not evaluated.** Recall of the four types on documents of the type; the cross-machine bar; cost on a Mac, on Windows and
|
| 188 |
+
on customer machines; engine time per file; a three-seed repeat; the fp32 graph; the PII output at batch sizes above 1;
|
| 189 |
+
`hr.personnel`; and, beyond the false records above, every type other than the four.
|
| 190 |
+
- **Training data.** Six types have no real training document, only generated text: `corporate.strategy`,
|
| 191 |
+
`finance.billing`, `health.patient_record`, `sales.pricing`, `sales.proposal` and `sales.rfp`. For the ten other types
|
| 192 |
+
that have real training documents, all the real documents are in English and their other-language training documents
|
| 193 |
+
are machine translations and generated text; only `public.published`, the examples of no type, has real documents in
|
| 194 |
+
other languages.
|
| 195 |
+
|
| 196 |
+
### Training data
|
| 197 |
+
|
| 198 |
+
47,824 training documents and 6,185 development documents. The training documents are 20,561 real, 6,680 machine
|
| 199 |
+
translations and 20,583 generated. The data is **not published**. Provenance, as it is:
|
| 200 |
+
|
| 201 |
+
- **Real public documents**, under open licences or in the public domain: agreements from CUAD, ContractNLI, the KL3M
|
| 202 |
+
collection of EDGAR agreements, Common Paper and Bonterms (CC BY 4.0), SEC EDGAR filings, official legislation (EU, French
|
| 203 |
+
and Italian texts), United Nations documents, European Union and UK government pages (the GOV.UK pages are used as
|
| 204 |
+
examples of documents that are none of the types, under the Open Government Licence v3.0), software and
|
| 205 |
+
security documentation and advisories, Python Enhancement Proposals, US drug labels, payslip examples, and Wikinews.
|
| 206 |
+
- **Machine translations** of registered training documents, made with OPUS-MT (CC BY 4.0).
|
| 207 |
+
- **Generated text** written by an open-weight language model released under a permissive open licence, from prompts
|
| 208 |
+
written for each type, and documents based on templates written by ScienceSoft.
|
| 209 |
+
|
| 210 |
+
Attributions that the sources ask for are in `THIRD_PARTY_NOTICES.md`. Text was cut into 450-character windows, pooled by the
|
| 211 |
+
frozen encoder and used only to fit the head; no source text is redistributed with the model.
|
| 212 |
+
|
| 213 |
+
## Licence
|
| 214 |
+
|
| 215 |
+
Proprietary to ScienceSoft. The weights are published so that the results on this page can be
|
| 216 |
+
evaluated and reproduced; no licence to use them in production, to modify them or to
|
| 217 |
+
redistribute them is granted. For licensing, contact ScienceSoft (https://www.scnsoft.com).
|
| 218 |
+
|
| 219 |
+
**Basis for these terms.** The model is derived from
|
| 220 |
+
[`FacebookAI/xlm-roberta-base`](https://huggingface.co/FacebookAI/xlm-roberta-base) (MIT
|
| 221 |
+
licence). Its head was fitted on documents from the sources listed in
|
| 222 |
+
[THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md) and described above. The licences those sources carry that this release has
|
| 223 |
+
checked (CC BY 4.0, the Open Government Licence v3.0, the Licence Ouverte 2.0, the European Commission's reuse policy for
|
| 224 |
+
EUR-Lex, the United Nations corpus terms, and public-domain and permissive software and documentation licences) permit a
|
| 225 |
+
derivative work to be distributed under proprietary terms, provided notices are retained and attribution is given; none
|
| 226 |
+
is share-alike and none restricts commercial use. The SEC EDGAR filings used for training come through a dataset whose card
|
| 227 |
+
says only that EDGAR data is publicly available; this release has not confirmed a licence statement for them beyond that.
|
THIRD_PARTY_NOTICES.md
ADDED
|
@@ -0,0 +1,24 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
|
| 2 |
+
## Document-type head, V3a: training-data sources
|
| 3 |
+
|
| 4 |
+
The document-type head was trained on documents from the following sources, which ask for a notice or an attribution; the model card describes the other sources. Text was cut into 450-character windows, pooled by the frozen encoder and used only to fit a small two-layer head; no source text is redistributed with the model.
|
| 5 |
+
|
| 6 |
+
- Bonterms Standard Agreements, Bonterms (https://bonterms.com/legal-center), CC BY 4.0.
|
| 7 |
+
- Common Paper standard agreements, Common Paper (https://commonpaper.com/standards/), CC BY 4.0.
|
| 8 |
+
- ContractNLI dataset, Stanford NLP Group (https://stanfordnlp.github.io/contract-nli/), CC BY 4.0.
|
| 9 |
+
- Contract Understanding Atticus Dataset (CUAD), The Atticus Project (https://zenodo.org/records/4595826), CC BY 4.0.
|
| 10 |
+
- European Commission topic pages (https://commission.europa.eu/), (c) European Union, CC BY 4.0.
|
| 11 |
+
- GitHub Advisory Database, GitHub (https://github.com/github/advisory-database), CC BY 4.0.
|
| 12 |
+
- Kubernetes documentation, The Kubernetes Authors (https://kubernetes.io/docs/), CC BY 4.0.
|
| 13 |
+
- KL3M Data Project, EDGAR agreements, ALEA Institute (https://huggingface.co/datasets/alea-institute/kl3m-data-edgar-agreements), CC BY 4.0; text decoded with the kl3m-004-128k-cased tokenizer, ALEA Institute, CC BY 4.0.
|
| 14 |
+
- OPUS-MT translation model opus-mt-en-de, Helsinki-NLP (https://huggingface.co/Helsinki-NLP/opus-mt-en-de), CC BY 4.0, used to machine-translate registered training documents.
|
| 15 |
+
- Wikinews, Wikinews contributors (https://www.wikinews.org/). The English, Spanish and Italian editions are CC BY 4.0 today (earlier articles CC BY 2.5); the German edition is CC BY 2.5. The Arabic edition has been CC BY-SA 4.0 for text written since 2024-08-21; this head was trained only on Arabic text written before that date, which is CC BY 2.5. No French Wikinews text was used.
|
| 16 |
+
- Your Europe (https://europa.eu/youreurope/), (c) European Union, CC BY 4.0.
|
| 17 |
+
- Contains text from the United Nations Parallel Corpus (https://www.un.org/dgacm/en/content/uncorpus); the United Nations is acknowledged as the source of the information.
|
| 18 |
+
- Service Public fiches (https://entreprendre.service-public.gouv.fr/), Direction de l'information legale et administrative, Licence Ouverte / Open Licence 2.0 (etalab-2.0): source identified, texts as of the crawl date recorded in the curation.
|
| 19 |
+
- Contains public sector information published on GOV.UK (https://www.gov.uk/), licensed under the Open Government Licence v3.0 (https://www.nationalarchives.gov.uk/doc/open-government-licence/version/3/); used as examples of documents that are none of the types.
|
| 20 |
+
- EUR-Lex legal documents (https://eur-lex.europa.eu/), (c) European Union, re-used under the European Commission's reuse policy (Decision 2011/833/EU); editorial and consolidated content CC BY 4.0.
|
| 21 |
+
- Normattiva, Istituto Poligrafico e Zecca dello Stato (IPZS) (https://dati.normattiva.it/), CC BY 4.0.
|
| 22 |
+
- Legifrance codes in force (LEGI), Direction de l'information legale et administrative (DILA) (https://echanges.dila.gouv.fr/), Licence Ouverte / Open Licence 2.0 (etalab-2.0): source identified, download URL, file name and date as recorded in the curation.
|
| 23 |
+
- Python documentation: Copyright © 2001 Python Software Foundation; All Rights Reserved. Used under the Python Software Foundation License Version 2 (https://docs.python.org/3/license.html).
|
| 24 |
+
- "The Rust Programming Language" and its translations, The Rust Project Developers, under the MIT or Apache-2.0 licence.
|
doc_labels.json
ADDED
|
@@ -0,0 +1,150 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"version": 2,
|
| 3 |
+
"encoder_revision": "b1a36210f99ef293a4d42697ef6b2ba548dd9208",
|
| 4 |
+
"encoder_onnx_sha256": "c996fc5ad1118e0571263ee637f541b3d0981bec648e57211ad4676a3dc1ab47",
|
| 5 |
+
"labels": [
|
| 6 |
+
"legal.nda",
|
| 7 |
+
"legal.master_agreement",
|
| 8 |
+
"legal.sow",
|
| 9 |
+
"legal.contract_other",
|
| 10 |
+
"sales.rfp",
|
| 11 |
+
"sales.proposal",
|
| 12 |
+
"sales.pricing",
|
| 13 |
+
"tech.source_code",
|
| 14 |
+
"tech.design",
|
| 15 |
+
"tech.security",
|
| 16 |
+
"finance.report",
|
| 17 |
+
"finance.billing",
|
| 18 |
+
"finance.personal",
|
| 19 |
+
"hr.personnel",
|
| 20 |
+
"corporate.strategy",
|
| 21 |
+
"public.published",
|
| 22 |
+
"health.patient_record"
|
| 23 |
+
],
|
| 24 |
+
"types": {
|
| 25 |
+
"legal.nda": {
|
| 26 |
+
"mode": "add",
|
| 27 |
+
"t_add": 0.84,
|
| 28 |
+
"t_confirm": null,
|
| 29 |
+
"delta": 0,
|
| 30 |
+
"status": "false-record line met on one host; recall and other machines not evaluated"
|
| 31 |
+
},
|
| 32 |
+
"legal.master_agreement": {
|
| 33 |
+
"mode": "add",
|
| 34 |
+
"t_add": 0.82,
|
| 35 |
+
"t_confirm": null,
|
| 36 |
+
"delta": 0,
|
| 37 |
+
"status": "unvalidated suggestion: not evaluated, no accuracy claimed"
|
| 38 |
+
},
|
| 39 |
+
"legal.sow": {
|
| 40 |
+
"mode": "add",
|
| 41 |
+
"t_add": 0.87,
|
| 42 |
+
"t_confirm": null,
|
| 43 |
+
"delta": 0,
|
| 44 |
+
"status": "unvalidated suggestion: not evaluated, no accuracy claimed"
|
| 45 |
+
},
|
| 46 |
+
"legal.contract_other": {
|
| 47 |
+
"mode": "rules",
|
| 48 |
+
"t_add": null,
|
| 49 |
+
"t_confirm": null,
|
| 50 |
+
"delta": 0,
|
| 51 |
+
"status": "not yet monitoring-grade: the false-record line was missed"
|
| 52 |
+
},
|
| 53 |
+
"sales.rfp": {
|
| 54 |
+
"mode": "add",
|
| 55 |
+
"t_add": 0.7,
|
| 56 |
+
"t_confirm": null,
|
| 57 |
+
"delta": 0,
|
| 58 |
+
"status": "unvalidated suggestion: not evaluated, no accuracy claimed"
|
| 59 |
+
},
|
| 60 |
+
"sales.proposal": {
|
| 61 |
+
"mode": "add",
|
| 62 |
+
"t_add": 0.72,
|
| 63 |
+
"t_confirm": null,
|
| 64 |
+
"delta": 0,
|
| 65 |
+
"status": "unvalidated suggestion: not evaluated, no accuracy claimed"
|
| 66 |
+
},
|
| 67 |
+
"sales.pricing": {
|
| 68 |
+
"mode": "add",
|
| 69 |
+
"t_add": 0.9,
|
| 70 |
+
"t_confirm": null,
|
| 71 |
+
"delta": 0,
|
| 72 |
+
"status": "unvalidated suggestion: not evaluated, no accuracy claimed"
|
| 73 |
+
},
|
| 74 |
+
"tech.source_code": {
|
| 75 |
+
"mode": "add",
|
| 76 |
+
"t_add": 0.99,
|
| 77 |
+
"t_confirm": null,
|
| 78 |
+
"delta": 0,
|
| 79 |
+
"status": "unvalidated suggestion: not evaluated, no accuracy claimed"
|
| 80 |
+
},
|
| 81 |
+
"tech.design": {
|
| 82 |
+
"mode": "add",
|
| 83 |
+
"t_add": 0.97,
|
| 84 |
+
"t_confirm": null,
|
| 85 |
+
"delta": 0,
|
| 86 |
+
"status": "false-record line met on one host; recall and other machines not evaluated"
|
| 87 |
+
},
|
| 88 |
+
"tech.security": {
|
| 89 |
+
"mode": "add",
|
| 90 |
+
"t_add": 0.97,
|
| 91 |
+
"t_confirm": null,
|
| 92 |
+
"delta": 0,
|
| 93 |
+
"status": "false-record line met on one host; recall and other machines not evaluated"
|
| 94 |
+
},
|
| 95 |
+
"finance.report": {
|
| 96 |
+
"mode": "add",
|
| 97 |
+
"t_add": 0.98,
|
| 98 |
+
"t_confirm": null,
|
| 99 |
+
"delta": 0,
|
| 100 |
+
"status": "unvalidated suggestion: not evaluated, no accuracy claimed"
|
| 101 |
+
},
|
| 102 |
+
"finance.billing": {
|
| 103 |
+
"mode": "add",
|
| 104 |
+
"t_add": 0.69,
|
| 105 |
+
"t_confirm": null,
|
| 106 |
+
"delta": 0,
|
| 107 |
+
"status": "unvalidated suggestion: not evaluated, no accuracy claimed"
|
| 108 |
+
},
|
| 109 |
+
"finance.personal": {
|
| 110 |
+
"mode": "add",
|
| 111 |
+
"t_add": 0.81,
|
| 112 |
+
"t_confirm": null,
|
| 113 |
+
"delta": 0,
|
| 114 |
+
"status": "false-record line met on one host; recall and other machines not evaluated"
|
| 115 |
+
},
|
| 116 |
+
"hr.personnel": {
|
| 117 |
+
"mode": "add",
|
| 118 |
+
"t_add": 0.93,
|
| 119 |
+
"t_confirm": null,
|
| 120 |
+
"delta": 0,
|
| 121 |
+
"status": "unvalidated suggestion: not evaluated, no accuracy claimed"
|
| 122 |
+
},
|
| 123 |
+
"corporate.strategy": {
|
| 124 |
+
"mode": "add",
|
| 125 |
+
"t_add": 0.66,
|
| 126 |
+
"t_confirm": null,
|
| 127 |
+
"delta": 0,
|
| 128 |
+
"status": "unvalidated suggestion: not evaluated, no accuracy claimed"
|
| 129 |
+
},
|
| 130 |
+
"public.published": {
|
| 131 |
+
"mode": "rules",
|
| 132 |
+
"t_add": null,
|
| 133 |
+
"t_confirm": null,
|
| 134 |
+
"delta": 0,
|
| 135 |
+
"status": "no reporting value: the rules decide"
|
| 136 |
+
},
|
| 137 |
+
"health.patient_record": {
|
| 138 |
+
"mode": "add",
|
| 139 |
+
"t_add": 0.95,
|
| 140 |
+
"t_confirm": null,
|
| 141 |
+
"delta": 0,
|
| 142 |
+
"status": "unvalidated suggestion: not evaluated, no accuracy claimed"
|
| 143 |
+
}
|
| 144 |
+
},
|
| 145 |
+
"pooling": "mean",
|
| 146 |
+
"window_chars": 450,
|
| 147 |
+
"pii_only_graph": null,
|
| 148 |
+
"evidence": "lead2",
|
| 149 |
+
"thresholds_kind": "t_add is set equal to the type's reporting value, a mapping that is not a registered rule; every finding is a suggestion for supportive monitoring, never enforcement"
|
| 150 |
+
}
|
onnx/model_quantized.onnx
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:0c9c1780673d719310afd1bce2538d8e078ea2adf0bc3a6e397c564ac6486b6e
|
| 3 |
+
size 279041365
|