Card: metadata, evaluation results, load from Hugging Face
Browse files
README.md
CHANGED
|
@@ -13,10 +13,80 @@ tags:
|
|
| 13 |
- pii
|
| 14 |
- privacy
|
| 15 |
- ner
|
|
|
|
| 16 |
- anonymization
|
|
|
|
| 17 |
- on-device
|
| 18 |
- browser
|
| 19 |
- typescript
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 20 |
---
|
| 21 |
|
| 22 |
# Veil
|
|
@@ -35,7 +105,7 @@ import { loadModel } from "@tars-technologies/veil/model"
|
|
| 35 |
const veil = createVeil({ model: loadModel() })
|
| 36 |
|
| 37 |
await veil.mask({
|
| 38 |
-
text: "I'm Laura Bianchi, call me on +33 6 12 34
|
| 39 |
types: ["PERSON", "PHONE", "EMAIL"]
|
| 40 |
})
|
| 41 |
// "I'm [NAME], call me on [PHONE] or write to [EMAIL]"
|
|
@@ -44,11 +114,20 @@ await veil.mask({
|
|
| 44 |
## Install
|
| 45 |
|
| 46 |
```sh
|
| 47 |
-
npm install github:Tars-Technologies/veil
|
| 48 |
```
|
| 49 |
|
| 50 |
The package ships TypeScript source. Use it with a bundler or runtime that compiles TypeScript, such as Vite,
|
| 51 |
-
esbuild, tsx or Bun.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 52 |
|
| 53 |
## Usage
|
| 54 |
|
|
@@ -117,19 +196,21 @@ const { data, spans, isPending, error } = useMask({ text: draft, types: ["PERSON
|
|
| 117 |
|---|---|
|
| 118 |
| Author | Vaibhav Kubre, Tars Technologies |
|
| 119 |
| Labels | `PERSON`, `ADDRESS`, `DOB`, `ID`, `FINANCIAL`, `PHONE`, `EMAIL` |
|
| 120 |
-
| Architecture | Hashed character n-gram embeddings (32,768 buckets × 64, 2-bit codes), word-shape features and three word-list flags, into a 2-layer bidirectional LSTM with 96 units.
|
| 121 |
| Size | 1,177,096 bytes (`veil-0.3.1.bin`) |
|
| 122 |
| Languages | Latin script. Most training text is English, German, French, Spanish, Dutch, Portuguese or Italian. |
|
| 123 |
| Input | Windows of 256 tokens with a 64-token overlap. Up to 300 characters of `context`. |
|
| 124 |
-
| Licence | Apache-2.0 for code and model. `NOTICE` credits the data sources. |
|
| 125 |
|
| 126 |
The model file carries its name, version and format in a header and ends with a checksum. The library reads format 3
|
| 127 |
and rejects a file whose checksum does not match. Each new model gets a new file name.
|
| 128 |
|
| 129 |
## Evaluation
|
| 130 |
|
| 131 |
-
We measured `PERSON` on five public test sets. None of them is in the training data.
|
| 132 |
-
|
|
|
|
|
|
|
| 133 |
|
| 134 |
| Test set | Names | Veil 0.3.1 (1.2 MB) | spaCy sm (15 MB) | newmean nano (2.6 MB) | newmean small (7.1 MB) | newmean large (20.7 MB) |
|
| 135 |
|---|---|---|---|---|---|---|
|
|
@@ -164,7 +245,7 @@ The training data includes these public datasets:
|
|
| 164 |
|
| 165 |
| Source | Rows | Licence |
|
| 166 |
|---|---|---|
|
| 167 |
-
| Wikipedia articles (DE, FR, EN), labelled by a larger model | 181,689 | CC BY-SA |
|
| 168 |
| Wikipedia talk pages (FR, NL, ES, PT, DE, IT) | 40,371 | CC BY-SA 4.0 |
|
| 169 |
| ABCD customer-service dialogues | 74,792 | MIT |
|
| 170 |
| Taskmaster dialogues | 39,723 | CC BY 4.0 |
|
|
@@ -174,8 +255,8 @@ The training data includes these public datasets:
|
|
| 174 |
| French public-service texts | 7,206 | Licence Ouverte 2.0 |
|
| 175 |
| NVIDIA Nemotron-PII | 96,050 | CC BY 4.0 |
|
| 176 |
|
| 177 |
-
The word-list flags come from three lists in `training/assets/gazetteer.json.gz`: given names and surnames
|
| 178 |
-
|
| 179 |
published. [`training/`](https://github.com/Tars-Technologies/veil/blob/main/training/README.md) has the trainer, the checkpoint and the export script, which rebuilds
|
| 180 |
`veil-0.3.1.bin` byte for byte.
|
| 181 |
|
|
@@ -188,16 +269,6 @@ published. [`training/`](https://github.com/Tars-Technologies/veil/blob/main/tra
|
|
| 188 |
- Titles such as Mr or Dr are not part of the span.
|
| 189 |
- Only `PERSON` has been measured on public data. The other six labels have no published evaluation yet.
|
| 190 |
|
| 191 |
-
## Development
|
| 192 |
-
|
| 193 |
-
```sh
|
| 194 |
-
pnpm install
|
| 195 |
-
pnpm test:ci
|
| 196 |
-
pnpm type-check
|
| 197 |
-
pnpm demo # demo page on http://localhost:5173
|
| 198 |
-
pnpm embed # after a model change, regenerates src/model.ts from the .bin file
|
| 199 |
-
```
|
| 200 |
-
|
| 201 |
## Citation
|
| 202 |
|
| 203 |
```bibtex
|
|
|
|
| 13 |
- pii
|
| 14 |
- privacy
|
| 15 |
- ner
|
| 16 |
+
- redaction
|
| 17 |
- anonymization
|
| 18 |
+
- multilingual
|
| 19 |
- on-device
|
| 20 |
- browser
|
| 21 |
- typescript
|
| 22 |
+
datasets:
|
| 23 |
+
- wikimedia/wikipedia
|
| 24 |
+
- nvidia/Nemotron-PII
|
| 25 |
+
- OpenAssistant/oasst2
|
| 26 |
+
- AmazonScience/massive
|
| 27 |
+
- google-research-datasets/taskmaster1
|
| 28 |
+
metrics:
|
| 29 |
+
- f1
|
| 30 |
+
model-index:
|
| 31 |
+
- name: veil-0.3.1
|
| 32 |
+
results:
|
| 33 |
+
- task:
|
| 34 |
+
type: token-classification
|
| 35 |
+
name: Person name detection
|
| 36 |
+
dataset:
|
| 37 |
+
type: eriktks/conll2003
|
| 38 |
+
name: CoNLL-2003 English
|
| 39 |
+
split: test
|
| 40 |
+
metrics:
|
| 41 |
+
- type: f1
|
| 42 |
+
value: 73.1
|
| 43 |
+
name: F1, PERSON, exact span
|
| 44 |
+
- task:
|
| 45 |
+
type: token-classification
|
| 46 |
+
name: Person name detection
|
| 47 |
+
dataset:
|
| 48 |
+
type: leondz/wnut_17
|
| 49 |
+
name: WNUT-17
|
| 50 |
+
split: test
|
| 51 |
+
metrics:
|
| 52 |
+
- type: f1
|
| 53 |
+
value: 51.5
|
| 54 |
+
name: F1, PERSON, exact span
|
| 55 |
+
- task:
|
| 56 |
+
type: token-classification
|
| 57 |
+
name: Person name detection
|
| 58 |
+
dataset:
|
| 59 |
+
type: eriktks/conll2002
|
| 60 |
+
name: CoNLL-2002 Spanish
|
| 61 |
+
config: es
|
| 62 |
+
split: test
|
| 63 |
+
metrics:
|
| 64 |
+
- type: f1
|
| 65 |
+
value: 75.0
|
| 66 |
+
name: F1, PERSON, exact span
|
| 67 |
+
- task:
|
| 68 |
+
type: token-classification
|
| 69 |
+
name: Person name detection
|
| 70 |
+
dataset:
|
| 71 |
+
type: eriktks/conll2002
|
| 72 |
+
name: CoNLL-2002 Dutch
|
| 73 |
+
config: nl
|
| 74 |
+
split: test
|
| 75 |
+
metrics:
|
| 76 |
+
- type: f1
|
| 77 |
+
value: 70.5
|
| 78 |
+
name: F1, PERSON, exact span
|
| 79 |
+
- task:
|
| 80 |
+
type: token-classification
|
| 81 |
+
name: Person name detection
|
| 82 |
+
dataset:
|
| 83 |
+
type: ildpil/text-anonymization-benchmark
|
| 84 |
+
name: Text Anonymization Benchmark
|
| 85 |
+
split: test
|
| 86 |
+
metrics:
|
| 87 |
+
- type: f1
|
| 88 |
+
value: 88.1
|
| 89 |
+
name: F1, PERSON, leading title ignored
|
| 90 |
---
|
| 91 |
|
| 92 |
# Veil
|
|
|
|
| 105 |
const veil = createVeil({ model: loadModel() })
|
| 106 |
|
| 107 |
await veil.mask({
|
| 108 |
+
text: "I'm Laura Bianchi, call me on +33 6 39 98 12 34 or write to laura.bianchi@example.com",
|
| 109 |
types: ["PERSON", "PHONE", "EMAIL"]
|
| 110 |
})
|
| 111 |
// "I'm [NAME], call me on [PHONE] or write to [EMAIL]"
|
|
|
|
| 114 |
## Install
|
| 115 |
|
| 116 |
```sh
|
| 117 |
+
npm install github:Tars-Technologies/veil#v0.3.1
|
| 118 |
```
|
| 119 |
|
| 120 |
The package ships TypeScript source. Use it with a bundler or runtime that compiles TypeScript, such as Vite,
|
| 121 |
+
esbuild, tsx or Bun. Node's built-in TypeScript support does not compile packages in `node_modules`, so in Node run
|
| 122 |
+
your code with tsx, as an ES module (`"type": "module"` or a `.mts` file).
|
| 123 |
+
|
| 124 |
+
To load the model straight from this page:
|
| 125 |
+
|
| 126 |
+
```ts
|
| 127 |
+
const veil = createVeil({
|
| 128 |
+
model: "https://huggingface.co/tars-technologies/veil/resolve/main/veil-0.3.1.bin"
|
| 129 |
+
})
|
| 130 |
+
```
|
| 131 |
|
| 132 |
## Usage
|
| 133 |
|
|
|
|
| 196 |
|---|---|
|
| 197 |
| Author | Vaibhav Kubre, Tars Technologies |
|
| 198 |
| Labels | `PERSON`, `ADDRESS`, `DOB`, `ID`, `FINANCIAL`, `PHONE`, `EMAIL` |
|
| 199 |
+
| Architecture | Hashed character n-gram embeddings (32,768 buckets × 64, 2-bit codes), word-shape features and three word-list flags, into a 2-layer bidirectional LSTM with 96 units. LSTM and output weights are int8 with one scale per row. |
|
| 200 |
| Size | 1,177,096 bytes (`veil-0.3.1.bin`) |
|
| 201 |
| Languages | Latin script. Most training text is English, German, French, Spanish, Dutch, Portuguese or Italian. |
|
| 202 |
| Input | Windows of 256 tokens with a 64-token overlap. Up to 300 characters of `context`. |
|
| 203 |
+
| Licence | Apache-2.0 for code and model. `NOTICE` credits the data sources. The word lists keep their source licences (Wikidata CC0, GeoNames CC BY 4.0). |
|
| 204 |
|
| 205 |
The model file carries its name, version and format in a header and ends with a checksum. The library reads format 3
|
| 206 |
and rejects a file whose checksum does not match. Each new model gets a new file name.
|
| 207 |
|
| 208 |
## Evaluation
|
| 209 |
|
| 210 |
+
We measured `PERSON` on five public test sets. None of them is in the training data. The model that labelled the
|
| 211 |
+
Wikipedia rows was trained on the CoNLL-2003 English and CoNLL-2002 Spanish and Dutch training splits; their test
|
| 212 |
+
splits were not used. All models use their default settings. F1 on exact span match, except that the TAB row ignores
|
| 213 |
+
a leading title (see below):
|
| 214 |
|
| 215 |
| Test set | Names | Veil 0.3.1 (1.2 MB) | spaCy sm (15 MB) | newmean nano (2.6 MB) | newmean small (7.1 MB) | newmean large (20.7 MB) |
|
| 216 |
|---|---|---|---|---|---|---|
|
|
|
|
| 245 |
|
| 246 |
| Source | Rows | Licence |
|
| 247 |
|---|---|---|
|
| 248 |
+
| Wikipedia articles (DE, FR, EN), labelled by a larger model | 181,689 | CC BY-SA 4.0 |
|
| 249 |
| Wikipedia talk pages (FR, NL, ES, PT, DE, IT) | 40,371 | CC BY-SA 4.0 |
|
| 250 |
| ABCD customer-service dialogues | 74,792 | MIT |
|
| 251 |
| Taskmaster dialogues | 39,723 | CC BY 4.0 |
|
|
|
|
| 255 |
| French public-service texts | 7,206 | Licence Ouverte 2.0 |
|
| 256 |
| NVIDIA Nemotron-PII | 96,050 | CC BY 4.0 |
|
| 257 |
|
| 258 |
+
The word-list flags come from three lists in `training/assets/gazetteer.json.gz`: given names and surnames,
|
| 259 |
+
places from GeoNames, and common words from the training text. The training set itself is not
|
| 260 |
published. [`training/`](https://github.com/Tars-Technologies/veil/blob/main/training/README.md) has the trainer, the checkpoint and the export script, which rebuilds
|
| 261 |
`veil-0.3.1.bin` byte for byte.
|
| 262 |
|
|
|
|
| 269 |
- Titles such as Mr or Dr are not part of the span.
|
| 270 |
- Only `PERSON` has been measured on public data. The other six labels have no published evaluation yet.
|
| 271 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 272 |
## Citation
|
| 273 |
|
| 274 |
```bibtex
|