Upload gbert-base-contact-ner (company_research.contact gpu model)

README.md

@@ -0,0 +1,126 @@
+---
+license: mit
+language: de
+base_model: deepset/gbert-base
+pipeline_tag: token-classification
+library_name: transformers
+tags:
+  - token-classification
+  - ner
+  - german
+  - contact-information
+  - imprint
+  - web-extraction
+---
+
+# gbert-base-contact-ner
+
+A German token-classification model that finds contact details in the text of company web pages: street, postal
+code, city, country, phone, fax, email, opening hours, location label, department and contact person. It is
+[`deepset/gbert-base`](https://huggingface.co/deepset/gbert-base) fine-tuned on LLM-labeled contact pages. It is the
+`gpu` extraction model of the `company_research.contact` package.
+
+## Labels
+
+11 fields in BIO format, 23 labels in total (`O` plus `B-`/`I-` for each field):
+
+`street`, `plz` (postal code), `city`, `country`, `phone`, `fax`, `email`, `opening_hours`, `location_label`,
+`department`, `person`.
+
+The model outputs character spans of the input text only. It never rewrites or normalizes a value.
+
+## Training data
+
+- Pages of 800 German-language company websites, labeled by LLM agents (Claude Opus) against a written labeling
+  guide. Every labeled value appears verbatim on its page. The labels were not verified by humans; one clean-up pass
+  covered flagged sites only.
+- The sites are German (399), Swiss (346) and Austrian (55), split by domain: 480 train, 160 dev, 160 test.
+  This model was trained on the train split (565 labeled pages) and tuned on dev. The 55 Austrian sites are all in
+  the test split, so the model never saw AT data during training.
+- Input is the page as markdown text (not raw HTML).
+
+## Training
+
+- Base model `deepset/gbert-base`, token classification head with 23 labels.
+- Pages are cut into windows of 512 tokens (510 plus `[CLS]` and `[SEP]`) with an overlap of 128 tokens. Training
+  uses the windows that contain a labeled value plus an equal number of randomly drawn windows without any.
+- AdamW, learning rate 3e-5, weight decay 0.01, batch size 4 with 4 gradient-accumulation steps, up to 8 epochs
+  with early stopping (patience 2) on dev seqeval F1.
+
+## Evaluation
+
+All numbers are agreement with the LLM-made labels on the 160 held-out test sites (153 of them have contact data),
+from the experiments report of the Impressum project (section 12). Scores are set-based: per site, the predicted
+values are compared with the labeled values after normalization (phones to E.164, emails lowercased, PLZ digits only,
+text casefolded with whitespace and punctuation cleaned). Micro F1 pools all fields. The differences under about 3
+points between rows of the report are noise at this test size.
+
+**Extraction on the right pages** (labeled contact pages given; this model alone, F1):
+
+| micro | street | plz | phone | fax | email | opening hours | person |
+|---|---|---|---|---|---|---|---|
+| 70.2 | 64.9 | 88.0 | 88.9 | 80.6 | 70.7 | 48.9 | 80.5 |
+
+The report does not list per-field scores for city, country, location label or department for this model alone.
+Street and email are its weaker fields among those listed (a regex and a CRF do better there; the pipeline routes those
+fields to them), and opening hours are hard for every method tried.
+
+**End to end** (report section 12.7). These numbers are for the whole `company_research.contact` `gpu` pipeline,
+not for this model alone: contact-page detection, hybrid extraction (this model for PLZ, city, phone, hours,
+department and person; a CRF for street and label; regex for fax, email and country), grouping into locations and
+departments, structured-data parsing and social links. Micro F1 over all site fields: **73.6** overall;
+71.0 on DE, 80.0 on CH, 72.9 on AT.
+
+## How to use
+
+### With `company_research.contact`
+
+```python
+from company_research import contact
+
+result = contact.from_site("https://www.example.de", device="gpu")
+```
+
+`device="gpu"` needs the package's `gpu` extra (torch and transformers); the package is not on PyPI.
+It downloads this model on first use and runs on CUDA when available (otherwise on the CPU, much slower). The `cpu` device uses a different, smaller
+set of models without a transformer.
+
+### With `transformers`
+
+The model was trained on windows, so long pages must be windowed at inference as well. The reference implementation
+is `company_research/contact/gbert.py` (`windows`, `decode`, `predict`). In short:
+
+1. Tokenize the page text without special tokens, keeping character offsets
+   (`tok(text, add_special_tokens=False, return_offsets_mapping=True)`).
+2. Make windows of 510 tokens whose starts are 382 tokens apart (overlap 128); add `[CLS]` and `[SEP]` to each.
+3. Run the model, apply softmax, and average the probabilities of tokens that are covered by several windows.
+4. Take the argmax per token and merge `B-x`/`I-x` runs into character spans using the offsets.
+
+```python
+from transformers import AutoModelForTokenClassification, AutoTokenizer
+
+tok = AutoTokenizer.from_pretrained("andyP/gbert-base-contact-ner")
+model = AutoModelForTokenClassification.from_pretrained("andyP/gbert-base-contact-ner").eval()
+```
+
+## Limitations
+
+- German-language company pages (imprint, contact and similar pages). Other languages and page types are untested.
+- The labels come from an LLM and were not verified by humans, so the scores measure agreement with those labels,
+  not with human judgment.
+- Weaker fields: street (64.9) and email (70.7) in the scores above; opening hours (48.9) are the hardest.
+  Typical errors in the project's spot-check of the full pipeline were missed values, values of third parties (for
+  example the web agency named in the imprint), spans cut inside words and navigation text.
+- The model does not group values into locations or departments, and it does not tell contact pages from other
+  pages. The pipeline does that.
+- Input must be the page text as markdown, as the training data was. Raw HTML will perform worse.
+
+## Privacy
+
+The training pages are public company websites. They contain names and business contact details of contact persons,
+so the training data includes personal data of that kind. The model outputs spans copied from the text you give it
+and does not generate text.
+
+## License
+
+MIT, as the base model.

config.json

@@ -0,0 +1,78 @@
+{
+  "add_cross_attention": false,
+  "architectures": [
+    "BertForTokenClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": null,
+  "classifier_dropout": null,
+  "dtype": "float32",
+  "eos_token_id": null,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "id2label": {
+    "0": "O",
+    "1": "B-street",
+    "2": "I-street",
+    "3": "B-plz",
+    "4": "I-plz",
+    "5": "B-city",
+    "6": "I-city",
+    "7": "B-country",
+    "8": "I-country",
+    "9": "B-phone",
+    "10": "I-phone",
+    "11": "B-fax",
+    "12": "I-fax",
+    "13": "B-email",
+    "14": "I-email",
+    "15": "B-opening_hours",
+    "16": "I-opening_hours",
+    "17": "B-location_label",
+    "18": "I-location_label",
+    "19": "B-department",
+    "20": "I-department",
+    "21": "B-person",
+    "22": "I-person"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "is_decoder": false,
+  "label2id": {
+    "B-city": 5,
+    "B-country": 7,
+    "B-department": 19,
+    "B-email": 13,
+    "B-fax": 11,
+    "B-location_label": 17,
+    "B-opening_hours": 15,
+    "B-person": 21,
+    "B-phone": 9,
+    "B-plz": 3,
+    "B-street": 1,
+    "I-city": 6,
+    "I-country": 8,
+    "I-department": 20,
+    "I-email": 14,
+    "I-fax": 12,
+    "I-location_label": 18,
+    "I-opening_hours": 16,
+    "I-person": 22,
+    "I-phone": 10,
+    "I-plz": 4,
+    "I-street": 2,
+    "O": 0
+  },
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 0,
+  "tie_word_embeddings": true,
+  "transformers_version": "5.16.1",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 31102
+}

impressum.json

@@ -0,0 +1 @@
+{"name": "gbert_ner", "hf_id": "deepset/gbert-base", "max_len": 512, "stride": 128, "fields": ["street", "plz", "city", "country", "phone", "fax", "email", "opening_hours", "location_label", "department", "person"]}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:66f501c8644499d84d80e2848c6843f02b3f08a56ceada192a4f0256c4fccb36
+size 437442444

tokenizer_config.json

@@ -0,0 +1,16 @@
+{
+  "backend": "tokenizers",
+  "cls_token": "[CLS]",
+  "do_lower_case": false,
+  "is_local": false,
+  "local_files_only": false,
+  "mask_token": "[MASK]",
+  "max_len": 512,
+  "model_max_length": 512,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "strip_accents": false,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "unk_token": "[UNK]"
+}