Upload gbert-base-contact-ner (company_research.contact gpu model)

5955a93 verified 8 days ago

6.2 kB

---
license: mit
language: de
base_model: deepset/gbert-base
pipeline_tag: token-classification
library_name: transformers
tags:
  - token-classification
  - ner
  - german
  - contact-information
  - imprint
  - web-extraction
---
# gbert-base-contact-ner

A German token-classification model that finds contact details in the text of company web pages: street, postal
code, city, country, phone, fax, email, opening hours, location label, department and contact person. It is
[`deepset/gbert-base`](https://huggingface.co/deepset/gbert-base) fine-tuned on LLM-labeled contact pages. It is the
`gpu` extraction model of the `company_research.contact` package.

## Labels

11 fields in BIO format, 23 labels in total (`O` plus `B-`/`I-` for each field):

`street`, `plz` (postal code), `city`, `country`, `phone`, `fax`, `email`, `opening_hours`, `location_label`,
`department`, `person`.

The model outputs character spans of the input text only. It never rewrites or normalizes a value.

## Training data

- Pages of 800 German-language company websites, labeled by LLM agents (Claude Opus) against a written labeling
  guide. Every labeled value appears verbatim on its page. The labels were not verified by humans; one clean-up pass
  covered flagged sites only.
- The sites are German (399), Swiss (346) and Austrian (55), split by domain: 480 train, 160 dev, 160 test.
  This model was trained on the train split (565 labeled pages) and tuned on dev. The 55 Austrian sites are all in
  the test split, so the model never saw AT data during training.
- Input is the page as markdown text (not raw HTML).

## Training

- Base model `deepset/gbert-base`, token classification head with 23 labels.
- Pages are cut into windows of 512 tokens (510 plus `[CLS]` and `[SEP]`) with an overlap of 128 tokens. Training
  uses the windows that contain a labeled value plus an equal number of randomly drawn windows without any.
- AdamW, learning rate 3e-5, weight decay 0.01, batch size 4 with 4 gradient-accumulation steps, up to 8 epochs
  with early stopping (patience 2) on dev seqeval F1.

## Evaluation

All numbers are agreement with the LLM-made labels on the 160 held-out test sites (153 of them have contact data),
from the experiments report of the Impressum project (section 12). Scores are set-based: per site, the predicted
values are compared with the labeled values after normalization (phones to E.164, emails lowercased, PLZ digits only,
text casefolded with whitespace and punctuation cleaned). Micro F1 pools all fields. The differences under about 3
points between rows of the report are noise at this test size.

**Extraction on the right pages** (labeled contact pages given; this model alone, F1):

| micro | street | plz | phone | fax | email | opening hours | person |
|---|---|---|---|---|---|---|---|
| 70.2 | 64.9 | 88.0 | 88.9 | 80.6 | 70.7 | 48.9 | 80.5 |

The report does not list per-field scores for city, country, location label or department for this model alone.
Street and email are its weaker fields among those listed (a regex and a CRF do better there; the pipeline routes those
fields to them), and opening hours are hard for every method tried.

**End to end** (report section 12.7). These numbers are for the whole `company_research.contact` `gpu` pipeline,
not for this model alone: contact-page detection, hybrid extraction (this model for PLZ, city, phone, hours,
department and person; a CRF for street and label; regex for fax, email and country), grouping into locations and
departments, structured-data parsing and social links. Micro F1 over all site fields: **73.6** overall;
71.0 on DE, 80.0 on CH, 72.9 on AT.

## How to use

### With `company_research.contact`
```python
from company_research import contact

result = contact.from_site("https://www.example.de", device="gpu")
```
`device="gpu"` needs the package's `gpu` extra (torch and transformers); the package is not on PyPI.
It downloads this model on first use and runs on CUDA when available (otherwise on the CPU, much slower). The `cpu` device uses a different, smaller
set of models without a transformer.
### With `transformers`
The model was trained on windows, so long pages must be windowed at inference as well. The reference implementation
is `company_research/contact/gbert.py` (`windows`, `decode`, `predict`). In short:

1. Tokenize the page text without special tokens, keeping character offsets
   (`tok(text, add_special_tokens=False, return_offsets_mapping=True)`).
2. Make windows of 510 tokens whose starts are 382 tokens apart (overlap 128); add `[CLS]` and `[SEP]` to each.
3. Run the model, apply softmax, and average the probabilities of tokens that are covered by several windows.
4. Take the argmax per token and merge `B-x`/`I-x` runs into character spans using the offsets.

```python
from transformers import AutoModelForTokenClassification, AutoTokenizer
tok = AutoTokenizer.from_pretrained("andyP/gbert-base-contact-ner")
model = AutoModelForTokenClassification.from_pretrained("andyP/gbert-base-contact-ner").eval()
```

## Limitations

- German-language company pages (imprint, contact and similar pages). Other languages and page types are untested.
- The labels come from an LLM and were not verified by humans, so the scores measure agreement with those labels,
  not with human judgment.
- Weaker fields: street (64.9) and email (70.7) in the scores above; opening hours (48.9) are the hardest.
  Typical errors in the project's spot-check of the full pipeline were missed values, values of third parties (for
  example the web agency named in the imprint), spans cut inside words and navigation text.
- The model does not group values into locations or departments, and it does not tell contact pages from other
  pages. The pipeline does that.
- Input must be the page text as markdown, as the training data was. Raw HTML will perform worse.

## Privacy

The training pages are public company websites. They contain names and business contact details of contact persons,
so the training data includes personal data of that kind. The model outputs spans copied from the text you give it
and does not generate text.

## License

MIT, as the base model.

	---
	license: mit
	language: de
	base_model: deepset/gbert-base
	pipeline_tag: token-classification
	library_name: transformers
	tags:
	- token-classification
	- ner
	- german
	- contact-information
	- imprint
	- web-extraction
	---

	# gbert-base-contact-ner

	A German token-classification model that finds contact details in the text of company web pages: street, postal
	code, city, country, phone, fax, email, opening hours, location label, department and contact person. It is
	[`deepset/gbert-base`](https://huggingface.co/deepset/gbert-base) fine-tuned on LLM-labeled contact pages. It is the
	`gpu` extraction model of the `company_research.contact` package.

	## Labels

	11 fields in BIO format, 23 labels in total (`O` plus `B-`/`I-` for each field):

	`street`, `plz` (postal code), `city`, `country`, `phone`, `fax`, `email`, `opening_hours`, `location_label`,
	`department`, `person`.

	The model outputs character spans of the input text only. It never rewrites or normalizes a value.

	## Training data

	- Pages of 800 German-language company websites, labeled by LLM agents (Claude Opus) against a written labeling
	guide. Every labeled value appears verbatim on its page. The labels were not verified by humans; one clean-up pass
	covered flagged sites only.
	- The sites are German (399), Swiss (346) and Austrian (55), split by domain: 480 train, 160 dev, 160 test.
	This model was trained on the train split (565 labeled pages) and tuned on dev. The 55 Austrian sites are all in
	the test split, so the model never saw AT data during training.
	- Input is the page as markdown text (not raw HTML).

	## Training

	- Base model `deepset/gbert-base`, token classification head with 23 labels.
	- Pages are cut into windows of 512 tokens (510 plus `[CLS]` and `[SEP]`) with an overlap of 128 tokens. Training
	uses the windows that contain a labeled value plus an equal number of randomly drawn windows without any.
	- AdamW, learning rate 3e-5, weight decay 0.01, batch size 4 with 4 gradient-accumulation steps, up to 8 epochs
	with early stopping (patience 2) on dev seqeval F1.

	## Evaluation

	All numbers are agreement with the LLM-made labels on the 160 held-out test sites (153 of them have contact data),
	from the experiments report of the Impressum project (section 12). Scores are set-based: per site, the predicted
	values are compared with the labeled values after normalization (phones to E.164, emails lowercased, PLZ digits only,
	text casefolded with whitespace and punctuation cleaned). Micro F1 pools all fields. The differences under about 3
	points between rows of the report are noise at this test size.

	Extraction on the right pages (labeled contact pages given; this model alone, F1):

	\| micro \| street \| plz \| phone \| fax \| email \| opening hours \| person \|
	\|---\|---\|---\|---\|---\|---\|---\|---\|
	\| 70.2 \| 64.9 \| 88.0 \| 88.9 \| 80.6 \| 70.7 \| 48.9 \| 80.5 \|

	The report does not list per-field scores for city, country, location label or department for this model alone.
	Street and email are its weaker fields among those listed (a regex and a CRF do better there; the pipeline routes those
	fields to them), and opening hours are hard for every method tried.

	End to end (report section 12.7). These numbers are for the whole `company_research.contact` `gpu` pipeline,
	not for this model alone: contact-page detection, hybrid extraction (this model for PLZ, city, phone, hours,
	department and person; a CRF for street and label; regex for fax, email and country), grouping into locations and
	departments, structured-data parsing and social links. Micro F1 over all site fields: 73.6 overall;
	71.0 on DE, 80.0 on CH, 72.9 on AT.

	## How to use

	### With `company_research.contact`

	```python
	from company_research import contact

	result = contact.from_site("https://www.example.de", device="gpu")
	```

	`device="gpu"` needs the package's `gpu` extra (torch and transformers); the package is not on PyPI.
	It downloads this model on first use and runs on CUDA when available (otherwise on the CPU, much slower). The `cpu` device uses a different, smaller
	set of models without a transformer.

	### With `transformers`

	The model was trained on windows, so long pages must be windowed at inference as well. The reference implementation
	is `company_research/contact/gbert.py` (`windows`, `decode`, `predict`). In short:

	1. Tokenize the page text without special tokens, keeping character offsets
	(`tok(text, add_special_tokens=False, return_offsets_mapping=True)`).
	2. Make windows of 510 tokens whose starts are 382 tokens apart (overlap 128); add `[CLS]` and `[SEP]` to each.
	3. Run the model, apply softmax, and average the probabilities of tokens that are covered by several windows.
	4. Take the argmax per token and merge `B-x`/`I-x` runs into character spans using the offsets.

	```python
	from transformers import AutoModelForTokenClassification, AutoTokenizer

	tok = AutoTokenizer.from_pretrained("andyP/gbert-base-contact-ner")
	model = AutoModelForTokenClassification.from_pretrained("andyP/gbert-base-contact-ner").eval()
	```

	## Limitations

	- German-language company pages (imprint, contact and similar pages). Other languages and page types are untested.
	- The labels come from an LLM and were not verified by humans, so the scores measure agreement with those labels,
	not with human judgment.
	- Weaker fields: street (64.9) and email (70.7) in the scores above; opening hours (48.9) are the hardest.
	Typical errors in the project's spot-check of the full pipeline were missed values, values of third parties (for
	example the web agency named in the imprint), spans cut inside words and navigation text.
	- The model does not group values into locations or departments, and it does not tell contact pages from other
	pages. The pipeline does that.
	- Input must be the page text as markdown, as the training data was. Raw HTML will perform worse.

	## Privacy

	The training pages are public company websites. They contain names and business contact details of contact persons,
	so the training data includes personal data of that kind. The model outputs spans copied from the text you give it
	and does not generate text.

	## License

	MIT, as the base model.