Text Classification
Transformers
Safetensors
English
modernbert
legal
document-ai
page-classification
text-embeddings-inference
Instructions to use RayJackson30/clawbert-149 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use RayJackson30/clawbert-149 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="RayJackson30/clawbert-149")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForSequenceClassification tokenizer = AutoTokenizer.from_pretrained("RayJackson30/clawbert-149") model = AutoModelForSequenceClassification.from_pretrained("RayJackson30/clawbert-149", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -49,8 +49,8 @@ overlap, but not always.
|
|
| 49 |
Trained on about 13K pages of human-labelled page images. Each page image was
|
| 50 |
OCR'd five times, by five different OCR engines. Play around with the tabs
|
| 51 |
below — the same cover page, five very different transcriptions, one label.
|
| 52 |
-
Note that each page is part of a document, and the
|
| 53 |
-
previous/subsequent
|
| 54 |
|
| 55 |
<img src="docs/examples/cover_page.jpg" alt="cover page" width="300" align="right">
|
| 56 |
|
|
@@ -362,6 +362,19 @@ probs = torch.softmax(model(**enc).logits, -1)[0]
|
|
| 362 |
print(model.config.id2label[int(probs.argmax())], float(probs.max()))
|
| 363 |
```
|
| 364 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 365 |
## Know what you're getting
|
| 366 |
|
| 367 |
- **Assumes one document.** Won't work on an appendix or combined record —
|
|
|
|
| 49 |
Trained on about 13K pages of human-labelled page images. Each page image was
|
| 50 |
OCR'd five times, by five different OCR engines. Play around with the tabs
|
| 51 |
below — the same cover page, five very different transcriptions, one label.
|
| 52 |
+
Note that each page is part of a document, and the whole-document mode below
|
| 53 |
+
takes previous/subsequent pages into account.
|
| 54 |
|
| 55 |
<img src="docs/examples/cover_page.jpg" alt="cover page" width="300" align="right">
|
| 56 |
|
|
|
|
| 362 |
print(model.config.id2label[int(probs.argmax())], float(probs.max()))
|
| 363 |
```
|
| 364 |
|
| 365 |
+
## Whole-document mode
|
| 366 |
+
|
| 367 |
+
The per-page model classifies one page at a time. The repo also ships a small
|
| 368 |
+
context layer (`docxf/`, 1.8M params) that runs over a whole document's page
|
| 369 |
+
embeddings — every page attends to every other page — so ambiguous pages get
|
| 370 |
+
decided with document context. Pooled test macro-F1 goes 0.937 → 0.949;
|
| 371 |
+
`subsequent_cover_page` F1 0.68 → 0.77. Feed it ONE document's pages, in order.
|
| 372 |
+
|
| 373 |
+
```python
|
| 374 |
+
from clawbert149_doc import score_document
|
| 375 |
+
labels, probs = score_document([page1_text, page2_text, page3_text])
|
| 376 |
+
```
|
| 377 |
+
|
| 378 |
## Know what you're getting
|
| 379 |
|
| 380 |
- **Assumes one document.** Won't work on an appendix or combined record —
|