File size: 4,166 Bytes
82fc76d de537b2 82fc76d de537b2 82fc76d de537b2 82fc76d de537b2 82fc76d de537b2 82fc76d de537b2 82fc76d | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 | ---
license: apache-2.0
base_model: facebook/detr-resnet-50
tags:
- object-detection
- transformers
- detr
- computer-vision
- text-detection
- historical-documents
widget:
- enable_torchscript: false
datasets:
- biglam/loc_beyond_words
library_name: transformers
pipeline_tag: object-detection
model-index:
- name: opencode-r2
results:
- task:
type: object-detection
name: Object Detection
dataset:
type: biglam/loc_beyond_words
name: Beeyond Words (Beyond Words) fine-grained illustration detection
split: validation
metrics:
- type: mean_average_precision
value: 0.2842
name: COCO mAP @ [0.5:0.95]
- type: coco_ap_50
value: 0.4189
name: COCO AP@IoU0.5
- type: coco_ap_75
value: 0.3196
name: COCO AP@IoU0.75
- type: cascade_average_recall
value: 0.4302
name: AR@maxDets=100
---
# opencode-r2
A fine-tuned object-detection model for fine-grained content detection in historical newspaper
and periodical pages, built on the [Beyond Words](https://huggingface.co/datasets/biglam/loc_beyond_words)
dataset (BigLam / Library of Congress). It detects the 7 page-layout content classes listed below.
## Model
- **Base model:** [`facebook/detr-resnet-50`](https://huggingface.co/facebook/detr-resnet-50) (DETR, **Apache-2.0**)
- **Task:** Object detection (COCO-style bounding boxes)
- **Classes (7):** `Photograph`, `Illustration`, `Map`, `Comics/Cartoon`, `Editorial Cartoon`, `Headline`, `Advertisement`
## Dataset
- **Source:** [`biglam/loc_beyond_words`](https://huggingface.co/datasets/biglam/loc_beyond_words)
- **Training:** 2,846 examples
- **Validation:** 712 examples
## Training
- Detector: DETR (`facebook/detr-resnet-50`) with the classification head re-initialized to 8 outputs (7 classes + no-object).
- Loss: DETR set-prediction loss (classification + L1 box + GIoU).
- Optimizer: AdamW (lr 1e-4), cosine/linear schedule with warmup, grad clip 1.0.
- Image size: shortest edge 560 px (longest edge 800 px).
- Backbone, transformer, and classifier all fine-tuned end-to-end.
### Validation results
Evaluated on the 712-example `biglam/loc_beyond_words` validation split (COCO metrics via pycocotools).
| Metric | Value |
|---|---|
| COCO mAP @ IoU [0.5:0.95] | **0.2842** |
| COCO AP @ IoU 0.5 | **0.4189** |
| COCO AP @ IoU 0.75 | **0.3196** |
| AR@maxDets=100 | **0.4302** |
Per-class COCO mAP @ [0.5:0.95]:
| Class | mAP |
|---|---|
| Photograph | 0.4027 |
| Illustration | 0.0485 |
| Map | 0.0431 |
| Comics/Cartoon | 0.2761 |
| Editorial Cartoon | 0.1702 |
| Headline | 0.5226 |
| Advertisement | 0.5259 |
Training loss: 21.06 (epoch 0) → 7.44 (epoch 4).
## Usage
```python
from transformers import AutoImageProcessor, AutoModelForObjectDetection
import torch
from PIL import Image
processor = AutoImageProcessor.from_pretrained("harness-race/opencode-r2")
model = AutoModelForObjectDetection.from_pretrained("harness-race/opencode-r2")
img = Image.open("page.png").convert("RGB")
inputs = processor(images=img, return_tensors="pt")
with torch.no_grad():
outputs = model(**inputs)
target_sizes = torch.tensor([[img.height, img.width]])
results = processor.post_process_object_detection(outputs, target_sizes=target_sizes, threshold=0.5)[0]
for score, label, box in zip(results["scores"], results["labels"], results["boxes"]):
box = [round(i, 1) for i in box.tolist()]
print(f"{model.config.id2label[label.item()]}: {round(score.item(),3)} {box}")
```
## License and attribution
- The fine-tuned weights in this repository are released under **Apache-2.0** (same as the base DETR model), so the model is free to share and use.
- Base detector: *End-to-End Object Detection with Transformers* (Carion et al., 2020).
- Data: Beyond Words (Library of Congress) via `biglam/loc_beyond_words`.
## Caveats
- Model was fine-tuned on a single GPU with a limited training budget; results reflect that constraint.
- Detection resolution and accuracy trade-offs exist for very small text blocks.
|