metadata
license: apache-2.0
base_model: facebook/detr-resnet-50
tags:
- object-detection
- transformers
- detr
- computer-vision
- text-detection
- historical-documents
widget:
- enable_torchscript: false
datasets:
- biglam/loc_beyond_words
library_name: transformers
pipeline_tag: object-detection
model-index:
- name: opencode-r2
results:
- task:
type: object-detection
name: Object Detection
dataset:
type: biglam/loc_beyond_words
name: Beeyond Words (Beyond Words) fine-grained illustration detection
split: validation
metrics:
- type: mean_average_precision
value: 0.2842
name: COCO mAP @ [0.5:0.95]
- type: coco_ap_50
value: 0.4189
name: COCO AP@IoU0.5
- type: coco_ap_75
value: 0.3196
name: COCO AP@IoU0.75
- type: cascade_average_recall
value: 0.4302
name: AR@maxDets=100
opencode-r2
A fine-tuned object-detection model for fine-grained content detection in historical newspaper and periodical pages, built on the Beyond Words dataset (BigLam / Library of Congress). It detects the 7 page-layout content classes listed below.
Model
- Base model:
facebook/detr-resnet-50(DETR, Apache-2.0) - Task: Object detection (COCO-style bounding boxes)
- Classes (7):
Photograph,Illustration,Map,Comics/Cartoon,Editorial Cartoon,Headline,Advertisement
Dataset
- Source:
biglam/loc_beyond_words - Training: 2,846 examples
- Validation: 712 examples
Training
- Detector: DETR (
facebook/detr-resnet-50) with the classification head re-initialized to 8 outputs (7 classes + no-object). - Loss: DETR set-prediction loss (classification + L1 box + GIoU).
- Optimizer: AdamW (lr 1e-4), cosine/linear schedule with warmup, grad clip 1.0.
- Image size: shortest edge 560 px (longest edge 800 px).
- Backbone, transformer, and classifier all fine-tuned end-to-end.
Validation results
Evaluated on the 712-example biglam/loc_beyond_words validation split (COCO metrics via pycocotools).
| Metric | Value |
|---|---|
| COCO mAP @ IoU [0.5:0.95] | 0.2842 |
| COCO AP @ IoU 0.5 | 0.4189 |
| COCO AP @ IoU 0.75 | 0.3196 |
| AR@maxDets=100 | 0.4302 |
Per-class COCO mAP @ [0.5:0.95]:
| Class | mAP |
|---|---|
| Photograph | 0.4027 |
| Illustration | 0.0485 |
| Map | 0.0431 |
| Comics/Cartoon | 0.2761 |
| Editorial Cartoon | 0.1702 |
| Headline | 0.5226 |
| Advertisement | 0.5259 |
Training loss: 21.06 (epoch 0) → 7.44 (epoch 4).
Usage
from transformers import AutoImageProcessor, AutoModelForObjectDetection
import torch
from PIL import Image
processor = AutoImageProcessor.from_pretrained("harness-race/opencode-r2")
model = AutoModelForObjectDetection.from_pretrained("harness-race/opencode-r2")
img = Image.open("page.png").convert("RGB")
inputs = processor(images=img, return_tensors="pt")
with torch.no_grad():
outputs = model(**inputs)
target_sizes = torch.tensor([[img.height, img.width]])
results = processor.post_process_object_detection(outputs, target_sizes=target_sizes, threshold=0.5)[0]
for score, label, box in zip(results["scores"], results["labels"], results["boxes"]):
box = [round(i, 1) for i in box.tolist()]
print(f"{model.config.id2label[label.item()]}: {round(score.item(),3)} {box}")
License and attribution
- The fine-tuned weights in this repository are released under Apache-2.0 (same as the base DETR model), so the model is free to share and use.
- Base detector: End-to-End Object Detection with Transformers (Carion et al., 2020).
- Data: Beyond Words (Library of Congress) via
biglam/loc_beyond_words.
Caveats
- Model was fine-tuned on a single GPU with a limited training budget; results reflect that constraint.
- Detection resolution and accuracy trade-offs exist for very small text blocks.