README.md

---
license: apache-2.0
base_model: facebook/detr-resnet-50
tags:
  - object-detection
  - transformers
  - detr
  - computer-vision
  - text-detection
  - historical-documents
widget:
  - enable_torchscript: false
datasets:
  - biglam/loc_beyond_words
library_name: transformers
pipeline_tag: object-detection
model-index:
  - name: opencode-r2
    results:
      - task:
          type: object-detection
          name: Object Detection
        dataset:
          type: biglam/loc_beyond_words
          name: Beeyond Words (Beyond Words) fine-grained illustration detection
          split: validation
        metrics:
          - type: mean_average_precision
            value: 0.2842
            name: COCO mAP @ [0.5:0.95]
          - type: coco_ap_50
            value: 0.4189
            name: COCO AP@IoU0.5
          - type: coco_ap_75
            value: 0.3196
            name: COCO AP@IoU0.75
          - type: cascade_average_recall
            value: 0.4302
            name: AR@maxDets=100
---

# opencode-r2

A fine-tuned object-detection model for fine-grained content detection in historical newspaper
and periodical pages, built on the [Beyond Words](https://huggingface.co/datasets/biglam/loc_beyond_words)
dataset (BigLam / Library of Congress). It detects the 7 page-layout content classes listed below.

## Model

- **Base model:** [`facebook/detr-resnet-50`](https://huggingface.co/facebook/detr-resnet-50) (DETR, **Apache-2.0**)
- **Task:** Object detection (COCO-style bounding boxes)
- **Classes (7):** `Photograph`, `Illustration`, `Map`, `Comics/Cartoon`, `Editorial Cartoon`, `Headline`, `Advertisement`

## Dataset

- **Source:** [`biglam/loc_beyond_words`](https://huggingface.co/datasets/biglam/loc_beyond_words)
- **Training:** 2,846 examples
- **Validation:** 712 examples

## Training

- Detector: DETR (`facebook/detr-resnet-50`) with the classification head re-initialized to 8 outputs (7 classes + no-object).
- Loss: DETR set-prediction loss (classification + L1 box + GIoU).
- Optimizer: AdamW (lr 1e-4), cosine/linear schedule with warmup, grad clip 1.0.
- Image size: shortest edge 560 px (longest edge 800 px).
- Backbone, transformer, and classifier all fine-tuned end-to-end.

### Validation results

Evaluated on the 712-example `biglam/loc_beyond_words` validation split (COCO metrics via pycocotools).

| Metric | Value |
|---|---|
| COCO mAP @ IoU [0.5:0.95] | **0.2842** |
| COCO AP @ IoU 0.5 | **0.4189** |
| COCO AP @ IoU 0.75 | **0.3196** |
| AR@maxDets=100 | **0.4302** |

Per-class COCO mAP @ [0.5:0.95]:

| Class | mAP |
|---|---|
| Photograph | 0.4027 |
| Illustration | 0.0485 |
| Map | 0.0431 |
| Comics/Cartoon | 0.2761 |
| Editorial Cartoon | 0.1702 |
| Headline | 0.5226 |
| Advertisement | 0.5259 |

Training loss: 21.06 (epoch 0) → 7.44 (epoch 4).

## Usage

```python
from transformers import AutoImageProcessor, AutoModelForObjectDetection
import torch
from PIL import Image

processor = AutoImageProcessor.from_pretrained("harness-race/opencode-r2")
model = AutoModelForObjectDetection.from_pretrained("harness-race/opencode-r2")

img = Image.open("page.png").convert("RGB")
inputs = processor(images=img, return_tensors="pt")
with torch.no_grad():
    outputs = model(**inputs)

target_sizes = torch.tensor([[img.height, img.width]])
results = processor.post_process_object_detection(outputs, target_sizes=target_sizes, threshold=0.5)[0]
for score, label, box in zip(results["scores"], results["labels"], results["boxes"]):
    box = [round(i, 1) for i in box.tolist()]
    print(f"{model.config.id2label[label.item()]}: {round(score.item(),3)} {box}")
```

## License and attribution

- The fine-tuned weights in this repository are released under **Apache-2.0** (same as the base DETR model), so the model is free to share and use.
- Base detector: *End-to-End Object Detection with Transformers* (Carion et al., 2020).
- Data: Beyond Words (Library of Congress) via `biglam/loc_beyond_words`.

## Caveats

- Model was fine-tuned on a single GPU with a limited training budget; results reflect that constraint.
- Detection resolution and accuracy trade-offs exist for very small text blocks.