README.md

---
language:
  - uk
license: agpl-3.0
datasets:
  - UkrainianCatholicUniversity/rukopys
base_model:
  - Kansallisarkisto/cyrillic-htr-model
tags:
  - optical-character-recognition
  - handwriting-recognition
  - document-ai
  - object-detection
  - rtdetr
  - trocr
  - ukrainian
---

# Mark One OCR Weights

This repository contains the custom inference weights for **Mark One OCR**, the
second-place solution in the final of the
[Handwritten to Data challenge](https://www.kaggle.com/competitions/handwritten-to-data).
The models detect, classify, and transcribe regions in Ukrainian handwritten
documents. They were trained or fine-tuned on the open
[RUKOPYS dataset](https://huggingface.co/datasets/UkrainianCatholicUniversity/rukopys).

The complete, supported inference pipeline—including detector ensembling,
geometry correction, TrOCR, optional Qwen page-context correction, final
post-processing, CLI, Docker, and the web app—is available at
[Mark One OCR](https://github.com/Isterikus/markone-ocr).

## Repository contents

| Path | Purpose |
|---|---|
| `detector_old.pt` | First fine-tuned Ultralytics RT-DETR-L checkpoint |
| `detector_new.pt` | Complementary fine-tuned RT-DETR-L checkpoint |
| `recognizer/` | Fine-tuned TrOCR `VisionEncoderDecoderModel` inference files |
| `recognizer_processor/` | Matching TrOCR image processor and tokenizer |
| `warp_selector.pkl` | LightGBM gate used by the conservative UVDoc geometry stage |
| `SHA256SUMS` | Checksums for every inference artifact |

The detector class order is:

```text
0 handwritten, 1 printed, 2 formula, 3 table,
4 annotation, 5 image, 6 graph
```

The two detector checkpoints are an ensemble. Running either checkpoint alone
is useful for inspection, but does not reproduce the challenge pipeline.
This is a multi-component weight bundle, so the repository root is not intended
to be loaded by a single `from_pretrained()` call or Hugging Face inference
widget. Load a component as shown below or use the complete pipeline.

## Use the complete pipeline

```bash
git clone https://github.com/Isterikus/markone-ocr
cd markone-ocr
python -m venv .venv
source .venv/bin/activate
python -m pip install -e '.[all]'
python scripts/download_models.py

# Base detector + geometry + greedy TrOCR inference
markone-ocr infer /path/to/pages --output outputs/base

# Optional page-context correction through a running Ollama service
markone-ocr infer /path/to/pages \
  --enhance --backend ollama --model qwen3.6:latest \
  --output outputs/enhanced
```

See the GitHub README for Ollama, vLLM, Docker, web app, batch, and low-VRAM
profiles. Qwen and UVDoc weights are external dependencies and are not included
in this repository.

## Load the recognizer by itself

The recognizer expects a **cropped text region**, not a full document page.
Greedy decoding (`num_beams=1`) is the supported configuration.

```python
from pathlib import Path

import torch
from huggingface_hub import snapshot_download
from PIL import Image
from transformers import TrOCRProcessor, VisionEncoderDecoderModel

repo_id = "dkovalen/markone-ocr-weights"
root = Path(
    snapshot_download(
        repo_id,
        allow_patterns=["recognizer/*", "recognizer_processor/*"],
    )
)

processor = TrOCRProcessor.from_pretrained(root / "recognizer_processor")
model = VisionEncoderDecoderModel.from_pretrained(root / "recognizer").to("cuda").eval()

crop = Image.open("text-crop.png").convert("RGB")
pixel_values = processor(images=crop, return_tensors="pt").pixel_values.to("cuda")
with torch.inference_mode():
    token_ids = model.generate(pixel_values, num_beams=1, max_new_tokens=128)

print(processor.batch_decode(token_ids, skip_special_tokens=True)[0])
```

## Load a detector checkpoint by itself

```python
from huggingface_hub import hf_hub_download
from ultralytics import RTDETR

repo_id = "dkovalen/markone-ocr-weights"
checkpoint = hf_hub_download(repo_id, filename="detector_old.pt")
detector = RTDETR(checkpoint)
results = detector.predict("page.jpg", imgsz=1536, device=0)

for result in results:
    print(result.boxes.xyxy, result.boxes.cls, result.boxes.conf)
```

This example is raw single-checkpoint inference. Use the full Mark One pipeline
for its two-checkpoint, full-page plus sliding-window ensemble, class-aware
filters, geometry safeguards, coordinate mapping, and final post-processing.

## Evaluation

The competition metric was:

```text
score = 0.15 × Detection F1
      + 0.05 × Classification Accuracy
      + 0.30 × (1 − Region CER)
      + 0.50 × (1 − Page CER)
```

| Full pipeline configuration | Public test composite |
|---|---:|
| RT-DETR ensemble + geometry + greedy TrOCR | 0.862 |
| + Qwen3.5 9B | 0.896 |
| + Qwen3.6 35B-A3B | **0.905** |
| + Qwen3.5 122B-A10B | 0.914 |

These are results from the challenge test distribution. They are not guaranteed
for unrelated document collections. The Qwen models in the enhanced rows are
served separately and are not part of this weight bundle.

## Intended use and limitations

The release is intended for research, evaluation, and OCR applications involving
Ukrainian handwritten documents. It is especially suited to pages structurally
similar to RUKOPYS.

- The recognizer is specialized for Ukrainian Cyrillic handwriting and may
  perform poorly on other languages, scripts, layouts, or modern camera photos.
- Large tables are a weak point for the base TrOCR-only profile.
- Handwriting is ambiguous. Human review is required for legal, medical,
  identity, archival, or other high-impact uses.
- A Qwen enhancer may replace an incorrect transcription with plausible but
  still incorrect text.
- `warp_selector.pkl` is a Python pickle. Load it only from this trusted,
  checksum-verified repository; pickle files can execute code when deserialized.

## Training data and upstream models

- Dataset: [UkrainianCatholicUniversity/rukopys](https://huggingface.co/datasets/UkrainianCatholicUniversity/rukopys),
  licensed CC-BY-4.0.
- Recognizer base: [Kansallisarkisto/cyrillic-htr-model](https://huggingface.co/Kansallisarkisto/cyrillic-htr-model),
  licensed Apache-2.0.
- Detector architecture/runtime: Ultralytics RT-DETR-L.

The compact training path and recorded profiles for both released RT-DETR
checkpoints are available in
[`training/detectors`](https://github.com/Isterikus/markone-ocr/tree/main/training/detectors).
Data-curation experiments are kept separate from the model trainer. See
`THIRD_PARTY_NOTICES.md` for dependency attribution.

## License

The Mark One code and custom weights are released under **GNU AGPL-3.0-only**
because the detector pipeline uses Ultralytics RT-DETR under AGPL-3.0.
Third-party components retain their respective licenses; see `LICENSE` and
`THIRD_PARTY_NOTICES.md` for details.

## Citation

If you use Mark One OCR in research, you can cite the software as:

```bibtex
@misc{markoneocr2026,
  title        = {Mark One OCR},
  author       = {Mark One},
  year         = {2026},
  url          = {https://github.com/Isterikus/markone-ocr}
}
```