code/PIPELINE.md

# HITIT-OCR v1 Pipeline

**SOTA-üstü 4-stage cascaded cuneiform OCR pipeline** — 2024-2026 literature'da en iyi yaklaşımların cuneiform-spesifik kombinasyonu.

## Hedef

Ham tablet görüntüsü → **transliterasyon** (Hittite/Akkadian/Sumerian/Elamite):

```
raw tablet image → sign detection → ABZ classification → line assembly → transliteration
```

### SOTA baseline'ları geçmek
- **DeepScribe** (ACM JOCCH 2025): end-to-end top-5 %80 (Elamite, 141 sınıf)
- **CHURRO** (EMNLP 2025): Qwen2.5-VL-3B normalized Levenshtein %82.3
- **PreP-OCR** (ACL 2025): CER -%63.9 post-correct
- **HATFormer** (2024): CER %8.6 Arabic historical

**Bizim hedef**: 3300 sınıf, IR 13,000×, 4 dil, tier-stratified macro-F1.

## Mimari

```
┌─────────────────────────────────────────────────────────────────────┐
│                    INPUT: Tablet image (any res)                    │
└──────────────────────────┬──────────────────────────────────────────┘
                           ▼
┌─────────────────────────────────────────────────────────────────────┐
│ Stage 0 — Preprocessing & Restoration                               │
│   • Canonical rotation (canonical_rotation_deg)                     │
│   • Illumination normalization (Retinex / Stötzner 2023)            │
│   • NAFNet restoration (if quality_gate_pass=False)                 │
│   • Multi-scale letterbox: 224 (cls) / 1280 (det)                   │
│   • Dataset-specific normalization (mean/std in config)             │
└──────────────────────────┬──────────────────────────────────────────┘
                           ▼
┌─────────────────────────────────────────────────────────────────────┐
│ Stage 1 — Sign Detection (YOLO11-m + P2 + SAHI)                     │
│   Training: tablet_view_fold 5-fold CV                              │
│   Output: bbox, objectness, top-k class candidates                  │
│   Loss: focal + CIoU + copy-paste aug                               │
│   TTA: multi-scale {0.8, 1.0, 1.2} + WBF                            │
└──────────────────────────┬──────────────────────────────────────────┘
                           ▼ crop 224 × 224 letterbox
┌─────────────────────────────────────────────────────────────────────┐
│ Stage 2 — Classification (DINOv3-B/14 + LoRA + Hierarchical)        │
│   Pretrain: continual DINOv3 SSL on in_curated_pretrain (70K)       │
│   Head: 2-level hierarchical                                         │
│     • top: ABZ super-class (≈500 base sign)                         │
│     • sub: sign_variant_code (OH/MH/NH stil)                        │
│   Rare tier: prototype learning (5-shot)                            │
│   Loss schedule (2-phase):                                          │
│     phase1 (0-40): CB-Focal γ=2, β=0.9999                           │
│     phase2 (40-80): LDAM-DRW decoupling (Kang 2020)                 │
│   Sampler: tiered — head uniform, mid/tail sqrt, rare prototype     │
│   Augmentation: Mixup + TailMix + CutMix + elastic + illum          │
│   Output: top-k ABZ + calibrated logits                             │
└──────────────────────────┬──────────────────────────────────────────┘
                           ▼ sign lattice (top-5 per bbox)
┌─────────────────────────────────────────────────────────────────────┐
│ Stage 3 — Line Assembly & Open-set Filter                           │
│   • Baseline fit (RANSAC) → row clustering                          │
│   • Column split (cuneiform reverse-Z reading order)                │
│   • Energy score (Liu 2020) → OOD threshold                         │
│   • Rare tier + low energy → ABZ_UNK token                          │
│   Evaluation: open-set AUROC on oltr_holdout                        │
└──────────────────────────┬──────────────────────────────────────────┘
                           ▼ ordered sign lattice
┌─────────────────────────────────────────────────────────────────────┐
│ Stage 4 — Transliteration & Post-correction                         │
│   LM: ByT5-small fine-tuned on TLHdig (21K) + fragments (23K)       │
│   Rescoring: Viterbi lattice = posterior × LM prior                 │
│   N-gram backoff: KenLM 5-gram char-level                           │
│   Optional: Qwen2.5-VL-3B LoRA reranker (CHURRO-style)              │
│   Output: transliteration (hit/akk/sum/elx)                         │
│   Evaluation: CER stratified by language × period                   │
└─────────────────────────────────────────────────────────────────────┘
```

## Stage detayları

### Stage 0 — Preprocessing

Mevcut `hitit_ocr/src/preprocessing/` modülü kullanılır (pipeline.py: CLAHE + gamma + letterbox + MSII proxy). Eklenenler:

**Yeni**: Retinex MSR illumination norm + NAFNet gating.

```python
# Conditional restoration flow
if r['quality_gate_pass'] is False:  # blur/contrast/exposure fail
    img = nafnet_restore(img)  # GPU, offline
# Always
img = retinex_msr(img)  # CPU, MSR(σ=[15,80,250])
img = canonical_rotate(img, r['canonical_rotation_deg'])
img = letterbox(img, target=224, margin=0.10, fill='median_border')
img = normalize(img, mean=[0.489,0.448,0.424], std=[0.362,0.359,0.364])
```

Config: `hitit_ocr/configs/preprocessing.yaml` (v1.0 hazır).

### Stage 1 — Detection (YOLO11-m + P2)

Neden **YOLO11-m + P2 head**:
- Small-object benchmark (arXiv 2504.09900): P2 head 15-25px wedge'lerde YOLOv10'dan +%2-4
- 3300 sınıflı closed-set için DETR-family gereksiz
- SAHI slicing ile 3000×4000 tablet foto'ta +%3-8 recall

```yaml
# hitit_ocr/configs/detection_hitit_v1.yaml
model: yolo11m-p2.yaml
imgsz: 1280
epochs: 150
batch: 4               # A100 40GB × 4 GPU
optimizer: AdamW
lr0: 0.001
momentum: 0.937
weight_decay: 0.0005
warmup_epochs: 3
box: 7.5
cls: 0.5
dfl: 1.5
mosaic: 0.8
mixup: 0.15
copy_paste: 0.15       # Sign copy-paste tail için kritik
fliplr: 0.0            # Cuneiform yön-duyarlı
flipud: 0.0
degrees: 5             # canonical_rotation_deg ile baş çözüldü
hsv_h: 0.015
hsv_s: 0.7
hsv_v: 0.4
close_mosaic: 10       # son 10 epoch mosaic kapat
```

**Data**: `datasets/unified/detection/manifest.parquet` + `tablet_view_fold` (leakage-free 5-fold).

**Training**:
```bash
# 5-fold CV, her fold 150 epoch
for fold in 0 1 2 3 4; do
  sbatch hitit_ocr/scripts/train_yolo11_fold.slurm $fold
done
```

**Inference + SAHI**:
```python
from sahi import AutoDetectionModel
from sahi.predict import get_sliced_prediction

model = AutoDetectionModel.from_pretrained("yolo11", model_path="best.pt")
result = get_sliced_prediction(
    image,
    detection_model=model,
    slice_height=640, slice_width=640,
    overlap_height_ratio=0.2, overlap_width_ratio=0.2,
    postprocess_type="NMS"
)
```

**TTA**: `eval_det_tta_slurm.sh` mevcut — multi-scale {0.8, 1.0, 1.2} + WBF.

**Target**: mAP50-95 ≥ 0.80 (DeepScribe 0.78'i geçmek).

### Stage 2 — Classification (DINOv3 + LoRA + Hierarchical)

#### 2.1 Continual SSL pretraining (hafta 2-3)

DINOv3 ViT-B/14 teacher'ı `in_curated_pretrain=True` 70K image üzerinde continual SSL:
- iBOT + DINO loss
- Teacher EMA 0.996
- 20 epoch, 4×A100, bs=256
- Output: `dinov3_vitb14_hitit_pretrained.pth`

```yaml
# hitit_ocr/configs/ssl_dinov3_continual.yaml
teacher: facebook/dinov3-vitb14
epochs: 20
batch: 256
lr: 1e-4
warmup: 3
ema_teacher: 0.996
local_crops_number: 8
global_crops_scale: [0.4, 1.0]
local_crops_scale: [0.05, 0.4]
patch_drop: 0.0
use_fp16: true
data:
  filter: "in_curated_pretrain == True"
  source: datasets/unified/classification/manifest.parquet
```

#### 2.2 Hierarchical classifier fine-tune (hafta 4-7)

```python
class HierarchicalClassifier(nn.Module):
    def __init__(self, backbone, n_abz_base=500, n_variants=None):
        self.backbone = backbone  # DINOv3-B/14 frozen or LoRA
        self.apply_lora(rank=16, alpha=32, target=['qkv','proj'])
        self.head_abz = nn.Linear(768, n_abz_base)
        self.head_variant = nn.Linear(768 + n_abz_base, n_variants)
        self.prototype_head = PrototypeHead(emb_dim=768)
    
    def forward(self, x):
        feat = self.backbone(x)  # (B, 768)
        abz_logits = self.head_abz(feat)
        variant_logits = self.head_variant(torch.cat([feat, abz_logits], -1))
        proto_logits = self.prototype_head(feat)  # rare tier için
        return abz_logits, variant_logits, proto_logits
```

**Loss schedule (2-phase decoupling, Kang 2020)**:
```python
# Phase 1 (epoch 0-40): representation learning
loss = cb_focal(logits, y, beta=0.9999, gamma=2.0)

# Phase 2 (epoch 40-80): classifier re-balancing (DRW)
loss = 0.3 * cb_focal(logits, y) + 0.7 * ldam(logits, y, max_m=0.5, s=30)
```

**CB-Focal** (`effective_num_weight` manifold alanı zaten hesaplı):
```python
class_weights = torch.tensor([r['effective_num_weight'] for r in manifest])
# ya da: (1 - beta) / (1 - beta^n_c)
loss = F.cross_entropy(logits * s, y, weight=class_weights, reduction='none')
loss = loss * ((1 - pt)**gamma)  # focal
```

**LDAM** (Cao 2019):
```python
def ldam_loss(logits, y, class_n, max_m=0.5, s=30):
    m_list = 1.0 / torch.sqrt(torch.sqrt(class_n.float()))
    m_list = m_list * max_m / m_list.max()
    index = torch.zeros_like(logits, dtype=torch.bool)
    index.scatter_(1, y.view(-1,1), True)
    x_m = logits - m_list[None, :]
    output = torch.where(index, x_m, logits) * s
    return F.cross_entropy(output, y)
```

**Tiered sampler**:
```python
# class_frequency_tier alanı kullanarak
# head (tier='head'): uniform sampling (regular)
# mid/tail: WeightedRandomSampler(sampler_weight)
# rare: ProtoNet 5-shot episode sampler
```

**Augmentation**:
- Mixup α=0.2 (head tier pairs only)
- **TailMix**: aynı unified_label içi pair mixup (tail tier)
- CutMix α=1.0
- Elastic + GridDistortion (from `augment_recipe.py`)
- Dataset-specific normalize

**Config**:
```yaml
# hitit_ocr/configs/classification_hitit_v1.yaml
backbone: dinov3_vitb14_hitit_pretrained.pth
lora: {r: 16, alpha: 32, targets: ['qkv', 'proj']}
head: hierarchical
n_abz_base: 500
n_variants: 3300
prototype_for_rare: true
rare_threshold: 5
loss:
  phase_1: {epochs: [0, 40], cb_focal: {beta: 0.9999, gamma: 2.0, weight: 1.0}, ldam: {weight: 0.0}}
  phase_2: {epochs: [40, 80], cb_focal: {weight: 0.3}, ldam: {max_m: 0.5, s: 30, weight: 0.7}}
sampler: tiered
augment:
  mixup: {alpha: 0.2, p: 0.5}
  tailmix: {alpha: 0.5, p: 0.3, intra_class: true}
  cutmix: {alpha: 1.0, p: 0.3}
  elastic: true
  grid_distortion: true
  color_jitter: true
  horizontal_flip: false        # cuneiform yön-duyarlı
optimizer: AdamW
lr: {backbone: 1e-5, lora: 1e-4, head: 1e-3}
scheduler: cosine_warm_restarts
T_0: 20, T_mult: 2, eta_min: 1e-6
epochs: 80
batch: 48
```

**Target**: macro-F1 tier-stratified:
- head: ≥ 0.92
- mid: ≥ 0.75
- tail: ≥ 0.50
- rare (OLTR): ≥ 0.25 + open-set AUROC ≥ 0.85

### Stage 3 — Line Assembly & Open-set

**Line assembly**:
1. Tüm bbox merkezleri → DBSCAN clustering (eps=median_sign_height × 1.5)
2. Her satırda x-coord sıralama (cuneiform left-to-right)
3. Tablet view aware: obverse/reverse sıralaması farklı

**Open-set energy**:
```python
def energy_score(logits, T=1.0):
    return -T * torch.logsumexp(logits / T, dim=-1)

# Threshold calibration on oltr_holdout
# OLTR holdout'ta FPR=%5 veren threshold seç
```

### Stage 4 — Transliteration (ByT5 + Viterbi)

**Training**:
- Input: Stage 3 sign lattice (top-5 per position + confidence)
- Output: transliteration string
- Data: TLHdig corpus (21K Hittite) + fragments (23K Akkadian)

```yaml
# hitit_ocr/configs/transliteration_byt5.yaml
model: google/byt5-small
max_input_len: 512         # byte-level
max_output_len: 256
epochs: 20
batch: 32
lr: 5e-5
warmup: 500
beam_size: 5
length_penalty: 0.6
```

**Viterbi rescoring**:
```python
def viterbi_lattice(lattice, lm_model, lambda_lm=0.3):
    """lattice: list of (top-k ABZ, log-probs) per position.
       LM prior: ByT5 score over candidate sequences."""
    best_path = viterbi(
        emissions=lattice,
        lm_scores=lm_model.score_all_paths(lattice, beam=10),
        lambda_lm=lambda_lm
    )
    return best_path
```

**Target**: CER stratified by language
- hit: < 8%
- akk: < 9%
- sum: < 12%
- elx: < 10%

**Optional reranker**: Qwen2.5-VL-3B LoRA sadece top-3 disagreement durumunda devreye gir (CHURRO stili).

## Training schedule (SLURM orchestration)

| Hafta | Aşama | Config | GPU-saat |
|---|---|---|---|
| 1 | Sentetik restoration pretrain (NAFNet) | Stage 0 | 50 |
| 2-3 | DINOv3 continual SSL on curated 70K | Stage 2.1 | 400 |
| 3-5 | YOLO11-P2 detection 5-fold CV | Stage 1 | 600 |
| 4-7 | Classifier 2-phase LDAM-DRW | Stage 2.2 | 800 |
| 6 | Prototype head rare tier | Stage 2 | 40 |
| 7-8 | ByT5 transliteration | Stage 4 | 100 |
| 9 | Qwen2.5-VL reranker (opt.) | Stage 4+ | 200 |
| 10 | End-to-end eval + TTA | all | 100 |

**Toplam**: ~2300 GPU-saat A100, 4-GPU akya-cuda node × 24 gün.

## Neden SOTA'yı geçer

1. **Long-tail SOTA**: CB-Focal → LDAM-DRW decoupling + prototype rare head (arXiv 2404.15593 survey'de en güçlü combo)
2. **Paleografik**: DINOv3 SSL + hierarchical variant head — aynı ABZ farklı era ayrımında ResNet flat'tan +%4-6
3. **Small-object**: YOLO11-P2 + SAHI + copy-paste aug — DeepScribe RetinaNet'ten küçük sign'larda +%3-8
4. **Multi-hypothesis**: ByT5 lattice Viterbi — PreP-OCR single-best'ten tail error -%20
5. **Domain adaptation**: DINOv3 continual SSL on mixed render+photo — Stötzner'ın explicit domain adapt'inden implicit olarak daha iyi
6. **Evaluation rigor**: tablet_view_fold leakage-free, tier-stratified macro-F1, OLTR AUROC, gold eval 3253

## Data flow haritası

| Manifest alan | Stage | Kullanım |
|---|---|---|
| `unified_label` | Stage 2 target | y (3300-way) |
| `sign_variant_code` | Stage 2 subhead | Hierarchical variant branch |
| `fold` / `tablet_view_fold` | All | 5-fold CV split |
| `class_frequency_tier` | Stage 2 | Tiered sampler, LDAM margin |
| `sampler_weight` | Stage 2 | WeightedRandomSampler |
| `effective_num_weight` | Stage 2 | CB-Loss |
| `quality_gate_pass` | Stage 0 | Restoration gate |
| `blur_score, exposure_mean, contrast_std` | Stage 0 | Illum aug params |
| `canonical_rotation_deg` | Stage 0 | Rectify |
| `phonetic_reading` | Stage 4 | ByT5 target |
| `visual_phash, is_duplicate` | All | Dedup sanity |
| `oltr_holdout` | Stage 3 eval | Open-set AUROC |
| `in_curated_pretrain` | Stage 2.1 | DINOv3 SSL pool |
| `in_gold_eval` | Final eval | Hitit gold 3253 |
| `period, language, script_era` | Eval | Stratified breakdown |
| `uncertainty_score` | Active learning | Next-round candidate |

## Evaluation pipeline

```python
# hitit_ocr/src/evaluate_e2e.py
metrics = {
    "detection": {
        "mAP50_95_macro": ...,
        "per_source_mAP": {...},
    },
    "classification": {
        "top1_macro_F1": ...,
        "per_tier_F1": {"head": ..., "mid": ..., "tail": ..., "rare": ...},
        "top5_accuracy": ...,
        "balanced_accuracy": ...,
    },
    "openset": {
        "AUROC_oltr": ...,  # oltr_holdout
        "FPR_at_95_TPR": ...,
    },
    "transliteration": {
        "CER_hit": ..., "CER_akk": ..., "CER_sum": ..., "CER_elx": ...,
        "BLEU": ...,
        "exact_match": ...,
    },
    "e2e_tablet": {
        "gold_accuracy": ...,  # in_gold_eval 3253 subset
        "per_period": {...},   # OH, MH, NH, OB, NA, NB, Ur-III, ACH
    }
}
```

**Baselines for comparison**:
- DeepScribe reproduced on our data
- CHURRO zero-shot on our gold set
- CuReD on transliteration OCR subset
- PreP-OCR with same ByT5 but without Viterbi

## Kaynaklar

- [CHURRO (EMNLP 2025)](https://arxiv.org/abs/2509.19768)
- [PreP-OCR (ACL 2025)](https://arxiv.org/abs/2505.20429)
- [DeepScribe (ACM JOCCH 2025)](https://arxiv.org/abs/2306.01268)
- [HATFormer (2024)](https://arxiv.org/abs/2410.02179)
- [DINOv3 (Meta 2025)](https://arxiv.org/abs/2508.10104)
- [Stötzner cuneiform 3D (ICCVW 2023)](https://arxiv.org/abs/2308.11277)
- [Long-tail survey (2024)](https://arxiv.org/pdf/2404.15593)
- [Small object YOLO (2025)](https://arxiv.org/html/2504.09900v1)
- [CuReD (ACL ML4AL 2024)](https://aclanthology.org/2024.ml4al-1.14/)
- [Qwen2.5-VL (2025)](https://arxiv.org/pdf/2502.13923)
- [LDAM (Cao 2019)](https://arxiv.org/abs/1906.07413)
- [Kang 2020 decoupling](https://arxiv.org/abs/1910.09217)
- [CB-Loss Cui 2019](https://arxiv.org/abs/1901.05555)
- [SAHI](https://github.com/obss/sahi)