code/PIPELINE.md · savastakan/hitit-cuneiform-ocr at main

Initial upload: code + 5 record checkpoints + fuse

f211247 verified 6 months ago

18.3 kB

# HITIT-OCR v1 Pipeline

**SOTA-üstü 4-stage cascaded cuneiform OCR pipeline** — 2024-2026 literature'da en iyi yaklaşımların cuneiform-spesifik kombinasyonu.

## Hedef

Ham tablet görüntüsü → **transliterasyon** (Hittite/Akkadian/Sumerian/Elamite):

```
raw tablet image → sign detection → ABZ classification → line assembly → transliteration
```

### SOTA baseline'ları geçmek
- **DeepScribe** (ACM JOCCH 2025): end-to-end top-5 %80 (Elamite, 141 sınıf)
- **CHURRO** (EMNLP 2025): Qwen2.5-VL-3B normalized Levenshtein %82.3
- **PreP-OCR** (ACL 2025): CER -%63.9 post-correct
- **HATFormer** (2024): CER %8.6 Arabic historical

**Bizim hedef**: 3300 sınıf, IR 13,000×, 4 dil, tier-stratified macro-F1.

## Mimari

```
┌─────────────────────────────────────────────────────────────────────┐
│                    INPUT: Tablet image (any res)                    │
└──────────────────────────┬──────────────────────────────────────────┘
                           ▼
┌─────────────────────────────────────────────────────────────────────┐
│ Stage 0 — Preprocessing & Restoration                               │
│   • Canonical rotation (canonical_rotation_deg)                     │
│   • Illumination normalization (Retinex / Stötzner 2023)            │
│   • NAFNet restoration (if quality_gate_pass=False)                 │
│   • Multi-scale letterbox: 224 (cls) / 1280 (det)                   │
│   • Dataset-specific normalization (mean/std in config)             │
└──────────────────────────┬──────────────────────────────────────────┘
                           ▼
┌─────────────────────────────────────────────────────────────────────┐
│ Stage 1 — Sign Detection (YOLO11-m + P2 + SAHI)                     │
│   Training: tablet_view_fold 5-fold CV                              │
│   Output: bbox, objectness, top-k class candidates                  │
│   Loss: focal + CIoU + copy-paste aug                               │
│   TTA: multi-scale {0.8, 1.0, 1.2} + WBF                            │
└──────────────────────────┬──────────────────────────────────────────┘
                           ▼ crop 224 × 224 letterbox
┌─────────────────────────────────────────────────────────────────────┐
│ Stage 2 — Classification (DINOv3-B/14 + LoRA + Hierarchical)        │
│   Pretrain: continual DINOv3 SSL on in_curated_pretrain (70K)       │
│   Head: 2-level hierarchical                                         │
│     • top: ABZ super-class (≈500 base sign)                         │
│     • sub: sign_variant_code (OH/MH/NH stil)                        │
│   Rare tier: prototype learning (5-shot)                            │
│   Loss schedule (2-phase):                                          │
│     phase1 (0-40): CB-Focal γ=2, β=0.9999                           │
│     phase2 (40-80): LDAM-DRW decoupling (Kang 2020)                 │
│   Sampler: tiered — head uniform, mid/tail sqrt, rare prototype     │
│   Augmentation: Mixup + TailMix + CutMix + elastic + illum          │
│   Output: top-k ABZ + calibrated logits                             │
└──────────────────────────┬──────────────────────────────────────────┘
                           ▼ sign lattice (top-5 per bbox)
┌─────────────────────────────────────────────────────────────────────┐
│ Stage 3 — Line Assembly & Open-set Filter                           │
│   • Baseline fit (RANSAC) → row clustering                          │
│   • Column split (cuneiform reverse-Z reading order)                │
│   • Energy score (Liu 2020) → OOD threshold                         │
│   • Rare tier + low energy → ABZ_UNK token                          │
│   Evaluation: open-set AUROC on oltr_holdout                        │
└──────────────────────────┬──────────────────────────────────────────┘
                           ▼ ordered sign lattice
┌─────────────────────────────────────────────────────────────────────┐
│ Stage 4 — Transliteration & Post-correction                         │
│   LM: ByT5-small fine-tuned on TLHdig (21K) + fragments (23K)       │
│   Rescoring: Viterbi lattice = posterior × LM prior                 │
│   N-gram backoff: KenLM 5-gram char-level                           │
│   Optional: Qwen2.5-VL-3B LoRA reranker (CHURRO-style)              │
│   Output: transliteration (hit/akk/sum/elx)                         │
│   Evaluation: CER stratified by language × period                   │
└─────────────────────────────────────────────────────────────────────┘
```

## Stage detayları

### Stage 0 — Preprocessing

Mevcut `hitit_ocr/src/preprocessing/` modülü kullanılır (pipeline.py: CLAHE + gamma + letterbox + MSII proxy). Eklenenler:

**Yeni**: Retinex MSR illumination norm + NAFNet gating.

```python
# Conditional restoration flow
if r['quality_gate_pass'] is False:  # blur/contrast/exposure fail
    img = nafnet_restore(img)  # GPU, offline
# Always
img = retinex_msr(img)  # CPU, MSR(σ=[15,80,250])
img = canonical_rotate(img, r['canonical_rotation_deg'])
img = letterbox(img, target=224, margin=0.10, fill='median_border')
img = normalize(img, mean=[0.489,0.448,0.424], std=[0.362,0.359,0.364])
```

Config: `hitit_ocr/configs/preprocessing.yaml` (v1.0 hazır).

### Stage 1 — Detection (YOLO11-m + P2)

Neden **YOLO11-m + P2 head**:
- Small-object benchmark (arXiv 2504.09900): P2 head 15-25px wedge'lerde YOLOv10'dan +%2-4
- 3300 sınıflı closed-set için DETR-family gereksiz
- SAHI slicing ile 3000×4000 tablet foto'ta +%3-8 recall

```yaml
# hitit_ocr/configs/detection_hitit_v1.yaml
model: yolo11m-p2.yaml
imgsz: 1280
epochs: 150
batch: 4               # A100 40GB × 4 GPU
optimizer: AdamW
lr0: 0.001
momentum: 0.937
weight_decay: 0.0005
warmup_epochs: 3
box: 7.5
cls: 0.5
dfl: 1.5
mosaic: 0.8
mixup: 0.15
copy_paste: 0.15       # Sign copy-paste tail için kritik
fliplr: 0.0            # Cuneiform yön-duyarlı
flipud: 0.0
degrees: 5             # canonical_rotation_deg ile baş çözüldü
hsv_h: 0.015
hsv_s: 0.7
hsv_v: 0.4
close_mosaic: 10       # son 10 epoch mosaic kapat
```

**Data**: `datasets/unified/detection/manifest.parquet` + `tablet_view_fold` (leakage-free 5-fold).

**Training**:
```bash
# 5-fold CV, her fold 150 epoch
for fold in 0 1 2 3 4; do
  sbatch hitit_ocr/scripts/train_yolo11_fold.slurm $fold
done
```

**Inference + SAHI**:
```python
from sahi import AutoDetectionModel
from sahi.predict import get_sliced_prediction
model = AutoDetectionModel.from_pretrained("yolo11", model_path="best.pt")
result = get_sliced_prediction(
    image,
    detection_model=model,
    slice_height=640, slice_width=640,
    overlap_height_ratio=0.2, overlap_width_ratio=0.2,
    postprocess_type="NMS"
)
```

**TTA**: `eval_det_tta_slurm.sh` mevcut — multi-scale {0.8, 1.0, 1.2} + WBF.

**Target**: mAP50-95 ≥ 0.80 (DeepScribe 0.78'i geçmek).

### Stage 2 — Classification (DINOv3 + LoRA + Hierarchical)

#### 2.1 Continual SSL pretraining (hafta 2-3)

DINOv3 ViT-B/14 teacher'ı `in_curated_pretrain=True` 70K image üzerinde continual SSL:
- iBOT + DINO loss
- Teacher EMA 0.996
- 20 epoch, 4×A100, bs=256
- Output: `dinov3_vitb14_hitit_pretrained.pth`

```yaml
# hitit_ocr/configs/ssl_dinov3_continual.yaml
teacher: facebook/dinov3-vitb14
epochs: 20
batch: 256
lr: 1e-4
warmup: 3
ema_teacher: 0.996
local_crops_number: 8
global_crops_scale: [0.4, 1.0]
local_crops_scale: [0.05, 0.4]
patch_drop: 0.0
use_fp16: true
data:
  filter: "in_curated_pretrain == True"
  source: datasets/unified/classification/manifest.parquet
```

#### 2.2 Hierarchical classifier fine-tune (hafta 4-7)

```python
class HierarchicalClassifier(nn.Module):
    def __init__(self, backbone, n_abz_base=500, n_variants=None):
        self.backbone = backbone  # DINOv3-B/14 frozen or LoRA
        self.apply_lora(rank=16, alpha=32, target=['qkv','proj'])
        self.head_abz = nn.Linear(768, n_abz_base)
        self.head_variant = nn.Linear(768 + n_abz_base, n_variants)
        self.prototype_head = PrototypeHead(emb_dim=768)
    
    def forward(self, x):
        feat = self.backbone(x)  # (B, 768)
        abz_logits = self.head_abz(feat)
        variant_logits = self.head_variant(torch.cat([feat, abz_logits], -1))
        proto_logits = self.prototype_head(feat)  # rare tier için
        return abz_logits, variant_logits, proto_logits
```

**Loss schedule (2-phase decoupling, Kang 2020)**:
```python
# Phase 1 (epoch 0-40): representation learning
loss = cb_focal(logits, y, beta=0.9999, gamma=2.0)
# Phase 2 (epoch 40-80): classifier re-balancing (DRW)
loss = 0.3 * cb_focal(logits, y) + 0.7 * ldam(logits, y, max_m=0.5, s=30)
```

**CB-Focal** (`effective_num_weight` manifold alanı zaten hesaplı):
```python
class_weights = torch.tensor([r['effective_num_weight'] for r in manifest])
# ya da: (1 - beta) / (1 - beta^n_c)
loss = F.cross_entropy(logits * s, y, weight=class_weights, reduction='none')
loss = loss * ((1 - pt)**gamma)  # focal
```

**LDAM** (Cao 2019):
```python
def ldam_loss(logits, y, class_n, max_m=0.5, s=30):
    m_list = 1.0 / torch.sqrt(torch.sqrt(class_n.float()))
    m_list = m_list * max_m / m_list.max()
    index = torch.zeros_like(logits, dtype=torch.bool)
    index.scatter_(1, y.view(-1,1), True)
    x_m = logits - m_list[None, :]
    output = torch.where(index, x_m, logits) * s
    return F.cross_entropy(output, y)
```

**Tiered sampler**:
```python
# class_frequency_tier alanı kullanarak
# head (tier='head'): uniform sampling (regular)
# mid/tail: WeightedRandomSampler(sampler_weight)
# rare: ProtoNet 5-shot episode sampler
```

**Augmentation**:
- Mixup α=0.2 (head tier pairs only)
- **TailMix**: aynı unified_label içi pair mixup (tail tier)
- CutMix α=1.0
- Elastic + GridDistortion (from `augment_recipe.py`)
- Dataset-specific normalize

**Config**:
```yaml
# hitit_ocr/configs/classification_hitit_v1.yaml
backbone: dinov3_vitb14_hitit_pretrained.pth
lora: {r: 16, alpha: 32, targets: ['qkv', 'proj']}
head: hierarchical
n_abz_base: 500
n_variants: 3300
prototype_for_rare: true
rare_threshold: 5
loss:
  phase_1: {epochs: [0, 40], cb_focal: {beta: 0.9999, gamma: 2.0, weight: 1.0}, ldam: {weight: 0.0}}
  phase_2: {epochs: [40, 80], cb_focal: {weight: 0.3}, ldam: {max_m: 0.5, s: 30, weight: 0.7}}
sampler: tiered
augment:
  mixup: {alpha: 0.2, p: 0.5}
  tailmix: {alpha: 0.5, p: 0.3, intra_class: true}
  cutmix: {alpha: 1.0, p: 0.3}
  elastic: true
  grid_distortion: true
  color_jitter: true
  horizontal_flip: false        # cuneiform yön-duyarlı
optimizer: AdamW
lr: {backbone: 1e-5, lora: 1e-4, head: 1e-3}
scheduler: cosine_warm_restarts
T_0: 20, T_mult: 2, eta_min: 1e-6
epochs: 80
batch: 48
```

**Target**: macro-F1 tier-stratified:
- head: ≥ 0.92
- mid: ≥ 0.75
- tail: ≥ 0.50
- rare (OLTR): ≥ 0.25 + open-set AUROC ≥ 0.85

### Stage 3 — Line Assembly & Open-set

**Line assembly**:
1. Tüm bbox merkezleri → DBSCAN clustering (eps=median_sign_height × 1.5)
2. Her satırda x-coord sıralama (cuneiform left-to-right)
3. Tablet view aware: obverse/reverse sıralaması farklı

**Open-set energy**:
```python
def energy_score(logits, T=1.0):
    return -T * torch.logsumexp(logits / T, dim=-1)
# Threshold calibration on oltr_holdout
# OLTR holdout'ta FPR=%5 veren threshold seç
```

### Stage 4 — Transliteration (ByT5 + Viterbi)

**Training**:
- Input: Stage 3 sign lattice (top-5 per position + confidence)
- Output: transliteration string
- Data: TLHdig corpus (21K Hittite) + fragments (23K Akkadian)

```yaml
# hitit_ocr/configs/transliteration_byt5.yaml
model: google/byt5-small
max_input_len: 512         # byte-level
max_output_len: 256
epochs: 20
batch: 32
lr: 5e-5
warmup: 500
beam_size: 5
length_penalty: 0.6
```

**Viterbi rescoring**:
```python
def viterbi_lattice(lattice, lm_model, lambda_lm=0.3):
    """lattice: list of (top-k ABZ, log-probs) per position.
       LM prior: ByT5 score over candidate sequences."""
    best_path = viterbi(
        emissions=lattice,
        lm_scores=lm_model.score_all_paths(lattice, beam=10),
        lambda_lm=lambda_lm
    )
    return best_path
```

**Target**: CER stratified by language
- hit: < 8%
- akk: < 9%
- sum: < 12%
- elx: < 10%

**Optional reranker**: Qwen2.5-VL-3B LoRA sadece top-3 disagreement durumunda devreye gir (CHURRO stili).

## Training schedule (SLURM orchestration)

| Hafta | Aşama | Config | GPU-saat |
|---|---|---|---|
| 1 | Sentetik restoration pretrain (NAFNet) | Stage 0 | 50 |
| 2-3 | DINOv3 continual SSL on curated 70K | Stage 2.1 | 400 |
| 3-5 | YOLO11-P2 detection 5-fold CV | Stage 1 | 600 |
| 4-7 | Classifier 2-phase LDAM-DRW | Stage 2.2 | 800 |
| 6 | Prototype head rare tier | Stage 2 | 40 |
| 7-8 | ByT5 transliteration | Stage 4 | 100 |
| 9 | Qwen2.5-VL reranker (opt.) | Stage 4+ | 200 |
| 10 | End-to-end eval + TTA | all | 100 |

**Toplam**: ~2300 GPU-saat A100, 4-GPU akya-cuda node × 24 gün.

## Neden SOTA'yı geçer

1. **Long-tail SOTA**: CB-Focal → LDAM-DRW decoupling + prototype rare head (arXiv 2404.15593 survey'de en güçlü combo)
2. **Paleografik**: DINOv3 SSL + hierarchical variant head — aynı ABZ farklı era ayrımında ResNet flat'tan +%4-6
3. **Small-object**: YOLO11-P2 + SAHI + copy-paste aug — DeepScribe RetinaNet'ten küçük sign'larda +%3-8
4. **Multi-hypothesis**: ByT5 lattice Viterbi — PreP-OCR single-best'ten tail error -%20
5. **Domain adaptation**: DINOv3 continual SSL on mixed render+photo — Stötzner'ın explicit domain adapt'inden implicit olarak daha iyi
6. **Evaluation rigor**: tablet_view_fold leakage-free, tier-stratified macro-F1, OLTR AUROC, gold eval 3253

## Data flow haritası

| Manifest alan | Stage | Kullanım |
|---|---|---|
| `unified_label` | Stage 2 target | y (3300-way) |
| `sign_variant_code` | Stage 2 subhead | Hierarchical variant branch |
| `fold` / `tablet_view_fold` | All | 5-fold CV split |
| `class_frequency_tier` | Stage 2 | Tiered sampler, LDAM margin |
| `sampler_weight` | Stage 2 | WeightedRandomSampler |
| `effective_num_weight` | Stage 2 | CB-Loss |
| `quality_gate_pass` | Stage 0 | Restoration gate |
| `blur_score, exposure_mean, contrast_std` | Stage 0 | Illum aug params |
| `canonical_rotation_deg` | Stage 0 | Rectify |
| `phonetic_reading` | Stage 4 | ByT5 target |
| `visual_phash, is_duplicate` | All | Dedup sanity |
| `oltr_holdout` | Stage 3 eval | Open-set AUROC |
| `in_curated_pretrain` | Stage 2.1 | DINOv3 SSL pool |
| `in_gold_eval` | Final eval | Hitit gold 3253 |
| `period, language, script_era` | Eval | Stratified breakdown |
| `uncertainty_score` | Active learning | Next-round candidate |

## Evaluation pipeline

```python
# hitit_ocr/src/evaluate_e2e.py
metrics = {
    "detection": {
        "mAP50_95_macro": ...,
        "per_source_mAP": {...},
    },
    "classification": {
        "top1_macro_F1": ...,
        "per_tier_F1": {"head": ..., "mid": ..., "tail": ..., "rare": ...},
        "top5_accuracy": ...,
        "balanced_accuracy": ...,
    },
    "openset": {
        "AUROC_oltr": ...,  # oltr_holdout
        "FPR_at_95_TPR": ...,
    },
    "transliteration": {
        "CER_hit": ..., "CER_akk": ..., "CER_sum": ..., "CER_elx": ...,
        "BLEU": ...,
        "exact_match": ...,
    },
    "e2e_tablet": {
        "gold_accuracy": ...,  # in_gold_eval 3253 subset
        "per_period": {...},   # OH, MH, NH, OB, NA, NB, Ur-III, ACH
    }
}
```

**Baselines for comparison**:
- DeepScribe reproduced on our data
- CHURRO zero-shot on our gold set
- CuReD on transliteration OCR subset
- PreP-OCR with same ByT5 but without Viterbi

## Kaynaklar

- [CHURRO (EMNLP 2025)](https://arxiv.org/abs/2509.19768)
- [PreP-OCR (ACL 2025)](https://arxiv.org/abs/2505.20429)
- [DeepScribe (ACM JOCCH 2025)](https://arxiv.org/abs/2306.01268)
- [HATFormer (2024)](https://arxiv.org/abs/2410.02179)
- [DINOv3 (Meta 2025)](https://arxiv.org/abs/2508.10104)
- [Stötzner cuneiform 3D (ICCVW 2023)](https://arxiv.org/abs/2308.11277)
- [Long-tail survey (2024)](https://arxiv.org/pdf/2404.15593)
- [Small object YOLO (2025)](https://arxiv.org/html/2504.09900v1)
- [CuReD (ACL ML4AL 2024)](https://aclanthology.org/2024.ml4al-1.14/)
- [Qwen2.5-VL (2025)](https://arxiv.org/pdf/2502.13923)
- [LDAM (Cao 2019)](https://arxiv.org/abs/1906.07413)
- [Kang 2020 decoupling](https://arxiv.org/abs/1910.09217)
- [CB-Loss Cui 2019](https://arxiv.org/abs/1901.05555)
- [SAHI](https://github.com/obss/sahi)

	# HITIT-OCR v1 Pipeline

	SOTA-üstü 4-stage cascaded cuneiform OCR pipeline — 2024-2026 literature'da en iyi yaklaşımların cuneiform-spesifik kombinasyonu.

	## Hedef

	Ham tablet görüntüsü → transliterasyon (Hittite/Akkadian/Sumerian/Elamite):

	```
	raw tablet image → sign detection → ABZ classification → line assembly → transliteration
	```

	### SOTA baseline'ları geçmek
	- DeepScribe (ACM JOCCH 2025): end-to-end top-5 %80 (Elamite, 141 sınıf)
	- CHURRO (EMNLP 2025): Qwen2.5-VL-3B normalized Levenshtein %82.3
	- PreP-OCR (ACL 2025): CER -%63.9 post-correct
	- HATFormer (2024): CER %8.6 Arabic historical

	Bizim hedef: 3300 sınıf, IR 13,000×, 4 dil, tier-stratified macro-F1.

	## Mimari

	```
	┌─────────────────────────────────────────────────────────────────────┐
	│ INPUT: Tablet image (any res) │
	└──────────────────────────┬──────────────────────────────────────────┘
	▼
	┌─────────────────────────────────────────────────────────────────────┐
	│ Stage 0 — Preprocessing & Restoration │
	│ • Canonical rotation (canonical_rotation_deg) │
	│ • Illumination normalization (Retinex / Stötzner 2023) │
	│ • NAFNet restoration (if quality_gate_pass=False) │
	│ • Multi-scale letterbox: 224 (cls) / 1280 (det) │
	│ • Dataset-specific normalization (mean/std in config) │
	└──────────────────────────┬──────────────────────────────────────────┘
	▼
	┌─────────────────────────────────────────────────────────────────────┐
	│ Stage 1 — Sign Detection (YOLO11-m + P2 + SAHI) │
	│ Training: tablet_view_fold 5-fold CV │
	│ Output: bbox, objectness, top-k class candidates │
	│ Loss: focal + CIoU + copy-paste aug │
	│ TTA: multi-scale {0.8, 1.0, 1.2} + WBF │
	└──────────────────────────┬──────────────────────────────────────────┘
	▼ crop 224 × 224 letterbox
	┌─────────────────────────────────────────────────────────────────────┐
	│ Stage 2 — Classification (DINOv3-B/14 + LoRA + Hierarchical) │
	│ Pretrain: continual DINOv3 SSL on in_curated_pretrain (70K) │
	│ Head: 2-level hierarchical │
	│ • top: ABZ super-class (≈500 base sign) │
	│ • sub: sign_variant_code (OH/MH/NH stil) │
	│ Rare tier: prototype learning (5-shot) │
	│ Loss schedule (2-phase): │
	│ phase1 (0-40): CB-Focal γ=2, β=0.9999 │
	│ phase2 (40-80): LDAM-DRW decoupling (Kang 2020) │
	│ Sampler: tiered — head uniform, mid/tail sqrt, rare prototype │
	│ Augmentation: Mixup + TailMix + CutMix + elastic + illum │
	│ Output: top-k ABZ + calibrated logits │
	└──────────────────────────┬──────────────────────────────────────────┘
	▼ sign lattice (top-5 per bbox)
	┌─────────────────────────────────────────────────────────────────────┐
	│ Stage 3 — Line Assembly & Open-set Filter │
	│ • Baseline fit (RANSAC) → row clustering │
	│ • Column split (cuneiform reverse-Z reading order) │
	│ • Energy score (Liu 2020) → OOD threshold │
	│ • Rare tier + low energy → ABZ_UNK token │
	│ Evaluation: open-set AUROC on oltr_holdout │
	└──────────────────────────┬──────────────────────────────────────────┘
	▼ ordered sign lattice
	┌─────────────────────────────────────────────────────────────────────┐
	│ Stage 4 — Transliteration & Post-correction │
	│ LM: ByT5-small fine-tuned on TLHdig (21K) + fragments (23K) │
	│ Rescoring: Viterbi lattice = posterior × LM prior │
	│ N-gram backoff: KenLM 5-gram char-level │
	│ Optional: Qwen2.5-VL-3B LoRA reranker (CHURRO-style) │
	│ Output: transliteration (hit/akk/sum/elx) │
	│ Evaluation: CER stratified by language × period │
	└─────────────────────────────────────────────────────────────────────┘
	```

	## Stage detayları

	### Stage 0 — Preprocessing

	Mevcut `hitit_ocr/src/preprocessing/` modülü kullanılır (pipeline.py: CLAHE + gamma + letterbox + MSII proxy). Eklenenler:

	Yeni: Retinex MSR illumination norm + NAFNet gating.

	```python
	# Conditional restoration flow
	if r['quality_gate_pass'] is False: # blur/contrast/exposure fail
	img = nafnet_restore(img) # GPU, offline
	# Always
	img = retinex_msr(img) # CPU, MSR(σ=[15,80,250])
	img = canonical_rotate(img, r['canonical_rotation_deg'])
	img = letterbox(img, target=224, margin=0.10, fill='median_border')
	img = normalize(img, mean=[0.489,0.448,0.424], std=[0.362,0.359,0.364])
	```

	Config: `hitit_ocr/configs/preprocessing.yaml` (v1.0 hazır).

	### Stage 1 — Detection (YOLO11-m + P2)

	Neden YOLO11-m + P2 head:
	- Small-object benchmark (arXiv 2504.09900): P2 head 15-25px wedge'lerde YOLOv10'dan +%2-4
	- 3300 sınıflı closed-set için DETR-family gereksiz
	- SAHI slicing ile 3000×4000 tablet foto'ta +%3-8 recall

	```yaml
	# hitit_ocr/configs/detection_hitit_v1.yaml
	model: yolo11m-p2.yaml
	imgsz: 1280
	epochs: 150
	batch: 4 # A100 40GB × 4 GPU
	optimizer: AdamW
	lr0: 0.001
	momentum: 0.937
	weight_decay: 0.0005
	warmup_epochs: 3
	box: 7.5
	cls: 0.5
	dfl: 1.5
	mosaic: 0.8
	mixup: 0.15
	copy_paste: 0.15 # Sign copy-paste tail için kritik
	fliplr: 0.0 # Cuneiform yön-duyarlı
	flipud: 0.0
	degrees: 5 # canonical_rotation_deg ile baş çözüldü
	hsv_h: 0.015
	hsv_s: 0.7
	hsv_v: 0.4
	close_mosaic: 10 # son 10 epoch mosaic kapat
	```

	Data: `datasets/unified/detection/manifest.parquet` + `tablet_view_fold` (leakage-free 5-fold).

	Training:
	```bash
	# 5-fold CV, her fold 150 epoch
	for fold in 0 1 2 3 4; do
	sbatch hitit_ocr/scripts/train_yolo11_fold.slurm $fold
	done
	```

	Inference + SAHI:
	```python
	from sahi import AutoDetectionModel
	from sahi.predict import get_sliced_prediction

	model = AutoDetectionModel.from_pretrained("yolo11", model_path="best.pt")
	result = get_sliced_prediction(
	image,
	detection_model=model,
	slice_height=640, slice_width=640,
	overlap_height_ratio=0.2, overlap_width_ratio=0.2,
	postprocess_type="NMS"
	)
	```

	TTA: `eval_det_tta_slurm.sh` mevcut — multi-scale {0.8, 1.0, 1.2} + WBF.

	Target: mAP50-95 ≥ 0.80 (DeepScribe 0.78'i geçmek).

	### Stage 2 — Classification (DINOv3 + LoRA + Hierarchical)

	#### 2.1 Continual SSL pretraining (hafta 2-3)

	DINOv3 ViT-B/14 teacher'ı `in_curated_pretrain=True` 70K image üzerinde continual SSL:
	- iBOT + DINO loss
	- Teacher EMA 0.996
	- 20 epoch, 4×A100, bs=256
	- Output: `dinov3_vitb14_hitit_pretrained.pth`

	```yaml
	# hitit_ocr/configs/ssl_dinov3_continual.yaml
	teacher: facebook/dinov3-vitb14
	epochs: 20
	batch: 256
	lr: 1e-4
	warmup: 3
	ema_teacher: 0.996
	local_crops_number: 8
	global_crops_scale: [0.4, 1.0]
	local_crops_scale: [0.05, 0.4]
	patch_drop: 0.0
	use_fp16: true
	data:
	filter: "in_curated_pretrain == True"
	source: datasets/unified/classification/manifest.parquet
	```

	#### 2.2 Hierarchical classifier fine-tune (hafta 4-7)

	```python
	class HierarchicalClassifier(nn.Module):
	def __init__(self, backbone, n_abz_base=500, n_variants=None):
	self.backbone = backbone # DINOv3-B/14 frozen or LoRA
	self.apply_lora(rank=16, alpha=32, target=['qkv','proj'])
	self.head_abz = nn.Linear(768, n_abz_base)
	self.head_variant = nn.Linear(768 + n_abz_base, n_variants)
	self.prototype_head = PrototypeHead(emb_dim=768)

	def forward(self, x):
	feat = self.backbone(x) # (B, 768)
	abz_logits = self.head_abz(feat)
	variant_logits = self.head_variant(torch.cat([feat, abz_logits], -1))
	proto_logits = self.prototype_head(feat) # rare tier için
	return abz_logits, variant_logits, proto_logits
	```

	Loss schedule (2-phase decoupling, Kang 2020):
	```python
	# Phase 1 (epoch 0-40): representation learning
	loss = cb_focal(logits, y, beta=0.9999, gamma=2.0)

	# Phase 2 (epoch 40-80): classifier re-balancing (DRW)
	loss = 0.3 * cb_focal(logits, y) + 0.7 * ldam(logits, y, max_m=0.5, s=30)
	```

	CB-Focal (`effective_num_weight` manifold alanı zaten hesaplı):
	```python
	class_weights = torch.tensor([r['effective_num_weight'] for r in manifest])
	# ya da: (1 - beta) / (1 - beta^n_c)
	loss = F.cross_entropy(logits * s, y, weight=class_weights, reduction='none')
	loss = loss * ((1 - pt)**gamma) # focal
	```

	LDAM (Cao 2019):
	```python
	def ldam_loss(logits, y, class_n, max_m=0.5, s=30):
	m_list = 1.0 / torch.sqrt(torch.sqrt(class_n.float()))
	m_list = m_list * max_m / m_list.max()
	index = torch.zeros_like(logits, dtype=torch.bool)
	index.scatter_(1, y.view(-1,1), True)
	x_m = logits - m_list[None, :]
	output = torch.where(index, x_m, logits) * s
	return F.cross_entropy(output, y)
	```

	Tiered sampler:
	```python
	# class_frequency_tier alanı kullanarak
	# head (tier='head'): uniform sampling (regular)
	# mid/tail: WeightedRandomSampler(sampler_weight)
	# rare: ProtoNet 5-shot episode sampler
	```

	Augmentation:
	- Mixup α=0.2 (head tier pairs only)
	- TailMix: aynı unified_label içi pair mixup (tail tier)
	- CutMix α=1.0
	- Elastic + GridDistortion (from `augment_recipe.py`)
	- Dataset-specific normalize

	Config:
	```yaml
	# hitit_ocr/configs/classification_hitit_v1.yaml
	backbone: dinov3_vitb14_hitit_pretrained.pth
	lora: {r: 16, alpha: 32, targets: ['qkv', 'proj']}
	head: hierarchical
	n_abz_base: 500
	n_variants: 3300
	prototype_for_rare: true
	rare_threshold: 5
	loss:
	phase_1: {epochs: [0, 40], cb_focal: {beta: 0.9999, gamma: 2.0, weight: 1.0}, ldam: {weight: 0.0}}
	phase_2: {epochs: [40, 80], cb_focal: {weight: 0.3}, ldam: {max_m: 0.5, s: 30, weight: 0.7}}
	sampler: tiered
	augment:
	mixup: {alpha: 0.2, p: 0.5}
	tailmix: {alpha: 0.5, p: 0.3, intra_class: true}
	cutmix: {alpha: 1.0, p: 0.3}
	elastic: true
	grid_distortion: true
	color_jitter: true
	horizontal_flip: false # cuneiform yön-duyarlı
	optimizer: AdamW
	lr: {backbone: 1e-5, lora: 1e-4, head: 1e-3}
	scheduler: cosine_warm_restarts
	T_0: 20, T_mult: 2, eta_min: 1e-6
	epochs: 80
	batch: 48
	```

	Target: macro-F1 tier-stratified:
	- head: ≥ 0.92
	- mid: ≥ 0.75
	- tail: ≥ 0.50
	- rare (OLTR): ≥ 0.25 + open-set AUROC ≥ 0.85

	### Stage 3 — Line Assembly & Open-set

	Line assembly:
	1. Tüm bbox merkezleri → DBSCAN clustering (eps=median_sign_height × 1.5)
	2. Her satırda x-coord sıralama (cuneiform left-to-right)
	3. Tablet view aware: obverse/reverse sıralaması farklı

	Open-set energy:
	```python
	def energy_score(logits, T=1.0):
	return -T * torch.logsumexp(logits / T, dim=-1)

	# Threshold calibration on oltr_holdout
	# OLTR holdout'ta FPR=%5 veren threshold seç
	```

	### Stage 4 — Transliteration (ByT5 + Viterbi)

	Training:
	- Input: Stage 3 sign lattice (top-5 per position + confidence)
	- Output: transliteration string
	- Data: TLHdig corpus (21K Hittite) + fragments (23K Akkadian)

	```yaml
	# hitit_ocr/configs/transliteration_byt5.yaml
	model: google/byt5-small
	max_input_len: 512 # byte-level
	max_output_len: 256
	epochs: 20
	batch: 32
	lr: 5e-5
	warmup: 500
	beam_size: 5
	length_penalty: 0.6
	```

	Viterbi rescoring:
	```python
	def viterbi_lattice(lattice, lm_model, lambda_lm=0.3):
	"""lattice: list of (top-k ABZ, log-probs) per position.
	LM prior: ByT5 score over candidate sequences."""
	best_path = viterbi(
	emissions=lattice,
	lm_scores=lm_model.score_all_paths(lattice, beam=10),
	lambda_lm=lambda_lm
	)
	return best_path
	```

	Target: CER stratified by language
	- hit: < 8%
	- akk: < 9%
	- sum: < 12%
	- elx: < 10%

	Optional reranker: Qwen2.5-VL-3B LoRA sadece top-3 disagreement durumunda devreye gir (CHURRO stili).

	## Training schedule (SLURM orchestration)

	\| Hafta \| Aşama \| Config \| GPU-saat \|
	\|---\|---\|---\|---\|
	\| 1 \| Sentetik restoration pretrain (NAFNet) \| Stage 0 \| 50 \|
	\| 2-3 \| DINOv3 continual SSL on curated 70K \| Stage 2.1 \| 400 \|
	\| 3-5 \| YOLO11-P2 detection 5-fold CV \| Stage 1 \| 600 \|
	\| 4-7 \| Classifier 2-phase LDAM-DRW \| Stage 2.2 \| 800 \|
	\| 6 \| Prototype head rare tier \| Stage 2 \| 40 \|
	\| 7-8 \| ByT5 transliteration \| Stage 4 \| 100 \|
	\| 9 \| Qwen2.5-VL reranker (opt.) \| Stage 4+ \| 200 \|
	\| 10 \| End-to-end eval + TTA \| all \| 100 \|

	Toplam: ~2300 GPU-saat A100, 4-GPU akya-cuda node × 24 gün.

	## Neden SOTA'yı geçer

	1. Long-tail SOTA: CB-Focal → LDAM-DRW decoupling + prototype rare head (arXiv 2404.15593 survey'de en güçlü combo)
	2. Paleografik: DINOv3 SSL + hierarchical variant head — aynı ABZ farklı era ayrımında ResNet flat'tan +%4-6
	3. Small-object: YOLO11-P2 + SAHI + copy-paste aug — DeepScribe RetinaNet'ten küçük sign'larda +%3-8
	4. Multi-hypothesis: ByT5 lattice Viterbi — PreP-OCR single-best'ten tail error -%20
	5. Domain adaptation: DINOv3 continual SSL on mixed render+photo — Stötzner'ın explicit domain adapt'inden implicit olarak daha iyi
	6. Evaluation rigor: tablet_view_fold leakage-free, tier-stratified macro-F1, OLTR AUROC, gold eval 3253

	## Data flow haritası

	\| Manifest alan \| Stage \| Kullanım \|
	\|---\|---\|---\|
	\| `unified_label` \| Stage 2 target \| y (3300-way) \|
	\| `sign_variant_code` \| Stage 2 subhead \| Hierarchical variant branch \|
	\| `fold` / `tablet_view_fold` \| All \| 5-fold CV split \|
	\| `class_frequency_tier` \| Stage 2 \| Tiered sampler, LDAM margin \|
	\| `sampler_weight` \| Stage 2 \| WeightedRandomSampler \|
	\| `effective_num_weight` \| Stage 2 \| CB-Loss \|
	\| `quality_gate_pass` \| Stage 0 \| Restoration gate \|
	\| `blur_score, exposure_mean, contrast_std` \| Stage 0 \| Illum aug params \|
	\| `canonical_rotation_deg` \| Stage 0 \| Rectify \|
	\| `phonetic_reading` \| Stage 4 \| ByT5 target \|
	\| `visual_phash, is_duplicate` \| All \| Dedup sanity \|
	\| `oltr_holdout` \| Stage 3 eval \| Open-set AUROC \|
	\| `in_curated_pretrain` \| Stage 2.1 \| DINOv3 SSL pool \|
	\| `in_gold_eval` \| Final eval \| Hitit gold 3253 \|
	\| `period, language, script_era` \| Eval \| Stratified breakdown \|
	\| `uncertainty_score` \| Active learning \| Next-round candidate \|

	## Evaluation pipeline

	```python
	# hitit_ocr/src/evaluate_e2e.py
	metrics = {
	"detection": {
	"mAP50_95_macro": ...,
	"per_source_mAP": {...},
	},
	"classification": {
	"top1_macro_F1": ...,
	"per_tier_F1": {"head": ..., "mid": ..., "tail": ..., "rare": ...},
	"top5_accuracy": ...,
	"balanced_accuracy": ...,
	},
	"openset": {
	"AUROC_oltr": ..., # oltr_holdout
	"FPR_at_95_TPR": ...,
	},
	"transliteration": {
	"CER_hit": ..., "CER_akk": ..., "CER_sum": ..., "CER_elx": ...,
	"BLEU": ...,
	"exact_match": ...,
	},
	"e2e_tablet": {
	"gold_accuracy": ..., # in_gold_eval 3253 subset
	"per_period": {...}, # OH, MH, NH, OB, NA, NB, Ur-III, ACH
	}
	}
	```

	Baselines for comparison:
	- DeepScribe reproduced on our data
	- CHURRO zero-shot on our gold set
	- CuReD on transliteration OCR subset
	- PreP-OCR with same ByT5 but without Viterbi

	## Kaynaklar

	- [CHURRO (EMNLP 2025)](https://arxiv.org/abs/2509.19768)
	- [PreP-OCR (ACL 2025)](https://arxiv.org/abs/2505.20429)
	- [DeepScribe (ACM JOCCH 2025)](https://arxiv.org/abs/2306.01268)
	- [HATFormer (2024)](https://arxiv.org/abs/2410.02179)
	- [DINOv3 (Meta 2025)](https://arxiv.org/abs/2508.10104)
	- [Stötzner cuneiform 3D (ICCVW 2023)](https://arxiv.org/abs/2308.11277)
	- [Long-tail survey (2024)](https://arxiv.org/pdf/2404.15593)
	- [Small object YOLO (2025)](https://arxiv.org/html/2504.09900v1)
	- [CuReD (ACL ML4AL 2024)](https://aclanthology.org/2024.ml4al-1.14/)
	- [Qwen2.5-VL (2025)](https://arxiv.org/pdf/2502.13923)
	- [LDAM (Cao 2019)](https://arxiv.org/abs/1906.07413)
	- [Kang 2020 decoupling](https://arxiv.org/abs/1910.09217)
	- [CB-Loss Cui 2019](https://arxiv.org/abs/1901.05555)
	- [SAHI](https://github.com/obss/sahi)