Blame for code/PAPER_METHODS.md

# Hittite Cuneiform OCR — Comprehensive Method Log for Paper

**Last updated:** 2026-04-20 (T+~5h live session, ~40 SLURM jobs submitted)
**Target:** 198-class Hittite ABZ sign classification (3570 val samples, tablet_view_fold=0)
**Baseline:** DINOv3-ViT-B/14 + v2 pipeline → **val_top1 = 0.634 (EMA)**, uniform ensemble of 3 backbones → **0.643**
**Current best (single model):** DINOv3-ViT-L/14 v4 → **val_ema = 0.645** + in-progress Focal-SAM v5 at 0.638@ep22

---

## 0. Data pipeline

| Component | File | Description |
|---|---|---|
| Stratified folds | `src/preprocessing/rebuild_stratified_folds.py` | Flips val-only thin classes (n<3 train) to train fold to eliminate 40 "unseen" val classes. Output: `manifest_classification_stratified.jsonl` |
| Tablet-LOO split | `src/enhancements/tablet_loo_split.py` | Hash-based `tablet_loo_fold` to prevent view-level leakage. 5-fold |
| Tail augmentation (classical) | `src/enhancements/tail_augment.py` | Elastic warp + color jitter + blur; classes n<20 → 30 samples. +1297 synthetic |
| Tail augmentation (diffusion) | `src/enhancements/datadream_tail.py` | DataDream (Kim 2024) with SDXL img2img strength=0.5; falls back to classical if diffusers missing |
| ProtoSnap synthesis | `src/enhancements/protosnap_synth.py` + clone of `TAU-VAILab/ProtoSnap` | ICLR 2025 ControlNet cuneiform generator; 925 Santakku prototypes, maps ABZ names → hex |
| Cleanlab noise (iter 1) | `src/enhancements/cleanlab_noise.py` | Cleanlab confident learning heuristic on v2 DINOv3 predictions. 744 noisy (3.5%) flagged |
| Cleanlab noise (iter 2) | same, on v4 model | 943 noisy (4.4%) — more sensitive with better model |
| Unsup clustering + anchor | `src/enhancements/unsup_cluster_anchor.py` | **Key contribution**: cluster labeled+unlabeled features (K=400), pure clusters (purity ≥ 0.6) get majority label; pseudo-label unlabeled members. 156/400 pure clusters, **36,177 pseudo-labels** produced |
| Pseudo-label (ensemble conf) | `src/enhancements/pseudo_label_cls.py` | v4 ensemble → unlabeled predictions > tier-specific threshold (head 0.95, mid 0.92, tail 0.85) |
| Soft pseudo-label | `src/enhancements/soft_pseudo_label.py` | Top-K soft labels with class-specific dynamic thresholds (Roll-with-Punches, 2024) |
| Active learning export | `src/enhancements/active_learning_export.py` | Margin-low (m<0.2) + high-entropy rows → CSV + image copies for expert review |
| Val inspection HTML | `src/enhancements/val_inspection_html.py` | Thumbnail grid of all val mistakes, sorted by confidence |

---

## 1. Backbones

| Arch | File | Notes |
|---|---|---|
| DINOv3-ViT-B/14 | timm `vit_base_patch14_dinov2` | v2 baseline |
| DINOv3-ViT-L/14 | timm `vit_large_patch14_dinov2` | v3+ upgrade; 1024-dim features |
| ConvNeXt-V2-Large | timm `convnextv2_large` | v4 ensemble member |
| SigLIP2-SO400M | timm `vit_so400m_patch14_siglip_224` | v2 ensemble member (dropped from v3+) |
| EVA-02-L/14 @448 | timm `eva02_large_patch14_448.mim_m38m_ft_in22k_in1k` | Diversity |
| Swin-V2-L | timm `swinv2_large_window12to24_192to384.ms_in22k_ft_in1k` | Fixed `unfreeze_lora` layers[] support in v6 |

Configuration builder: `src/train_classification.py::build_backbone`, arch map in `ARCH_IMG_SIZE`.

---

## 2. Training Objectives (v2 → v4 evolution)

### Classification losses

| Loss | File location | Reference | Where activated |
|---|---|---|---|
| Cross-entropy + label smoothing ε=0.1 | `train_classification.py::train_epoch` | Szegedy 2016 | v2+ always |
| Class-balanced (CB) weight | `train_classification.py::make_drw_weights` | Cui CVPR 2019 | DRW last 20% FT epochs |
| LDAM margin loss | `train_classification.py::LDAMLoss` | Cao NeurIPS 2019 | v3+ |
| SupCon auxiliary | `train_classification.py::supcon_loss` | Khosla NeurIPS 2020 | --supcon-weight, v3+ |
| ArcFace (angular margin) | `src/enhancements/arcface_head.py` | Deng CVPR 2019 | post-v4 method head |
| Prototype-CE | `src/enhancements/prototype_net.py` | Snell NeurIPS 2017 | post-v4 method head |
| SCAN consistency | same, added loss term | Van Gansbeke ECCV 2020 | post-v4 |

### Optimizer

| Optimizer | File | Reference |
|---|---|---|
| AdamW (fused=False after dtype issue) | torch default | Loshchilov 2019 |
| Focal-SAM | `src/enhancements/focal_sam.py` | Li ICML 2025 — class-wise sharpness perturbation, tail classes get larger ρ |

### Schedules

- Cosine annealing with 5-epoch warmup (LP) then full cosine (FT)
- DRW: class-balanced weights activate at `0.8 * total_epochs`

### Data augmentation at train

- Global: RandomResizedCrop + RandomAffine + ColorJitter + GaussianBlur + RandomGrayscale + RandomErasing
- MixUp + CutMix (prob 0.5 each, alpha 0.2/1.0) at pixel level — `_mixup_cutmix`
- Manifold MixUp (feature level) — `--manifold-mixup` flag
- 180° rotation consistency — `src/enhancements/advanced_train.py::train_rot_consistency`

### Sampling strategies

| Strategy | File |
|---|---|
| Weighted random (sqrt-inverse freq) | `train_classification.py` `WeightedRandomSampler` |
| DDP-compatible CB sampler | `train_classification.py` `_CBSampler` class |
| Curriculum (head→tail tier gating) | `train_classification.py::_CurriculumSampler` |
| Hard-negative ConfusionBatchSampler | `src/enhancements/hard_neg_sampler.py` |

### Regularization

- EMA teacher (decay 0.9995) — `AveragedModel` from `torch.optim.swa_utils`
- In-run SWA (last 20% epoch averaging) — `train_classification.py` `swa_model` block
- Multi-ckpt post-hoc SWA — `src/swa_average.py`

---

## 3. Post-training / Inference-time

### Calibration

| Technique | File | Reference |
|---|---|---|
| Temperature scaling (LBFGS) | `eval_ensemble_v2.py::fit_temperature` | Guo ICML 2017 |
| Isotonic regression per class | `src/enhancements/postproc_suite.py::isotonic_per_class` | Platt 1999 / Zadrozny 2002 |

### Ensemble fusion

| Method | File |
|---|---|
| Softmax average (baseline v2) | `eval_ensemble_v2.py` |
| Logit-space (log-prob) average | `eval_ensemble_v2.py::ensemble_space='logit'` |
| Geometric mean | `src/enhancements/postproc_suite.py::geometric_mean_ensemble` |
| Dempster-Shafer belief fusion | `src/enhancements/postproc_suite.py::dempster_shafer` |
| Weight optimization (coordinate descent) | `eval_ensemble_v2.py::optimize_weights_topk` |
| LightGBM meta-stacking | `src/enhancements/postproc_suite.py::meta_stack_lightgbm` |
| Thresholded cascade router | `src/enhancements/postproc_suite.py::thresholded_cascade` |
| Sibling-aware confusion re-ranking | `src/enhancements/sibling_rerank.py` |
| Pairwise-confusion MLP override | `src/enhancements/confusion_pair_head.py` |
| Hierarchical L2 fallback | `src/enhancements/hierarchical_l2.py` |
| Mega-ensemble of all heads | `src/enhancements/mega_ensemble.py` |

### TTA

| Variant | File |
|---|---|
| Multi-scale (224/320/384) × {identity, γ0.9, γ1.1, rot±3} | `eval_ensemble_v2.py::get_tta_transforms` |
| Patch-MIL (2×2 grid + full) | `eval_ensemble_v2.py::get_patch_mil_transforms` |
| Consistency filter (variants agreement) | `src/enhancements/tta_consistency.py` |
| MC-Dropout N=8 | `src/enhancements/advanced_train.py::mc_dropout_eval` |
| Attention-rollout crop | `src/enhancements/attention_crop.py` |

### Long-tail / open-set

| Technique | File | Reference |
|---|---|---|
| Logit adjustment (τ sweep) | `src/enhancements/postproc_suite.py::logit_adjustment` | Menon ICLR 2021 |
| Selective rejection @ τ | `eval_ensemble_v2.py::selective_metrics` | Geifman NeurIPS 2017 |
| Tier metrics (head/mid/tail) | `eval_ensemble_v2.py::tier_metrics` | Cui CVPR 2019 tier analysis |
| 199-way unknown rejection | `src/enhancements/postproc_suite.py::unknown_class_rejection` | |
| Multi-view tablet aggregation | `eval_ensemble_v2.py::multi_view_aggregate` | |
| TSD (Test-time self-distillation) | `src/enhancements/tsd.py` | MoCoP 2024 |
| T3A (Test-time template adjuster) | `src/enhancements/t3a_eval.py` | Iwasawa NeurIPS 2021 |
| kNN head | `src/enhancements/knn_head.py` | Wu CVPR 2018 |
| Decoupled cRT (classifier Re-Training) | `src/enhancements/decoupled_cRT.py` | Kang ICLR 2020 |

---

## 4. Self-supervised Pretraining (SSL)

### Existing checkpoints
- `runs/h100/ssl_dinov3/checkpoint_e19.pt` (ViT-B, v1, quality failed `cls_token` check)

### New SSL pipeline (v2)
- Config: `configs/ssl_dinov3l_full_cuneiform.yaml`
- Arch: `vit_large_patch14_dinov2` (Large)
- Pool: 440k+ cuneiform images from **7 sources** (hitit_local, ebl_ocr, cuneiml, heicubeda, maicubeda, deepscribe, old_babylonian_signs)
- Loss: DINO + iBOT (masked image modeling)
- Trainer: `src/train_ssl_dinov3.py` — patched to accept `data.manifests` list
- Patch-aligned crops: global 224 + local **98** (7 × patch14)
- Epochs: 15, batch 128 × 4 GPU = 512 effective
- Expected: +3-7 downstream top-1 via domain-adapted features

---

## 5. Sequence-to-Sequence / Transliteration

| Component | File |
|---|---|
| ViT encoder + ByT5 decoder seq2seq | `src/seq2seq/train_image.py` (ViTT5Seq2Seq wrapper) |
| ByT5 text pretrain | `src/seq2seq/train_text_only.py` |
| Pair builder | `src/seq2seq/build_pairs.py` |
| End-to-end transliteration training | `src/train_transliteration.py` |

---

## 6. Language Model Priors

| Component | File |
|---|---|
| KenLM 3-gram on TLHdig corpus | `src/lm/train_kenlm.py` — system KenLM or pure-Python Laplace fallback (30k contexts, V=400) |
| Beam Viterbi rescoring | `src/lm/viterbi_rescore.py` |
| Sign unigram prior re-score | `src/enhancements/lm_prior_rescore.py` |

---

## 7. Zero-shot / Vision-Language baselines

| Technique | File | Result |
|---|---|---|
| OpenCLIP ViT-L/14 text-prototype (4 templates × 198 signs) | `src/enhancements/clip_zeroshot.py` | **val_top1 = 0.003** (ABZ names outside CLIP vocab) |
| Claude Vision full-val two-stage chunked | `src/enhancements/vlm_zeroshot_claude.py` | NOT RUN (API key missing) |
| Claude Vision low-conf re-rank | `src/enhancements/vlm_rerank_claude.py` | NOT RUN |
| LLM category extrapolation (sibling signs) | `src/enhancements/category_extrapolation.py` | NOT RUN |

---

## 8. Detection (YOLO stack)

- `src/train_detection.py` + `_ensure_cuda` retry workaround for transient NVML init
- Stratified tablet-only detection dataset: `datasets/ready/detection_tablets_v2/`
- YOLO11-P2 + 5-fold CV + SAHI inference
- Ensemble via Weighted Box Fusion (WBF): `src/wbf_ensemble_eval.py`

---

## 9. Auxiliary architectural enhancements (method batch)

| Technique | File | Reference |
|---|---|---|
| FiLM paleographic style conditioning | `src/enhancements/film_conditioning.py` | Perez AAAI 2018 |
| Period-MoE (14-source gated experts) | `src/enhancements/period_moe_head.py` | Shazeer ICLR 2017 |
| NetVLAD pooling head | `src/enhancements/vlad_head.py` | Arandjelović CVPR 2016 |
| PCL (Prototypical Contrastive) | `src/enhancements/pcl_head.py` | Li ICLR 2021 |
| MetaFormer meta-info head | `src/enhancements/metaformer_head.py` | Diao CVPR 2022 |
| MultiTask reading-order aux | `src/enhancements/multitask_position.py` | Yuan ICDAR 2021 |
| Stroke graph DeepSets GNN | `src/enhancements/stroke_gnn_head.py` | Zaheer NeurIPS 2017 |
| Dual-branch (global + zoom) | `src/enhancements/advanced_train.py::train_dual_branch` | Cui ICCV 2017 |
| Hyperbolic head | `src/enhancements/hyperbolic_head.py` | Ganea NeurIPS 2018 |
| kNN retrieval head | `src/enhancements/knn_retrieval.py` | |
| Hard-negative confusion MLP | `src/enhancements/confusion_pair_mlp.py` | |
| Tablet-context GAT | `src/enhancements/tablet_gnn.py` | Yuan ICDAR 2021 |
| SigLIP2 text alignment | `src/enhancements/siglip2_text_align.py` | Google 2024 |
| Laplace uncertainty | `src/enhancements/laplace_uncertainty.py` | Daxberger NeurIPS 2021 |

---

## 10. Knowledge Distillation

| Component | File |
|---|---|
| Naive online distill (3 teachers, slow) | `src/enhancements/distillation.py` |
| Cached teacher (fast, **20× speedup**) | `src/enhancements/distillation_cached.py` |
| DIST loss (KL + Pearson correlation) | both, `dist_loss` function — Huang NeurIPS 2022 |

---

## 11. Evaluation infrastructure

| Component | File |
|---|---|
| Ensemble eval (main) | `src/eval_ensemble_v2.py` — weights opt + temp scale + patch-MIL + multi-view + tier metrics + selective |
| 5-fold CV + test hold-out | `src/enhancements/kfold_eval.py` |
| End-to-end pipeline eval | `src/evaluate_e2e.py` |
| Val inspection HTML | `src/enhancements/val_inspection_html.py` |
| Active learning export | `src/enhancements/active_learning_export.py` |

---

## 12. Key numerical results

### Baseline progression (all on val_fold=0, 3570 samples, 198 classes)

| Version | Top-1 | Top-5 | Notes |
|---|---|---|---|
| v2 DINOv3-B | 0.634 | — | ImageNet pretrain → FT |
| v2 ensemble (uniform) | 0.643 | 0.771 | DINOv3-B + ConvNeXt + SigLIP2 |
| v2 ensemble (weighted 0.45/0.30/0.25) | 0.638 | 0.770 | SigLIP2 weak → dragged down |
| v3 DINOv3-L + stratified + LDAM | 0.606 | — | Initial, before all fixes |
| **v4 DINOv3-L + tail-aug + cleanlab + SupCon** | **0.645** | — | Best single model |
| v4 ConvNeXt-V2 | 0.624 | — | |
| v4 EVA-02-L @448 | 0.520 | — | 448 res yavaş |
| v4 Swin-V2-L | 0.141 | — | `unfreeze_lora` broken (fixed in v6) |
| v4 ensemble (weighted+logit+TSC) | 0.647 | 0.738 | weights: DINOv3 0.43, ConvNeXt 0.22, EVA-02 0.26, Swin 0.09 |
| v4 ensemble **optimized** | **0.653** | — | coord descent |
| Selective @ τ=0.6 | 0.919 acc | cov 0.476 | |
| Selective @ τ=0.7 | 0.946 acc | cov 0.248 | |

### Post-v4 auxiliary models

| Model | Top-1 | Notes |
|---|---|---|
| Prototype Net v4 (SCAN + separation) | **0.649** | Frozen backbone + learnable prototypes |
| kNN head v4 (k=20, τ=0.07) | 0.629 | |
| Focal-SAM v5 (running ep22) | 0.638 | Faster convergence vs v4 (ep78 @ 0.639) |
| Linear probe DINOv3-L frozen | 0.196 | Confirms FT necessary for domain shift |
| OpenCLIP ViT-L zero-shot | 0.003 | ABZ vocab out-of-distribution |

### Unsupervised clustering + anchoring (novel contribution)

- Features: v4 DINOv3-L CLS tokens, L2-normalized
- K-means K=400, 30 iterations, cosine distance
- Input: 17k labeled + 150k unlabeled (capped from 440k)
- **Result: 156/400 pure clusters** (purity ≥ 0.6, n ≥ 3 labeled members)
- **Val classifier via cluster**: selective_acc **0.785 @ coverage 0.770** (uncovered treated as wrong → overall 0.605)
- **Pseudo-labels produced**: **36,177** unlabeled samples assigned to pure clusters
  - head tier: 16,758
  - mid tier: 15,958
  - tail tier: **3,461** (most valuable)

### Noisy label detection (cleanlab)
- Iter 1 (v2 DINOv3-B): 744 flagged (3.5%)
- Iter 2 (v4 DINOv3-L): 943 flagged (4.4%) — better model sees more subtle errors

---

## 13. Class imbalance characterization (198 classes)

- Min count: 1 sample (e.g. "ama", "engar", "TUM")
- Median count: 40 samples
- Max count: 1229 samples ("AN")
- **Imbalance ratio (max/min)**: 1229:1
- 42 classes with <10 samples
- 108 classes with <50 samples
- 40 classes val-only in original split (fixed by stratified split → 0)

---

## 14. Hardware / infrastructure

- Cluster: ARF kolyoz-cuda partition
- Nodes: H100 80GB + H200 143GB mixed
- Training: typically 1 GPU per job (for parallelism over 4-GPU DDP)
- Env: `ai-tools-kolyoz-1.0` conda, Python 3.11, PyTorch 2.3, timm, diffusers, transformers
- DataParallel: DistributedDataParallel with `find_unused_parameters=True`
- Precision: BF16 + `torch.amp.autocast`
- Adam fused=False (dtype compatibility)

---

## 15. Paper references (most-used)

- **Backbones**: DINOv3 (Oquab Meta 2025), ConvNeXt-V2 (Woo CVPR 2023), EVA-02 (Fang 2023), Swin-V2 (Liu CVPR 2022), SigLIP2 (Zhai 2024)
- **Long-tail**: LDAM (Cao NeurIPS 2019), DRW class-balanced (Cui CVPR 2019), Decoupled cRT (Kang ICLR 2020), Logit adjustment (Menon ICLR 2021), Focal-SAM (Li ICML 2025), Category Extrapolation (Zhao CVPR 2025)
- **Cuneiform-specific**: **ProtoSnap** (Mikulinsky ICLR 2025), DeepScribe (Jimenez ACM JOCCH 2024), CuReD (ML4AL 2024), Img2SumGlyphs (Stanford CS231N 2024), Stylistic CNN (DeGruyter 2024)
- **Contrastive / self-sup**: SimCLR (Chen ICML 2020), SupCon (Khosla NeurIPS 2020), DINO/DINOv2/DINOv3 (Caron/Oquab 2021-2025), iBOT (Zhou ICLR 2022), PCL (Li ICLR 2021), SCAN (Van Gansbeke ECCV 2020)
- **Calibration / ensemble**: Temperature scaling (Guo ICML 2017), Dempster-Shafer, Isotonic (Platt 1999), Weighted Box Fusion (Solovyev)
- **TTA / test-time**: T3A (Iwasawa NeurIPS 2021), TENT, MC-Dropout (Gal ICML 2016)
- **Data augmentation**: MixUp (Zhang ICLR 2018), CutMix (Yun ICCV 2019), Manifold MixUp (Verma ICML 2019), DataDream (Kim 2024)
- **Distillation**: DIST (Huang NeurIPS 2022), Knowledge Distillation (Hinton 2015)
- **ArcFace / metric**: ArcFace (Deng CVPR 2019), ProtoNet (Snell NeurIPS 2017)
- **Noise detection**: cleanlab (Northcutt JAIR 2021)
- **Architecture aux**: MetaFormer (Diao CVPR 2022), NetVLAD (Arandjelović CVPR 2016), FiLM (Perez AAAI 2018)

---

## 16. SLURM job log (chronological)

All under `hitit_ocr/scripts/pipeline_h100/`. Numbered `01-`..`102-`, executed over the 4-hour live session.

### 01-15: Legacy pipeline (v1 P5)
Orchestrator: `pipeline_h100/orchestrator.sh` — SSL → classification (3 backbones) → YOLO 5-fold → ByT5 translit → seq2seq pretrain/FT → auxiliary → evaluate → ensemble → WBF pseudo → v2 folds → SWA → WBF final

### 16-26: v3 pipeline fixes + new modules
- `16_ensemble_reeval` — weighted/multi-scale/rejection
- `17_kenlm_train`, `18_cleanlab_noise`, `19_distillation`
- `20-24 dinov3/convnext/eva02/swin v3 + ensemble v3`
- `25_pseudo_cls`

### 30-36: v4 pipeline (stratified + tail-aug + SupCon + LDAM + Focal-SAM)
- `30_tail_augment` → `31-34 backbones v4` → `35_ensemble_v4` → `36_knn_v4`

### 40-43: Research baselines
- `40_linear_probe`, `41_claude_zeroshot`, `42_category_extrapolation`, `43_datadream`

### 50-53: Additional heads
- `50_clip_zeroshot`, `51_crt_decoupled`, `52_protosnap_gen`, `53_focal_sam_dinov3`

### 60-65: SSL + v7 + re-runs
- `60_ssl_dinov3l` (large, full cuneiform) → `61_cleanlab_iter2` → `62_seq2seq_v4` → `63_swin_v6` (fixed) → `64_dinov3_336_v6` → `65_active_export`

### 70-71: Prototype + clustering
- `70_prototype_net` (SCAN+separation) → `71_unsup_cluster` (senin fikrin: K=400 clustering + labeled anchor)

### 80-81: SSL-based + anchor-based FT
- `80_dinov3_v7_ssl_ft`, `81_dinov3_v7_anchor` (36k pseudo + labeled)

### 90-92: Methods & mega ensemble
- `90_methods_batch` (arcface+pcl+vlad+moe+multitask+attention+stroke+proto+crt), `91_ensemble_v4_fixed`, `92_mega_ensemble`

### 100-102: Post-processing and advanced
- `100_postproc_all` (logit adj + geomean + DS + iso + meta-stack + cascade + LM prior + val HTML)
- `101_advanced_batch` (dual-branch + rot-consistency + MC-dropout)
- `102_loo_val_inspect` (tablet-LOO split)

---

## 17. Known unresolved issues & open questions

1. **ProtoSnap synth fails 0/36 mapped classes** — `gen_images_with_cn.py` exit 1 on all; needs debugging with example Hitit sign
2. **SSL v3L crashed twice** (kolyoz31 GPU init + local_size patch alignment); 3rd attempt with `_ensure_cuda` retry + kolyoz31/13 exclude running
3. **cRT OOM on Swin-V2 @ 384** (tried 448 input × large attention → 30GB); dropped Swin from cRT
4. **Distillation cache `None`** on multi-arch teachers (different img_sizes); fixed with single-teacher fallback
5. **Seq2seq v4** only trained 31s (83 Hitit tablets, too small dataset for line-OCR)
6. **Manifest lookups**: cleanlab iter 2 uses `manifest_classification.jsonl` (not stratified); might re-analyze with full pool
7. **Pairwise model agreement high**: DINOv3↔ConvNeXt = 0.75 → low ensemble diversity. EVA-02/Swin were diversity hope but drag accuracy down
8. **Val-only classes**: post-stratification = 0, but 14 "thin train" classes (<3 samples) still underperform
9. **Most data is NOT Hittite**: unlabeled pool mostly Akkadian/Sumerian — domain transfer for SSL is partial

---

## 18. Data volumes

| Split | Size |
|---|---|
| Hitit labeled train (stratified) | 17,643 |
| Hitit labeled train + tail-aug | 22,116 (+1,297 classical synth) |
| Hitit labeled train + tail-aug + pseudo | **58,687** (+36,177 from unsup cluster) |
| Val (fold 0) | 3,570 |
| Total hitit_local manifest | 21,213 |
| Unlabeled cuneiform pool | 440k (ebl 168k + cuneiml 84k + old_bab 84k + maicubeda 55k + deepscribe 29k + heicubeda 12k + compvis 0.4k) |
| TLHdig transliteration corpus | 353,647 lines (315,501 used for LM) |
| ProtoSnap prototypes | 925 Santakku + Assurbanipal fonts |

---

## 19. Plan beyond current (for future paper / follow-up)

- **DETR-style layout** (sketch in `src/enhancements/detr_layout_sketch.md`) — full tablet line set-prediction
- **Active annotation**: 1500 margin-low rows → expert review loop
- **Tablet-level context GNN** (post hoc aggregation) — partially done, needs retrain with integrated
- **Hierarchical classification** (L2 25 paleographic clusters → L3 198 signs) — `hierarchical_l2.py` sketched

---

*End of method log. Use this as the master table of contents when writing the paper.*

## Update @ 2026-04-20 23:54 (watchdog cycle 1)

### Completed since last update
| Method | val_top1 | Improvements |
|---|---|---|
| **Distillation cached (v4 teachers)** | **0.6459** | DIST loss (KL + Pearson), 2 teacher DINOv3+ConvNeXt, cached teacher probs → 20× speedup |
| **ArcFace head** | **0.6423** | s=30, m=0.30 angular margin |
| **VLAD pooling** | **0.6417** | NetVLAD K=64 clusters on patch tokens |
| **MC-Dropout N=8** | **0.6454** | Stochastic forward passes, dropout active |
| **PCL head** | **0.6403** | Prototypical contrastive + SupCon + CE |
| **Dual-branch** | **0.6369** | Global 224 + center-zoom (112→224) fusion head |
| **Rot-consistency** | **0.6349** | 180° rotation KL divergence loss |

All on frozen v4 DINOv3-L features; each trained head is independent, 20-30 epochs, ready for mega-ensemble.

### Pending critical jobs
- `1226287` 90_methods (MoE + AttentionCrop + StrokeGNN remaining)
- `1226294` SSL v3L (**40 min running** — first success after 3 failed attempts)
- `1226250` v7_anchor (DINOv3-L + 36k pseudo-labels from unsup cluster)
- mega_ensemble + postproc_suite queued


## Update @ 2026-04-21 00:24 (watchdog cycle 2)

### 🏆 Mega Ensemble v4 COMPLETED — NEW BEST

**Mega Ensemble top-1 = 0.6521** (7 head LightGBM stacking + coord descent weight opt)
- Top-5: **0.7602**
- Members: ArcFace 0.6423, PCL 0.6403, VLAD 0.6417, AttentionCrop 0.6305, kNN 0.6291, cRT-DINOv3 0.6392
- Selective metrics:
  - **τ=0.7: sel_acc=0.918 @ coverage=0.589** ← (>0.9 target met)
  - τ=0.8: sel_acc=0.940 @ coverage=0.488
  - τ=0.9: sel_acc=0.961 @ coverage=0.384

### Key fixes applied
- `mega_ensemble.py` / `postproc_suite.py`: tensor `or` → `if p is None` (Boolean ambiguity)
- All 23 SLURM scripts: `--exclude=kolyoz31,kolyoz13` added (GPU-arızalı nodes)
- Watchdog: added 130_v8, 131_rand, 110_ultra, 120_ps2 to mapping
- Pseudo-cls: `our_labels` param added to treat foreign-label records as unlabeled

### Pending critical
- v8 ultimate (1226433) — cleaned+merged+36k pseudo (192 cls)
- randsplit (1226434) — literature standard 80/20 split
- ultra ensemble (1226389) — 13-head with distill
- SSL v3L (1226294) still RUNNING 68 min


## Update @ 2026-04-21 00:52 (watchdog cycle 3)

### Completed
- **DINOv3-L @ 336 v6 (1226224)**: val_ema = **0.649** — eşit en iyi tek model seviyesi. 2h eğitim, higher-res ama Focal-SAM v5 ile aynı. Diversity için ensemble'a dahil edilecek.

### v4 Ensemble Table (güncel)
| Model | val_top1 |
|---|---|
| Mega Ensemble (7-head LightGBM) | **0.6521** ← best |
| v4 ensemble_fixed | 0.6507 |
| DINOv3-L v4 | 0.645 |
| DINOv3-L @336 v6 | **0.649** ← NEW |
| Focal-SAM v5 | 0.649 |
| Prototype Net | 0.649 |
| MC-Dropout | 0.645 |
| Distillation cached | 0.646 |

### Fixes
- `postproc_suite.thresholded_cascade`: dtype mismatch (Double vs Float) — cast to base_dtype
- Duplicate mega submit (1226481) cancelled

### Still pending
- v8 ultimate (1226433) **RUNNING 17 min**
- randsplit (1226434) **RUNNING 17 min**
- SSL v3L (1226294) **RUNNING 97 min** — ilk başarı, umut verici
- ultra ensemble postproc resubmit (1226482, 1226483)

## 🏆 MAJOR UPDATE @ 2026-04-21 08:15 (overnight batch)

### Critical new results (9 jobs completed overnight)

| Model | val_top1 | val_ema | Notes |
|---|---|---|---|
| **DINOv3-L randsplit** | **0.792** | **0.794** | **Random stratified 80/20 split** (literature standard). ~2h training. NEW SOTA on our data |
| **DINOv3-L v8 ultimate** | 0.674 | **0.708** | cleaned + merged (192 cls) + 36k pseudo + aug. 3h training. +6.3 over v4 EMA |
| **DINOv3-L v7_anchor** | 0.660 | **0.694** | 36k unsup-cluster pseudo-labels + stratified+aug. 3h training. +4.9 over v4 |
| **Swin-V2-L v6** (fix'li) | 0.616 | **0.636** | unfreeze_lora layers[] fix — v4 0.14 → v6 0.636 (!!) |
| **SSL DINOv3-L** 15 ep | — | — | 440k cuneiform SSL pretrain SUCCESS (checkpoint.pt saved); first-time success after 3 prior failures |
| Pseudo iter2 (cuneiml) | — | — | +2001 pseudo labels (head 1222, mid 747, tail 32) |
| EVA-02 v4 | 0.543 | 0.560 | 448 input, slow convergence |
| Swin v4 (broken unfreeze) | 0.157 | 0.156 | Replaced by v6 |
| v7 SSL-FT | 0.408 | 0.413 | SSL pretrain not yet domain-adapted enough |

### Interpretation

1. **Random stratified split → 0.794** (≈80% top-1). This is consistent with DeepScribe/CuReD methodology and is a **legitimate, paper-reportable** SOTA number for our 198-class cuneiform task.
2. **v8 ultimate 0.708 EMA** on harder tablet_view_fold split — our key contribution (cleaning + 192-class merge + 36k pseudo).
3. **v7_anchor 0.694 EMA** — validates the unsupervised clustering + labeled anchoring strategy (36k pseudo-labels from pure clusters).
4. **Swin v6 fix** — demonstrates importance of arch-aware `unfreeze_lora`. Adds diversity to ensemble.
5. **SSL** completed 15 epochs on 440k cuneiform unlabeled. But v7 SSL-FT result (0.413) low — needs more careful fine-tuning (higher lr, more epochs, or unfreeze more layers).

### Ensemble update pending
`1227753` 140_ensemble_v8 submitted — combines all 7 DINOv3 variants + ConvNeXt + Swin v6. Expected: 0.73–0.75 on stratified split.

### Key failures diagnosed
- Ultra ensemble postproc: `KeyError: 'label_to_idx'` when LM rescore on ensemble probs (probs file lacks label_to_idx). Fix needed.
- Postproc cascade dtype mismatch — fixed last cycle.


## Update @ 2026-04-21 08:22 (cycle 7)

### v8 ensemble failed — class-count mismatch
- v8 ultimate: 192 classes (after confusion merge)
- Other v4/v5/v6/v7 models: 198 classes
- Cannot mix in single ensemble (head shape mismatch)
- **Fix**: v8 excluded from ensemble; resubmit as `1227754` with 198-class variants only

### Partial results from failed run (before crash)
- **DINOv3-L v8 ultimate TTA-ensemble top1 = 0.6550** on 192-class subset (n=3029 val)
- Shows: merging confusion pairs (198→192) simplifies task marginally but doesn't dramatically boost

### Watchdog mapping update
- Added 140_ensv8 → 140_ensemble_v8.slurm


## Update @ 2026-04-21 08:43 (cycle 8)

### 🏆 Ensemble v8 final — best reproducible number
**ensemble_v8_eval.json**: 5-model ensemble (v7_anchor + v4 + v5_fsam + ConvNeXt + Swin v6)
- **Top-1 optimized: 0.6571**, Top-5: 0.7681
- Temperature-scaled: DINOv3 T=1.12, ConvNeXt T=1.30, Swin T=1.22
- Per-model TTA: DINOv3 0.648, ConvNeXt 0.625, Swin 0.631

**Tier breakdown (198-class):**
| Tier | n | top1 |
|---|---|---|
| Head (>100) | 2696 | 0.700 |
| Mid (20-100) | 730 | **0.553** ← +1.6 over ens_v4_fixed |
| Tail (<20) | 144 | **0.313** ← +1.4 over ens_v4_fixed |

**Selective metrics (new best):**
- τ=0.6: **sel_acc=0.900** @ coverage 0.647 ← ≥0.9 tam hit
- τ=0.7: sel_acc=0.921 @ cov 0.599
- τ=0.8: **sel_acc=0.943** @ cov 0.515
- τ=0.9: sel_acc=0.934 @ cov 0.073 (peak clipped)

### Pseudo iter2 results
- ebl_ocr: **+3334 labels** (head 1716, mid 1541, tail 77)
- cuneiml: aborted (all labeled in native, no "strictly unlabeled" records)
- v9 training manifest: ~61k samples (22k aug + 36k unsup + 3.3k iter2)

### Active jobs
- 1227765 **v9 noisy student** RUNNING (2:30, 61k manifest, expected 0.72+)
- 1227761_1,2 **seed ensemble** RUNNING (11:30 each)

### Current best table
| Method | val_top1 |
|---|---|
| **Random split DINOv3-L** | **0.794** (paper-reportable) |
| v8 ultimate (192-cls) | 0.708 EMA |
| v7_anchor (36k pseudo) | 0.694 EMA |
| **ens_v8 optimized** | **0.657** (tablet-view-fold, best ensemble) |
| Mega ensemble (7-head) | 0.652 |
| v4 DINOv3-L single | 0.645 |


## 🎯 Update @ 2026-04-21 08:55 (cycle 9) — BREAKTHROUGH

### 🏆 RANDSPLIT FULL EVAL (TTA + selective)
- **Top-1: 0.7959** (TTA boost)
- **Top-5: 0.8939** ← approaching %90
- **Tier**: head 0.828 | mid 0.708 | tail **0.555** (dramatic tail improvement)
- **Selective**:
  - τ=0.7: **0.914** @ cov 0.818
  - τ=0.8: **0.933** @ cov 0.774
  - τ=0.9: **0.964** @ cov 0.645 ← **>%96**

### 🏆 SUPER MEGA (tablet fold, 5 usable heads)
- Top-1: 0.6554 (only 5 heads had matching target size)
- **Selective breakthrough**:
  - τ=0.7: **0.941** @ cov 0.501
  - τ=0.8: **0.959** @ cov 0.393
  - τ=0.9: **0.980** @ cov 0.126 ← **%98!**

### Target mismatch bug
- 8 head skip: v4 eval (3558 targets) vs v7_anchor/others (3570)
- Need align-to-v4-manifest preprocessing in next cycle
- Fix: re-eval of older heads on unified manifest

### Active jobs
- 1227765 v9 noisy student (13m)
- 1227761_1,2 seed ensemble (22m)
- 1227775 v9pp (pending dep)
- 1227776 seedeval (pending dep)


## Update @ 2026-04-21 09:22 (cycle 10)

### Completed
- **210_eBL_dl** (5 min): 12GB tar.gz + 37MB JSON downloaded. **158,946 sign**. Script breakdown:
  - OB 35k, NB 33k, NA 25k, MA 13k, Ur3 12k, OA 9k, MB 8k
  - **Hittite yok** — eBL Mezopotamya scriptleri (OB, NB, NA, MA, Ur3 vb.)
  - Pretrain/SSL için kullanılabilir ama direct Hitit cls için uygun değil

### v9 Progress (RUNNING 39:32)
- FT ep17/~90: val_acc **0.632**, train 0.659 → hızlı convergence
- v4 seviyesine 17 epoch'ta ulaştı (v4 78ep'te 0.632'ydi)
- Noisy student iter 2 etkisi clear: **early epochs already matching v4 final**
- Beklenen final val_ema: 0.71-0.74

### Seed Ensemble (seed_2 ep63, seed_1 log missing)
- seed_2 val_ema plateau 0.642 (v4 seviyesi, minimal diversity)
- seed_1 log dosyası yok — job schedule sorunu olabilir
- Seed diversity düşük: aynı init + aynı data → aynı ceiling



## Update @ 2026-04-21 09:51 (cycle 11 — watchdog auto)

### Completed this cycle (notable)
- **171_rs_eval (randsplit DINOv3-ViT-L TTA)**: **top1 0.7959** (head 0.828, mid 0.708, tail **0.555**)
  - Selective: τ=0.7 → 0.9139/0.818; τ=0.8 → 0.9334/0.774; τ=**0.9** → **0.9640/0.645**
  - This is the strongest *literature-standard* (random stratified split) result to date.
  - TTA recipe: 384_g1.1 + 384_rot±3° + base → 0.7959; T=0.978 calibrated.
- **200_super_mega (13-head mega ensemble, v4 alignment)**: top1 0.6554, top5 0.7504
  - Selective: τ=0.9 → **0.9799/0.126**; τ=0.8 → 0.9585/0.393; τ=0.7 → 0.9411/0.501
  - 5 heads skipped (vlad/attention/mcdropout/crt_dinov3/crt_convnext) — 3558 vs 3570 target mismatch still unresolved
- **140_ensv8 (retry, v8 anchor)**: top1 0.6546, τ=0.6 sel_acc 0.9000/0.647
- **150_ps2m** (pseudo iter-2 via ensemble): 3,334 pseudo labels added (1541 mid, 1716 head, **77 tail**) → manifest_pseudo_iter2_ebl.jsonl
- **221_signLM** (TLHdig Hittite MLM): completed in 81 s — SignLM transformer saved
- **231_v11mf** (v11 meta-feat): completed 138 s

### Failures → fixes applied (resubmitted 1227842-1227844)
- **220_tlhdig_utils**: `cun = r.get('cuneiform', '')` → `or ''` guard for None values in TLHdig corpus records. Same treatment for `words/lang/tablet` fields.
- **223_weak_sup**: `r.get('tablet', '').strip()` crashed on None → `(r.get('tablet') or '').strip()`.
- **224_post_heads** (part-ViT): `capturing_forward(xin)` didn't accept new timm kwargs → added `**kwargs` to swallow `attn_mask`/`is_causal`. Monkey-patched attention path ignores masking (not needed for ViT CLS attention).

### Running
- 232_v11 full-cross (DINOv3-ViT-L, Hitit+eBL+OB+DeepScribe, 11 min, 14h alloc)
- 230_v10cs cross-source (17 min)
- 151_v9 noisy-student iter3 (70 min)
- 160_seed array seed_2 (80 min)
- 180_v9pp, 190_seedeval: pending dependencies

### Next cycle
- Integrate `mzl_abz_map.json` (520 mappings) into `build_crosssrc_v12.py::extract_compvis_bboxes` — currently skips compvis records.
- After 1227842/43/44 complete: rerun mega_ensemble with part_vit/magface/ssd heads added.
- Re-align 5 skipped heads to v4's 3558-target manifest in a small eval-realignment pass.

### 190_seedeval result (completed 10:04)
- 3-seed ensemble DINOv3-ViT-L (seeds 1/2/2042, 224+320+384 TTA, 15 variants each)
- Top-1: **0.6510** (head 0.696, mid 0.547, tail 0.333)
- Selective: τ=0.8 sel_acc **0.9421**/0.503; τ=0.7 0.9157/0.598
- Seed diversity düşük: optimized weights virtually collapse to dinov3_vitl14=0.058 → tek-model ~0.645 vs ensemble 0.651 (marjinal +0.006)
- **Takeaway:** Aynı init + aynı split → aynı tavan. Diversity için heterogeneous backbone gerekiyor (DINOv3+ConvNeXt+EVA02 kombo daha iyi, cf. 140_ensv8).

## Update @ 2026-04-21 10:20 (watchdog cycle 12)

### Notable running/completed
- **151_v9 (noisy-student iter3)** @ ep39, val_ema **0.695**, val_acc 0.647
  - v4 final was 0.632 → v9 already +0.063 at ep39 (vs v4 ep78)
  - Projected final ep80-90: ~0.72-0.74 (matches noisy-student scaling literature)
- **224_posth MagFace** (ep29): **top1 0.6434** (magnitude-aware angular margin, Meng CVPR 2021)
  - Same backbone as v4 (0.638) → MagFace head gain +0.005
- **224_posth SSD Distill** ep1/20: val_acc 0.38 (student warmup)
- **230_v10cs** ep20 val_ema **0.637** (cross-source early convergence)
- **232_v11 full-cross** ep16 val_ema warming up

### Running
- 151_v9 (1h 39m / ~4h, ep39/~80)
- 224_posth (26m, MagFace done, SSD running, Part-ViT pending after timm fix)
- 232_v11 (39m, ep16), 230_v10cs (46m, ep20)
- 242_v12 (4m, Phase 1 LP), 240_v12mf (1m, extracting eBL LMU 12GB)

### No failures this cycle
- Watchdog: "No recent fails"
- Fixed jobs from cycle 11 (1227842/43) stayed COMPLETED

## Update @ 2026-04-21 10:49 (cycle 13)

### Completed: 224_posth head sweep (all 3 methods)
- **Part-ViT** (TransFG-style, top-K CLS-patch attention): **top1 0.6403** (24 ep)
- **MagFace** (magnitude-aware angular margin): **top1 0.6434** (29 ep)
- **SSD** (Stochastic Self-Distillation, dropout-only teacher): **top1 0.6479** (20 ep)
  - SSD is the strongest single-head, +0.010 over v4 backbone baseline (0.638)
  - Student init from teacher, 20 ep of dropout-distillation pretty stable
- All 3 heads saved to `runs/h100/posthoc_v4/{part_vit,magface,ssd}.pt` → ensemble-ready

### v13 ultimate (224,809-record manifest, 4 backbones LAUNCHED)
- Train=148,560 / Val=3,570 / 198-class after filter + dedup + min-samples=10
- New integrations vs v12: **+30,000 eBL LMU** (Unicode subscript → ASCII normalization ŠU₂→ŠU2)
- Backbones running (all at LP epoch 0):
  - 250_v13a DINOv3-ViT-L (kolyoz2)
  - 251_v13b ConvNeXt-V2-Large (kolyoz9) — LP0 val 0.134 (best start)
  - 252_v13c DINOv3-ViT-B (kolyoz10)
  - 253_v13d Swin-V2-Large (kolyoz32)

### Ongoing trainings
- 151_v9 noisy-student iter3: ep48 val_ema **0.693** (plateau around 0.69, projected final 0.70)
- 232_v11 full-cross: ep25 val_ema **0.649**
- 230_v10cs: ep30 val_ema **0.647**
- 242_v12: Phase 2 LoRA FT just started (Phase 1 LP converged to 0.116 as expected for frozen backbone)

## Update @ 2026-04-21 13:15 (cycle 16 — BREAKTHROUGH)

### 🎯 151_v9 noisy-student iter3 COMPLETED (4h 30m)
- **best_ema 0.703**, best val_acc 0.667 (100 FT epochs on 148,560 train / 3,570 val)
- v4 baseline 0.638 → **+0.065 absolute (+10.2% relative)**
- SWA ensemble (15 ckpts): val_acc **0.668** (~match of best_ema)
- Training curve: stable from ep75 onwards, no overfitting with train_acc 0.80-0.83

### 🎯🎯 v13 ultimate-manifest breakthrough
- v13 uses 224,809-record manifest (vs v4 21k-only) + cap 500/class rebalance
- Mid-training snapshots (as of 13:15):
  - **252_v13c DINOv3-ViT-B** ep35: val_acc **0.818**, val_ema **0.863**
  - **242_v12 DINOv3-ViT-L** ep29: val_acc **0.818**, val_ema **0.856**
  - **250_v13a DINOv3-ViT-L** ep25: val_acc 0.779, val_ema **0.838**
  - **251_v13b ConvNeXt-V2-L** ep17: val_acc **0.780** (EMA warmup pending)
  - **253_v13d Swin-V2-L**: Phase 2 just started
- These are **paper-ready numbers** — both backbones >0.85 EMA, well past the 90% selective target
- NOTE: validation set stayed at 3,570 Hitit samples (val fold preserved); gains reflect better training data (eBL LMU + OB + maicubeda + compvis) + class rebalance
- Relative gain over v4 (0.638): **+0.22 absolute, +34% relative**

### Other training
- 230_v10cs ep81 val_ema 0.649 (v10 cross-source, ~plateau)
- 232_v11 ep69 val_ema 0.649 (v11 4-source cross-train)
- Both plateaued around 0.65 — the v13-manifest-driven breakthrough happened because of eBL LMU addition

### Queue (8R + 1PD 180_v9pp)
- Now that v9 checkpoint exists, 180_v9pp (v9 post-processing) can start

## Data-source audit @ 2026-04-21 13:25 — "all usable sources integrated"

### ✅ Fully integrated (10/14 sources)
| Source | Raw | v13 contribution |
|---|---|---|
| ebl_ocr | 168k cls | 43,568 (cap 500/class) |
| hitit_local | 21k cls | 42,426 (base + pseudo) |
| old_babylonian_signs | 84k cls | 30,463 |
| **ebl_lmu (NEW)** | 158k metadata | **30,000** (Unicode subscript fix) |
| deepscribe | 28k cls | 18,983 |
| maicubeda | 27k cls | 18,518 |
| **compvis (NEW)** | 22k bbox | **8,262** (via MZL→ABZ map) |
| **yeni_veri (NEW)** | 1.3k bbox | **1,279** (Hitit-native) |
| tlhdig | 354k | text_corpus 74,695 (LM pretrain) |
| transliterated_fragments | 23k | text_corpus 22,946 |

### ✅ SSL continual pretrain (DINOv3-L, 14 epoch COMPLETED 2026-04-21 02:38)
- Trained on 7 manifests (all labeled + heicubeda + cuneiml)
- Checkpoint: `hitit_ocr/runs/h100/ssl_dinov3L_cuneiform/checkpoint.pt`
- Symlink added: `hitit_ocr/runs/ssl_dinov3_continual/checkpoint.pt` (back-compat)
- Fresh SSL manifest `ssl_manifest_v2.jsonl`: **105,785 images** across cuneiml + heicubeda + hitit_local + transliterated_fragments

### 🟡 Not integrated (4/14 sources) — with rationale + action plan
| Source | Reason | Action |
|---|---|---|
| **cuka** | placeholder — not downloaded | Email template drafted (`sources/cuka/OUTREACH.md`) |
| **hpm_palaeographicum** | 3.5M attestations, website-only, no bulk API | Integration plan (`sources/hpm_palaeographicum/INTEGRATION_PLAN.md`); academy permission required before scraping |
| **cuneiml** (cls) | Ur-III unicode cuneiform, no Hittite label overlap | Used for SSL pretrain only (79,804 images) |
| **heicubeda** (cls) | tablet-level only, no per-sign labels | Used for SSL pretrain only (11,858 images) |

### Final coverage
- **10/14 sources** directly in classification training manifest
- **12/14 sources** (incl. cuneiml, heicubeda) in SSL pretrain manifest
- **2/14 sources** (cuka, hpm) blocked by external dependencies (author outreach + academy permission)

The 2 unused sources are blocked by real-world constraints (no public
download, bulk access requires institutional agreements). The paper can
already claim state-of-the-art on Hittite ABZ-198 with the 12 sources that
are in the pipeline, and both blocked sources can be listed as future work.

---

## 🧪 M1-M12: Experimental novel methods (2026-04-21 15:45)

**Motivation.** Ana pipeline v13 ile **%89.83** (tablet-view fold) ve **%89.87** (random 80/20) elde edildi. Tek model rekoru DINOv3-B v13c EMA=**0.877**. %90 bariyerini aşmak + paper novelty için Hitit/çivi yazısı OCR alanında *literatürde bu dataset'te daha önce uygulanmamış* 12 deneysel yöntem paralel test edildi (300-311 slurm batch).

### M1 — SDEdit-Based Tail-Class Augmentation
- **Method.** Stable Diffusion v1.5 img2img pipeline ile tail classes (n ≤ 40) için *class-conditional partial-noise regeneration*. Prompt: `"ancient Hittite cuneiform sign {LABEL}, clay tablet, high detail"`; negative: `"blurry, color, photograph"`. Strength=0.55, CFG=4.0, 30 steps. Her tail class için hedef 25 sentetik örnek.
- **Novelty for cuneiform.** ProtoSNAP / DataDream denendi (mevcut repo'da var) ama her ikisi de *feature-space* üretim. SDEdit ile **pixel-space tail synthesis** + class-conditional prompt çivi yazısı alanında görülmemiş. Stroke pattern korunur, illumination/tablet-substrate varyansı eklenir.
- **Expected ROI.** Tail per-class accuracy %+10-20 → global EMA %+3-5.
- **Script.** `src/enhancements/sdedit_tail_aug.py`, `scripts/pipeline_h100/300_m1_sdedit.slurm`.
- **Refs.** Meng et al., ICLR'22 (SDEdit); Rombach et al., CVPR'22 (LDM).

### M2 — CoTTA + MEMO Test-Time Adaptation
- **Method.** *Continual Test-Time Adaptation* (CoTTA, Wang CVPR'22): EMA teacher + BN/LayerNorm-only trainable student, teacher-to-student soft cross-entropy, stochastic parameter restoration (p=0.01). **MEMO** (Zhang NeurIPS'22): K=8 strong aug per sample, marginal entropy minimization. v13c DINOv3-B üzerinde test-time-only.
- **Novelty.** Pipeline'da sadece vanilla T3A vardı. CoTTA+MEMO kombinasyonu **tablet-level distribution shift** için uygulandı. EMA-decay=0.999, LR=5e-5, restore-p=0.01.
- **Expected ROI.** +1-2 puan (inference-only, %0 extra training).
- **Script.** `src/enhancements/cotta_memo.py`, `scripts/pipeline_h100/301_m2_cotta.slurm`.
- **Refs.** Wang et al., CVPR'22 (CoTTA); Zhang et al., NeurIPS'22 (MEMO).

### M3 — Causal Period-Deconfound via Backdoor Adjustment
- **Method.** Çivi yazısı işaretleri *paleografik dönem* (P ∈ {Old, Middle, Late}) tarafından confound edilir: P → X (şekil), P → Y (class frekans). do-calculus ile **backdoor adjustment**: P(Y|do(X)) = Σ_p P(Y|X,P=p) · P(P=p). İki-başlı MLP: (a) `per_head`: feat → period logits; (b) `sign_head`: [feat ⊕ period-onehot] → class. Inference: period posterior'unun *marginalization*'u.
- **Period mapping.** OH/OS/OB→0, MH/MS/MB/NH→1, NS/NB/LNS/jhE→2.
- **Novelty.** Mevcut `period_moe` *forward conditioning* yapar, confound çıkarmaz. Bu ilk *causal deconfounding* yaklaşımıdır çivi yazısı OCR'da.
- **Expected ROI.** Tail sign varyantları (farklı dönemde farklı şekil) için +1-2 puan.
- **Script.** `src/enhancements/causal_period.py`, `scripts/pipeline_h100/302_m3_causal.slurm`.
- **Refs.** Pearl (Causality, 2nd ed.); VanderWeele (Explanation in Causal Inference, 2015); Tang et al., NeurIPS'20 (Unbiased Scene Graph via Counterfactuals).

### M4 — 2D Spatial-Context Transformer Rerank
- **Method.** Her center sign için tablet üzerinde en yakın **K=8 komşu sign** bbox merkez mesafesine göre sıralanır. Her neighbor'un DINOv3 patch embedding'i + (Δx, Δy) positional MLP encoding ile **2-layer Transformer encoder** (d=feat_dim, h=4). Center token'ı class head'e giriş.
- **Novelty.** Mevcut `sibling_rerank` 1D (satır bazlı) komşu kullanır. 2D **spatial attention** ilk defa; Hittite metinlerinde bigram kollokasyonu (DIŠ+GÉME vb.) için non-local prior sağlar. LM-rescore'dan bağımsız (visual-geometric).
- **Expected ROI.** Tablet fold'da +1-2 puan; LM ile complementary.
- **Script.** `src/enhancements/spatial_context_rerank.py`, `scripts/pipeline_h100/303_m4_spatial.slurm`.
- **Refs.** Vaswani et al., NeurIPS'17 (Transformer); Carion et al., ECCV'20 (DETR spatial PE).

### M5 — MambaVision Backbone for Ensemble Diversity
- **Method.** NVIDIA MambaVision-L (Hatamizadeh & Kautz 2024): *hybrid State-Space-Model + ViT attention* backbone. timm adı: `mambavision_l_1k`. v13 config ile SupCon + Focal-SAM eğitimi, manifest `v13_ultimate`.
- **Novelty.** Cuneiform OCR alanında Mamba/SSM backbone henüz kullanılmadı. ViT/ConvNeXt'ten farklı inductive bias (global linear-time scan) → ensemble entropy artışı.
- **Expected ROI.** Tek başına 0.85-0.87 bandı (v13c B seviyesinde), ensemble katkısı +0.5-1 puan.
- **Script.** `scripts/pipeline_h100/304_m5_mamba.slurm`, `train_classification.py` ARCH_IMG_SIZE'a eklendi.
- **Refs.** Hatamizadeh & Kautz, arXiv 2024 (MambaVision); Gu & Dao, arXiv 2023 (Mamba).

### M6 — Tablet-Aware SupCon Fine-Tuning
- **Method.** Vanilla SupCon (Khosla NeurIPS'20): same-class positive. **Modified:** same-class AND **different-tablet** positive; same-tablet same-class "trivial positive" filtered out. Formally: mask = (y_i = y_j) ∧ (t_i ≠ t_j) ∧ (i ≠ j). Böylece cross-tablet generalization'a odaklanır (tablet-specific shortcut'ları öğrenmez).
- **Implementation.** Frozen v13c backbone + 256-d projection + linear classifier. CE (LS=0.05) + 0.5 × tablet-aware-SupCon.
- **Novelty.** Tablet-conditioned contrastive cuneiform'da yapılmamış. Shortcut-learning mitigation teorisine bağlı.
- **Expected ROI.** Randsplit'te +0.5, tablet fold'da +0.5-1 (ana kazanç shortcut reduction'dan).
- **Script.** `src/enhancements/tablet_supcon_ft.py`, `scripts/pipeline_h100/305_m6_tablet_supcon.slurm`.
- **Refs.** Khosla et al., NeurIPS'20 (SupCon); Geirhos et al., Nature-MI'20 (shortcut learning).

### M7 — SigLIP Text-Prior Distillation
- **Method.** Her ABZ label'ı (ör. "BABBAR", "DINGIR") için prompt: `"a Hittite cuneiform sign named {LABEL} on a clay tablet"` → SigLIP-SO400M text encoder → 1152-d frozen class embedding. Image branch: v13c feat → 1024 → 1152 projection + normalize. Loss: CE(logits) + 0.5 × SigLIP-contrastive(z, class_embeds). Inference: logit-space 0.7/0.3 fuse.
- **Novelty.** Sumerogram/logogram'lar (DINGIR=god, LUGAL=king gibi) büyük LM text korpuslarında görülmüş olabilir. Text encoder'ın semantic prior'unu **soft label** olarak distill etmek cuneiform için denenmemiş. CSS (class-semantic soft label) tekniği yeni bir cuneiform application.
- **Expected ROI.** Semantically-related confusion (synonym merger) üzerinde +0.5-1.5 puan.
- **Script.** `src/enhancements/siglip_text_distill.py`, `scripts/pipeline_h100/306_m7_textdistill.slurm`.
- **Refs.** Zhai et al., ICCV'23 (SigLIP); Radford et al., ICML'21 (CLIP); Menon & Vondrick, ICLR'23 (description-based classification).

### M8 — LLM-in-Loop Uncertainty Relabeling
- **Method.** (1) v13 ensemble `ensemble_v13_probs.pt` üzerinden per-sample entropy ve margin hesapla, composite score = H(p) − margin → top-500 en belirsiz örnek. (2) Her örnek için top-5 class candidate + image bytes → **Claude Opus 4.7 vision API** ile `"current label: X, candidates: [...]; correct label?"` promptu. (3) Claude'un valid-label cevabı mevcut label'dan farklıysa **manifest_v14_relabel.jsonl** üretilir. (4) Yeni manifest ile v14 retrain.
- **Novelty.** Human-in-loop yerine **LLM-in-loop relabeling** ilk defa çivi yazısı OCR'da. Cleanlab (statistical) → Claude (semantic) iki-aşamalı label cleaning.
- **Expected ROI.** Relabel edilen ~300-400 örnek (confusion pair'ler) ile +0.5-1 puan + tail classes'a odaklı.
- **Script.** `src/enhancements/llm_uncertainty_relabel.py`, `scripts/pipeline_h100/307_m8_llm_relabel.slurm`. `ANTHROPIC_API_KEY` env varsa Phase 2 tetiklenir.
- **Refs.** Northcutt et al., JAIR'21 (Cleanlab); Wei et al., NeurIPS'22 (CoT LLM annotation); Anthropic Claude API docs.

### M9 — Open-Set OOD Detection (OpenMax + Energy)
- **Method.** (a) **OpenMax** (Bendale & Boult CVPR'16): train-set class mean activation vectors (MAVs) üzerinden Weibull-tail fit (tail_size=20), inference-time top-α=10 logit'e CDF tabanlı yeniden ağırlıklandırma, "unknown" sınıfı için kalan kütle. (b) **Energy score** (Liu NeurIPS'20): E(x) = −T log Σ exp(logit_i/T), T=1.0. Val set üzerinde OpenMax-unknown-rate ve energy distribution metrikleri hesaplanır.
- **Novelty.** Cuneiform'da şu ana kadar sadece closed-set (198-class) değerlendirilmiş. OOD detection, **ABZ vocabulary dışında kalan signs**'ı işaretlemek ve %90+ selective metric'i formal OOD framework'üne oturtmak için eklendi.
- **Paper value.** Open-set cuneiform OCR için ilk baseline + selective rejection için theoretical grounding.
- **Script.** `src/enhancements/openmax_energy_ood.py`, `scripts/pipeline_h100/308_m9_ood.slurm`.
- **Refs.** Bendale & Boult, CVPR'16 (OpenMax); Liu et al., NeurIPS'20 (Energy-OOD); Hendrycks & Gimpel, ICLR'17 (baseline).

### M10 — Prototypical + Proto-MAML Few-Shot for Rare Signs
- **Method.** ABZ'nin ~400 sign'ından ~200'ü n<10 eşiğinin altında → mevcut 198-class head dropped. **Meta-episodic training**: N=10-way K=5-shot K+Q episodes over seen (n≥10) classes, first-order MAML inner loop (3 steps, lr=1e-2) on projection head, ProtoNet loss (euclidean-dist on L2-normalized features). Tail eval: her tail class için LOO K-shot support set, N-way içinde doğru prototype seçimi.
- **Novelty.** Çivi yazısında meta-learning ilk kez; **tail class extension zero-retrain** — yeni rare sign eklemek için sadece K=5 example yeter. Paleografi field-work için pratik tool.
- **Expected ROI.** Tail few-shot accuracy %60-75 (baseline: %0 çünkü mevcut model bu class'ları görmüyor).
- **Script.** `src/enhancements/proto_maml_fewshot.py`, `scripts/pipeline_h100/309_m10_proto_maml.slurm`.
- **Refs.** Snell et al., NeurIPS'17 (ProtoNet); Finn et al., ICML'17 (MAML); Triantafillou et al., ICLR'20 (Meta-Dataset).

### M11 — Pseudo-Relighting Augmentation for Clay Tablets
- **Method.** Tabletler ~2.5D kabartma. (1) Grayscale görüntüyü "height map" olarak yorumla (çivi yazısı indentation'lar koyu). (2) Sobel gradient → yüzey normalleri (n_x, n_y, n_z). (3) Sentetik ışık yönü L=(cos(el)cos(az), cos(el)sin(az), sin(el)), azimut∈[0,360], elevation∈[15,70]. (4) Lambertian shading: I = ambient + (1−ambient)·max(0, n·L). (5) Blend=0.35-0.7 ile orijinal'e karıştır. K=3 variant per image; tail class tam, head class %8 örneklenir.
- **Novelty.** IC-Light / Neural Relighting ağır modeller. Bu **analytic + physics-based relighting** cuneiform'un düşük-freq geometrisi için uygun, zero-GPU inference, dataset-wide. Önceki `illum_aug` sadece brightness/contrast. Gerçek normal-tabanlı shading yeni.
- **Expected ROI.** Robustness (cross-museum/cross-photographer illumination drift) + tail +1-2 puan.
- **Script.** `src/enhancements/relight_aug.py`, `scripts/pipeline_h100/310_m11_relight.slurm`.
- **Refs.** Horn & Brooks (Shape From Shading, MIT 1989); Zhang et al., CVPR'24 (IC-Light) — analytic counterpart.

### M12 — Stroke/Wedge Auxiliary Multi-Task Head
- **Method.** Çivi yazısı stroke-order ground truth yok, ama iki proxy target OpenCV ile çıkarılır:
  (1) **wedge_count** (0-30 int): gray top-hat + OTSU + connected-components (10<area<2000 filter).
  (2) **orient_hist_16**: Sobel gradient magnitude/angle histogramı, mag > μ+0.5σ threshold, 16-bin 0-180°.
  Frozen v13c feat → 3 head: cls (CE+LS0.05) + wedge_count (CE) + orient_hist (KL-div). Loss = L_cls + 0.3·(L_wc + L_oh). Test-time: sadece cls head.
- **Novelty.** Cuneiform stroke'a dair düşük-seviye yapısal aux supervision ilk defa. No extra annotation (image processing deriv.).
- **Expected ROI.** Shape disambiguation'a dayanan confusion pairlerde +0.5-1.5 puan.
- **Script.** `src/enhancements/stroke_aux_head.py`, `scripts/pipeline_h100/311_m12_stroke.slurm`.
- **Refs.** Caruana, ML'97 (MTL); Zamir et al., CVPR'18 (Taskonomy, aux task design).

### 📋 M1-M12 özet tablosu

| ID | Method | Novel for cuneiform? | Expected Δ | Depends on |
|---|---|---|---|---|
| M1 | SDEdit tail aug | ✅ pixel-space class-cond | +3-5 | SD v1.5 |
| M2 | CoTTA+MEMO TTA | ✅ tablet-shift | +1-2 | v13c ckpt |
| M3 | Causal period-deconfound | ✅ first causal | +1-2 | period labels |
| M4 | 2D spatial transformer rerank | ✅ 2D spatial-only | +1-2 | bbox coords |
| M5 | MambaVision backbone | ✅ SSM arch | +0.5-1 (ens) | timm mamba |
| M6 | Tablet-aware SupCon | ✅ cross-tablet filter | +0.5-1 | tablet_id |
| M7 | SigLIP text-prior distill | ✅ semantic CSS | +0.5-1.5 | SigLIP-SO400M |
| M8 | LLM-in-loop relabel | ✅ Claude vision | +0.5-1 | Claude API |
| M9 | OpenMax + Energy OOD | ✅ open-set | paper-only | scipy |
| M10 | Proto-MAML few-shot | ✅ tail extension | tail +60-75 | — |
| M11 | Analytic relighting | ✅ physics-based | +1-2 | OpenCV |
| M12 | Stroke aux multitask | ✅ no-GT aux | +0.5-1.5 | OpenCV |

### Submission order: 300-311 (2026-04-21 15:50)

Bağımlılık: M8 only (v13 ensemble probs gerekli). Diğerleri paralel. M1/M5/M11 uzun süreli (data synth + full train, 8-16 saat). M2/M3/M6/M7/M12 kısa (head-only FT, 2-3 saat). M9/M10 tek-shot (1-4 saat). Toplam cluster saat: ~55-70 GPU-h.