Instructions to use kesha-humonen/runic-reader-2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use kesha-humonen/runic-reader-2 with PEFT:
Task type is invalid.
- Notebooks
- Google Colab
- Kaggle
Add files using upload-large-folder tool
Browse files- README.md +74 -0
- adapters/mt4b/adapter_model.safetensors +3 -0
- adapters/mt4b_noise/adapter_model.safetensors +3 -0
- adapters/ocr4b_crop/adapter_model.safetensors +3 -0
- gamla_runor/fetch_gamla_runor.py +71 -0
- preds/e2e_ocr4b_cropcut_mt4b_fromocr.jsonl +0 -0
- preds/mt_test_retrieval.jsonl +0 -0
- splits/images.jsonl +0 -0
- splits/pairs.jsonl +0 -0
README.md
ADDED
|
@@ -0,0 +1,74 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: other
|
| 3 |
+
license_name: cc-by-sa-4.0-and-odbl-1.0
|
| 4 |
+
base_model: Qwen/Qwen3.5-4B
|
| 5 |
+
library_name: peft
|
| 6 |
+
tags:
|
| 7 |
+
- runic
|
| 8 |
+
- epigraphy
|
| 9 |
+
- cultural-heritage
|
| 10 |
+
- ocr
|
| 11 |
+
- translation
|
| 12 |
+
---
|
| 13 |
+
|
| 14 |
+
# Runic Reader 2: data, code and adapters
|
| 15 |
+
|
| 16 |
+
Release accompanying the paper *From Photograph to Translation: An End-to-End Pipeline for Runic
|
| 17 |
+
Inscriptions with Open Vision-Language Models* (IEEE Access, under review). The pipeline locates the
|
| 18 |
+
inscription in a photograph with the base model `Qwen/Qwen3.5-4B` (revision `851bf6e`), reads it into a
|
| 19 |
+
Rundata-style transliteration with a LoRA recognizer, and translates the transliteration into English or
|
| 20 |
+
Swedish with a LoRA translator.
|
| 21 |
+
|
| 22 |
+
## Contents
|
| 23 |
+
|
| 24 |
+
| Path | Content |
|
| 25 |
+
|---|---|
|
| 26 |
+
| `adapters/ocr4b_crop` | recognizer: LoRA on Qwen3.5-4B, trained on crops of real photographs and procedural synthetic images |
|
| 27 |
+
| `adapters/mt4b` | translator: LoRA on Qwen3.5-4B, trained on the parallel corpus |
|
| 28 |
+
| `adapters/mt4b_noise` | translator trained with recognition-like character noise on the input |
|
| 29 |
+
| `splits/pairs.jsonl` | all 12,496 transliteration–translation pairs with source, record identifier, signum, language, inscription cluster and split (train 10,640, validation 592, test 1,264) |
|
| 30 |
+
| `splits/images.jsonl` | all 4,949 real images with record, split, cluster, source URL, visual domain and the localization box (0–1000 coordinates) |
|
| 31 |
+
| `preds/` | test-set predictions of the recognizers, translators, end-to-end pipeline and retrieval baseline |
|
| 32 |
+
| `code/` | corpus construction, normalization, splitting, synthetic image generation, localization, training, inference and evaluation; `code/jobs/` holds the exact training and evaluation commands |
|
| 33 |
+
| `gamla_runor/fetch_gamla_runor.py` | retrieves the Gamla runor records from the public website and checks them against the hashes |
|
| 34 |
+
|
| 35 |
+
Splits are inscription-disjoint: pairs and images of one inscription cluster are always in the same split.
|
| 36 |
+
Evaluation in the paper uses only validation and test pairs that are not linked to any training pair; the
|
| 37 |
+
released splits already exclude the others.
|
| 38 |
+
|
| 39 |
+
## Sources and restrictions
|
| 40 |
+
|
| 41 |
+
- **Rundata**: the rows with source `rundata` in `splits/pairs.jsonl` are a modified database based on the
|
| 42 |
+
Scandinavian Runic-text Database (Samnordisk runtextdatabas, Uppsala University,
|
| 43 |
+
http://www.nordiska.uu.se/forskn/samnord.htm). They are released under the Open Database License (ODbL 1.0);
|
| 44 |
+
their individual contents are under the Database Contents License (DbCL 1.0). This release is not the
|
| 45 |
+
Scandinavian Runic-text Database itself.
|
| 46 |
+
- **RuneS-DB** (Göttingen Academy of Sciences and Humanities, https://www.runesdb.de/): the rows with source
|
| 47 |
+
`runes` are released under CC BY-SA 4.0, the licence of the RuneS data. RuneS photographs are linked, not
|
| 48 |
+
copied; each is under the licence stated on its find page.
|
| 49 |
+
- **Gamla runor** (christerhamp.se, Christer Hamp): texts and photographs are protected by copyright and are
|
| 50 |
+
**not** redistributed. Their rows in `splits/pairs.jsonl` give the record identifier, page and image links
|
| 51 |
+
and SHA-1 hashes of the normalized transliteration and of the translation. Predictions that would reproduce
|
| 52 |
+
these texts are given as SHA-1 hashes. To rebuild the full corpus for research use:
|
| 53 |
+
|
| 54 |
+
```
|
| 55 |
+
python gamla_runor/fetch_gamla_runor.py --pairs splits/pairs.jsonl --out gamla_runor_local [--images]
|
| 56 |
+
```
|
| 57 |
+
|
| 58 |
+
## Usage
|
| 59 |
+
|
| 60 |
+
```python
|
| 61 |
+
from peft import PeftModel
|
| 62 |
+
from transformers import AutoModelForImageTextToText, AutoProcessor
|
| 63 |
+
base = AutoModelForImageTextToText.from_pretrained("Qwen/Qwen3.5-4B", revision="851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a")
|
| 64 |
+
model = PeftModel.from_pretrained(base, "kesha-humonen/runic-reader-2", subfolder="adapters/ocr4b_crop")
|
| 65 |
+
```
|
| 66 |
+
|
| 67 |
+
Prompts, decoding settings (greedy; 320 new tokens for recognition, 200 for translation) and the
|
| 68 |
+
repetition truncation of recognizer output are in `code/qwen_common.py`, `code/generate.py` and
|
| 69 |
+
`code/loopcut.py`.
|
| 70 |
+
|
| 71 |
+
## Licence
|
| 72 |
+
|
| 73 |
+
Code, adapters, predictions and the RuneS rows are released under CC BY-SA 4.0. The Rundata rows are under
|
| 74 |
+
ODbL 1.0 / DbCL 1.0 (see above); reference texts in `preds/` follow the licence of their source. The base model is under its own licence.
|
adapters/mt4b/adapter_model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d396d4fd1dbe1a87f8eea112b82be3181d6d263f2f87af926ea79ffdf4dcc5dc
|
| 3 |
+
size 156077736
|
adapters/mt4b_noise/adapter_model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:aceee5648c07a627c0102b59591b39ea74072362facb7321b957bd0a2c8f4266
|
| 3 |
+
size 156077736
|
adapters/ocr4b_crop/adapter_model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f21218432e0752f85e926fc0a2e316830253895f2e78a256620add044119c4fb
|
| 3 |
+
size 156077736
|
gamla_runor/fetch_gamla_runor.py
ADDED
|
@@ -0,0 +1,71 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Retrieve the Gamla runor records of the corpus from christerhamp.se.
|
| 2 |
+
|
| 3 |
+
The texts and photographs of Gamla runor (Christer Hamp) are protected by copyright and are not redistributed.
|
| 4 |
+
splits/pairs.jsonl lists, for every Gamla runor pair, its page and image links and the SHA-1 hashes of the
|
| 5 |
+
normalized transliteration and the translation used in the paper. This script downloads each page and photograph
|
| 6 |
+
for personal research use, extracts the transliteration (<P class="trans">) and Swedish translation
|
| 7 |
+
(<P class="overs">), normalizes the transliteration as in the paper and reports how many records match the hashes.
|
| 8 |
+
|
| 9 |
+
python fetch_gamla_runor.py --pairs ../splits/pairs.jsonl --out gamla_runor_local [--images] [--delay 1.0]
|
| 10 |
+
"""
|
| 11 |
+
import argparse
|
| 12 |
+
import hashlib
|
| 13 |
+
import html
|
| 14 |
+
import json
|
| 15 |
+
import os
|
| 16 |
+
import re
|
| 17 |
+
import sys
|
| 18 |
+
import time
|
| 19 |
+
import urllib.request
|
| 20 |
+
|
| 21 |
+
sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)), "..", "code"))
|
| 22 |
+
from runic_norm import normalize # noqa: E402
|
| 23 |
+
|
| 24 |
+
UA = {"User-Agent": "runic-access-corpus-rebuild/1.0 (research use)"}
|
| 25 |
+
|
| 26 |
+
|
| 27 |
+
def get(url):
|
| 28 |
+
with urllib.request.urlopen(urllib.request.Request(url, headers=UA), timeout=30) as r:
|
| 29 |
+
return r.read()
|
| 30 |
+
|
| 31 |
+
|
| 32 |
+
def field(page, cls):
|
| 33 |
+
parts = re.findall(rf'<P class="{cls}">(.*?)(?=<P|</TD|</td)', page, flags=re.S | re.I)
|
| 34 |
+
txt = " ".join(re.sub(r"<[^>]+>", " ", p) for p in parts)
|
| 35 |
+
return re.sub(r"\s+", " ", html.unescape(txt)).strip()
|
| 36 |
+
|
| 37 |
+
|
| 38 |
+
def main():
|
| 39 |
+
ap = argparse.ArgumentParser()
|
| 40 |
+
ap.add_argument("--pairs", required=True)
|
| 41 |
+
ap.add_argument("--out", required=True)
|
| 42 |
+
ap.add_argument("--images", action="store_true")
|
| 43 |
+
ap.add_argument("--delay", type=float, default=1.0)
|
| 44 |
+
a = ap.parse_args()
|
| 45 |
+
os.makedirs(a.out, exist_ok=True)
|
| 46 |
+
rows = [json.loads(l) for l in open(a.pairs) if '"gamla_runor"' in l]
|
| 47 |
+
ok = 0
|
| 48 |
+
with open(os.path.join(a.out, "gamla_runor.jsonl"), "w") as f:
|
| 49 |
+
for i, r in enumerate(rows):
|
| 50 |
+
b = get(r["page"])
|
| 51 |
+
try:
|
| 52 |
+
page = b.decode("utf-8")
|
| 53 |
+
except UnicodeDecodeError:
|
| 54 |
+
page = b.decode("cp1252", "replace")
|
| 55 |
+
raw, tr = field(page, "trans"), field(page, "overs")
|
| 56 |
+
norm = normalize(raw)
|
| 57 |
+
match = hashlib.sha1(norm.encode()).hexdigest() == r["sha1_transliteration"]
|
| 58 |
+
ok += match
|
| 59 |
+
f.write(json.dumps(dict(id=r["id"], split=r["split"], cluster=r["cluster"], transliteration_raw=raw,
|
| 60 |
+
transliteration=norm, translation=tr, matches_paper=match), ensure_ascii=False) + "\n")
|
| 61 |
+
if a.images and r.get("image_urls"):
|
| 62 |
+
for k, u in enumerate(r["image_urls"].split("; ")):
|
| 63 |
+
open(os.path.join(a.out, f"{r['id']}_{k}.jpg"), "wb").write(get(u))
|
| 64 |
+
time.sleep(a.delay)
|
| 65 |
+
if i % 100 == 0:
|
| 66 |
+
print(f"{i}/{len(rows)} matching {ok}", flush=True)
|
| 67 |
+
print(f"done: {len(rows)} records, {ok} match the paper's transliterations")
|
| 68 |
+
|
| 69 |
+
|
| 70 |
+
if __name__ == "__main__":
|
| 71 |
+
main()
|
preds/e2e_ocr4b_cropcut_mt4b_fromocr.jsonl
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
preds/mt_test_retrieval.jsonl
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
splits/images.jsonl
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
splits/pairs.jsonl
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|