EuroLLM-22B-grc-ro (preview, v2)

Koine / Byzantine (patristic) Greek → Romanian translation, obtained by LoRA fine-tuning utter-project/EuroLLM-22B-Instruct-2512 on 5,810 paragraph-level Greek–Romanian pairs. The Romanian side follows the register of the Orthodox translations in the training data (archaic / ecclesiastical, e.g. întru, cela ce), not modern Romanian.

Status: preview. Scores below are on development rows only: the 134-row development split that was also used to choose this model and its input arm, and 91 development rows from the conciliar acts added in v2. A 106-row test split, 74 test rows from the conciliar acts and a 160-row held-out source were not touched by training or selection and have not been evaluated yet. There has been no human evaluation.

Versions

Revision	Training pairs	Notes
main, tag v2	5,810	The v1 data plus 2,233 pairs from the acts of the ecumenical councils.
tag v1	3,577	First release. Load it with revision="v1".

What is in this repository

Path	Content
*.safetensors, config.json, tokenizer files	Merged bf16 weights (LoRA merged into the base). Loads directly in transformers and vLLM.
adapter/	The LoRA adapter alone (rank 32, fp32, 702 MB) and run.json with the training settings.
grc_normalize.py	Required input normalization (see below).
example.py	Minimal transformers usage.

Input format

The model was trained with the Greek folded to monotonic form. Apply monotonic() from grc_normalize.py to every Greek input: it drops breathings, iota subscript, macron and breve, and maps grave and circumflex to acute. Then send a single user message:

Translate the following Koine/Ancient Greek text into Romanian. Output only the Romanian translation.

{greek}

All reported results use greedy decoding. Training and evaluation were paragraph-level (median about 610 Greek characters, maximum about 2,300); behaviour on much longer inputs is untested.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from grc_normalize import monotonic

REPO = "icxcn/EuroLLM-22B-grc-ro"
PROMPT = "Translate the following Koine/Ancient Greek text into Romanian. Output only the Romanian translation.\n\n{grc}"
tok = AutoTokenizer.from_pretrained(REPO)
model = AutoModelForCausalLM.from_pretrained(REPO, dtype=torch.bfloat16, device_map="auto")

messages = [{"role": "user", "content": PROMPT.format(grc=monotonic(greek))}]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=512, do_sample=False)
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

vLLM: serve the merged weights (vllm serve icxcn/EuroLLM-22B-grc-ro) and normalize on the client side. To use the adapter instead: PeftModel.from_pretrained(base_model, "icxcn/EuroLLM-22B-grc-ro", subfolder="adapter").

Results

Development split (134 rows: Bible, liturgical and patristic text)

System	chrF++	COMET-22	chrF++ Bible rows	chrF++ patristic rows
Base model, zero-shot (same input format)	47.61	not measured	52.17	44.67
v1	58.99 (95% CI 56.95–61.17)	0.8018 (0.7899–0.8129)	71.22	51.19
v2 (this model)	59.97 (57.85–62.19)	0.8034 (0.7904–0.8153)	71.91	52.42

Paired difference v2 − v1: +0.98 chrF++ (95% CI −0.12 to +2.27) and +0.0016 COMET (−0.0064 to +0.0094), both within noise.

Conciliar acts, development rows (91 rows)

System	chrF++	COMET-22
v1	53.66 (95% CI 51.14–55.84)	0.7513 (0.7367–0.7649)
v2 (this model)	64.76 (62.74–66.81)	0.7822 (0.7662–0.7958)

Paired difference v2 − v1: +11.09 chrF++ (95% CI 9.10 to 13.33), COMET +0.0309 (0.0153 to 0.0450).

• chrF++ is sacrebleu CHRF(word_order=2); COMET is Unbabel/wmt22-comet-da; intervals are 95% bootstrap intervals over rows (1,000 resamples).
• "Bible" rows are the New Testament and two Psalm editions (65 dev rows); "patristic" rows are everything else (69 rows).
• Read the conciliar-acts gain with care. The references for those rows are the Romanian text that came with the data, whose origin is not documented and which may be machine translation (see Training). The gain shows that v2 has learned this material and the style of its references; it is not evidence of the same gain against a vetted human translation.
• Under greedy decoding both versions sometimes run away into a repetition loop: v2 on 3 of the 225 development rows, v1 on 2. Those rows are included in the scores above. Without the 4 rows of the 134-row split where either version's output is more than 1.5 times the reference length, the difference is +1.15 chrF++ (95% CI 0.42 to 1.87).
• Greedy decoding is not bit-for-bit reproducible across runs: the same merged model scored 54.99 and 54.37 chrF++ on two vLLM runs in a separate test of a different model, so differences below about 1 chrF++ should not be read as real.
• In the v1 bake-off of four fine-tuned base models (also Gemma 4 31B, Gemma 4 12B, Krikri-8B) this base tied the best on dev (Gemma 4 31B: 59.43 chrF++, COMET 0.8019; difference not significant) and beat the 12B and 8B models by about 4–5 chrF++.

Learning curve (same settings, dev split; the first four rows are nested subsets of the v1 data):

Training pairs	Dev loss	chrF++	Bible	Patristic
894 (25%)	0.912	55.73	66.99	48.53
1,788 (50%)	0.865	55.92	69.61	47.36
2,683 (75%)	0.829	57.41	70.78	48.97
3,577 (v1)	0.802	58.99	71.22	51.19
5,810 (v2: v1 + conciliar acts)	0.797	59.97	71.91	52.42

Dev loss was still falling at about 0.055 per doubling of data when the v1 data ran out. The 2,233 added pairs are a different genre from anything in this split and moved its loss by 0.005.

Training

• Method: LoRA, rank 32, alpha 64, dropout 0.05, on the attention and MLP projections of the language model. Base weights bf16, adapter weights fp32, bf16 compute. No quantization.
• Optimization: 2 epochs (726 steps), learning rate 1e-4 with cosine schedule and 3% warmup, effective batch 16 sequences (2 × 8 gradient accumulation), maximum 2,560 tokens, seed 0, gradient checkpointing. Loss on the Romanian answer only. Same settings as v1.
• Cost / software: about 84 minutes of training on one H200 (5,011 s); transformers 5.22.2, peft 0.21.2, torch 2.15.0.
• Data: 5,810 Greek–Romanian paragraph pairs (about 572k Greek words, 665k Romanian words). The training data is not distributed with this model.
  • 3,577 pairs (the v1 data) from 13 bilingual sources: the New Testament, two editions of the Psalms, a liturgical book, and patristic and ascetic works.
  • 2,233 pairs from 14 volumes of the Greek acts of the ecumenical councils in the Acta Conciliorum Oecumenicorum edition (Ephesus, Chalcedon, Constantinople III, the Council in Trullo, Nicaea II). The origin of the Romanian side of these pairs is not documented in the dataset; it may be machine translation, and no human checked the pairs.
  • About a third of the Greek is properly polytonic Unicode (New Testament, Psalms, one volume of the acts); the rest comes from OCR'd PDFs and has collapsed diacritics and OCR errors, which is why the model uses the monotonic-folded input form.
• Cleaning: NFC normalization; folio markers, editorial brackets, scripture-citation parentheticals, section numbers and footnote digits removed; rows with length-ratio outliers, merged tokens, mixed-script Greek, embedded chapter headings or duplicates dropped; training rows overlapping the held-out rows removed. For the conciliar acts also: line numbers, manuscript sigla and apparatus tokens removed from the Greek, paragraph numbers removed from the Romanian. The two sides of these pairs were not cut at the same points, so each pair was trimmed automatically to the passage both sides share (an LLM alignment pass followed by an LLM audit of the trimmed pair); about a quarter of the delivered rows did not survive.
• Held out: a development split (134 rows, used for monitoring and selection), a test split (106 rows) and one whole source (160 rows) used as an out-of-distribution test; from the conciliar acts, 91 development rows and 74 test rows.

Limitations

• Patristic text is the weak spot. About 52 chrF++ on patristic rows of the development split against about 72 on Bible rows. Expect noticeably worse output on patristic and philosophical prose than on Gospel or Psalm text.
• Register. Output imitates the Orthodox Romanian of the training translations. It is not modern Romanian, and it will reproduce that translation tradition's wording choices.
• Fluent errors. Translations can be fluent and wrong (mistranslated terms, dropped clauses, garbled phrases). Do not use the output as authoritative text or for liturgical or theological purposes without review by a reader of both languages.
• Runaway repetition. On about 1 in 75 development paragraphs greedy decoding repeats a phrase until max_new_tokens. Set max_new_tokens in proportion to the input and check the length of what comes back.
• Unverified training targets. The conciliar-acts pairs (38% of the training pairs) have Romanian of undocumented origin, aligned and filtered automatically.
• Evidence is limited. Single training run per configuration, development splits of 134 and 91 rows, automatic metrics only, no human evaluation, and the test and out-of-distribution evaluations are still pending.
• Input. Only Koine / Byzantine Greek paragraphs normalized with grc_normalize.py were used. Classical or dialectal Greek, modern Greek, other target languages and very long inputs were not evaluated.