| --- |
| language: |
| - el |
| library_name: transformers |
| pipeline_tag: text-generation |
| license: cc-by-nc-4.0 |
| tags: |
| - greek |
| - text-generation |
| - instruction-tuned |
| - llama |
| base_model: Konstantinos/LexisLM-1B |
| --- |
| |
| # LexisLM-1B-Instruct |
|
|
| LexisLM-1B-Instruct is a Greek-focused instruction model fine-tuned from LexisLM-1B. It is a 1.093-billion-parameter decoder-only model exported in BF16 and compatible with the Transformers Llama architecture. |
|
|
| ## Model details |
|
|
| | Property | Value | |
| |---|---| |
| | Layers | 20 | |
| | Hidden size | 2,048 | |
| | Attention heads | 16 | |
| | Context length | 4,096 tokens | |
| | Vocabulary | 32,004 tokens | |
| | Tokenizer | Greek byte-level BPE with chat tokens | |
| | Fine-tuning checkpoint | `runs/greek_1b_22bn_sft_mixed/best.pt`, step 3,802 | |
| | Fine-tuning data | Greek EuroAlpaca, Aya Greek SFT, and Greek chat SFT mixtures | |
|
|
| The chat template prepends `<eos>`, formats each message as `<|role|>` followed by a newline and content, and ends each turn with `<|endofturn|>`. Generation stops at `<eos>` or `<|endofturn|>`. |
|
|
| ## Usage |
|
|
| ```python |
| import torch |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| |
| model_id = "ComputationalLinguisticsPlatform/LexisLM-1B-Instruct" |
| tokenizer = AutoTokenizer.from_pretrained(model_id) |
| torch.set_num_threads(4) |
| model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float32) |
| torch.manual_seed(42) |
| messages = [{"role": "user", "content": "Η πρωτεύουσα της Ελλάδας είναι"}] |
| inputs = tokenizer.apply_chat_template( |
| messages, add_generation_prompt=True, tokenize=True, |
| return_tensors="pt", return_dict=True, |
| ) |
| inputs = {key: value.to(model.device) for key, value in inputs.items() |
| if key in ("input_ids", "attention_mask")} |
| output = model.generate( |
| **inputs, max_new_tokens=64, do_sample=True, temperature=0.1, top_p=0.9, |
| eos_token_id=[tokenizer.eos_token_id, |
| tokenizer.convert_tokens_to_ids("<|endofturn|>")], |
| pad_token_id=tokenizer.pad_token_id, |
| ) |
| print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True)) |
| ``` |
|
|
| Output from this example (CPU FP32, seed 42): |
|
|
| ```text |
| Η Αθήνα είναι η πρωτεύουσα της Ελλάδας από το 1834. |
| ``` |
|
|
| ## Evaluation |
|
|
|
|
| | Benchmark | Metric | Score | |
| |---|---|---:| |
| | ARC-Easy | Character-normalized accuracy | 44.15% | |
| | ARC-Challenge | Character-normalized accuracy | 27.91% | |
| | MMLU | Accuracy | 24.67% | |
| | TruthfulQA | MC1 accuracy | 22.52% | |
| | TruthfulQA | MC2 score | 40.90% | |
|
|
| ## Limitations |
|
|
| The model may produce incorrect, biased, or unsafe text. Greek is the primary target language. The benchmarks above do not establish reliability for a specific application. |
|
|
| ## Provenance |
|
|
| Pretraining used Greek Wikipedia and FineWeb-2. Supervised fine-tuning used the three data mixtures listed above, including [EuroAlpaca](https://huggingface.co/datasets/Telefonica/EuroAlpaca). EuroAlpaca is licensed under CC BY-NC 4.0, so this instruction-tuned model is released for noncommercial use under CC BY-NC 4.0. |
|
|
| ## Citation |
| ``` |
| @misc{perifanos2026compactgreeklanguagemodels, |
| title={Compact Greek Language Models from Scratch: Pretraining 500M- and 1B-Parameter Decoder-Only Transformers}, |
| author={Konstantinos Perifanos and Anne Laure Poite and Giannis Tsilikas and Ilias Katis and Stavros Deligiannidis and Dionysis Goutsos}, |
| year={2026}, |
| eprint={2611.10209}, |
| archivePrefix={arXiv}, |
| primaryClass={cs.CL}, |
| url={https://arxiv.org/abs/2611.10209}, |
| } |
| ``` |
