LexisLM-1B-Instruct

LexisLM-1B-Instruct is a Greek-focused instruction model fine-tuned from LexisLM-1B. It is a 1.093-billion-parameter decoder-only model exported in BF16 and compatible with the Transformers Llama architecture.

Model details

Property	Value
Layers	20
Hidden size	2,048
Attention heads	16
Context length	4,096 tokens
Vocabulary	32,004 tokens
Tokenizer	Greek byte-level BPE with chat tokens
Fine-tuning checkpoint	runs/greek_1b_22bn_sft_mixed/best.pt, step 3,802
Fine-tuning data	Greek EuroAlpaca, Aya Greek SFT, and Greek chat SFT mixtures

The chat template prepends <eos>, formats each message as <|role|> followed by a newline and content, and ends each turn with <|endofturn|>. Generation stops at <eos> or <|endofturn|>.

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "ComputationalLinguisticsPlatform/LexisLM-1B-Instruct"
tokenizer = AutoTokenizer.from_pretrained(model_id)
torch.set_num_threads(4)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float32)
torch.manual_seed(42)
messages = [{"role": "user", "content": "Η πρωτεύουσα της Ελλάδας είναι"}]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=True,
    return_tensors="pt", return_dict=True,
)
inputs = {key: value.to(model.device) for key, value in inputs.items()
          if key in ("input_ids", "attention_mask")}
output = model.generate(
    **inputs, max_new_tokens=64, do_sample=True, temperature=0.1, top_p=0.9,
    eos_token_id=[tokenizer.eos_token_id,
                  tokenizer.convert_tokens_to_ids("<|endofturn|>")],
    pad_token_id=tokenizer.pad_token_id,
)
print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Output from this example (CPU FP32, seed 42):

Η Αθήνα είναι η πρωτεύουσα της Ελλάδας από το 1834.

Evaluation

Benchmark	Metric	Score
ARC-Easy	Character-normalized accuracy	44.15%
ARC-Challenge	Character-normalized accuracy	27.91%
MMLU	Accuracy	24.67%
TruthfulQA	MC1 accuracy	22.52%
TruthfulQA	MC2 score	40.90%

Limitations

The model may produce incorrect, biased, or unsafe text. Greek is the primary target language. The benchmarks above do not establish reliability for a specific application.

Provenance

Pretraining used Greek Wikipedia and FineWeb-2. Supervised fine-tuning used the three data mixtures listed above, including EuroAlpaca. EuroAlpaca is licensed under CC BY-NC 4.0, so this instruction-tuned model is released for noncommercial use under CC BY-NC 4.0.

Citation

@misc{perifanos2026compactgreeklanguagemodels,
      title={Compact Greek Language Models from Scratch: Pretraining 500M- and 1B-Parameter Decoder-Only Transformers}, 
      author={Konstantinos Perifanos and Anne Laure Poite and Giannis Tsilikas and Ilias Katis and Stavros Deligiannidis and Dionysis Goutsos},
      year={2026},
      eprint={2611.10209},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2611.10209}, 
}