Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,84 @@
+---
+language:
+  - el
+library_name: transformers
+pipeline_tag: text-generation
+license: cc-by-nc-4.0
+tags:
+  - greek
+  - text-generation
+  - instruction-tuned
+  - llama
+---
+
+# LexisLM-500m-Instruct
+
+LexisLM-500m-Instruct is a Greek-focused instruction model fine-tuned from the 502-million-parameter LexisLM-500M base checkpoint. It is a decoder-only model exported in BF16 and compatible with the Transformers Llama architecture.
+
+## Model details
+
+| Property | Value |
+|---|---|
+| Layers | 16 |
+| Hidden size | 1,536 |
+| Attention heads | 16 |
+| Context length | 4,096 tokens |
+| Vocabulary | 32,004 tokens |
+| Tokenizer | Greek byte-level BPE with chat tokens |
+| Fine-tuning checkpoint | `runs/greek_500m_22bn_ctx4k_sft_mixed/best.pt`, step 3,802 |
+| Fine-tuning data | Greek EuroAlpaca, Aya Greek SFT, and Greek chat SFT mixtures |
+
+The chat template prepends `<eos>`, formats each message as `<|role|>` followed by a newline and content, and ends each turn with `<|endofturn|>`. Generation stops at `<eos>` or `<|endofturn|>`.
+
+## Usage
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_id = "ComputationalLinguisticsPlatform/LexisLM-500m-Instruct"
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+torch.set_num_threads(4)
+model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float32)
+torch.manual_seed(42)
+messages = [{"role": "user", "content": "Η πρωτεύουσα της Ελλάδας είναι"}]
+inputs = tokenizer.apply_chat_template(
+    messages, add_generation_prompt=True, tokenize=True,
+    return_tensors="pt", return_dict=True,
+)
+inputs = {key: value.to(model.device) for key, value in inputs.items()
+          if key in ("input_ids", "attention_mask")}
+output = model.generate(
+    **inputs, max_new_tokens=64, do_sample=True, temperature=0.1, top_p=0.9,
+    eos_token_id=[tokenizer.eos_token_id,
+                  tokenizer.convert_tokens_to_ids("<|endofturn|>")],
+    pad_token_id=tokenizer.pad_token_id,
+)
+print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
+```
+
+Output from this example (CPU FP32, seed 42):
+
+```text
+Η πρωτεύουσα της Ελλάδας είναι η Αθήνα.
+```
+
+## Evaluation
+
+The selected SFT checkpoint was evaluated with this project's Greek answer-likelihood protocol. ARC used 25 demonstrations, MMLU used 5, and TruthfulQA used 0. These scores are specific to that protocol and are not instruction-following evaluations.
+
+| Benchmark | Metric | Score |
+|---|---|---:|
+| ARC-Easy | Character-normalized accuracy | 41.58% |
+| ARC-Challenge | Character-normalized accuracy | 25.26% |
+| MMLU | Accuracy | 26.04% |
+| TruthfulQA | MC1 accuracy | 23.13% |
+| TruthfulQA | MC2 score | 41.35% |
+
+## Limitations
+
+The model may produce incorrect, biased, or unsafe text. Greek is the primary target language. The benchmarks above do not establish reliability for a specific application.
+
+## Provenance and license
+
+Pretraining used Greek Wikipedia and [FineWeb2](https://huggingface.co/datasets/HuggingFaceFW/fineweb-2), which is released under ODC-By 1.0. Supervised fine-tuning used the three data mixtures listed above, including [EuroAlpaca](https://huggingface.co/datasets/Telefonica/EuroAlpaca). EuroAlpaca is licensed under CC BY-NC 4.0, so this instruction-tuned model is released for noncommercial use under CC BY-NC 4.0. The export was produced with `scripts/export_hf.py` from the selected training checkpoint.