| --- |
| language: grc |
| license: apache-2.0 |
| inference: false |
| --- |
| # GrεTa |
|
|
| The paper [Exploring Language Models for Classical Philology](https://todo.com) is the first effort to systematically provide state-of-the-art language models for Classical Philology. GrεTa is a T5-base sized, monolingual, encoder-decoder variant. |
|
|
| This model was trained in two stages. Initially, it was pre-trained on a recently acquired corpus that leverages OCR scans obtained from the [Internet Archive](https://archive.org/). Subsequently, the model was further trained using data from the [Open Greek & Latin Project](https://opengreekandlatin.org/), the CLARIN corpus [Greek Medieval Texts](https://inventory.clarin.gr/corpus/890), and the [Patrologia Graeca](https://patristica.net/graeca/). |
|
|
|
|
| Further information can be found in our paper or in our [GitHub repository](https://github.com/Heidelberg-NLP/ancient-language-models). |
|
|
| ## Usage |
| ```python |
| from transformers import AutoTokenizer, AutoModelForConditionalGeneration |
| |
| tokenizer = AutoTokenizer.from_pretrained('bowphs/GreTa') |
| model = AutoModelForConditionalGeneration.from_pretrained('bowphs/GreTa') |
| ``` |
| Please check out the awesome Hugging Face tutorials on how to fine-tune our models. |
|
|
| ## Evaluation Results |
| When fine-tuned on data from [Universal Dependencies 2.10](https://universaldependencies.org/), GrεTa achieves the following results on the Ancient Greek Perseus dataset: |
|
|
| | Task | XPoS | UPoS | UAS | LAS | Lemma | |
| |:--:|:--:|:--:|:--:|:--:|:--:| |
| | |94.44|89.03|87.32|83.06|91.14| |
|
|
| Please note that the PoS tagging and dependency parsing results are obtained using _only_ the encoder component of the model. |
|
|
| ## Contact |
| If you have any questions or problems, feel free to [reach out](mailto:riemenschneider@cl.uni-heidelberg.de). |
|
|
| ## Citation |
| ```bibtex |
| @incollection{riemenschneiderfrank:2023, |
| address = "Toronto, Canada", |
| author = "Riemenschneider, Frederick and Frank, Anette", |
| booktitle = "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL’23)", |
| note = "to appear", |
| pubType = "incollection", |
| publisher = "Association for Computational Linguistics", |
| title = "Exploring Large Language Models for Classical Philology", |
| url = "https://arxiv.org/abs/2305.13698", |
| year = "2023", |
| key = "riemenschneiderfrank:2023" |
| } |
| ``` |
