| --- |
| license: mit |
| language: |
| - ba |
| tags: |
| - bashkir |
| - full-fine-tuning |
| - gpt2 |
| - low-resource |
| --- |
| |
| # Bashkir Full Fine-Tuning Models |
|
|
| Full fine-tuned GPT-2 family models on **Bashkir Web Corpus** (71,567 documents). |
|
|
| Part of the BashkirNLP project. See the paper for details. |
|
|
| ## Structure |
|
|
| Each top-level subfolder `<model>_seed<N>/` contains a full fine-tuned language model. |
| The `classification/` folder contains sequence-classification heads trained on top of the same base models. |
|
|
| ``` |
| distilgpt2_seed42/ |
| distilgpt2_seed123/ |
| distilgpt2_seed2024/ |
| gpt2_seed42/ |
| ... |
| classification/ |
| cls_binary_distilgpt2_seed42/ |
| cls_multiclass_distilgpt2_seed42/ |
| ... |
| ``` |
|
|
| ## Training |
|
|
| - Base models: `distilgpt2` (82M), `gpt2` (124M), `gpt2-medium` (355M) |
| - Corpus: Bashkir Web Corpus (57,254 train / 7,157 val / 7,156 test) |
| - Epochs: 3 |
| - Seeds: 42, 123, 2024 |
| - `max_seq_len`: 128 |
|
|
| ## Evaluation |
|
|
| See `step1_report.txt` in the paper repository for metrics (PPL, BPB, catastrophic forgetting, target-language ratio). |
|
|
