DART-FT / README.md

Update README.md

f3834b9 verified about 2 months ago

18.1 kB

---
license: cc-by-nc-4.0
language:
- grc
base_model:
- pranaydeeps/Ancient-Greek-BERT
pipeline_tag: token-classification
tags:
- morphological-tagging
- ancient-greek
- nlp
- universal-dependencies
- low-resource-language
- digital-humanities
metrics:
- accuracy
- f1
datasets:
- proiel
---
# DART-FT: Decomposed, Ambiguity-Aware, Risk-Targeted Fine-Tuning

**DART-FT** is an Ancient Greek fine-grained morphological tagging model and targeted fine-tuning methodology developed as part of a Doctor of Engineering praxis at The George Washington University.

📄 **Full doctoral praxis:**  
**[DART-FT: Decomposed, Ambiguity-Aware, Risk-Targeted Fine-Tuning of Transformer Models for Ancient Greek Morphological Tagging](https://scholarspace.library.gwu.edu/etd/4x51hk097)**

DART-FT addresses a specific challenge in low-resource, morphologically rich language modeling: strong fine-tuned transformer models can achieve high aggregate performance while retaining systematic errors concentrated in difficult morphological feature-value patterns.

Rather than broadly retraining the model, DART-FT identifies an empirically defined **high-risk slice** associated with recurrent baseline tagging errors and performs targeted fine-tuning in which training examples associated with the high-risk patterns are selectively oversampled.

Across five random seeds, DART-FT significantly improved fine-grained morphological tagging accuracy on the predefined high-risk slice relative to corresponding standard cross-entropy fine-tuned Ancient-Greek-BERT baselines. The best-performing configuration improved high-risk-slice accuracy by **6.4 percentage points**, from **63.4% to 69.9%** (**p < .001**), while global macro-F1 showed no observed degradation.

## Model Details

- **Developed by:** Matthew K. Weber
- **Model:** DART-FT
- **Model type:** BERT encoder with decomposed morphological classification heads
- **Base model:** [`pranaydeeps/Ancient-Greek-BERT`](https://huggingface.co/pranaydeeps/Ancient-Greek-BERT)
- **Language:** Ancient Greek (`grc`)
- **Primary task:** Fine-grained morphological tagging
- **Framework:** PyTorch / Hugging Face Transformers
- **Dataset:** UD Ancient Greek PROIEL v2.12
- **Doctoral program:** Doctor of Engineering, The George Washington University
- **Praxis:** [Full text in GW ScholarSpace](https://scholarspace.library.gwu.edu/etd/4x51hk097)


## What DART-FT Predicts

DART-FT predicts fine-grained morphology using **decomposed classification** rather than treating every complete morphological bundle as a single class.

The released schema contains a UPOS head and separate heads for 15 Universal Dependencies morphological attributes:

- UPOS
- Case
- Gender
- Number
- Definite
- PronType
- Aspect
- Tense
- VerbForm
- Voice
- Degree
- Mood
- Person
- Polarity
- Reflex
- Poss

Individual predictions can then be reconstructed into a complete fine-grained morphological representation such as:

```text
NOUN|Case=Acc|Gender=Masc|Number=Sing
```

This decomposition is intended to reduce the data-sparsity problems that arise when every possible combination of morphological features is represented as an independent class.

---

# DART-FT Methodology

DART-FT combines three principal ideas:

1. **Decomposed morphological classification**
2. **Empirical identification of a high-risk slice**
3. **Risk-targeted fine-tuning through oversampling**

## 1. Decomposed Morphological Classification

Instead of representing a complete FEATS bundle as one classification label, DART-FT decomposes each bundle into individual morphological attributes for independent per-feature classification.

For example:

```text
Case=Nom
Gender=Masc
Number=Sing
```

are modeled as separate classification decisions.

The model additionally predicts UPOS.

This architecture reduces sparsity associated with treating the full Cartesian combination of morphological attributes as distinct labels.

## 2. Empirical High-Risk Slice Identification

The **high-risk slice is not a predefined dictionary of difficult Ancient Greek words**.

Instead, it is identified empirically from recurrent errors made by a standard fine-tuned Ancient-Greek-BERT baseline.

The methodology uses a **three-pass process**.

### Pass 1 — Error-Driven Mining

The baseline model predicts morphological labels on the validation set. Predicted feature-value pairs are compared with gold labels to identify recurrent error patterns.

### Pass 2 — Feature-Value Filtering and Diversity Constraint

Candidate feature-value pairs are filtered to isolate sufficiently frequent and systematic error patterns while preventing a small number of broad categories from dominating the high-risk slice.

### Pass 3 — Slice Freezing and Mask Creation

The resulting feature-value definition is frozen. A mask based on this frozen definition is then applied to held-out test data for consistent evaluation across models.

The validation set is used to define the high-risk slice so that the test set remains held out from training and model-selection activities.

Although morphological ambiguity motivates DART-FT, membership in the high-risk slice is determined by **observed model error behavior at the feature-value level**, rather than lexical identity alone.

## 3. Risk-Targeted Fine-Tuning

After the high-risk feature-value patterns are identified, DART-FT performs an additional stage of targeted fine-tuning.

Training sentences containing tokens associated with the high-risk patterns are selectively **oversampled**, increasing model exposure to those difficult cases while preserving sentence-level context.

The research evaluated:

- oversampling factors of **2x, 3x, and 4x**
- five random seeds
- cross-entropy loss
- class-balanced loss
- class-balanced + focal loss
- multiple class-balanced β values
- multiple focal-loss γ values

A total of **90 distinct experimental models** were evaluated.

---

# Released Model Configuration

The model artifact in this repository corresponds to the highest-performing DART-FT configuration evaluated in the research:

- **Random seed:** 43
- **Oversampling factor:** 3x
- **Loss:** Class-balanced
- **β:** 0.993
- **Learning rate:** 3 × 10⁻⁵
- **Batch size:** 8
- **Training epochs:** 5
- **Maximum sequence length:** 512
- **Optimizer:** AdamW
- **Weight decay:** 0.01
- **Dropout:** 0.1
- **Mixed precision:** FP16

The repository includes the fine-tuned Ancient-Greek-BERT encoder and separately stored decomposed classification heads.

Important files include:

```text
model.safetensors
heads.pt
schema.json
meta.json
test_pred.conllu
```

### Repository Components

**`model.safetensors`**  
Fine-tuned transformer encoder weights.

**`heads.pt`**  
Trained decomposed morphological classification heads.

**`schema.json`**  
Label inventory for UPOS and each morphological feature head.

**`meta.json`**  
Metadata describing the released experimental configuration.

**`test_pred.conllu`**  
Model predictions in CoNLL-U-compatible form.
---
# Training and Evaluation Data
DART-FT was trained and evaluated using the **Universal Dependencies Ancient Greek PROIEL treebank, version 2.12**.
The pre-existing PROIEL train, validation, and test splits were preserved.
| Split | Sentences | Tokens | UPOS Tags | UFEATS Attributes |
|---|---:|---:|---:|---:|
| Train | 15,016 | 187,039 | 13 | 15 |
| Dev / Validation | 1,019 | 13,652 | 13 | 15 |
| Test | 1,047 | 13,314 | 13 | 15 |
| **Total** | **17,082** | **214,005** | **13** | **15** |
The Ancient Greek PROIEL corpus used in the study includes the **27 books of the Ancient Greek New Testament and portions of Herodotus**.
The New Testament portion of PROIEL itself contains approximately **140,000 tokens**. This should not be confused with the full experimental dataset, which contains **214,005 tokens across the train, validation, and test splits**.
---
# Tokenization and Label Alignment
DART-FT uses the WordPiece tokenization employed by BERT.
Because one source word can be divided into multiple subword tokens:
- the **first subword** receives the corresponding UPOS and morphological labels;
- subsequent subwords are **excluded from the classification loss**.
This prevents a word that is split into multiple WordPieces from contributing duplicate morphological training labels and promotes learning at the word level.
---
# Evaluation
DART-FT was evaluated from two complementary perspectives:
1. **Fine-grained morphological accuracy on the high-risk slice**
2. **Global macro-F1**
## High-Risk Exact-Match Accuracy
Fine-grained accuracy uses the complete morphological prediction for a word.
The prediction must match the corresponding gold morphological representation for the token; partial corrections do not count as an exact match.
## Macro-F1
Macro-F1 gives classes equal weight rather than allowing highly frequent morphological categories to dominate the evaluation.
Global macro-F1 was used specifically to evaluate whether targeted intervention on the high-risk slice caused broader model degradation.
## Statistical Testing
Paired baseline and DART-FT predictions were evaluated using the **exact two-sided McNemar test**.
Confidence intervals were also calculated for the paired accuracy differences.
---
# Results
## High-Risk Slice Performance
For the highest-performing DART-FT model associated with each random seed:
| Seed | Baseline Accuracy | DART-FT Accuracy | Change | p-value |
|---:|---:|---:|---:|---:|
| 41 | 59.3% | 63.9% | +4.5 pp | < .001 |
| 42 | 65.6% | 70.7% | +5.2 pp | .003 |
| 43 | 63.4% | 69.9% | **+6.4 pp** | **< .001** |
| 44 | 57.5% | 63.2% | +5.7 pp | < .001 |
| 45 | 64.5% | 69.8% | +5.3 pp | < .001 |
The largest observed improvement was produced by **seed 43, 3x oversampling, class-balanced loss, β = 0.993**:
```text
Baseline high-risk accuracy: 63.4%
DART-FT high-risk accuracy:  69.9%
Absolute improvement:        +6.4 percentage points
p-value:                     < .001
95% confidence interval:     3.38–9.14 pp
```
All five seed comparisons exhibited statistically significant increases in high-risk-slice accuracy.
## Global Model Stability
Targeted fine-tuning did **not** produce an observed decrease in global macro-F1 for any of the five random seeds.
| Seed | Baseline Global Macro-F1 | DART-FT Global Macro-F1 | Change |
|---:|---:|---:|---:|
| 41 | 0.921 | 0.939 | +0.018 |
| 42 | 0.919 | 0.940 | +0.021 |
| 43 | 0.917 | 0.939 | +0.022 |
| 44 | 0.919 | 0.938 | +0.019 |
| 45 | 0.920 | 0.937 | +0.017 |
The mean global macro-F1 improvement was **+0.0194**.
These results support the central DART-FT objective: targeted repair of systematic fine-grained morphological tagging errors without observed degradation of overall model performance.
## Error Analysis
Error analysis showed that targeted fine-tuning corrected recurring errors across multiple morphological features.
A particularly strong effect was observed for the `Gender=Masc,Neut` feature-value pair, where the standard baseline frequently predicted one of the more common individual gender categories rather than the syncretic `Masc,Neut` value.
The research also identified localized regressions in some tightly coupled features, demonstrating that targeted fine-tuning can introduce tradeoffs even when aggregate high-risk and global performance improve.
---
# Oversampling Sensitivity
DART-FT explicitly evaluated whether performance was sensitive to the amount of high-risk oversampling.
The tested oversampling factors were:
```text
2x
3x
4x
```
Performance varied across seeds and configurations.
Across the experiments, intermediate oversampling generally produced the strongest mean improvement, and the highest-performing individual configuration used **3x oversampling**.
This supports the conclusion that targeted fine-tuning effectiveness depends on the strength of the intervention and that simply maximizing oversampling is not necessarily optimal.
---
# Intended Uses
## Fine-Grained Ancient Greek Morphological Tagging
DART-FT can support prediction of grammatical information including:
- part of speech
- case
- gender
- number
- tense
- mood
- voice
- person
- aspect
- degree
- pronoun type
- definiteness
- polarity
- reflexivity
- possession
## Digital Humanities
Potential applications include automated annotation and analysis of Ancient Greek corpora, including historical and biblical texts.
## NLP Pipelines
Fine-grained morphological predictions may provide structured linguistic information for downstream tasks such as:
- syntactic parsing
- corpus analysis
- information extraction
- search
- experimental machine translation workflows
Downstream benefits should be independently evaluated for each application.
## Research on Targeted Model Repair
DART-FT may also be useful as an example of **error-driven targeted fine-tuning**.
The broader methodology identifies a reproducible model weakness through validation-set behavior and selectively increases exposure to corresponding training examples rather than indiscriminately increasing training across the entire corpus.
---
# Downstream Translation Pilot
The praxis also included a small-scale exploratory Ancient Greek-to-Latin translation pilot.
The pilot compared morphological information produced by the DART-FT model with the corresponding baseline model when supplied to a downstream translation system.
The results provided limited evidence of downstream improvement:
- **chrF++:** +0.24, p = 0.0408
- **TER:** 59.42 → 59.27
- for the subset in which baseline and DART-FT morphological sequences differed (`N = 232`), chrF++ improved by +0.37 and TER decreased by 0.23
This pilot was **secondary and exploratory**. DART-FT should not be interpreted as a production machine-translation system, and the translation results should not be generalized beyond the limited experiment.
---
# Limitations
## Corpus Scope
DART-FT was trained and evaluated on the Ancient Greek PROIEL treebank, consisting principally of New Testament Greek and portions of Herodotus.
Performance should not be assumed to transfer unchanged to:
- Homeric Greek
- Classical Attic Greek
- Septuagint Greek
- papyri
- inscriptions
- patristic Greek
- Byzantine Greek
- other Ancient Greek treebanks or annotation schemes
## Generalizability
Generalizability of the methodology to other languages and domains is plausible but was **not empirically validated** in the praxis.
Applying DART-FT to another problem requires, at minimum:
- a gold-labeled dataset with token-level annotations;
- a pretrained model capable of task-specific fine-tuning; and
- a finite, decomposable feature set suitable for identifying systematic feature-level errors.
## High-Risk Slice Definition
The high-risk slice is empirically defined and depends on methodological thresholds selected during the research.
Different thresholds or filtering rules may result in different high-risk populations.
## Model Dependence
The high-risk slice reflects weaknesses exhibited by a particular model and dataset.
Different pretrained models, datasets, or annotation systems may produce different concentrations of errors.
## Oversampling Tradeoffs
Oversampling increases model exposure to difficult examples but can also increase the risk of overfitting or localized negative transfer.
DART-FT therefore treats oversampling strength as a hyperparameter rather than assuming that greater oversampling will always yield greater performance.
---
# How to Use
> **Important:** The released DART-FT repository is not packaged as a conventional single-head Hugging Face `AutoModelForTokenClassification` checkpoint.
DART-FT uses a BERT encoder plus **separately stored decomposed morphological classification heads**.
Therefore, a call such as:
```python
AutoModelForTokenClassification.from_pretrained("zootsewt/DART-FT")
```
does **not by itself reproduce the complete DART-FT inference architecture**.
The transformer encoder and tokenizer can be loaded with:
```python
from transformers import AutoTokenizer, AutoModel
model_name = "zootsewt/DART-FT"
tokenizer = AutoTokenizer.from_pretrained(model_name)
encoder = AutoModel.from_pretrained(model_name)
```
The decomposed classification heads stored in:
```text
heads.pt
```
must also be loaded and applied according to the label inventory in:
```text
schema.json
```
The individual head predictions are then combined to reconstruct a complete fine-grained morphological analysis for each source token.
A reference inference wrapper may be provided in a future repository update to automate loading of the encoder, classification heads, schema, token alignment, and output reconstruction.
---
# Research Context
DART-FT was developed as the principal research contribution of the Doctor of Engineering praxis:
** _DART-FT: Decomposed, Ambiguity-Aware, Risk-Targeted Fine-Tuning of Transformer Models for Ancient Greek Morphological Tagging._ The George Washington University.**

### Read the full praxis

**https://scholarspace.library.gwu.edu/etd/4x51hk097**



---

# Citation

If you use DART-FT or the associated methodology in academic work, please cite the doctoral praxis:

```bibtex
@phdthesis{weber2026dartft,
  author = {Weber, Matthew K.},
  title = {DART-FT: Decomposed, Ambiguity-Aware, Risk-Targeted Fine-Tuning of Transformer Models for Ancient Greek Morphological Tagging},
  school = {The George Washington University},
  year = {2026},
  type = {Doctor of Engineering Praxis},
  url = {https://scholarspace.library.gwu.edu/etd/4x51hk097}
}
```

**Praxis:**  
https://scholarspace.library.gwu.edu/etd/4x51hk097

---

# Disclaimer

DART-FT is a research artifact developed and evaluated within the scope of a Doctor of Engineering praxis.

Reported results reflect the specific datasets, baseline model, experimental design, high-risk slice definition, hyperparameters, and evaluation methods used in that research.

Results should not be assumed to generalize unchanged to other corpora, models, languages, annotation systems, or downstream applications.

	---
	license: cc-by-nc-4.0
	language:
	- grc
	base_model:
	- pranaydeeps/Ancient-Greek-BERT
	pipeline_tag: token-classification
	tags:
	- morphological-tagging
	- ancient-greek
	- nlp
	- universal-dependencies
	- low-resource-language
	- digital-humanities
	metrics:
	- accuracy
	- f1
	datasets:
	- proiel
	---

	# DART-FT: Decomposed, Ambiguity-Aware, Risk-Targeted Fine-Tuning

	DART-FT is an Ancient Greek fine-grained morphological tagging model and targeted fine-tuning methodology developed as part of a Doctor of Engineering praxis at The George Washington University.

	📄 Full doctoral praxis:
	[DART-FT: Decomposed, Ambiguity-Aware, Risk-Targeted Fine-Tuning of Transformer Models for Ancient Greek Morphological Tagging](https://scholarspace.library.gwu.edu/etd/4x51hk097)

	DART-FT addresses a specific challenge in low-resource, morphologically rich language modeling: strong fine-tuned transformer models can achieve high aggregate performance while retaining systematic errors concentrated in difficult morphological feature-value patterns.

	Rather than broadly retraining the model, DART-FT identifies an empirically defined high-risk slice associated with recurrent baseline tagging errors and performs targeted fine-tuning in which training examples associated with the high-risk patterns are selectively oversampled.

	Across five random seeds, DART-FT significantly improved fine-grained morphological tagging accuracy on the predefined high-risk slice relative to corresponding standard cross-entropy fine-tuned Ancient-Greek-BERT baselines. The best-performing configuration improved high-risk-slice accuracy by 6.4 percentage points, from 63.4% to 69.9% (p < .001), while global macro-F1 showed no observed degradation.

	## Model Details

	- Developed by: Matthew K. Weber
	- Model: DART-FT
	- Model type: BERT encoder with decomposed morphological classification heads
	- Base model: [`pranaydeeps/Ancient-Greek-BERT`](https://huggingface.co/pranaydeeps/Ancient-Greek-BERT)
	- Language: Ancient Greek (`grc`)
	- Primary task: Fine-grained morphological tagging
	- Framework: PyTorch / Hugging Face Transformers
	- Dataset: UD Ancient Greek PROIEL v2.12
	- Doctoral program: Doctor of Engineering, The George Washington University
	- Praxis: [Full text in GW ScholarSpace](https://scholarspace.library.gwu.edu/etd/4x51hk097)


	## What DART-FT Predicts

	DART-FT predicts fine-grained morphology using decomposed classification rather than treating every complete morphological bundle as a single class.

	The released schema contains a UPOS head and separate heads for 15 Universal Dependencies morphological attributes:

	- UPOS
	- Case
	- Gender
	- Number
	- Definite
	- PronType
	- Aspect
	- Tense
	- VerbForm
	- Voice
	- Degree
	- Mood
	- Person
	- Polarity
	- Reflex
	- Poss

	Individual predictions can then be reconstructed into a complete fine-grained morphological representation such as:

	```text
	NOUN\|Case=Acc\|Gender=Masc\|Number=Sing
	```

	This decomposition is intended to reduce the data-sparsity problems that arise when every possible combination of morphological features is represented as an independent class.

	---

	# DART-FT Methodology

	DART-FT combines three principal ideas:

	1. Decomposed morphological classification
	2. Empirical identification of a high-risk slice
	3. Risk-targeted fine-tuning through oversampling

	## 1. Decomposed Morphological Classification

	Instead of representing a complete FEATS bundle as one classification label, DART-FT decomposes each bundle into individual morphological attributes for independent per-feature classification.

	For example:

	```text
	Case=Nom
	Gender=Masc
	Number=Sing
	```

	are modeled as separate classification decisions.

	The model additionally predicts UPOS.

	This architecture reduces sparsity associated with treating the full Cartesian combination of morphological attributes as distinct labels.

	## 2. Empirical High-Risk Slice Identification

	The high-risk slice is not a predefined dictionary of difficult Ancient Greek words.

	Instead, it is identified empirically from recurrent errors made by a standard fine-tuned Ancient-Greek-BERT baseline.

	The methodology uses a three-pass process.

	### Pass 1 — Error-Driven Mining

	The baseline model predicts morphological labels on the validation set. Predicted feature-value pairs are compared with gold labels to identify recurrent error patterns.

	### Pass 2 — Feature-Value Filtering and Diversity Constraint

	Candidate feature-value pairs are filtered to isolate sufficiently frequent and systematic error patterns while preventing a small number of broad categories from dominating the high-risk slice.

	### Pass 3 — Slice Freezing and Mask Creation

	The resulting feature-value definition is frozen. A mask based on this frozen definition is then applied to held-out test data for consistent evaluation across models.

	The validation set is used to define the high-risk slice so that the test set remains held out from training and model-selection activities.

	Although morphological ambiguity motivates DART-FT, membership in the high-risk slice is determined by observed model error behavior at the feature-value level, rather than lexical identity alone.

	## 3. Risk-Targeted Fine-Tuning

	After the high-risk feature-value patterns are identified, DART-FT performs an additional stage of targeted fine-tuning.

	Training sentences containing tokens associated with the high-risk patterns are selectively oversampled, increasing model exposure to those difficult cases while preserving sentence-level context.

	The research evaluated:

	- oversampling factors of 2x, 3x, and 4x
	- five random seeds
	- cross-entropy loss
	- class-balanced loss
	- class-balanced + focal loss
	- multiple class-balanced β values
	- multiple focal-loss γ values

	A total of 90 distinct experimental models were evaluated.

	---

	# Released Model Configuration

	The model artifact in this repository corresponds to the highest-performing DART-FT configuration evaluated in the research:

	- Random seed: 43
	- Oversampling factor: 3x
	- Loss: Class-balanced
	- β: 0.993
	- Learning rate: 3 × 10⁻⁵
	- Batch size: 8
	- Training epochs: 5
	- Maximum sequence length: 512
	- Optimizer: AdamW
	- Weight decay: 0.01
	- Dropout: 0.1
	- Mixed precision: FP16

	The repository includes the fine-tuned Ancient-Greek-BERT encoder and separately stored decomposed classification heads.

	Important files include:

	```text
	model.safetensors
	heads.pt
	schema.json
	meta.json
	test_pred.conllu
	```

	### Repository Components

	`model.safetensors`
	Fine-tuned transformer encoder weights.

	`heads.pt`
	Trained decomposed morphological classification heads.

	`schema.json`
	Label inventory for UPOS and each morphological feature head.

	`meta.json`
	Metadata describing the released experimental configuration.

	`test_pred.conllu`
	Model predictions in CoNLL-U-compatible form.

	---

	# Training and Evaluation Data

	DART-FT was trained and evaluated using the Universal Dependencies Ancient Greek PROIEL treebank, version 2.12.

	The pre-existing PROIEL train, validation, and test splits were preserved.

	\| Split \| Sentences \| Tokens \| UPOS Tags \| UFEATS Attributes \|
	\|---\|---:\|---:\|---:\|---:\|
	\| Train \| 15,016 \| 187,039 \| 13 \| 15 \|
	\| Dev / Validation \| 1,019 \| 13,652 \| 13 \| 15 \|
	\| Test \| 1,047 \| 13,314 \| 13 \| 15 \|
	\| Total \| 17,082 \| 214,005 \| 13 \| 15 \|

	The Ancient Greek PROIEL corpus used in the study includes the 27 books of the Ancient Greek New Testament and portions of Herodotus.

	The New Testament portion of PROIEL itself contains approximately 140,000 tokens. This should not be confused with the full experimental dataset, which contains 214,005 tokens across the train, validation, and test splits.

	---

	# Tokenization and Label Alignment

	DART-FT uses the WordPiece tokenization employed by BERT.

	Because one source word can be divided into multiple subword tokens:

	- the first subword receives the corresponding UPOS and morphological labels;
	- subsequent subwords are excluded from the classification loss.

	This prevents a word that is split into multiple WordPieces from contributing duplicate morphological training labels and promotes learning at the word level.

	---

	# Evaluation

	DART-FT was evaluated from two complementary perspectives:

	1. Fine-grained morphological accuracy on the high-risk slice
	2. Global macro-F1

	## High-Risk Exact-Match Accuracy

	Fine-grained accuracy uses the complete morphological prediction for a word.

	The prediction must match the corresponding gold morphological representation for the token; partial corrections do not count as an exact match.

	## Macro-F1

	Macro-F1 gives classes equal weight rather than allowing highly frequent morphological categories to dominate the evaluation.

	Global macro-F1 was used specifically to evaluate whether targeted intervention on the high-risk slice caused broader model degradation.

	## Statistical Testing

	Paired baseline and DART-FT predictions were evaluated using the exact two-sided McNemar test.

	Confidence intervals were also calculated for the paired accuracy differences.

	---

	# Results

	## High-Risk Slice Performance

	For the highest-performing DART-FT model associated with each random seed:

	\| Seed \| Baseline Accuracy \| DART-FT Accuracy \| Change \| p-value \|
	\|---:\|---:\|---:\|---:\|---:\|
	\| 41 \| 59.3% \| 63.9% \| +4.5 pp \| < .001 \|
	\| 42 \| 65.6% \| 70.7% \| +5.2 pp \| .003 \|
	\| 43 \| 63.4% \| 69.9% \| +6.4 pp \| < .001 \|
	\| 44 \| 57.5% \| 63.2% \| +5.7 pp \| < .001 \|
	\| 45 \| 64.5% \| 69.8% \| +5.3 pp \| < .001 \|

	The largest observed improvement was produced by seed 43, 3x oversampling, class-balanced loss, β = 0.993:

	```text
	Baseline high-risk accuracy: 63.4%
	DART-FT high-risk accuracy: 69.9%
	Absolute improvement: +6.4 percentage points
	p-value: < .001
	95% confidence interval: 3.38–9.14 pp
	```

	All five seed comparisons exhibited statistically significant increases in high-risk-slice accuracy.

	## Global Model Stability

	Targeted fine-tuning did not produce an observed decrease in global macro-F1 for any of the five random seeds.

	\| Seed \| Baseline Global Macro-F1 \| DART-FT Global Macro-F1 \| Change \|
	\|---:\|---:\|---:\|---:\|
	\| 41 \| 0.921 \| 0.939 \| +0.018 \|
	\| 42 \| 0.919 \| 0.940 \| +0.021 \|
	\| 43 \| 0.917 \| 0.939 \| +0.022 \|
	\| 44 \| 0.919 \| 0.938 \| +0.019 \|
	\| 45 \| 0.920 \| 0.937 \| +0.017 \|

	The mean global macro-F1 improvement was +0.0194.

	These results support the central DART-FT objective: targeted repair of systematic fine-grained morphological tagging errors without observed degradation of overall model performance.

	## Error Analysis

	Error analysis showed that targeted fine-tuning corrected recurring errors across multiple morphological features.

	A particularly strong effect was observed for the `Gender=Masc,Neut` feature-value pair, where the standard baseline frequently predicted one of the more common individual gender categories rather than the syncretic `Masc,Neut` value.

	The research also identified localized regressions in some tightly coupled features, demonstrating that targeted fine-tuning can introduce tradeoffs even when aggregate high-risk and global performance improve.

	---

	# Oversampling Sensitivity

	DART-FT explicitly evaluated whether performance was sensitive to the amount of high-risk oversampling.

	The tested oversampling factors were:

	```text
	2x
	3x
	4x
	```

	Performance varied across seeds and configurations.

	Across the experiments, intermediate oversampling generally produced the strongest mean improvement, and the highest-performing individual configuration used 3x oversampling.

	This supports the conclusion that targeted fine-tuning effectiveness depends on the strength of the intervention and that simply maximizing oversampling is not necessarily optimal.

	---

	# Intended Uses

	## Fine-Grained Ancient Greek Morphological Tagging

	DART-FT can support prediction of grammatical information including:

	- part of speech
	- case
	- gender
	- number
	- tense
	- mood
	- voice
	- person
	- aspect
	- degree
	- pronoun type
	- definiteness
	- polarity
	- reflexivity
	- possession

	## Digital Humanities

	Potential applications include automated annotation and analysis of Ancient Greek corpora, including historical and biblical texts.

	## NLP Pipelines

	Fine-grained morphological predictions may provide structured linguistic information for downstream tasks such as:

	- syntactic parsing
	- corpus analysis
	- information extraction
	- search
	- experimental machine translation workflows

	Downstream benefits should be independently evaluated for each application.

	## Research on Targeted Model Repair

	DART-FT may also be useful as an example of error-driven targeted fine-tuning.

	The broader methodology identifies a reproducible model weakness through validation-set behavior and selectively increases exposure to corresponding training examples rather than indiscriminately increasing training across the entire corpus.

	---

	# Downstream Translation Pilot

	The praxis also included a small-scale exploratory Ancient Greek-to-Latin translation pilot.

	The pilot compared morphological information produced by the DART-FT model with the corresponding baseline model when supplied to a downstream translation system.

	The results provided limited evidence of downstream improvement:

	- chrF++: +0.24, p = 0.0408
	- TER: 59.42 → 59.27
	- for the subset in which baseline and DART-FT morphological sequences differed (`N = 232`), chrF++ improved by +0.37 and TER decreased by 0.23

	This pilot was secondary and exploratory. DART-FT should not be interpreted as a production machine-translation system, and the translation results should not be generalized beyond the limited experiment.

	---

	# Limitations

	## Corpus Scope

	DART-FT was trained and evaluated on the Ancient Greek PROIEL treebank, consisting principally of New Testament Greek and portions of Herodotus.

	Performance should not be assumed to transfer unchanged to:

	- Homeric Greek
	- Classical Attic Greek
	- Septuagint Greek
	- papyri
	- inscriptions
	- patristic Greek
	- Byzantine Greek
	- other Ancient Greek treebanks or annotation schemes

	## Generalizability

	Generalizability of the methodology to other languages and domains is plausible but was not empirically validated in the praxis.

	Applying DART-FT to another problem requires, at minimum:

	- a gold-labeled dataset with token-level annotations;
	- a pretrained model capable of task-specific fine-tuning; and
	- a finite, decomposable feature set suitable for identifying systematic feature-level errors.

	## High-Risk Slice Definition

	The high-risk slice is empirically defined and depends on methodological thresholds selected during the research.

	Different thresholds or filtering rules may result in different high-risk populations.

	## Model Dependence

	The high-risk slice reflects weaknesses exhibited by a particular model and dataset.

	Different pretrained models, datasets, or annotation systems may produce different concentrations of errors.

	## Oversampling Tradeoffs

	Oversampling increases model exposure to difficult examples but can also increase the risk of overfitting or localized negative transfer.

	DART-FT therefore treats oversampling strength as a hyperparameter rather than assuming that greater oversampling will always yield greater performance.

	---

	# How to Use

	> Important: The released DART-FT repository is not packaged as a conventional single-head Hugging Face `AutoModelForTokenClassification` checkpoint.

	DART-FT uses a BERT encoder plus separately stored decomposed morphological classification heads.

	Therefore, a call such as:

	```python
	AutoModelForTokenClassification.from_pretrained("zootsewt/DART-FT")
	```

	does not by itself reproduce the complete DART-FT inference architecture.

	The transformer encoder and tokenizer can be loaded with:

	```python
	from transformers import AutoTokenizer, AutoModel

	model_name = "zootsewt/DART-FT"

	tokenizer = AutoTokenizer.from_pretrained(model_name)
	encoder = AutoModel.from_pretrained(model_name)
	```

	The decomposed classification heads stored in:

	```text
	heads.pt
	```

	must also be loaded and applied according to the label inventory in:

	```text
	schema.json
	```

	The individual head predictions are then combined to reconstruct a complete fine-grained morphological analysis for each source token.

	A reference inference wrapper may be provided in a future repository update to automate loading of the encoder, classification heads, schema, token alignment, and output reconstruction.

	---

	# Research Context

	DART-FT was developed as the principal research contribution of the Doctor of Engineering praxis:

	_DART-FT: Decomposed, Ambiguity-Aware, Risk-Targeted Fine-Tuning of Transformer Models for Ancient Greek Morphological Tagging._ The George Washington University.

	### Read the full praxis

	https://scholarspace.library.gwu.edu/etd/4x51hk097



	---

	# Citation

	If you use DART-FT or the associated methodology in academic work, please cite the doctoral praxis:

	```bibtex
	@phdthesis{weber2026dartft,
	author = {Weber, Matthew K.},
	title = {DART-FT: Decomposed, Ambiguity-Aware, Risk-Targeted Fine-Tuning of Transformer Models for Ancient Greek Morphological Tagging},
	school = {The George Washington University},
	year = {2026},
	type = {Doctor of Engineering Praxis},
	url = {https://scholarspace.library.gwu.edu/etd/4x51hk097}
	}
	```

	Praxis:
	https://scholarspace.library.gwu.edu/etd/4x51hk097

	---

	# Disclaimer

	DART-FT is a research artifact developed and evaluated within the scope of a Doctor of Engineering praxis.

	Reported results reflect the specific datasets, baseline model, experimental design, high-risk slice definition, hyperparameters, and evaluation methods used in that research.

	Results should not be assumed to generalize unchanged to other corpora, models, languages, annotation systems, or downstream applications.