Update README.md
Browse files
README.md
CHANGED
|
@@ -24,24 +24,12 @@ clf("She reached out and touched the rough bark.")
|
|
| 24 |
# -> [{'label': 'haptic_material', ...}]
|
| 25 |
```
|
| 26 |
|
| 27 |
-
The classifier takes the sentence alone - no surrounding context. Human annotators
|
| 28 |
-
saw a ~200-word context window, so borderline cases the model gets wrong are often
|
| 29 |
-
ones a reader would resolve from context.
|
| 30 |
|
| 31 |
## Performance
|
| 32 |
|
| 33 |
Held-out test set (n=580): micro-F1 **0.805** [0.772, 0.835], macro-F1 0.797.
|
| 34 |
5-fold CV: 0.808 +/- 0.022. Human inter-annotator agreement ceiling: Cohen's kappa 0.760.
|
| 35 |
|
| 36 |
-
**Read before citing:** these are post-adjudication figures. A review pass revised
|
| 37 |
-
387 gold labels (~10% of the dataset), but it only surfaced cases where the model
|
| 38 |
-
disagreed with the original label, so the gain over the pre-cleanup model
|
| 39 |
-
(test 0.750, CV 0.755) is partly selection bias. Report both.
|
| 40 |
-
|
| 41 |
-
Known limitation: `segmentation_error` is style-correlated - Dickens and Gissing
|
| 42 |
-
top the per-book rate at 15-20% - so it is not ignorable noise in author-level
|
| 43 |
-
aggregates.
|
| 44 |
-
|
| 45 |
## Data and code
|
| 46 |
|
| 47 |
Codebook, gold annotations, and reproduction notebooks:
|
|
|
|
| 24 |
# -> [{'label': 'haptic_material', ...}]
|
| 25 |
```
|
| 26 |
|
|
|
|
|
|
|
|
|
|
| 27 |
|
| 28 |
## Performance
|
| 29 |
|
| 30 |
Held-out test set (n=580): micro-F1 **0.805** [0.772, 0.835], macro-F1 0.797.
|
| 31 |
5-fold CV: 0.808 +/- 0.022. Human inter-annotator agreement ceiling: Cohen's kappa 0.760.
|
| 32 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 33 |
## Data and code
|
| 34 |
|
| 35 |
Codebook, gold annotations, and reproduction notebooks:
|