katrohrb commited on
Commit
c2e4ebf
·
verified ·
1 Parent(s): f821085

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +0 -12
README.md CHANGED
@@ -24,24 +24,12 @@ clf("She reached out and touched the rough bark.")
24
  # -> [{'label': 'haptic_material', ...}]
25
  ```
26
 
27
- The classifier takes the sentence alone - no surrounding context. Human annotators
28
- saw a ~200-word context window, so borderline cases the model gets wrong are often
29
- ones a reader would resolve from context.
30
 
31
  ## Performance
32
 
33
  Held-out test set (n=580): micro-F1 **0.805** [0.772, 0.835], macro-F1 0.797.
34
  5-fold CV: 0.808 +/- 0.022. Human inter-annotator agreement ceiling: Cohen's kappa 0.760.
35
 
36
- **Read before citing:** these are post-adjudication figures. A review pass revised
37
- 387 gold labels (~10% of the dataset), but it only surfaced cases where the model
38
- disagreed with the original label, so the gain over the pre-cleanup model
39
- (test 0.750, CV 0.755) is partly selection bias. Report both.
40
-
41
- Known limitation: `segmentation_error` is style-correlated - Dickens and Gissing
42
- top the per-book rate at 15-20% - so it is not ignorable noise in author-level
43
- aggregates.
44
-
45
  ## Data and code
46
 
47
  Codebook, gold annotations, and reproduction notebooks:
 
24
  # -> [{'label': 'haptic_material', ...}]
25
  ```
26
 
 
 
 
27
 
28
  ## Performance
29
 
30
  Held-out test set (n=580): micro-F1 **0.805** [0.772, 0.835], macro-F1 0.797.
31
  5-fold CV: 0.808 +/- 0.022. Human inter-annotator agreement ceiling: Cohen's kappa 0.760.
32
 
 
 
 
 
 
 
 
 
 
33
  ## Data and code
34
 
35
  Codebook, gold annotations, and reproduction notebooks: