rafmacalaba commited on
Commit
f510dfc
·
verified ·
1 Parent(s): 5c68570

README.md: instructions refresh

Browse files
Files changed (1) hide show
  1. README.md +26 -12
README.md CHANGED
@@ -107,15 +107,29 @@ Per-origin thresholds (published `thresholds.json`): `fcv_pads_east_africa`
107
  `ruling` is `DATA_MENTION` (keep) or `NON_MENTION` (drop) — same labels as the deck export.
108
  ## review.html (current)
109
 
110
- Multi-span review over `rafmacalaba/data-use-ner`: `gliner` holdout (gold 473)
111
- and `to_annotate` (pool + per-user `annotate_{paddy,aj,aivin,rafael}` sets,
112
- 240 passages each, 24 shared for agreement). Typing your name auto-loads
113
- your file; annotating without a name is blocked.
114
- Keys: `←`/`→` mention · `k` keep · `d` drop · `space` skip ·
115
- `[`/`]` slide window · `-`/`+` window size. Rulings key by span `key`.
116
-
117
- Export flow (no logins, no tokens): annotators press **export** (or `e`),
118
- which downloads `rulings-<name>-<queue>.jsonl`. They send the file back;
119
- drop it into `human_labeling/annotations/` and run `merge.py` → majority
120
- verdict per key (ties → REVIEW) in `annotations/adjudicated.jsonl`.
121
- Public share: https://rafmacalaba-data-use-annotate.static.hf.space/
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
107
  `ruling` is `DATA_MENTION` (keep) or `NON_MENTION` (drop) — same labels as the deck export.
108
  ## review.html (current)
109
 
110
+ Multi-span review over `rafmacalaba/data-use-ner`: `gliner` holdout (gold 473),
111
+ `to_annotate` pool, per-user `annotate_{paddy,aj,aivin,rafael}[_part2]` sets
112
+ (240 passages each, mutually exclusive, stratified by origin × probe tag).
113
+ Assignment manifests live in `assignment_manifest.json` (verified equal to
114
+ the Hub configs — queue files are append-only; never resampled in place).
115
+
116
+ ### For annotators
117
+
118
+ 1. Open your link (or pick your name on the landing page) — your file loads
119
+ automatically. Type your name if asked; it locks on first ruling.
120
+ 2. Judge each highlighted mention in its window: `→` next mention,
121
+ `k` keep, `d` drop, `space` skip, `[`/`]` slide the window,
122
+ `-`/`+` resize it. Optional freeform note per ruling.
123
+ 3. Before ruling you see only `probe: easy` or `probe: confusion`. After
124
+ ruling, Luna's verdict + reasoning appear, with a DISAGREE flag (also a
125
+ toast) when you differ.
126
+ 4. Every 10% you'll be nudged to **export** a checkpoint (`e` or the button,
127
+ header and footer). Send `rulings-<name>-<queue>.jsonl` to rafael/aivin.
128
+ To resume later, **import** the file back — you jump to the first
129
+ unreviewed mention.
130
+
131
+ ### For coordinators
132
+
133
+ Drop returned files into `human_labeling/annotations/` and run `merge.py` →
134
+ majority verdict per span `key` (ties → REVIEW) in `annotations/adjudicated.jsonl`.
135
+ Public app: https://rafmacalaba-data-use-annotate.static.hf.space/