Spaces:
Running
Running
README.md: instructions refresh
Browse files
README.md
CHANGED
|
@@ -107,15 +107,29 @@ Per-origin thresholds (published `thresholds.json`): `fcv_pads_east_africa`
|
|
| 107 |
`ruling` is `DATA_MENTION` (keep) or `NON_MENTION` (drop) — same labels as the deck export.
|
| 108 |
## review.html (current)
|
| 109 |
|
| 110 |
-
Multi-span review over `rafmacalaba/data-use-ner`: `gliner` holdout (gold 473)
|
| 111 |
-
|
| 112 |
-
240 passages each,
|
| 113 |
-
|
| 114 |
-
|
| 115 |
-
|
| 116 |
-
|
| 117 |
-
|
| 118 |
-
|
| 119 |
-
|
| 120 |
-
|
| 121 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 107 |
`ruling` is `DATA_MENTION` (keep) or `NON_MENTION` (drop) — same labels as the deck export.
|
| 108 |
## review.html (current)
|
| 109 |
|
| 110 |
+
Multi-span review over `rafmacalaba/data-use-ner`: `gliner` holdout (gold 473),
|
| 111 |
+
`to_annotate` pool, per-user `annotate_{paddy,aj,aivin,rafael}[_part2]` sets
|
| 112 |
+
(240 passages each, mutually exclusive, stratified by origin × probe tag).
|
| 113 |
+
Assignment manifests live in `assignment_manifest.json` (verified equal to
|
| 114 |
+
the Hub configs — queue files are append-only; never resampled in place).
|
| 115 |
+
|
| 116 |
+
### For annotators
|
| 117 |
+
|
| 118 |
+
1. Open your link (or pick your name on the landing page) — your file loads
|
| 119 |
+
automatically. Type your name if asked; it locks on first ruling.
|
| 120 |
+
2. Judge each highlighted mention in its window: `→` next mention,
|
| 121 |
+
`k` keep, `d` drop, `space` skip, `[`/`]` slide the window,
|
| 122 |
+
`-`/`+` resize it. Optional freeform note per ruling.
|
| 123 |
+
3. Before ruling you see only `probe: easy` or `probe: confusion`. After
|
| 124 |
+
ruling, Luna's verdict + reasoning appear, with a DISAGREE flag (also a
|
| 125 |
+
toast) when you differ.
|
| 126 |
+
4. Every 10% you'll be nudged to **export** a checkpoint (`e` or the button,
|
| 127 |
+
header and footer). Send `rulings-<name>-<queue>.jsonl` to rafael/aivin.
|
| 128 |
+
To resume later, **import** the file back — you jump to the first
|
| 129 |
+
unreviewed mention.
|
| 130 |
+
|
| 131 |
+
### For coordinators
|
| 132 |
+
|
| 133 |
+
Drop returned files into `human_labeling/annotations/` and run `merge.py` →
|
| 134 |
+
majority verdict per span `key` (ties → REVIEW) in `annotations/adjudicated.jsonl`.
|
| 135 |
+
Public app: https://rafmacalaba-data-use-annotate.static.hf.space/
|