annotation review app (per-user queues, Hub-backed rulings, static-safe direct commit)

.dockerignore

@@ -0,0 +1,2 @@
+__pycache__/
+*.pyc

Dockerfile

@@ -0,0 +1,5 @@
+FROM python:3.14-slim
+RUN pip install --no-cache-dir huggingface_hub
+COPY . /app/human_labeling
+EXPOSE 7860
+CMD ["python", "human_labeling/app.py", "--port", "7860"]

README.md

@@ -1,10 +1,98 @@
----
-title: Data Use Annotate
-emoji: 🔥
-colorFrom: purple
-colorTo: pink
-sdk: static
-pinned: false
----
-
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# human_labeling
+
+Fork of `analysis/v24_sample_review/audit_deck.html` as a hostable
+annotation pipeline. Same blind KEEP/DROP interaction, but the queue
+comes from the Hub instead of an 11MB embedded `ROWS` array.
+
+- Source: `rafmacalaba/datause-ner` ([viewer](https://huggingface.co/datasets/rafmacalaba/datause-ner/viewer/probe_splits))
+  for passages; **scores are singlepass only** — the infer head in
+  `rafmacalaba/gliner-datause-catchall-singlepass`
+  (`sigmoid(head([start; end; mean; ±64 window]))` on inference-native features).
+  No v3/encode-head scores anywhere in this tool.
+- Default queue: `probe_candidates/pool` — the only config with an
+  embedded `passage`. `probe_splits` train/val/holdout rows have **no
+  passage** (surface + word offsets only), so they annotate surface-only
+  unless joined to a chunk corpus (see caveat below).
+- Annotations land in `human_labeling/annotations/rulings.jsonl`
+  (one JSON object per ruling). On static hosting (github.io) there is
+  no writer: the app exports a JSON file the annotator drops into that
+  folder (PR). On Spaces the bundled `app.py` accepts `POST
+  /api/rulings` and appends server-side.
+
+## Quick start (local)
+
+```bash
+uv run python human_labeling/build_gliner_queue.py            # Luna spans + singlepass scores
+uv run python human_labeling/app.py  # serves index.html + /api/queue + POST /api/rulings
+```
+
+Data source: the `gliner` config of rafmacalaba/datause-ner (tokenized
+passages + catch-all entity spans with camp2 Luna traceability). The queue
+maps each Luna verdict onto its span, rescores with the singlepass head
+(probe score) and the GLiNER proposer (extractor score @0.1), and the UI
+reviews keep/drop with both signals. Retired experiments (pool, passage
+grouping, gliner2/human473 queues) live only in git history.
+
+## Hosting
+
+- **github.io (static):** publish `human_labeling/` as-is. `index.html`
+  fetches `./queue_gliner.json` (built by `build_gliner_queue.py`, committed). Rulings
+  persist in `localStorage`; Export downloads
+  `rulings-<annotator>.json`, which is committed under `annotations/`
+  and reconciled with `merge.py`.
+- **Hugging Face Spaces (SDK: Docker, port 7860):** same folder plus
+  `app.py`. The frontend feature-detects `POST /api/rulings`: if it
+  200s, rulings stream to `annotations/rulings.jsonl` on disk; if not,
+  it falls back to export. Dockerfile provided.
+
+## Files
+
+| file | purpose |
+|---|---|
+| `index.html` | annotation UI (passage view, per-mention ruling, luna×model filter) |
+| `build_gliner_queue.py` | builds + scores `queue_gliner.json` from the gliner config (singlepass probe + extractor on MPS) |
+| `queue_gliner.json` | served queue: 225 passages / 522 Luna-labeled spans, round-robin over origins, multi-mention first |
+| `probe_labels.py` | shared decision rule + per-origin singlepass thresholds |
+| `app.py` | stdlib server: static + `GET /api/queue` + `POST /api/rulings` |
+| `merge.py` | `annotations/rulings*.jsonl` → `annotations/adjudicated.jsonl` (majority per key) |
+| `annotations/` | ruling storage; every file is JSONL, one ruling per line |
+| `Dockerfile` | Spaces deploy (Docker SDK) |
+| `build_gliner2_queue.py`, `build_pool_queue.py`, `rescore_singlepass.py`, `build_passage_queue.py`, `build_gold_queue.py` | retired experiments (kept for reference; not served) |
+
+## Caveat: `probe_splits` has no context
+
+`probe_train/val/holdout.jsonl` rows are
+`key, origin, corpus_id, w_start, w_end, surface, label, source,
+head_score, split` — no passage text. The underlying chunk text lives
+in per-corpus repos (`rafmacalaba/fcv-extractions-meta[-tiered]`), and
+`general_prwp` (the majority origin) has no published chunk source at
+all (same gap as `demo/server.py: HOLDOUT_META_SOURCES`). So:
+
+- `build_pool_queue.py --config probe_candidates` (default): full passage, blind ruling quality = audit_deck.
+- `build_pool_queue.py --config probe_splits`: surface-only triage; items are
+  flagged `"ctx_missing": true` and the UI shows the surface + scores
+  without a context card. Fine for obvious keeps/drops, not for adjudication.
+
+## Bands (decision labels)
+
+Every queue item carries `band`, tagged from the singlepass score with the
+bundle's per-origin best-F1 thresholds (`probe_labels.py`):
+
+| band | rule |
+|---|---|
+| `keep` | `head_score >= per-origin threshold` — model-flagged keep, quick confirm |
+| `confusion` | `DROP_FLOOR < head_score < threshold` — the review zone (was "mid") |
+| `drop` | `head_score <= 0.05` — confident reject, quick confirm |
+| `unscored` | score missing (out-of-grid span) |
+
+Per-origin thresholds (published `thresholds.json`): `fcv_pads_east_africa`
+0.5 · `general_prwp` 0.4 · `jad_paddy_docs` 0.1 · `jdc_operational` 0.6 ·
+`refugee_pads` 0.5 · `reliefweb` 0.4; fallback `global_best` 0.5.
+
+## Ruling schema
+
+```json
+{"key": "sample:fcv_pads_east_africa:002082:7:1:0", "ruling": "DATA_MENTION",
+ "annotator": "rm", "ts": "2026-09-08T12:00:00Z", "note": "",
+ "head_score": 0.61, "origin": "fcv_pads_east_africa", "queue": "probe_candidates/pool"}
+`ruling` is `DATA_MENTION` (keep) or `NON_MENTION` (drop) — same labels as the deck export.

annotations/.gitkeep

@@ -0,0 +1 @@
+.gitkeep

app.py

@@ -0,0 +1,122 @@
+#!/usr/bin/env python3
+"""Static server + ruling sink for human_labeling.
+
+Serves index.html / queue.json and appends POSTed rulings to
+annotations/rulings.jsonl. Stdlib only. The frontend feature-detects
+POST /api/rulings: static hosts (github.io) 404 it and fall back to
+file export; Spaces (Dockerfile) runs this file so writes land on disk.
+
+    uv run python human_labeling/app.py [--port 7860]
+"""
+
+import argparse
+import datetime
+import http.server
+import json
+import socketserver
+from pathlib import Path
+HERE = Path(__file__).resolve().parent
+ANNOT = HERE / "annotations"
+RULINGS = ANNOT / "rulings.jsonl"
+ANN_REPO = "rafmacalaba/data-use-annotations"
+_hub_lines = None
+
+class Handler(http.server.SimpleHTTPRequestHandler):
+    def __init__(self, *a, **kw):
+        super().__init__(*a, directory=str(HERE), **kw)
+
+    def _json(self, obj, status=200):
+        body = json.dumps(obj).encode()
+        self.send_response(status)
+        self.send_header("Content-Type", "application/json")
+        self.send_header("Content-Length", str(len(body)))
+        self.end_headers()
+        self.wfile.write(body)
+
+    def do_GET(self):
+        if self.path == "/api/queue":
+            items = []
+            for name in ("queue_gliner.json",):
+                p = HERE / name
+                if p.exists():
+                    items.extend(json.loads(r) for r in p.read_text().splitlines() if r.strip())
+            return self._json({"items": items})
+        if self.path == "/api/health":
+            return self._json({"ok": True, "rulings": _count()})
+        return super().do_GET()
+
+    def do_POST(self):
+        if self.path != "/api/rulings":
+            return self._json({"error": "not found"}, 404)
+        try:
+            n = int(self.headers.get("Content-Length", 0))
+        except ValueError:
+            return self._json({"error": "bad length"}, 400)
+        try:
+            ruling = json.loads(self.rfile.read(n) or b"{}")
+        except json.JSONDecodeError:
+            return self._json({"error": "bad json"}, 400)
+        if not ruling.get("key") or ruling.get("ruling") not in ("DATA_MENTION", "NON_MENTION"):
+            return self._json({"error": "need key + ruling"}, 400)
+        if not (ruling.get("annotator") or "").strip():
+            return self._json({"error": "need annotator name"}, 400)
+        ruling["annotator"] = ruling["annotator"].strip()
+        ruling.setdefault("ts", datetime.datetime.now(datetime.timezone.utc).isoformat())
+        ANNOT.mkdir(exist_ok=True)
+        with open(RULINGS, "a") as f:
+            f.write(json.dumps(ruling) + "\n")
+        pushed = _push_hub(ruling)
+        return self._json({"ok": True, "n": _count(), "pushed": pushed})
+
+
+def _count() -> int:
+    if not RULINGS.exists():
+        return 0
+    with open(RULINGS) as f:
+        return sum(1 for line in f if line.strip())
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--port", type=int, default=7860)
+    a = ap.parse_args()
+    with socketserver.TCPServer(("", a.port), Handler) as httpd:
+        print(f"human_labeling on :{a.port} (rulings -> {RULINGS})")
+        httpd.serve_forever()
+def _push_hub(ruling) -> bool:
+    """Merge one ruling into the annotator's public Hub file
+    (rulings/<annotator>.jsonl). Last write per (queue, key) wins.
+    Local file stays source of truth; Hub failures never fail."""
+    global _hub_lines
+    try:
+        from huggingface_hub import HfApi, hf_hub_download
+        api = HfApi()
+        who = "".join(c if c.isalnum() or c in "-_" else "_" for c in ruling["annotator"])
+        path = f"rulings/{who}.jsonl"
+        if _hub_lines is None:
+            _hub_lines = {}
+        if who not in _hub_lines:
+            try:
+                p = hf_hub_download(ANN_REPO, path, repo_type="dataset")
+                with open(p) as f:
+                    _hub_lines[who] = [json.loads(l) for l in f if l.strip()]
+            except Exception:
+                _hub_lines[who] = []
+        rows = [r for r in _hub_lines[who]
+                if not (r.get("queue") == ruling.get("queue")
+                        and r.get("key") == ruling.get("key"))]
+        rows.append(ruling)
+        _hub_lines[who] = rows
+        api.upload_file(
+            path_or_fileobj="\n".join(json.dumps(r) for r in rows).encode(),
+            path_in_repo=path, repo_id=ANN_REPO, repo_type="dataset",
+            commit_message=f"ruling {who}: {ruling.get('key')}={ruling.get('ruling')}",
+        )
+        return True
+    except Exception as exc:  # noqa: BLE001
+        print(f"hub push failed: {exc}")
+        return False
+
+
+if __name__ == "__main__":
+    main()

build_gliner2_queue.py

@@ -0,0 +1,195 @@
+#!/usr/bin/env python3
+"""Build the gliner2 adjudication queue: Luna-labeled spans rescored with
+the singlepass bundle (rafmacalaba/gliner-datause-catchall-singlepass).
+
+Source: the gliner2 config of rafmacalaba/datause-ner (passage rows with
+spans[]: text, luna_label, char start/end, key). Local mirrors in
+hf_datause_ner/gliner2_*.jsonl are byte-identical to the Hub (verified:
+29,346/29,346 keys) and used as the fetch source.
+
+Per mention the queue carries:
+    luna            camp2 verdict, 1=keep / 0=drop
+    head_score      probe_score from the singlepass infer head (MPS)
+    extractor_score GLiNER proposer score @0.1 for the same grid cell
+                    (null when the proposer didn't fire on the span)
+    band            keep/confusion/drop via probe_labels.decide
+
+Sampling: round-robin over origins, multi-mention passages first, span
+budget --limit (default 520). Overlapping duplicate spans are kept as
+separate mentions (the UI renderer suppresses in-text doubles).
+
+    uv run python human_labeling/build_gliner2_queue.py [--limit 520] [--batch 8]
+"""
+
+import argparse
+import json
+import sys
+from collections import defaultdict
+from pathlib import Path
+
+HERE = Path(__file__).resolve().parent
+REPO = HERE.parent
+sys.path.insert(0, str(REPO))
+
+MIRROR = REPO / "hf_datause_ner"
+OUT = HERE / "queue_gliner2.json"
+
+
+def load_passages() -> list[dict]:
+    rows = []
+    for split in ("train", "val", "holdout"):
+        for line in (MIRROR / f"gliner2_{split}.jsonl").read_text().splitlines():
+            if line.strip():
+                r = json.loads(line)
+                r["split"] = split
+                rows.append(r)
+    return rows
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--model", default="rafmacalaba/gliner-datause-catchall-singlepass")
+    ap.add_argument("--limit", type=int, default=520, help="span budget")
+    ap.add_argument("--batch", type=int, default=8)
+    a = ap.parse_args()
+
+    import torch
+    from training.singlepass_infer import default_device, load_bundle
+    from training.probe_features_infer import char_to_infer_word
+    from probe_labels import decide
+
+    passages = load_passages()
+    # round-robin origins, multi-mention passages first within each origin
+    by_origin: dict[str, list[dict]] = defaultdict(list)
+    for p in passages:
+        by_origin[p["origin"]].append(p)
+    ordered: list[dict] = []
+    origins = sorted(by_origin)
+    for o in origins:
+        by_origin[o].sort(key=lambda p: -min(len(p["spans"]), 2))
+    i = 0
+    while any(by_origin[o] for o in origins):
+        o = origins[i % len(origins)]
+        if by_origin[o]:
+            ordered.append(by_origin[o].pop(0))
+        i += 1
+
+    # take passages until span budget; dedupe exact-duplicate keys in passage
+    chosen, n_spans = [], 0
+    for p in ordered:
+        if n_spans >= a.limit:
+            break
+        seen, spans = set(), []
+        for s in p["spans"]:
+            if s["key"] in seen:
+                continue
+            seen.add(s["key"])
+            spans.append(s)
+        chosen.append((p, spans))
+        n_spans += len(spans)
+
+    device = default_device()
+    print(f"device={device} model={a.model} passages={len(chosen)} spans={n_spans}",
+          flush=True)
+    model, head, bundle = load_bundle(
+        "rafmacalaba/gliner-datause-mentions-catch-all", a.model, device)
+    thresholds = bundle.get("thresholds") or {}
+    radius = bundle["radius"]
+
+    import torch.utils.data
+    INFER_LABELS = ["DATA_MENTION"]
+    texts = [p["input"] for p, _ in chosen]
+    prepared = model.prepare_batch(texts, INFER_LABELS)
+    collator = model.create_collator()
+
+    def collate_fn(batch):
+        return model.collate_batch(batch, prepared["entity_types"], collator)
+
+    loader = torch.utils.data.DataLoader(
+        prepared["input_x"], batch_size=a.batch, shuffle=False,
+        collate_fn=collate_fn)
+
+    v2o = prepared["valid_to_orig_idx"]
+    o2v = {o: v for v, o in enumerate(v2o)}
+    n_probe = n_ext = n_skip = 0
+    items: list[dict] = []
+
+    def flush(p, spans, probes, extractors):
+        mentions = []
+        for s, probe, ext in zip(spans, probes, extractors):
+            mentions.append({
+                "key": s["key"], "surface": s["text"],
+                "start": s["start"], "end": s["end"],
+                "luna": s.get("luna_label"),
+                "head_score": probe, "extractor_score": ext,
+                "band": decide(probe, p["origin"], thresholds),
+            })
+        bands = {m["band"] for m in mentions}
+        pband = ("unscored" if "unscored" in bands else
+                 "confusion" if "confusion" in bands else
+                 "mixed" if len(bands) > 1 else bands.pop())
+        items.append({
+            "queue": "gliner2", "origin": p["origin"], "split": p["split"],
+            "ctx": p["input"], "n": len(mentions), "band": pband,
+            "mentions": mentions, "scored_by": a.model,
+        })
+
+    row = 0
+    with torch.no_grad():
+        for batch in loader:
+            out = model.run_batch(batch, threshold=0.1, move_to_device=True)
+            W = out.words_embedding.detach().float()
+            mask = (out.mask.detach().cpu()
+                    if getattr(out, "mask", None) is not None else None)
+            decoded = model.decode_batch(out, batch, threshold=0.1,
+                                         flat_ner=True, multi_label=False)
+            B = W.shape[0]
+            for bi in range(B):
+                vi = row + bi
+                oi = v2o[vi]
+                p, spans = chosen[oi]
+                if o2v.get(oi) is None:  # empty passage filtered upstream
+                    flush(p, spans, [None] * len(spans), [None] * len(spans))
+                    n_skip += len(spans)
+                    continue
+                w = W[bi].to(device)
+                L = int(mask[bi].sum()) if mask is not None else w.shape[0]
+                starts = prepared["start_token_map"][vi]
+                proposals = [(int(sp.start), int(sp.end), float(sp.score))
+                             for sp in decoded[bi]]
+                probes, extractors = [], []
+                for s in spans:
+                    cs, ce = s["start"], s["end"]
+                    g0, g1 = char_to_infer_word(starts, cs, ce)
+                    probe = ext = None
+                    if g1 < L and g0 < L:
+                        idx = torch.arange(g0, g1 + 1, device=device)
+                        parts = [w[g0], w[g1], w[idx].mean(dim=0)]
+                        if radius > 0:
+                            w0, w1 = max(0, g0 - radius), min(g1 + radius, L - 1)
+                            parts.append(w[w0:w1 + 1].mean(dim=0))
+                        probe = float(torch.sigmoid(
+                            head(torch.cat(parts).unsqueeze(0))).item())
+                        n_probe += 1
+                        hit = [sc for (ps, pe, sc) in proposals
+                               if ps == g0 and pe == g1]
+                        if hit:
+                            ext = hit[0]
+                            n_ext += 1
+                    probes.append(probe)
+                    extractors.append(ext)
+                flush(p, spans, probes, extractors)
+            row += B
+
+    OUT.write_text("\n".join(json.dumps(it) for it in items) + "\n")
+    from collections import Counter
+    bands = Counter(m["band"] for it in items for m in it["mentions"])
+    lu = Counter(m["luna"] for it in items for m in it["mentions"])
+    print(f"queue: passages={len(items)} spans={sum(it['n'] for it in items)} "
+          f"probe_scored={n_probe} extractor_matched={n_ext} "
+          f"unscored={n_skip} bands={dict(bands)} luna={dict(lu)} -> {OUT}",
+          flush=True)
+
+
+if __name__ == "__main__":
+    main()

build_gliner_queue.py

@@ -0,0 +1,218 @@
+#!/usr/bin/env python3
+"""Build the annotation queue from the gliner config of
+rafmacalaba/datause-ner, with camp2 Luna verdicts mapped onto every
+entity span and scores from the singlepass bundle
+(rafmacalaba/gliner-datause-catchall-singlepass).
+
+gliner rows: tokenized_text + ner (catch-all DATA_MENTION token spans)
++ spans[] traceability (key, luna_label, text, source). Join is positional:
+ner[i] <-> spans[i] (keys are sequential per passage; verified).
+
+ctx = ' '.join(tokenized_text); char offsets are computed on that grid, so
+they are exact by construction. Per mention:
+    luna            camp2 verdict, 1=keep / 0=drop
+    head_score      probe_score from the singlepass infer head (MPS)
+    extractor_score GLiNER proposer score @0.1 matched on the inference grid
+    band            keep/confusion/drop via probe_labels.decide
+
+Sampling: round-robin over origins, multi-mention passages first, span
+budget --limit (default 520).
+
+    uv run python human_labeling/build_gliner_queue.py [--limit 520] [--batch 8]
+"""
+
+import argparse
+import json
+import sys
+from collections import defaultdict
+from pathlib import Path
+
+HERE = Path(__file__).resolve().parent
+REPO = HERE.parent
+sys.path.insert(0, str(REPO))
+
+MIRROR = REPO / "hf_datause_ner"
+OUT = HERE / "queue_gliner.json"
+
+
+def token_char_offsets(tokens: list[str]) -> list[int]:
+    offs, c = [], 0
+    for t in tokens:
+        offs.append(c)
+        c += len(t) + 1
+    return offs
+
+
+def load_passages() -> list[dict]:
+    rows = []
+    for split in ("train", "val", "holdout"):
+        for line in (MIRROR / f"gliner_{split}.jsonl").read_text().splitlines():
+            if line.strip():
+                r = json.loads(line)
+                r["split"] = split
+                rows.append(r)
+    return rows
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--model", default="rafmacalaba/gliner-datause-catchall-singlepass")
+    ap.add_argument("--limit", type=int, default=520, help="span budget")
+    ap.add_argument("--batch", type=int, default=8)
+    a = ap.parse_args()
+
+    import torch
+    import torch.utils.data
+    from training.singlepass_infer import default_device, load_bundle
+    from training.probe_features_infer import char_to_infer_word
+    from probe_labels import decide
+
+    passages = load_passages()
+    by_origin: dict[str, list[dict]] = defaultdict(list)
+    for p in passages:
+        if len(p.get("ner", [])) == len(p.get("spans", [])) and p["ner"]:
+            by_origin[p["origin"]].append(p)
+    origins = sorted(by_origin)
+    for o in origins:
+        by_origin[o].sort(key=lambda p: -min(len(p["ner"]), 2))
+    ordered: list[dict] = []
+    i = 0
+    while any(by_origin[o] for o in origins):
+        o = origins[i % len(origins)]
+        if by_origin[o]:
+            ordered.append(by_origin[o].pop(0))
+        i += 1
+
+    chosen, n_spans = [], 0
+    for p in ordered:
+        if n_spans >= a.limit:
+            break
+        # dedupe identical spans (upstream extractor proposed the same span
+        # twice under separate keys; 173 groups corpus-wide, 22 with
+        # CONFLICTING luna verdicts). First key wins; conflicting duplicates
+        # flag the survivor as luna_split for the UI.
+        first, ner_u, spans_u = {}, [], []
+        for i, (ner, s) in enumerate(zip(p["ner"], p["spans"])):
+            k = (ner[0], ner[1])
+            if k not in first:
+                first[k] = len(spans_u)
+                ner_u.append(ner)
+                spans_u.append(dict(s))
+            elif spans_u[first[k]].get("luna_label") != s.get("luna_label"):
+                spans_u[first[k]]["luna_split"] = True
+        q = dict(p, ner=ner_u, spans=spans_u)
+        chosen.append(q)
+        n_spans += len(ner_u)
+    device = default_device()
+    print(f"device={device} model={a.model} passages={len(chosen)} spans={n_spans}",
+          flush=True)
+    model, head, bundle = load_bundle(
+        "rafmacalaba/gliner-datause-mentions-catch-all", a.model, device)
+    thresholds = bundle.get("thresholds") or {}
+    radius = bundle["radius"]
+
+    INFER_LABELS = ["DATA_MENTION"]
+    texts, char_maps = [], []
+    for p in chosen:
+        toks = p["tokenized_text"]
+        texts.append(" ".join(toks))
+        char_maps.append(token_char_offsets(toks))
+    prepared = model.prepare_batch(texts, INFER_LABELS)
+    collator = model.create_collator()
+
+    def collate_fn(batch):
+        return model.collate_batch(batch, prepared["entity_types"], collator)
+
+    loader = torch.utils.data.DataLoader(
+        prepared["input_x"], batch_size=a.batch, shuffle=False,
+        collate_fn=collate_fn)
+
+    v2o = prepared["valid_to_orig_idx"]
+    n_probe = n_ext = n_skip = 0
+    items: list[dict] = []
+
+    def flush(p, ctx, char_offs, probes, extractors):
+        mentions = []
+        for i, (ner, s, probe, ext) in enumerate(
+                zip(p["ner"], p["spans"], probes, extractors)):
+            t0, t1 = ner[0], ner[1]
+            start = char_offs[t0]
+            end = char_offs[t1] + len(p["tokenized_text"][t1])
+            mentions.append({
+                "key": s["key"], "surface": " ".join(p["tokenized_text"][t0:t1 + 1]),
+                "start": start, "end": end,
+                "luna": s.get("luna_label"), "luna_split": s.get("luna_split", False),
+                "head_score": probe, "extractor_score": ext,
+                "band": decide(probe, p["origin"], thresholds),
+            })
+        bands = {m["band"] for m in mentions}
+        pband = ("unscored" if "unscored" in bands else
+                 "confusion" if "confusion" in bands else
+                 "mixed" if len(bands) > 1 else bands.pop())
+        items.append({
+            "queue": "gliner", "origin": p["origin"], "split": p["split"],
+            "ctx": ctx, "n": len(mentions), "band": pband,
+            "mentions": mentions, "scored_by": a.model,
+        })
+
+    row = 0
+    with torch.no_grad():
+        for batch in loader:
+            out = model.run_batch(batch, threshold=0.1, move_to_device=True)
+            W = out.words_embedding.detach().float()
+            mask = (out.mask.detach().cpu()
+                    if getattr(out, "mask", None) is not None else None)
+            decoded = model.decode_batch(out, batch, threshold=0.1,
+                                         flat_ner=True, multi_label=False)
+            B = W.shape[0]
+            for bi in range(B):
+                vi = row + bi
+                oi = v2o[vi]
+                p = chosen[oi]
+                if oi not in set(v2o):  # unreachable; v2o IS the valid map
+                    continue
+                w = W[bi].to(device)
+                L = int(mask[bi].sum()) if mask is not None else w.shape[0]
+                starts = prepared["start_token_map"][vi]
+                proposals = [(int(sp.start), int(sp.end), float(sp.score))
+                             for sp in decoded[bi]]
+                probes, extractors = [], []
+                for ner in p["ner"]:
+                    # token grid -> char grid -> inference word grid
+                    t0, t1 = ner[0], ner[1]
+                    char_offs = char_maps[oi]
+                    cs = char_offs[t0]
+                    ce = char_offs[t1] + len(p["tokenized_text"][t1])
+                    g0, g1 = char_to_infer_word(starts, cs, ce)
+                    probe = ext = None
+                    if g1 < L and g0 < L:
+                        idx = torch.arange(g0, g1 + 1, device=device)
+                        parts = [w[g0], w[g1], w[idx].mean(dim=0)]
+                        if radius > 0:
+                            w0, w1 = max(0, g0 - radius), min(g1 + radius, L - 1)
+                            parts.append(w[w0:w1 + 1].mean(dim=0))
+                        probe = float(torch.sigmoid(
+                            head(torch.cat(parts).unsqueeze(0))).item())
+                        n_probe += 1
+                        hit = [sc for (ps, pe, sc) in proposals
+                               if ps == g0 and pe == g1]
+                        if hit:
+                            ext = hit[0]
+                            n_ext += 1
+                    probes.append(probe)
+                    extractors.append(ext)
+                flush(p, texts[oi], char_maps[oi], probes, extractors)
+            row += B
+
+    OUT.write_text("\n".join(json.dumps(it) for it in items) + "\n")
+    from collections import Counter
+    bands = Counter(m["band"] for it in items for m in it["mentions"])
+    lu = Counter(m["luna"] for it in items for m in it["mentions"])
+    print(f"queue: passages={len(items)} spans={sum(it['n'] for it in items)} "
+          f"probe_scored={n_probe} extractor_matched={n_ext} "
+          f"unscored={n_skip} bands={dict(bands)} luna={dict(lu)} -> {OUT}",
+          flush=True)
+
+
+if __name__ == "__main__":
+    main()

build_gold_queue.py

@@ -0,0 +1,85 @@
+#!/usr/bin/env python3
+"""Build the gold-controls queue from local human-adjudicated spans.
+
+Joins outputs/gliner_datause_v3_probe_human473.jsonl (key, set, surface)
+to analysis/v24_sample_review/gliner2_review.jsonl
+(spans[].key -> input passage, start/end, origin). Scores are the
+singlepass infer-head H100 predictions
+(outputs/gliner-datause-catchall-infer-probe/human473_predictions.jsonl);
+truth labels are deliberately excluded (blind controls).
+Splits: annotator190 (set=annotator) + jdc283 (set=jdc).
+
+    uv run python human_labeling/build_gold_queue.py [--out human_labeling/queue_human473.json]
+"""
+
+import argparse
+import json
+import sys
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).resolve().parent))
+
+from probe_labels import decide  # noqa: E402
+
+REPO = Path(__file__).resolve().parent.parent
+HUMAN = REPO / "outputs" / "gliner_datause_v3_probe_human473.jsonl"
+REVIEW = REPO / "analysis" / "v24_sample_review" / "gliner2_review.jsonl"
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--out", default=str(REPO / "human_labeling" / "queue_human473.json"))
+    a = ap.parse_args()
+
+    # singlepass (infer-head) rescore, published H100 predictions; 23 of 473
+    # out-of-grid spans absent here -> head_score None, band "unscored"
+    SP = REPO / "outputs" / "gliner-datause-catchall-infer-probe" / "human473_predictions.jsonl"
+    sp = {}
+    if SP.exists():
+        for line in SP.read_text().splitlines():
+            if line.strip():
+                o = json.loads(line)
+                sp[o["key"]] = float(o.get("head_score"))
+    # review spans by key -> (passage, start, end, origin)
+    ctx: dict[str, tuple] = {}
+    for line in REVIEW.read_text().splitlines():
+        if not line.strip():
+            continue
+        row = json.loads(line)
+        for s in row.get("spans", []):
+            if s.get("key"):
+                ctx[s["key"]] = (row.get("input", ""), s.get("start"),
+                                 s.get("end"), row.get("origin"))
+    items: list[dict] = []
+    missing: list[str] = []
+    from collections import Counter
+    for line in HUMAN.read_text().splitlines():
+        if not line.strip():
+            continue
+        r = json.loads(line)
+        key = r["key"]
+        subset = "annotator190" if r.get("set") == "annotator" else "jdc283"
+        score = sp.get(key)  # singlepass infer-head (H100); None = out-of-grid
+        if key in ctx:
+            passage, start, end, origin = ctx[key]
+            items.append({
+                "key": key, "surface": r.get("surface"), "ctx": passage,
+                "ctx_missing": False, "head_score": score,
+                "start": start, "end": end,
+                "band": decide(score, origin), "origin": origin,
+                "specificity": "", "split": "holdout",
+                "queue": "human473", "subset": subset,
+                "scored_by": "rafmacalaba/gliner-datause-catchall-infer-probe (H100)",
+            })
+        else:
+            missing.append(key)
+    Path(a.out).write_text("\n".join(json.dumps(i) for i in items) + "\n")
+    print(f"queued={len(items)} missing_ctx={len(missing)} "
+          f"scored={sum(1 for i in items if i['head_score'] is not None)} "
+          f"{Counter(i['subset'] for i in items)} -> {a.out}")
+    if missing:
+        print("missing:", missing[:5])
+
+
+if __name__ == "__main__":
+    main()

build_passage_queue.py

@@ -0,0 +1,82 @@
+#!/usr/bin/env python3
+"""Group scored flat pool items into passage examples (multi-mention).
+
+Reads the flat rescored queue (build_pool_queue.py → rescore_singlepass.py)
+and emits one JSON object per distinct (origin, ctx):
+
+    {queue, origin, ctx, n, band, scored_by,
+     mentions: [{key, surface, start, end, head_score, band}...]}   # by start
+
+Passages with one mention stay as-is; multi stays multi — the UI drives
+them as one example with a per-mention ruling target. The passage-level
+band is the most-informative mention state:
+
+    unscored  any unscored
+    confusion any confusion
+    mixed     keep + drop, no confusion
+    else      the single uniform band
+
+    uv run python human_labeling/build_passage_queue.py \
+        [--input human_labeling/queue.json] [--out human_labeling/queue_passages.json]
+"""
+
+import argparse
+import json
+from collections import defaultdict
+from pathlib import Path
+
+HERE = Path(__file__).resolve().parent
+
+
+def passage_band(mentions: list[dict]) -> str:
+    bands = {m.get("band") for m in mentions}
+    if "unscored" in bands:
+        return "unscored"
+    if "confusion" in bands:
+        return "confusion"
+    if len(bands) == 1:
+        return bands.pop()
+    return "mixed"
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--input", default=str(HERE / "queue.json"))
+    ap.add_argument("--out", default=str(HERE / "queue_passages.json"))
+    a = ap.parse_args()
+
+    groups: dict[tuple, list[dict]] = defaultdict(list)
+    for line in Path(a.input).read_text().splitlines():
+        if not line.strip():
+            continue
+        r = json.loads(line)
+        if r.get("ctx"):
+            groups[(r.get("origin"), r["ctx"])].append(r)
+
+    passages = []
+    for (origin, ctx), rows in groups.items():
+        mentions = sorted(
+            ({"key": r["key"], "surface": r.get("surface"),
+              "start": r.get("start"), "end": r.get("end"),
+              "head_score": r.get("head_score"), "band": r.get("band")}
+             for r in rows),
+            key=lambda m: (m["start"] if isinstance(m["start"], int) else 0,
+                           m["surface"] or ""))
+        passages.append({
+            "queue": "probe_candidates/passages",
+            "origin": origin, "ctx": ctx, "n": len(mentions),
+            "band": passage_band(mentions), "mentions": mentions,
+            "scored_by": rows[0].get("scored_by"),
+        })
+
+    passages.sort(key=lambda p: (p["origin"] or "", p["ctx"] or ""))
+    Path(a.out).write_text(
+        "\n".join(json.dumps(p) for p in passages) + "\n")
+    n_multi = sum(1 for p in passages if p["n"] > 1)
+    from collections import Counter
+    print(f"passages={len(passages)} multi={n_multi} "
+          f"{Counter(p['band'] for p in passages)} -> {a.out}")
+
+
+if __name__ == "__main__":
+    main()

build_pool_queue.py

@@ -0,0 +1,127 @@
+#!/usr/bin/env python3
+"""Build an annotation queue from rafmacalaba/datause-ner via datasets-server.
+
+Stdlib only (urllib) so it runs anywhere, including Spaces builds.
+
+    uv run python human_labeling/build_pool_queue.py [--config probe_candidates] [--split pool]
+        [--limit 500] [--out human_labeling/queue.json]
+
+Defaults target the annotatable config: probe_candidates/pool is the only
+one with an embedded `passage`. --config probe_splits yields surface-only
+triage items flagged ctx_missing (no passage on the Hub; see README).
+Stratified sample: round-robin over origin x score-band so the queue
+covers keep/confusion/drop and every origin instead of the head-heavy
+random draw (proxy bands on the Hub score; true tags come from rescore).
+"""
+
+import argparse
+import json
+import urllib.parse
+import urllib.request
+from pathlib import Path
+
+REPO = Path(__file__).resolve().parent
+DATASET = "rafmacalaba/datause-ner"
+API = "https://datasets-server.huggingface.co/rows"
+
+BANDS = (("drop", 0.0, 0.05), ("confusion", 0.05, 0.9), ("keep", 0.9, 1.01))
+# Proxy strata on the Hub (v3-era) score, used ONLY to stratify the sample
+# across the eventual decision zones (same 0.05/0.9 edges as the audit
+# deck). True keep/confusion/drop labels are assigned later by
+# rescore_singlepass.probe_labels.decide; the written item band is always
+# "unscored" until then.
+def band_of(score: float) -> str:
+    for name, lo, hi in BANDS:
+        if lo <= score < hi:
+            return name
+    return "confusion"
+
+
+def fetch_rows(config: str, split: str, offset: int, length: int) -> list[dict]:
+    import time
+    import urllib.error
+    q = urllib.parse.urlencode(
+        {"dataset": DATASET, "config": config, "split": split,
+         "offset": offset, "length": length}
+    )
+    last = None
+    for attempt in range(6):
+        try:
+            with urllib.request.urlopen(f"{API}?{q}", timeout=60) as r:
+                payload = json.load(r)
+            return [d["row"] for d in payload.get("rows", [])]
+        except urllib.error.HTTPError as e:
+            last = e
+            if e.code not in (429, 500, 502, 503):
+                raise
+            time.sleep(2 ** attempt)
+    raise last
+def to_item(row: dict, config: str, split: str) -> dict:
+    score = float(row.get("head_score") or 0.0)
+    passage = row.get("passage")
+    start, end = row.get("start"), row.get("end")
+    item = {
+        "key": row.get("key"),
+        "surface": row.get("surface"),
+        "ctx": passage,  # None for probe_splits rows (no passage on Hub)
+        "ctx_missing": passage is None,
+        "head_score": score,  # Hub v3-era placeholder; rescore overwrites
+        "start": start, "end": end,
+        "band": "unscored",  # rescore_singlepass tags keep/confusion/drop
+        "origin": row.get("origin"),
+        "specificity": row.get("extractor_label") or row.get("stratum") or "",
+        "split": row.get("split", split),
+        "queue": f"{config}/{split}",
+    }
+    return item
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--config", default="probe_candidates")
+    ap.add_argument("--split", default="pool")
+    ap.add_argument("--limit", type=int, default=500)
+    ap.add_argument("--fetch", type=int, default=5000,
+                    help="rows scanned from the Hub for stratification")
+    ap.add_argument("--seed", type=int, default=7)
+    ap.add_argument("--out", default=str(REPO / "queue.json"))
+    a = ap.parse_args()
+
+    scanned: list[dict] = []
+    offset = 0
+    while len(scanned) < a.fetch:
+        batch = fetch_rows(a.config, a.split, offset, min(100, a.fetch - len(scanned)))
+        if not batch:
+            break
+        scanned.extend(batch)
+        offset += len(batch)
+
+    # stratify: round-robin over (origin, band)
+    import random
+    rng = random.Random(a.seed)
+    buckets: dict[tuple, list[dict]] = {}
+    for row in scanned:
+        item = to_item(row, a.config, a.split)
+        buckets.setdefault((item["origin"], item["band"]), []).append(item)
+    for b in buckets.values():
+        rng.shuffle(b)
+    queue: list[dict] = []
+    while len(queue) < min(a.limit, len(scanned)) and buckets:
+        for k in sorted(buckets):
+            if len(queue) >= a.limit:
+                break
+            if buckets[k]:
+                queue.append(buckets[k].pop())
+        buckets = {k: v for k, v in buckets.items() if v}
+    rng.shuffle(queue)
+
+    out = Path(a.out)
+    out.write_text("\n".join(json.dumps(q) for q in queue) + "\n")
+    origins = sorted({q["origin"] for q in queue})
+    missing = sum(1 for q in queue if q["ctx_missing"])
+    print(f"scanned={len(scanned)} queued={len(queue)} origins={origins} "
+          f"ctx_missing={missing} -> {out}")
+
+
+if __name__ == "__main__":
+    main()

index.html

@@ -1,19 +1,326 @@
 <!doctype html>
-<html>
-	<head>
-		<meta charset="utf-8" />
-		<meta name="viewport" content="width=device-width" />
-		<title>My static Space</title>
-		<link rel="stylesheet" href="style.css" />
-	</head>
-	<body>
-		<div class="card">
-			<h1>Welcome to your static Space!</h1>
-			<p>You can modify this app directly by editing <i>index.html</i> in the Files and versions tab.</p>
-			<p>
-				Also don't forget to check the
-				<a href="https://huggingface.co/docs/hub/spaces" target="_blank">Spaces documentation</a>.
-			</p>
-		</div>
-	</body>
-</html>
+<html lang="en"><head><meta charset="utf-8">
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<title>datause-ner labeling · passage queue</title>
+<style>
+:root{
+  --bg:#F6F4EF; --card:#FFFFFF; --line:#E7E3DD; --ink:#211D1A; --mut:#6F6A63;
+  --primary:#4F46E5; --primary-ink:#FFFFFF;
+  --keep:#0F7A4D; --keep-bg:#E2F5E8; --keep-line:#7BC7A2;
+  --drop:#C2412B; --drop-bg:#FCE8E3; --drop-line:#E8A394;
+  --conf:#B45309; --conf-bg:#FBF0DC; --conf-line:#E0B37A;
+  --unsc:#6B7280; --unsc-bg:#F1F3F5;
+  --mark-u:#FDF0DA; --mark-u-line:#D97706; --mark-k:#DCFCE7; --mark-k-line:#22A55B;
+  --mark-d:#FEE2E2; --mark-d-line:#DC2626;
+}
+*{box-sizing:border-box}
+body{margin:0;background:var(--bg);color:var(--ink);font-family:"Fira Sans","Segoe UI",system-ui,sans-serif;font-size:15px;line-height:1.6}
+header{position:sticky;top:0;z-index:30;background:rgba(255,255,255,.94);backdrop-filter:blur(6px);border-bottom:1px solid var(--line)}
+.hrow{display:flex;gap:10px;align-items:center;flex-wrap:wrap;padding:10px 20px}
+header h1{font-size:14px;margin:0;letter-spacing:.2px;font-weight:800}
+.sub{font-size:12.5px;color:var(--mut)}
+#bar{height:5px;background:var(--line);border-radius:3px;margin:2px 0}
+#bar>i{display:block;height:100%;width:0;background:linear-gradient(90deg,var(--primary),#22A55B);border-radius:3px;transition:width .25s}
+main{max-width:1120px;margin:0 auto;padding:18px 20px 110px;display:grid;grid-template-columns:1fr 356px;gap:22px;align-items:start}
+@media(max-width:920px){main{grid-template-columns:1fr}}
+.card{background:var(--card);border:1px solid var(--line);border-radius:16px;box-shadow:0 1px 2px rgba(33,29,26,.05),0 10px 28px rgba(33,29,26,.05)}
+.passage-card{padding:16px 20px 14px}
+.grid-2{display:grid;grid-template-columns:auto 1fr;gap:18px}
+.badges{display:flex;gap:6px;align-items:center;flex-wrap:wrap;margin-bottom:10px}
+.badge{font-size:11px;font-weight:700;padding:3px 10px;border-radius:999px;background:var(--unsc-bg);color:var(--unsc)}
+.b-keep{background:var(--keep-bg);color:var(--keep)}.b-drop{background:var(--drop-bg);color:var(--drop)}
+.b-confusion{background:var(--conf-bg);color:var(--conf)}.b-mixed{background:#E9E6FF;color:#4338CA}.b-unscored{background:var(--unsc-bg);color:var(--unsc)}
+.passage{max-height:min(54vh,560px);overflow-y:auto;font-size:14.5px;line-height:1.8;padding:14px 6px;border:1px solid var(--line);border-radius:12px;background:#FDFCFA}
+.passage mark{cursor:pointer;border-radius:3px;padding:0 2px;font-weight:600}
+mark.m-u{background:var(--mark-u);border-bottom:2px solid var(--mark-u-line)}
+mark.m-k{background:var(--mark-k);border-bottom:2px solid var(--mark-k-line)}
+mark.m-d{background:var(--mark-d);border-bottom:2px solid var(--mark-d-line)}
+mark.on{outline:2px solid var(--primary);outline-offset:1px}
+mark sup{font-size:9.5px;font-weight:800;color:var(--primary);margin-left:1px}
+.legend{display:flex;gap:8px;align-items:center;margin-top:8px;font-size:11.5px;color:var(--mut);flex-wrap:wrap}
+.legend i{display:inline-block;width:20px;height:4px;border-radius:2px}
+.legend .u i{background:var(--mark-u);border-bottom:2px solid var(--mark-u-line)}
+.legend .k i{background:var(--mark-k);border-bottom:2px solid var(--mark-k-line)}
+.legend .d i{background:var(--mark-d);border-bottom:2px solid var(--mark-d-line)}
+.legend .a i{outline:2px solid var(--primary);outline-offset:1px}
+.chips{display:flex;gap:7px;align-items:center;overflow-x:auto;margin-top:10px;padding:7px 4px;scrollbar-width:thin}
+.chips::-webkit-scrollbar{height:5px}.chips::-webkit-scrollbar-thumb{background:var(--line);border-radius:3px}
+.chip{font:inherit;font-size:11.5px;cursor:pointer;border:1px solid var(--line);background:var(--card);color:var(--ink);border-radius:999px;padding:5px 11px;white-space:nowrap;display:inline-flex;gap:5px;align-items:center;transition:all .15s}
+.chip b{width:18px;height:18px;border-radius:50%;font-size:10px;display:inline-grid;place-items:center;color:#fff}
+.chip .dot{width:7px;height:7px;border-radius:50%;background:var(--conf-line)}
+.chip.r-k{background:var(--keep-bg);border-color:var(--keep-line);color:var(--keep)}
+.chip.r-d{background:var(--drop-bg);border-color:var(--drop-line);color:var(--drop)}
+.chip.on{outline:2px solid var(--primary);outline-offset:1px}
+.chip .sc{font-variant-numeric:tabular-nums}
+.panel-card{padding:16px 18px;position:sticky;top:96px;display:flex;flex-direction:column;gap:12px}
+.panel-label{font-size:10.5px;text-transform:uppercase;letter-spacing:.8px;color:var(--mut);font-weight:800}
+.surface{font-size:20px;font-weight:800;line-height:1.35;padding:10px 12px;background:#FBF8F1;border:1px solid #EADFC4;border-radius:12px}
+.pstate{font-size:13px;font-weight:700}
+.pstate.un{color:var(--conf)}.pstate.k{color:var(--keep)}.pstate.d{color:var(--drop)}
+.scoreline{display:flex;gap:7px;align-items:center;flex-wrap:wrap;font-size:12.5px;color:var(--mut)}
+.tag{font-size:11px;font-weight:800;padding:2px 9px;border-radius:999px;letter-spacing:.3px}
+.tag.k{background:var(--keep-bg);color:var(--keep)}.tag.d{background:var(--drop-bg);color:var(--drop)}
+.tag.c{background:var(--conf-bg);color:var(--conf)}.tag.u{background:var(--unsc-bg);color:var(--unsc)}
+.note{font:inherit;font-size:13.5px;padding:9px 12px;border:1px solid var(--line);border-radius:10px;background:#FDFCFA;width:100%}
+.note:focus{outline:2px solid var(--primary);outline-offset:0;border-color:var(--primary)}
+.hint{font-size:11.5px;color:var(--mut);line-height:1.55}
+kbd{font-family:inherit;font-size:10.5px;background:var(--unsc-bg);border:1px solid var(--line);border-radius:5px;padding:1px 5px}
+#dock{position:fixed;left:0;right:0;bottom:0;z-index:30;background:rgba(255,255,255,.97);backdrop-filter:blur(6px);border-top:1px solid var(--line);padding:11px 20px;display:flex;gap:8px;align-items:center;justify-content:center;flex-wrap:wrap}
+button{font:inherit;cursor:pointer;border:1px solid var(--line);background:var(--card);color:var(--ink);border-radius:12px;padding:10px 18px;font-size:13.5px;transition:all .15s}
+button.nav:hover{border-color:var(--primary);color:var(--primary)}
+button:disabled{opacity:.4;cursor:default}
+button.keep{background:var(--keep);border-color:var(--keep);color:#fff;font-weight:800;min-width:132px;box-shadow:0 1px 3px rgba(15,122,77,.35)}
+button.keep:hover{filter:brightness(1.05)}
+button.drop{background:var(--drop);border-color:var(--drop);color:#fff;font-weight:800;min-width:132px;box-shadow:0 1px 3px rgba(194,65,43,.35)}
+button.drop:hover{filter:brightness(1.05)}
+button.primary{background:var(--primary);border-color:var(--primary);color:#fff}
+select,input[type=text]{font:inherit;font-size:12.5px;padding:6px 10px;border:1px solid var(--line);border-radius:9px;background:var(--card);color:var(--ink)}
+.state{font-size:12.5px;color:var(--mut);min-width:180px;text-align:center;font-variant-numeric:tabular-nums}
+#who{width:76px}
+.btnrow{display:flex;gap:8px;align-items:center}
+.foot{text-align:center;color:var(--mut);font-size:12px;padding-top:14px}
+.empty{text-align:center;color:var(--mut);padding:60px 0;font-size:16px}
+.warn{background:#FBF0DC;border:1px solid var(--conf-line);border-radius:10px;padding:8px 12px;font-size:12.5px}
+</style></head><body>
+<header>
+<div class="hrow">
+<h1>datause-ner labeling</h1>
+<span class="sub" id="qname">queue</span>
+<span class="sub" id="prog" style="font-weight:800;color:var(--ink)"></span><span style="flex:1"></span>
+<label class="sub">annotator <input type="text" id="who" placeholder="initials" style="width:76px"></label>
+<label class="sub"><input type="checkbox" id="rev"> scores</label>
+<select id="fQueue" title="queue"></select>
+<select id="fOrigin" title="origin filter"></select>
+<select id="fBand" title="band filter"></select>
+<select id="fLuna" title="Luna verdict vs model">
+<option value="">luna×model: all</option><option value="agree">agree</option>
+<option value="conflict">conflict</option><option value="unsure">model unsure</option></select>
+<button class="primary" onclick="exportRulings()">⬇ Export</button>
+<span class="sub" id="sink"></span>
+</div>
+<div id="bar"><i></i></div>
+</header>
+<main>
+<div class="card passage-card">
+<div class="badges"><span class="badge" id="bOrigin"></span><span class="badge" id="bQueue"></span><span class="badge" id="bBand"></span><span class="badge" id="bKey"></span></div>
+<div id="noctx" class="warn" style="display:none">surface-only item — no passage available.</div>
+<div class="passage" id="passage"></div>
+<div class="legend">
+<span class="u"><i></i> unruled</span><span class="k"><i></i> kept</span><span class="d"><i></i> dropped</span><span class="a"><i></i> active</span>
+</div>
+<div class="chips" id="chips"></div>
+<div style="margin-top:4px"></div>
+</div>
+<div class="card panel-card">
+<div class="panel-label">Ruling target</div>
+<div class="surface" id="surface"></div>
+<div class="scoreline"><span id="pos"></span><span class="sc" id="sc"></span></div>
+<div class="pstate" id="pstate">unruled</div>
+<div class="btnrow">
+<button class="keep" id="bKeep" onclick="rule('DATA_MENTION')">✓ Keep</button>
+<button class="drop" id="bDrop" onclick="rule('NON_MENTION')">✕ Drop</button>
+</div>
+<input class="note" id="note" placeholder="note (optional) — saved with the ruling">
+<div class="hint"><kbd>k</kbd> keep · <kbd>d</kbd> drop · <kbd>←</kbd><kbd>→</kbd> mention · <kbd>↑</kbd><kbd>↓</kbd> passage · <kbd>s</kbd> skip · <kbd>z</kbd> undo</div>
+</div>
+<div class="empty" id="empty" style="display:none">Nothing matches the active filters.</div>
+</main>
+<div id="dock">
+<button class="nav" id="bPPrev" onclick="moveP(-1)">⏮ P</button>
+<button class="nav" id="bMPrev" onclick="moveM(-1)">‹ M <span id="mPos"></span></button>
+<button class="keep" onclick="rule('DATA_MENTION')">✓ Keep</button>
+<button class="drop" onclick="rule('NON_MENTION')">✕ Drop</button>
+<button class="nav" id="bMNext" onclick="moveM(1)"><span id="mPos2"></span> M ›</button>
+<button class="nav" id="bPNext" onclick="moveP(1)">P ⏭</button>
+<button class="nav" onclick="moveP(1)">Skip <small>(s)</small></button>
+<button class="nav" id="bUndo" onclick="undo()">⟲ Undo</button>
+<span class="state" id="state"></span>
+</div>
+<script>
+let Q=[], R={}, pidx=0, mi=0, lastRuled=null;
+const $=id=>document.getElementById(id);
+const LS='datause_ner_rulings_v1';
+try{R=JSON.parse(localStorage.getItem(LS)||'{}')}catch(e){R={}}
+$('who').value=localStorage.getItem('datause_ner_who')||'';
+$('who').oninput=()=>localStorage.setItem('datause_ner_who',$('who').value);
+const save=()=>localStorage.setItem(LS,JSON.stringify(R));
+const esc=s=>String(s??'').replace(/&/g,'&amp;').replace(/</g,'&lt;').replace(/>/g,'&gt;');
+const mentionsOf=q=>Array.isArray(q.mentions)&&q.mentions.length?q.mentions:[{
+  key:q.key,surface:q.surface,start:q.start,end:q.end,
+  head_score:q.head_score,band:q.band}];
+const isFlat=q=>!Array.isArray(q.mentions);
+async function post(ruling){
+  try{const r=await fetch('/api/rulings',{method:'POST',headers:{'Content-Type':'application/json'},body:JSON.stringify(ruling)});
+    if(r.ok){$('sink').textContent='● server ('+(await r.json()).n+')';return}}catch(e){}
+  $('sink').textContent='● local export';
+}
+async function load(){
+  try{const r=await fetch('/api/queue');if(r.ok)Q=(await r.json()).items||[]}catch(e){}
+  if(!Q.length){
+    for(const f of ['./queue_gliner.json']){
+      try{const r=await fetch(f);if(r.ok)Q.push(...(await r.text()).split('\n').filter(Boolean).map(JSON.parse))}catch(e){}
+    }
+  }
+  fillFilters();render();
+}
+const lunaState=m=>{
+  if(m.luna==null||m.head_score==null)return null;
+  if(m.luna_split)return 'conflict'; // Luna itself split on the same span
+  if(m.band==='keep')return m.luna===1?'agree':'conflict';
+  if(m.band==='drop')return m.luna===0?'agree':'conflict';
+  return 'unsure';
+};
+const filtered=()=>{const qu=$('fQueue').value,o=$('fOrigin').value,b=$('fBand').value,lu=$('fLuna').value;
+  return Q.map((q,i)=>({q,i})).filter(({q})=>(!qu||q.queue===qu)&&(!o||q.origin===o)&&(!b||q.band===b)
+    &&(!lu||mentionsOf(q).some(m=>lunaState(m)===lu)))};
+const totalMentions=()=>Q.reduce((n,q)=>n+(q.mentions?.length||1),0);
+function fillFilters(){
+  const qs=[...new Set(Q.map(q=>q.queue))].sort();
+  $('fQueue').innerHTML=qs.map(q=>`<option value="${esc(q)}">${esc(q)} · ${Q.filter(x=>x.queue===q).length}</option>`).join('');
+  if(qs.includes('gliner'))$('fQueue').value='gliner';
+  else if(qs.includes('probe_candidates/pool'))$('fQueue').value='probe_candidates/pool';
+  else if(qs.includes('human473'))$('fQueue').value='human473';
+  const os=[...new Set(Q.map(q=>q.origin))].sort(),bs=[...new Set(Q.map(q=>q.band))].sort();
+  $('fOrigin').innerHTML='<option value="">all origins</option>'+os.map(o=>`<option>${esc(o)}</option>`).join('');
+  $('fBand').innerHTML='<option value="">all bands</option>'+bs.map(b=>`<option>${esc(b)}</option>`).join('');
+  for(const id of ['fQueue','fOrigin','fBand','fLuna']) $(id).onchange=()=>{pidx=0;mi=0;render()};
+  $('rev').onchange=()=>render();
+}
+function bandsHTML(m){
+  const t=m.band==='keep'?'k':m.band==='drop'?'d':m.band==='unscored'?'u':'c';
+  return `<span class="tag ${t}">${esc(m.band||'?')}</span>`;
+}
+function contextHTML(q){
+  const p=q.ctx||'';const ms=mentionsOf(q);
+  const marks=[];
+  for(let i=0;i<ms.length;i++){
+    const m=ms[i];let s=m.start,e=m.end;
+    if(!(Number.isInteger(s)&&Number.isInteger(e)&&e>s&&p.slice(s,e)===m.surface)){
+      s=p.indexOf(m.surface||'');e=s<0?0:s+(m.surface||'').length;
+    }
+    if(s<0)continue;
+    marks.push({s,e,i});
+  }
+  // char-once walk: overlapping spans never duplicate passage text; a mark
+  // starting inside the previous one is suppressed in-text (chip remains)
+  marks.sort((a,b)=>a.s-b.s||b.e-a.e);
+  let out='',cur=0,suppressed=0;
+  for(const mk of marks){
+    if(mk.s<cur){suppressed++;continue}
+    out+=esc(p.slice(cur,mk.s));
+    const r=R[ms[mk.i].key];
+    const cls=r? (r.ruling==='DATA_MENTION'?'m-k':'m-d') : 'm-u';
+    out+=`<mark class="${cls}" data-mi="${mk.i}" onclick="clickM(${mk.i})">${esc(p.slice(mk.s,mk.e))}${ms.length>1?`<sup>${mk.i+1}</sup>`:''}</mark>`;
+    cur=mk.e;
+  }
+  out+=esc(p.slice(cur));
+  return out;
+}
+function render(){
+  const F=filtered();
+  const ruled=Object.keys(R).length, tm=totalMentions();
+  $('prog').textContent=`${ruled}/${tm} ruled`;
+  $('qname').textContent=`${$('fQueue').value||'queue'} · ${F.length}`;
+  $('empty').style.display=F.length?'none':'block';
+  const bar=document.querySelector('#bar>i');bar.style.width=(tm?100*ruled/tm:0)+'%';
+  if(!F.length){$('passage').innerHTML='';$('chips').innerHTML='';$('state').textContent='adjust filters';return}
+  pidx=Math.max(0,Math.min(pidx,F.length-1));
+  const q=F[pidx].q, ms=mentionsOf(q);
+  mi=Math.max(0,Math.min(mi,ms.length-1));
+  const m=ms[mi],r=R[m.key];
+  $('bOrigin').textContent=[q.origin,q.subset].filter(Boolean).join(' · ');
+  const nk=ms.filter(x=>x.luna===1).length,nd=ms.filter(x=>x.luna===0).length;
+  $('bQueue').textContent=(isFlat(q)?'single mention':`${q.mentions.length} mentions`)
+    +(nk+nd?` · luna: ${nk}✓ / ${nd}✕`:'');
+  const bb=$('bBand');bb.textContent=`tag: ${q.band}`;bb.className='badge b-'+(q.band||'unscored');
+  $('bKey').textContent=q.mentions?m.key:q.key;
+  $('noctx').style.display=q.ctx?'none':'block';
+  $('passage').innerHTML=q.ctx?contextHTML(q):`<div class="empty">${esc(m.surface||'(no context)')}</div>`;
+  for(const el of $('passage').querySelectorAll('mark[data-mi]')){
+    if(el.dataset.mi===String(mi))el.classList.add('on');
+    else el.classList.remove('on');
+  }
+  $('passage').style.display='';
+  const act=$('passage').querySelector('mark.on');
+  if(act)act.scrollIntoView({block:'center',behavior:'smooth'});
+  $('chips').innerHTML=ms.map((mm,i)=>{
+    const rr=R[mm.key];
+    const cls=rr?(rr.ruling==='DATA_MENTION'?'r-k':'r-d'):'';
+    const on=i===mi?'on':'';
+    const dot=mm.luna_split?'var(--conf)':mm.luna===1?'var(--keep)':mm.luna===0?'var(--drop)':'var(--conf-line)';
+    return `<button class="chip ${cls} ${on}" data-ma="${i}" onclick="clickM(${i})"><b>${i+1}</b><span class="dot" style="background:${dot}"></span>${esc(mm.surface)}</button>`;
+  }).join('');
+  const actChip=$('chips').querySelector('button.on');
+  if(actChip)actChip.scrollIntoView({inline:'center',behavior:'smooth',block:'nearest'});
+  const sc=$('rev').checked?(m.head_score==null?'n/a':(+m.head_score).toFixed(3)):'•••';
+  $('surface').textContent=m.surface||'(no surface)';
+  $('pos').textContent=`M ${mi+1}/${ms.length}`;
+  $('sc').innerHTML=bandsHTML(m)
+    +(m.luna_split?`<span class="tag c">Luna split ✓/✕</span>`
+      :m.luna!=null?`<span class="tag ${m.luna===1?'k':'d'}">Luna ${m.luna===1?'✓ kept':'✕ dropped'}</span>`:'')
+    +`<span class="sc">probe ${sc}</span>`
+    +(m.extractor_score!=null?`<span class="sc">ext ${(+m.extractor_score).toFixed(2)}</span>`:'');
+  const st=$('pstate');
+  if(r){st.textContent=(r.ruling==='DATA_MENTION'?'✓ kept':'✕ dropped')+' — your ruling';st.className='pstate '+(r.ruling==='DATA_MENTION'?'k':'d')}
+  else{st.textContent='unruled — model tag above is a hint, you decide';st.className='pstate un'}
+  $('note').value=r?.note||'';
+  $('state').textContent=`P ${pidx+1}/${F.length} · M ${mi+1}/${ms.length}${r?' · ruled':''}`;
+  $('mPos').textContent=`${mi+1}/${ms.length}`;$('mPos2').textContent=`${mi+1}/${ms.length}`;
+  $('bPPrev').disabled=pidx===0;$('bPNext').disabled=pidx>=F.length-1;
+  $('bMPrev').disabled=mi===0&&pidx===0;$('bMNext').disabled=mi>=ms.length-1&&pidx>=F.length-1;
+  $('bUndo').disabled=!lastRuled;
+}
+function clickM(i){mi=i;render()}
+function moveM(d){
+  const F=filtered();if(!F.length)return;
+  const ms=mentionsOf(F[pidx].q);
+  if(d>0){if(mi<ms.length-1)mi++;else if(pidx<F.length-1){pidx++;mi=0} }
+  else{if(mi>0)mi--;else if(pidx>0){pidx--;mi=mentionsOf(F[pidx].q).length-1} }
+  render();
+}
+function moveP(d){
+  const F=filtered();if(!F.length)return;
+  pidx=Math.max(0,Math.min(pidx+d,F.length-1));mi=0;render();
+}
+async function rule(v){
+  const F=filtered();if(!F.length)return;
+  const q=F[pidx].q,ms=mentionsOf(q),m=ms[mi];
+  R[m.key]={key:m.key,ruling:v,annotator:$('who').value||'anon',ts:new Date().toISOString(),
+    note:$('note').value||'',head_score:m.head_score,origin:q.origin,surface:m.surface,queue:q.queue};
+  lastRuled=m.key;save();post(R[m.key]);
+  if(mi<ms.length-1)mi++;
+  else if(pidx<F.length-1){pidx++;mi=0}
+  render();
+}
+function undo(){
+  if(!lastRuled)return;
+  const F=filtered();
+  let found=false;
+  for(let fi=0;fi<F.length;fi++){
+    const ms=mentionsOf(F[fi].q);
+    for(let j=0;j<ms.length;j++){
+      if(ms[j].key===lastRuled){pidx=fi;mi=j;found=true;break}
+    }
+    if(found)break;
+  }
+  delete R[lastRuled];lastRuled=null;save();
+  if(!found){$('state').textContent='ruled item outside current filters';return}
+  render();
+}
+function exportRulings(){
+  const blob=new Blob([JSON.stringify({items:Object.values(R)},null,1)],{type:'application/json'});
+  const a=document.createElement('a');a.href=URL.createObjectURL(blob);
+  a.download=`rulings-${$('who').value||'anon'}-export.json`;a.click();
+}
+document.addEventListener('keydown',e=>{
+  if(/INPUT|SELECT|TEXTAREA/.test(e.target.tagName))return;
+  if(e.key==='k')rule('DATA_MENTION');
+  else if(e.key==='d')rule('NON_MENTION');
+  else if(e.key==='ArrowRight')moveM(1);
+  else if(e.key==='ArrowLeft')moveM(-1);
+  else if(e.key==='ArrowDown'||e.key==='s')moveP(1);
+  else if(e.key==='ArrowUp')moveP(-1);
+  else if(e.key==='z')undo();
+});
+load();
+</script></body></html>

merge.py

@@ -0,0 +1,64 @@
+#!/usr/bin/env python3
+"""Reconcile annotation files into one adjudicated row per key.
+
+    uv run python human_labeling/merge.py [--out annotations/adjudicated.jsonl]
+
+Reads every annotations/rulings*.jsonl + annotations/*-export.json
+(export format: {"items": [...]}, same ruling objects), majority-votes
+ties -> "REVIEW", and writes adjudicated.jsonl:
+{"key", "verdict", "n_keep", "n_drop", "annotators", "origins"?, ...}
+Kept inputs are never modified; rerun any time.
+"""
+
+import argparse
+import glob
+import json
+from pathlib import Path
+
+HERE = Path(__file__).resolve().parent
+ANNOT = HERE / "annotations"
+
+
+def load() -> list[dict]:
+    rulings: list[dict] = []
+    for f in sorted(glob.glob(str(ANNOT / "rulings*.jsonl"))):
+        for line in open(f):
+            line = line.strip()
+            if line:
+                rulings.append(json.loads(line))
+    for f in sorted(glob.glob(str(ANNOT / "*-export.json"))):
+        try:
+            payload = json.load(open(f))
+        except json.JSONDecodeError:
+            continue
+        items = payload.get("items", payload if isinstance(payload, list) else [])
+        rulings.extend(i for i in items if isinstance(i, dict) and i.get("key"))
+    return rulings
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--out", default=str(ANNOT / "adjudicated.jsonl"))
+    a = ap.parse_args()
+    by_key: dict[str, list[dict]] = {}
+    for r in load():
+        by_key.setdefault(r["key"], []).append(r)
+    n_tie = 0
+    with open(a.out, "w") as f:
+        for key in sorted(by_key):
+            rs = by_key[key]
+            keep = sum(1 for r in rs if r["ruling"] == "DATA_MENTION")
+            drop = sum(1 for r in rs if r["ruling"] == "NON_MENTION")
+            verdict = "DATA_MENTION" if keep > drop else "NON_MENTION" if drop > keep else "REVIEW"
+            n_tie += verdict == "REVIEW"
+            f.write(json.dumps({
+                "key": key, "verdict": verdict, "n_keep": keep, "n_drop": drop,
+                "n": len(rs), "annotators": sorted({r.get("annotator", "?") for r in rs}),
+                "surface": rs[0].get("surface"), "origin": rs[0].get("origin"),
+                "head_score": rs[0].get("head_score"),
+            }) + "\n")
+    print(f"keys={len(by_key)} ties={n_tie} -> {a.out}")
+
+
+if __name__ == "__main__":
+    main()

probe_labels.py

@@ -0,0 +1,47 @@
+"""Decision labels for queue items from the singlepass bundle thresholds.
+
+Rule (shared by build_gold_queue.py and rescore_singlepass.py so every
+queue files the same vocabulary):
+
+    keep        score >= per-origin best-F1 threshold (thresholds.json)
+    drop        score <= DROP_FLOOR  (confident reject, repo 0.05 edge)
+    confusion   everything between — the human-review zone (was "mid")
+    unscored    score missing (out-of-grid span / not yet rescored)
+
+Per-origin table (published singlepass bundle, best-F1):
+
+    fcv_pads_east_africa 0.5 · general_prwp 0.4 · jad_paddy_docs 0.1
+    jdc_operational 0.6 · refugee_pads 0.5 · reliefweb 0.4
+    fallback: global_best 0.5
+"""
+
+import json
+from pathlib import Path
+
+DROP_FLOOR = 0.05
+
+_THR: dict | None = None
+
+
+def load_thresholds() -> dict:
+    """Published singlepass thresholds; mirror of the bundle's thresholds.json."""
+    global _THR
+    if _THR is None:
+        p = (Path(__file__).resolve().parent.parent / "outputs"
+             / "gliner-datause-catchall-infer-probe" / "thresholds.json")
+        _THR = json.loads(p.read_text()) if p.exists() else {}
+    return _THR
+
+
+def decide(score, origin, thresholds: dict | None = None) -> str:
+    if score is None:
+        return "unscored"
+    t = thresholds if thresholds is not None else load_thresholds()
+    thr = (t.get("per_origin") or {}).get(origin, {}).get("thr")
+    if thr is None:
+        thr = (t.get("global_best") or {}).get("thr", 0.5)
+    if score >= thr:
+        return "keep"
+    if score <= DROP_FLOOR:
+        return "drop"
+    return "confusion"

rescore_singlepass.py

@@ -0,0 +1,93 @@
+#!/usr/bin/env python3
+"""Rescore labeling queues with the singlepass infer head (local MPS).
+
+Same weights the H100 fit (probe_head.pt in the singlepass bundle), applied
+to fixed queue spans via training/probe_features_infer.encode_split_infer:
+one prompt-conditioned forward per passage batch, score =
+sigmoid(head([start; end; mean; +-64 window])).
+
+Scoring happens only for items without scored_by (or --force). Every queue
+item — scored here or carrying authoritative H100 scores from
+build_gold_queue.py — is then tagged with the singlepass decision rule
+(human_labeling/probe_labels.py):
+
+    keep        score >= per-origin best-F1 threshold (thresholds.json)
+    drop        score <= 0.05
+    confusion   between threshold and the drop floor (was "mid")
+    unscored    score missing (out-of-grid span)
+
+    uv run python human_labeling/rescore_singlepass.py [--force] [--limit N]
+"""
+
+import argparse
+import json
+import sys
+from pathlib import Path
+
+REPO = Path(__file__).resolve().parent.parent
+sys.path.insert(0, str(REPO))
+
+MODEL = "rafmacalaba/gliner-datause-catchall-singlepass"
+QUEUES = ["human_labeling/queue.json", "human_labeling/queue_human473.json"]
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--model", default=MODEL)
+    ap.add_argument("--force", action="store_true")
+    ap.add_argument("--limit", type=int, default=0)
+    ap.add_argument("--batch", type=int, default=8)
+    a = ap.parse_args()
+
+    import torch
+    from training.singlepass_infer import default_device, load_bundle
+    from training.probe_features_infer import encode_split_infer
+    from probe_labels import decide, load_thresholds
+
+    device = default_device()
+    print(f"device={device} model={a.model}", flush=True)
+    model, head, bundle = load_bundle(
+        "rafmacalaba/gliner-datause-mentions-catch-all", a.model, device)
+    thresholds = bundle.get("thresholds") or load_thresholds()
+    args = argparse.Namespace(encode_batch_size=a.batch, context_radius=64)
+
+    for qname in QUEUES:
+        p = REPO / qname
+        if not p.exists():
+            continue
+        items = [json.loads(l) for l in p.read_text().splitlines() if l.strip()]
+        todo = [(i, r) for i, r in enumerate(items) if r.get("ctx")]
+        to_score = [(i, r) for i, r in todo if a.force or not r.get("scored_by")]
+        if a.limit:
+            to_score = to_score[:a.limit]
+
+        n_scored = n_skip = skipped = 0
+        if to_score:
+            texts = [r["ctx"] for _, r in to_score]
+            spans = [(k, r["start"], r["end"], 0, r["key"])
+                     for k, (_, r) in enumerate(to_score)]
+            feats, dim, skipped = encode_split_infer(model, texts, spans, args, device)
+            F = torch.from_numpy(feats).to(device)
+            with torch.no_grad():
+                probs = torch.sigmoid(head(F)).cpu().numpy().ravel()
+            for (i, r), frow, ps in zip(to_score, feats, probs):
+                if not frow.any():  # out-of-grid: zero row, not a real score
+                    r["head_score"] = None
+                    n_skip += 1
+                else:
+                    r["head_score"] = float(ps)
+                    n_scored += 1
+                r["scored_by"] = a.model
+
+        n_tag = 0
+        for i, r in todo:
+            r["band"] = decide(r.get("head_score"), r.get("origin"), thresholds)
+            n_tag += 1
+        if to_score or n_tag:
+            p.write_text("\n".join(json.dumps(r) for r in items) + "\n")
+        print(f"{qname}: rescored={n_scored} unscored={n_skip} "
+              f"tagged={n_tag} skipped_align={skipped}", flush=True)
+
+
+if __name__ == "__main__":
+    main()

review.html

@@ -0,0 +1,313 @@
+<!DOCTYPE html>
+<html lang="en" data-theme="light">
+<head>
+<meta charset="utf-8">
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<title>annotation review</title>
+<style>
+:root, [data-theme="light"] { --bg: #f6f8fa; --card: #ffffff; --fg: #1f2328; --mut: #59636e; --line: #d0d7de; --keep: #1a7f37; --drop: #d1242c; --hl: #0969da; --chip: #eaeef2; }
+[data-theme="dark"] { --bg: #0f1419; --card: #1a222c; --fg: #e6edf3; --mut: #8b98a5; --line: #30363d; --keep: #3fb950; --drop: #f85149; --hl: #1f6feb; --chip: #21262d; }
+* { box-sizing: border-box; }
+body { background: var(--bg); color: var(--fg); font: 15px/1.55 -apple-system, system-ui, sans-serif; margin: 0; }
+header { position: sticky; top: 0; background: var(--card); border-bottom: 1px solid var(--line); padding: 10px 16px; display: flex; gap: 10px; align-items: center; z-index: 5; }
+header .prog { font-variant-numeric: tabular-nums; white-space: nowrap; }
+#bar { flex: 1; height: 8px; background: var(--chip); border-radius: 4px; overflow: hidden; }
+#bar > div { height: 100%; background: var(--hl); width: 0; }
+select, button { font: inherit; color: var(--fg); }
+main { max-width: 860px; margin: 0 auto; padding: 20px 16px 90px; }
+.meta { color: var(--mut); font-size: 13px; margin-bottom: 8px; display: flex; gap: 10px; flex-wrap: wrap; }
+#passage { background: var(--card); border: 1px solid var(--line); border-radius: 10px; padding: 18px; font-size: 16px; }
+#passage mark { background: #9e6a03; color: #fff; border-radius: 3px; padding: 0 2px; cursor: pointer; }
+#passage mark.active { outline: 2px solid var(--hl); }
+#passage mark.done-k { background: var(--keep); }
+#passage mark.done-d { background: var(--drop); }
+#chips { display: flex; gap: 6px; flex-wrap: wrap; margin-top: 10px; }
+.chip { background: var(--chip); border: 1px solid var(--line); border-radius: 14px; padding: 2px 12px; font-size: 13px; cursor: pointer; max-width: 280px; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
+.chip.active { outline: 2px solid var(--hl); }
+.chip.k { border-color: var(--keep); } .chip.d { border-color: var(--drop); }
+#ref { margin-top: 10px; font-size: 13px; color: var(--mut); background: var(--card); border: 1px solid var(--line); border-radius: 10px; padding: 10px 14px; }
+#ref b.k { color: var(--keep); } #ref b.d { color: var(--drop); }
+.controls { display: flex; gap: 10px; margin-top: 14px; }
+button { cursor: pointer; border-radius: 8px; border: 1px solid var(--line); background: var(--card); padding: 10px 22px; }
+button.keep { border-color: var(--keep); } button.keep.on { background: var(--keep); color: #fff; }
+button.drop { border-color: var(--drop); } button.drop.on { background: var(--drop); color: #fff; }
+button.nav { padding: 10px 14px; }
+#mine { color: var(--mut); font-size: 13px; margin-top: 8px; min-height: 20px; }
+kbd { background: var(--chip); border-radius: 4px; padding: 1px 6px; font-size: 12px; }
+footer { position: fixed; bottom: 0; left: 0; right: 0; background: var(--card); border-top: 1px solid var(--line); padding: 8px 16px; display: flex; gap: 10px; align-items: center; font-size: 13px; color: var(--mut); }
+footer .sp { flex: 1; }
+</style>
+</head>
+<body>
+<header>
+  <span class="prog" id="prog">–/–</span>
+  <div id="bar"><div></div></div>
+  <select id="ds" onchange="loadDs()">
+    <option value="sample" selected>annotate sample · 60</option>
+    <option value="gold">gold · 473</option>
+    <option value="paddy">paddy · 240</option>
+    <option value="aj">aj · 240</option>
+    <option value="aivin">aivin · 240</option>
+    <option value="rafael">rafael · 240</option>
+  </select>
+  <select id="src" onchange="reset()"></select>
+  <input id="who" placeholder="your name to annotate" autocomplete="off"
+    style="font:inherit;color:var(--fg);background:var(--bg);border:1px solid var(--line);border-radius:8px;padding:8px 10px;max-width:190px">
+  <button onclick="theme()" title="toggle theme">◐</button>
+</header>
+<main>
+  <div class="meta"><span id="key"></span><span id="surface"></span></div>
+  <div id="passage"></div>
+  <div id="chips"></div>
+  <div id="ref"></div>
+  <div class="controls">
+    <button class="nav" onclick="move(-1)">← prev</button>
+    <button class="keep" id="bKeep" onclick="rule('DATA_MENTION')">keep <kbd>k</kbd></button>
+    <button class="drop" id="bDrop" onclick="rule('NON_MENTION')">drop <kbd>d</kbd></button>
+    <button class="nav" onclick="move(1)">next →</button>
+  </div>
+  <div id="mine"></div>
+<footer>
+  <span><kbd>←</kbd><kbd>→</kbd> prev/next mention · <kbd>k</kbd> keep · <kbd>d</kbd> drop · <kbd>space</kbd> skip · <kbd>[</kbd><kbd>]</kbd> slide · <kbd>-</kbd><kbd>+</kbd> size · <kbd>e</kbd> export</span>
+  <span class="sp"></span><span id="count"></span>
+  <input id="hubtok" type="password" placeholder="HF token" title="write token for data-use-annotations; only needed on static hosting without the bundled server"
+    style="font:inherit;color:var(--fg);background:var(--bg);border:1px solid var(--line);border-radius:8px;padding:8px 10px;max-width:130px">
+  <button id="hubBtn" onclick="exp()">export</button>
+</footer>
+<script>
+const DS = { sample: './queue_annotate_sample.json', gold: './queue_review473.json',
+  paddy: './queue_annotate_paddy.json', aj: './queue_annotate_aj.json',
+  aivin: './queue_annotate_aivin.json', rafael: './queue_annotate_rafael.json' };
+const USERS = ['paddy', 'aj', 'aivin', 'rafael'];
+const LS = 'review_rulings';
+let R = JSON.parse(localStorage.getItem(LS) || '{}');
+const $ = id => document.getElementById(id);
+const esc = s => String(s).replace(/[&<>]/g, c => ({'&':'&amp;','<':'&lt;','>':'&gt;'}[c]));
+const QD = document.documentElement;
+QD.dataset.theme = localStorage.getItem('review_theme') || 'light';
+function theme() { QD.dataset.theme = QD.dataset.theme === 'light' ? 'dark' : 'light'; localStorage.setItem('review_theme', QD.dataset.theme); }
+const who = () => ($('who').value || '').trim();
+function loadWho() {
+  $('who').value = localStorage.getItem('review_who') || '';
+  $('who').addEventListener('input', () => {
+    localStorage.setItem('review_who', who());
+    paintGate();
+    const w = who().toLowerCase();
+    if (USERS.includes(w) && $('ds').value !== w) { $('ds').value = w; loadDs(); }
+  });
+  $('hubtok').value = localStorage.getItem('review_hubtok') || '';
+  $('hubtok').addEventListener('input', () => localStorage.setItem('review_hubtok', $('hubtok').value.trim()));
+  paintGate();
+}
+function paintGate() {
+  const ok = who().length > 0;
+  $('who').style.borderColor = ok ? '' : 'var(--drop)';
+  $('bKeep').disabled = $('bDrop').disabled = !ok;
+  $('bKeep').style.opacity = $('bDrop').style.opacity = ok ? '' : '0.45';
+}
+function needName() {
+  if (who().length > 0) return false;
+  $('mine').textContent = 'enter your name above before annotating';
+  $('who').focus();
+  return true;
+}
+
+// normalize legacy single-span items -> spans[]
+function norm(o) {
+  if (o.spans) return o;
+  return {...o, spans: [{key: o.key, start: o.start, end: o.end, surface: o.surface,
+    luna_label: o.ref ? (o.ref.luna_label === 1 ? 1 : 0) : undefined,
+    human_verdict: o.ref && o.ref.human_verdict, human_note: o.ref && o.ref.human_note,
+    probe_tag: undefined, probe_score: o.ref && o.ref.head_score,
+    pred: o.ref && o.ref.pred}]};
+}
+async function loadDs() {
+  const q = await (await fetch(DS[$('ds').value])).json();
+  ALL = q.map(norm);
+  const srcs = [...new Set(ALL.map(o => o.source))].sort();
+  const total = ALL.length;
+  $('src').innerHTML = `<option value="all">all · ${total}</option>` +
+    srcs.map(x => `<option value="${esc(x)}">${esc(x)} · ${ALL.filter(o => o.source === x).length}</option>`).join('');
+  reset();
+}
+function reset() {
+  const f = $('src').value;
+  ITEMS = ALL.filter(o => f === 'all' || o.source === f);
+  i = 0; s = 0; needCenter = true; render();
+}
+function cur() { return ITEMS[i]; }
+function span() { return cur().spans[s]; }
+let WSIZES = [400, 800, 1600, Infinity], wi = 1, wc = null, plo = 0, phi = 0, needCenter = true;
+function wlabel() { return WSIZES[wi] === Infinity ? 'full' : '±' + WSIZES[wi] / 2; }
+function render() {
+  const o = cur();
+  if (!o) return;
+  s = Math.min(s, o.spans.length - 1);
+  const sp = span();
+  $('prog').textContent = `${i + 1}/${ITEMS.length} · span ${s + 1}/${o.spans.length}`;
+  document.querySelector('#bar > div').style.width = (100 * (i + 1) / ITEMS.length) + '%';
+  $('key').textContent = sp.key;
+  $('surface').textContent = '“' + sp.surface + '”';
+  // sliding assessment window around the active span
+  if (needCenter || wc === null) { wc = (sp.start + sp.end) / 2; needCenter = false; }
+  const W = WSIZES[wi], L = o.text.length;
+  let lo = 0, hi = L;
+  if (W !== Infinity && L > W) {
+    lo = Math.max(0, Math.floor(wc - W / 2)); hi = Math.min(L, lo + W);
+    if (hi - lo < W) lo = Math.max(0, hi - W);
+    const snap = (n, dir) => { while (n > 0 && n < L && o.text[n] !== ' ' && o.text[n] !== '\n') n += dir; return n; };
+    lo = snap(lo, -1); hi = snap(hi, 1);
+    lo = Math.min(lo, sp.start); hi = Math.max(hi, sp.end);
+  }
+  const pts = [];
+  if (lo > 0) pts.push('<span style="color:var(--mut)">…</span>');
+  let p = lo;
+  const ordered = [...o.spans].sort((a, b) => a.start - b.start).filter(x => x.end > lo && x.start < hi);
+  for (const x of ordered) {
+    const a = Math.max(x.start, lo), b = Math.min(x.end, hi);
+    pts.push(esc(o.text.slice(p, a)));
+    const cls = [x.key === sp.key ? 'active' : '', R[x.key] === 'DATA_MENTION' ? 'done-k' : '', R[x.key] === 'NON_MENTION' ? 'done-d' : ''].join(' ');
+    pts.push(`<mark data-k="${esc(x.key)}" class="${cls}">${esc(o.text.slice(a, b))}</mark>`);
+    p = b;
+  }
+  pts.push(esc(o.text.slice(p, hi)));
+  plo = lo; phi = hi;
+  $('passage').innerHTML = pts.join('');
+  bindMarks(o);
+  renderChips(o);
+  renderRef(span());
+  lightMeta();
+}
+function renderChips(o) {
+  $('chips').innerHTML = o.spans.length > 1 ? o.spans.map((x, n) =>
+    `<span class="chip${n === s ? ' active' : ''}${R[x.key] === 'DATA_MENTION' ? ' k' : ''}${R[x.key] === 'NON_MENTION' ? ' d' : ''}" data-n="${n}">#${n + 1} ${esc(x.surface.slice(0, 40))}</span>`).join('') : '';
+  document.querySelectorAll('.chip').forEach(c => c.onclick = () => show(i, +c.dataset.n));
+}
+function renderRef(sp) {
+  let ref;
+  if (sp.luna_label !== undefined && sp.human_verdict) {
+    const g = sp.luna_label === 1 ? '<b class="k">KEEP</b>' : '<b class="d">DROP</b>';
+    ref = `gold ${g} · human ${esc(sp.human_verdict)}${sp.human_note ? ' (' + esc(sp.human_note) + ')' : ''} · model ${sp.pred} @ ${sp.probe_score}`;
+  } else {
+    const g = sp.luna_label === 1 ? '<b class="k">KEEP</b>' : '<b class="d">DROP</b>';
+    ref = `luna ${g} · probe ${esc(sp.probe_tag)} @ ${sp.probe_score}`;
+  }
+  $('ref').innerHTML = ref;
+}
+let lastSaved = '';
+function lightMeta() {
+  const mine = R[span().key];
+  $('bKeep').className = 'keep' + (mine === 'DATA_MENTION' ? ' on' : '');
+  $('bDrop').className = 'drop' + (mine === 'NON_MENTION' ? ' on' : '');
+  $('mine').textContent = mine ? `your ruling: ${mine === 'DATA_MENTION' ? 'keep' : 'drop'}` : '';
+  $('count').textContent = Object.keys(R).length + ' ruled' + (lastSaved ? ' · last save: ' + lastSaved : '');
+}
+function bindMarks(o) {
+  document.querySelectorAll('#passage mark').forEach(m => m.onclick = () => {
+    show(i, o.spans.findIndex(x => x.key === m.dataset.k));
+  });
+}
+function light() {
+  // same passage, text stays fixed: only the active highlight changes
+  const o = cur(), sp = span();
+  $('prog').textContent = `${i + 1}/${ITEMS.length} · span ${s + 1}/${o.spans.length}`;
+  $('key').textContent = sp.key;
+  $('surface').textContent = '“' + sp.surface + '”';
+  document.querySelectorAll('#passage mark').forEach(m => {
+    m.classList.toggle('active', m.dataset.k === sp.key);
+    m.classList.toggle('done-k', R[m.dataset.k] === 'DATA_MENTION');
+    m.classList.toggle('done-d', R[m.dataset.k] === 'NON_MENTION');
+  });
+  renderChips(o);
+  renderRef(sp);
+  lightMeta();
+}
+function show(ni, ns) {
+  // same passage: text stays fixed, only the highlight moves.
+  // recenter only when the target span lies outside the shown window.
+  if (ni !== i) { i = ni; s = ns; needCenter = true; render(); return; }
+  s = Math.max(0, Math.min(ns, cur().spans.length - 1));
+  const c = (span().start + span().end) / 2;
+  if (c < plo || c > phi) { needCenter = true; render(); } else light();
+}
+function move(d) { show((i + d + ITEMS.length) % ITEMS.length, 0); }
+function smove(d) {
+  const o = cur();
+  if (s + d >= 0 && s + d < o.spans.length) show(i, s + d);
+  else if (d > 0 && i < ITEMS.length - 1) show(i + 1, 0);
+  else if (d < 0 && i > 0) show(i - 1, ITEMS[i - 1].spans.length - 1);
+  else show(i, d > 0 ? o.spans.length - 1 : 0);
+}
+async function rule(v) {
+  if (needName()) return;
+  const k = span().key;
+  R[k] = v;
+  localStorage.setItem(LS, JSON.stringify(R));
+  paintGate();
+  const entry = {key: k, ruling: v, queue: $('ds').value, annotator: who()};
+  let saved = 'this browser only';
+  try {
+    const r = await fetch('/api/rulings', {method: 'POST', headers: {'Content-Type': 'application/json'},
+      body: JSON.stringify(entry)});
+    if (r.ok) saved = (await r.json()).pushed ? 'server + Hub' : 'server';
+    else if (r.status === 404) saved = await hubCommit(entry);
+  } catch (e) {
+    saved = await hubCommit(entry);
+  }
+  lastSaved = saved;
+  // next unruled span in passage, else next passage
+  const o = cur();
+  const nx = o.spans.findIndex((x, n) => n > s && !R[x.key]);
+  if (nx >= 0) show(i, nx);
+  else if (i < ITEMS.length - 1) show(i + 1, 0);
+  else { light(); }
+}
+async function hubCommit(entry) {
+  // static hosting has no ruling sink: commit straight to the public
+  // dataset with the annotator's own write token (stored locally only).
+  const tok = ($('hubtok').value || '').trim();
+  if (!tok) return 'this browser only (add HF token to sync)';
+  try {
+    const repo = 'rafmacalaba/data-use-annotations';
+    const who = entry.annotator.replace(/[^a-zA-Z0-9-_]/g, '_');
+    const path = `rulings/${who}.jsonl`;
+    let rows = [];
+    try {
+      const r = await fetch(`https://huggingface.co/datasets/${repo}/resolve/main/${path}`);
+      if (r.ok) rows = (await r.text()).split('\n').filter(Boolean).map(JSON.parse);
+    } catch (e) {}
+    rows = rows.filter(x => !(x.queue === entry.queue && x.key === entry.key));
+    rows.push({...entry, ts: new Date().toISOString()});
+    const content = btoa(unescape(encodeURIComponent(rows.map(x => JSON.stringify(x)).join('\n') + '\n')));
+    const fd = new FormData();
+    fd.append('operations', JSON.stringify({key: 'headerFile', value: {path, content}}) + '\n');
+    fd.append('commitTitle', `ruling ${who}: ${entry.key}=${entry.ruling} (web)`);
+    const res = await fetch(`https://huggingface.co/api/datasets/${repo}/commit/main`, {
+      method: 'POST', headers: {Authorization: `Bearer ${tok}`}, body: fd});
+    if (!res.ok) throw new Error('hub ' + res.status);
+    return 'Hub';
+  } catch (e) { return 'this browser only (Hub sync failed)'; }
+}
+document.addEventListener('keydown', e => {
+  if (e.target.tagName === 'INPUT' || e.target.tagName === 'SELECT') return;
+  if (e.key === 'ArrowRight') smove(1);
+  else if (e.key === 'ArrowLeft') smove(-1);
+  else if (e.key === '[') { wc = Math.max(0, (wc ?? 0) - WSIZES[wi] / 2); render(); }
+  else if (e.key === ']') { wc = (wc ?? 0) + WSIZES[wi] / 2; render(); }
+  else if (e.key === '-' || e.key === '_') { wi = Math.max(0, wi - 1); render(); }
+  else if (e.key === 'k' || e.key === 'K') rule('DATA_MENTION');
+  else if (e.key === 'd' || e.key === 'D') rule('NON_MENTION');
+  else if (e.key === ' ') { e.preventDefault(); skip(); }
+  else if (e.key === 'e' || e.key === 'E') exp();
+});
+function skip() {
+  const o = cur();
+  const nx = o.spans.findIndex((x, n) => n > s && !R[x.key]);
+  if (nx >= 0) show(i, nx);
+  else if (i < ITEMS.length - 1) show(i + 1, 0);
+}
+loadWho();
+ loadDs();
+</script>
+</body>
+</html>