NagaYu commited on
Commit
b07f2d1
·
verified ·
1 Parent(s): 170a671

ClaimCheck v1.1.0: client-side deterministic LLM answer verifier

Browse files
Files changed (3) hide show
  1. README.md +153 -4
  2. claimcheck.js +1711 -0
  3. index.html +375 -17
README.md CHANGED
@@ -1,10 +1,159 @@
1
  ---
2
  title: ClaimCheck
3
- emoji: 🐢
4
- colorFrom: green
5
- colorTo: yellow
6
  sdk: static
 
7
  pinned: false
 
 
 
 
 
 
 
8
  ---
9
 
10
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  title: ClaimCheck
3
+ emoji: 🔎
4
+ colorFrom: yellow
5
+ colorTo: red
6
  sdk: static
7
+ app_file: index.html
8
  pinned: false
9
+ license: apache-2.0
10
+ short_description: Verify LLM answers against the context they came from
11
+ tags:
12
+ - hallucination-detection
13
+ - groundedness
14
+ - rag
15
+ - guardrails
16
  ---
17
 
18
+ # 🔎 ClaimCheck
19
+
20
+ **Verify LLM answers against their source context before they reach users.**
21
+
22
+ Paste an answer and the context it was supposed to be grounded in. ClaimCheck extracts the claims it can check as
23
+ strings — numbers, dates, quotes, entities, URLs — matches each against the context, and returns a verdict
24
+ (`pass` / `annotate` / `retry` / `block`), a grounding score, and **how much of the answer it did not check**.
25
+
26
+ **Everything runs in your browser.** No server, no network call, no telemetry — the verifier is ~1,700 lines of
27
+ dependency-free JavaScript executing in this tab. Paste real model output if you want to; it stays on your machine.
28
+
29
+ | | |
30
+ |---|---|
31
+ | 📦 Rule pack | [`NagaYu/claimcheck-rules`](https://huggingface.co/NagaYu/claimcheck-rules) |
32
+ | 📊 Evaluation set | [`NagaYu/claimcheck-eval`](https://huggingface.co/datasets/NagaYu/claimcheck-eval) |
33
+ | ⚡ Latency | ~0.2 ms typical · 15 ms on a 20,000-character worst case |
34
+ | 🌏 Languages | English and Japanese |
35
+
36
+ ---
37
+
38
+ ## Design philosophy
39
+
40
+ We cannot detect every hallucination. Nobody can, and a tool that claims to is lying to you.
41
+
42
+ But we do not have to. The observation this is built on:
43
+
44
+ > **Dangerous hallucinations are specific.**
45
+
46
+ A model that invents *"the contract terminates on 2023-11-04 under Article 12, with a penalty of 3,400,000 JPY"* is far
47
+ more harmful than one that says *"the contract may terminate at some point"*. And the specific one is **checkable as a
48
+ string**: numbers, dates, quotes, entities and URLs are literal tokens that either do, or do not, appear in (or follow
49
+ arithmetically from) the source.
50
+
51
+ So ClaimCheck verifies only what it can verify deterministically, and declares the rest out of scope — loudly.
52
+
53
+ ## ⚠️ Always read `coverage`
54
+
55
+ Two independent numbers, and reporting one without the other is the most dangerous way to misuse this tool.
56
+
57
+ | field | question it answers |
58
+ |---|---|
59
+ | `grounding_score` | Of the claims I checked, what share held up? |
60
+ | `coverage` | What share of the answer did I check at all? |
61
+
62
+ ```
63
+ grounding_score 1.00 + coverage 0.10
64
+ = one sentence checked out and nine were never looked at.
65
+ = NOT "this answer is verified".
66
+ ```
67
+
68
+ An answer of pure hedging produces **zero** claims and scores **1.00** on any naive groundedness metric.
69
+
70
+ ## Claim statuses
71
+
72
+ | status | colour | meaning |
73
+ |---|---|---|
74
+ | `supported` | 🟩 | Appears literally in the context after normalisation |
75
+ | `derived` | 🟦 | Computable from context numbers by 2–3 term arithmetic; the formula is shown |
76
+ | `approximate` | 🟨 | A near match exists — paraphrase, finer-grained date, related URL |
77
+ | `unsupported` | 🟧 | Nothing in the context backs it |
78
+ | `contradicted` | 🟥 | The context holds a **near but different** value |
79
+
80
+ `approximate` is deliberately **not** counted as grounded: a paraphrased quote is still an inaccurate quote.
81
+
82
+ ## Measured behaviour
83
+
84
+ On the [158-case bilingual evaluation set](https://huggingface.co/datasets/NagaYu/claimcheck-eval):
85
+
86
+ ```
87
+ answer-level recall 56/56 (100%) cases that should be flagged, and were
88
+ clean answers passing 44/46 (96%) the rest were annotated, never blocked
89
+ known false positives 22 (13.9%)
90
+ known false negatives 3 (1.9%)
91
+ JS/Python parity 158/158 (100%)
92
+ ```
93
+
94
+ The false-positive rate is published deliberately. It is the number that decides whether anyone keeps a verifier
95
+ switched on, and hiding it helps nobody.
96
+
97
+ ### Known limitation: semantic reversal
98
+
99
+ All three false negatives share a shape. A one-character change that reverses meaning keeps character similarity above
100
+ 0.9, so a reversed quote reads as a faithful paraphrase:
101
+
102
+ | context | answer | similarity | status |
103
+ |---|---|---|---|
104
+ | 売上高は7%**増加**した | 売上高は7%**減少**した | 0.97 | `approximate` ❌ |
105
+ | 提案は全会一致で**承認**された | 提案は全会一致で**否決**された | 0.92 | `approximate` ❌ |
106
+ | revenue **grew** 8% | revenue **declined** 8% | 0.97 | `approximate` ❌ |
107
+
108
+ Character matching cannot detect negation or antonym substitution. If your domain turns on approve/reject or
109
+ increase/decrease, **do not rely on quote status alone.** In all three cases the answer-level verdict was still
110
+ `retry`, because the NUMERIC and ENTITY claims inside the quote are the layer that catches it.
111
+
112
+ ### False positives
113
+
114
+ Faithful paraphrase · generic noun phrases ("Machine Learning") · acronym expansion (WHO ↔ World Health Organization) ·
115
+ rounding just outside tolerance · fiscal vs calendar years · identifier variants (ISO 9001:2015 vs ISO 9001) ·
116
+ URL query parameters.
117
+
118
+ **ENTITY is the noisiest check.** Set `{"enable_entity": false}` in the policy box first, and start from `annotate`,
119
+ never `block`.
120
+
121
+ ## Policy presets
122
+
123
+ Paste any of these into the **Policy JSON** box. Full set in the
124
+ [rule pack](https://huggingface.co/NagaYu/claimcheck-rules):
125
+
126
+ ```jsonc
127
+ // observe: nothing blocked but critical safety — start here
128
+ {"retry_on_contradicted": 9999, "retry_grounding": 0.0, "annotate_grounding": 1.0}
129
+
130
+ // numeric + date only: the lowest false-positive configuration
131
+ {"enable_entity": false, "enable_quote": false, "enable_url": false}
132
+
133
+ // tolerant of rounding
134
+ {"numeric_tolerance": 0.02, "contradiction_rel": 0.10}
135
+ ```
136
+
137
+ ## Running it yourself
138
+
139
+ The core is an ES module with no dependencies:
140
+
141
+ ```js
142
+ import { gate, verify, buildHighlight } from "./claimcheck.js";
143
+
144
+ const r = gate(answer, context, systemPrompt, userInput, schemaJson, policyJson, tagsJson);
145
+ console.log(r.verdict, r.grounding_score, r.coverage);
146
+ ```
147
+
148
+ A Python implementation with a Gradio UI and a REST API also exists and is behaviourally identical (verified at 100%
149
+ parity on all 158 evaluation cases); it is in the
150
+ [rule pack repository](https://huggingface.co/NagaYu/claimcheck-rules) for self-hosting.
151
+
152
+ ## Disclaimer
153
+
154
+ ClaimCheck is **one layer of defence, not a correctness guarantee.** It verifies numbers, dates, quotes, entities and
155
+ URLs against a context you supply. It says nothing about claims it could not extract, nothing about reasoning, and
156
+ nothing about whether the context itself was right. A `pass` verdict means *"the specific things I know how to check,
157
+ checked out"* — nothing more.
158
+
159
+ Apache-2.0.
claimcheck.js ADDED
@@ -0,0 +1,1711 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ /*
2
+ * ClaimCheck - browser port of the deterministic LLM answer verifier.
3
+ *
4
+ * DESIGN PHILOSOPHY
5
+ * =================
6
+ * We cannot detect every hallucination. But dangerous hallucinations are
7
+ * SPECIFIC, and specific claims can be matched as strings. So this verifies
8
+ * only what it can verify deterministically, and always reports how much it
9
+ * did NOT check (coverage).
10
+ *
11
+ * Two numbers, always together:
12
+ * grounding_score = (supported + derived) / verifiable claims
13
+ * coverage = sentences that produced a claim / all sentences
14
+ * An answer of pure hedging produces zero claims and scores 1.00 on any naive
15
+ * groundedness metric. coverage is what stops you reading that as "verified".
16
+ *
17
+ * This file is a faithful port of app.py (the Python source of truth) and is
18
+ * validated for parity against all 158 cases of NagaYu/claimcheck-eval.
19
+ * Everything runs client-side: no network, no server, nothing leaves the tab.
20
+ */
21
+
22
+ export const APP_VERSION = "1.1.0";
23
+ export const STATUSES = ["supported", "derived", "approximate", "unsupported", "contradicted"];
24
+ export const CLAIM_TYPES = ["NUMERIC", "DATE", "QUOTE", "ENTITY", "URL"];
25
+ export const VERDICTS = ["pass", "annotate", "retry", "block"];
26
+
27
+ export const DEFAULTS = {
28
+ MAX_TEXT_CHARS: 20000,
29
+ NGRAM_LEAK_N: 8,
30
+ LEAK_THRESHOLD: 0.15,
31
+ PRICE_IN_PER_1K: 0.0,
32
+ PRICE_OUT_PER_1K: 0.0,
33
+ ENTROPY_THRESHOLD: 3.6,
34
+ ENTROPY_MIN_LEN: 24,
35
+ CONTRADICTION_REL: 0.25,
36
+ APPROX_RATIO: 0.82,
37
+ QUOTE_CONTRADICTION_FLOOR: 0.65,
38
+ DERIVE_MAX_TERMS: 3,
39
+ DERIVE_BUDGET_MS: 120,
40
+ DERIVE_MAX_NUMBERS: 40,
41
+ FUZZY_BUDGET_MS: 250,
42
+ MAX_CLAIMS: 1500,
43
+ };
44
+
45
+ /* ===================================================================== (C) NORMALIZER */
46
+
47
+ // NOTE: "ー" (U+30FC) is deliberately absent. It is a katakana letter, not a
48
+ // dash: folding it turned データセンター into デ-タセンタ- and broke Japanese
49
+ // matching wholesale. This was a real bug in v1.0.0.
50
+ const PUNCT_MAP = {
51
+ ",": ",", "、": ",", "、": ",",
52
+ ".": ".", "。": ".", "。": ".",
53
+ ":": ":", ";": ";",
54
+ "(": "(", ")": ")", "[": "[", "]": "]", "{": "{", "}": "}",
55
+ "「": '"', "」": '"', "『": '"', "』": '"',
56
+ "“": '"', "”": '"', "„": '"', "‟": '"',
57
+ "‘": "'", "’": "'", "‚": "'",
58
+ "-": "-", "−": "-", "—": "-", "–": "-", "―": "-", "‐": "-", "‑": "-",
59
+ "%": "%", "$": "$", "¥": "¥", "/": "/", "\": "\\", "#": "#", "&": "&",
60
+ "!": "!", "?": "?", "〜": "~", "~": "~", "・": " ",
61
+ };
62
+
63
+ /** Guarantees: NFKC + lowercase + collapsed whitespace + unified punctuation; never throws. */
64
+ export function normText(s) {
65
+ try {
66
+ if (s === null || s === undefined) return "";
67
+ let t = String(s).replace(/[,、、.。。:;()[]{}「」『』“”„‟‘’‚-−—–―‐‑%$¥/\#&!?〜~・]/g,
68
+ (ch) => (PUNCT_MAP[ch] !== undefined ? PUNCT_MAP[ch] : ch));
69
+ t = t.normalize("NFKC").toLowerCase();
70
+ return t.replace(/\s+/g, " ").trim();
71
+ } catch (e) { return ""; }
72
+ }
73
+
74
+ const MAG_FACTORS = { "千": 1e3, "万": 1e4, "億": 1e8, "兆": 1e12, k: 1e3, K: 1e3, m: 1e6, M: 1e6, b: 1e9, B: 1e9 };
75
+
76
+ /** Guarantees: a finite Number for any recognised numeric literal, else null. */
77
+ export function normNumber(s) {
78
+ try {
79
+ if (s === null || s === undefined) return null;
80
+ if (typeof s === "number") return Number.isFinite(s) ? s : null;
81
+ let t = String(s).trim();
82
+ if (!t) return null;
83
+ t = t.normalize("NFKC").replace(/[−-]/g, "-").replace(/,/g, ",");
84
+ let sign = 1;
85
+ if (t[0] === "▲" || t[0] === "△") { sign = -1; t = t.slice(1).trim(); }
86
+ if (t.startsWith("(") && t.endsWith(")")) { sign = -sign; t = t.slice(1, -1).trim(); }
87
+ if (t[0] === "+" || t[0] === "-") { if (t[0] === "-") sign = -sign; t = t.slice(1).trim(); }
88
+ let low = t.toLowerCase();
89
+ low = low.replace(/^(?:us\$|usd|jpy|eur|gbp|krw|cny|[$¥€£₩])/, "")
90
+ .replace(/(?:usd|jpy|eur|gbp|円|ドル|ユーロ)$/, "").trim();
91
+ t = low;
92
+ for (const suf of ["%", "パーセント", "percent", "pct"]) {
93
+ if (t.endsWith(suf)) { t = t.slice(0, -suf.length).trim(); break; }
94
+ }
95
+ if (t.endsWith("割")) { // 3割 == 30%
96
+ const base = parseNum(t.slice(0, -1).replace(/[,\s_]/g, ""));
97
+ return base === null ? null : sign * base * 10;
98
+ }
99
+ let factor = 1, changed = true;
100
+ while (changed && t) {
101
+ changed = false;
102
+ for (const suf of Object.keys(MAG_FACTORS)) {
103
+ if (suf.length === 1 && t.endsWith(suf)) {
104
+ const rest = t.slice(0, -1).trim();
105
+ if (rest && /\d/.test(rest)) { factor *= MAG_FACTORS[suf]; t = rest; changed = true; break; }
106
+ }
107
+ }
108
+ }
109
+ t = t.replace(/[^0-9.eE+\-]+$/, "").trim().replace(/[,\s_]/g, "");
110
+ const base = parseNum(t);
111
+ if (base === null) return null;
112
+ return sign * base * factor;
113
+ } catch (e) { return null; }
114
+ }
115
+
116
+ function parseNum(t) {
117
+ if (!t || !/^[-+]?(?:\d+\.?\d*|\.\d+)(?:[eE][-+]?\d+)?$/.test(t)) return null;
118
+ const v = Number(t);
119
+ return Number.isFinite(v) ? v : null;
120
+ }
121
+
122
+ const MONTHS = { jan:1,january:1,feb:2,february:2,mar:3,march:3,apr:4,april:4,may:5,jun:6,june:6,
123
+ jul:7,july:7,aug:8,august:8,sep:9,sept:9,september:9,oct:10,october:10,nov:11,november:11,dec:12,december:12 };
124
+ // Japanese era -> offset such that year = offset + eraYear. 令和1 == 2019.
125
+ const ERAS = { "令和":2018, reiwa:2018, "平成":1988, heisei:1988, "昭和":1925, showa:1925,
126
+ "大正":1911, taisho:1911, "明治":1867, meiji:1867 };
127
+
128
+ function mkDate(y, m, d) {
129
+ y = parseInt(y, 10);
130
+ if (!(y >= 1000 && y <= 3000)) return null;
131
+ const p = (n, w) => String(n).padStart(w, "0");
132
+ if (m === undefined || m === null || m === "") return p(y, 4);
133
+ m = parseInt(m, 10);
134
+ if (!(m >= 1 && m <= 12)) return null;
135
+ if (d === undefined || d === null || d === "") return `${p(y,4)}-${p(m,2)}`;
136
+ d = parseInt(d, 10);
137
+ if (!(d >= 1 && d <= 31)) return null;
138
+ return `${p(y,4)}-${p(m,2)}-${p(d,2)}`;
139
+ }
140
+
141
+ /** Guarantees: an ISO string (YYYY / YYYY-MM / YYYY-MM-DD) or null. Ambiguity returns null, not a guess. */
142
+ export function normDate(s) {
143
+ try {
144
+ if (!s) return null;
145
+ let t = String(s).normalize("NFKC").trim().replace(/[−]/g, "-").replace(/./g, ".");
146
+ t = t.replace(/\s+/g, " ").trim().replace(/[.,]+$/, "");
147
+ if (!t) return null;
148
+ let m;
149
+ if ((m = /^(\d{4})\s*年\s*(\d{1,2})\s*月\s*(\d{1,2})\s*日?$/.exec(t))) return mkDate(m[1], m[2], m[3]);
150
+ if ((m = /^(\d{4})\s*年\s*(\d{1,2})\s*月$/.exec(t))) return mkDate(m[1], m[2]);
151
+ if ((m = /^(令和|平成|昭和|大正|明治)\s*(\d{1,2}|元)\s*年(?:\s*(\d{1,2})\s*月(?:\s*(\d{1,2})\s*日?)?)?$/.exec(t))) {
152
+ const ey = m[2] === "元" ? 1 : parseInt(m[2], 10);
153
+ return mkDate(ERAS[m[1]] + ey, m[3], m[4]);
154
+ }
155
+ if ((m = /^(\d{4})[-/.](\d{1,2})[-/.](\d{1,2})$/.exec(t))) return mkDate(m[1], m[2], m[3]);
156
+ if ((m = /^(\d{4})[-/.](\d{1,2})$/.exec(t))) return mkDate(m[1], m[2]);
157
+ if ((m = /^([A-Za-z]{3,9})\.?\s+(\d{1,2})(?:st|nd|rd|th)?,?\s+(\d{4})$/.exec(t))) {
158
+ const mo = MONTHS[m[1].toLowerCase().replace(/\.$/, "")];
159
+ if (mo) return mkDate(m[3], mo, m[2]);
160
+ }
161
+ if ((m = /^(\d{1,2})(?:st|nd|rd|th)?\s+([A-Za-z]{3,9})\.?,?\s+(\d{4})$/.exec(t))) {
162
+ const mo = MONTHS[m[2].toLowerCase().replace(/\.$/, "")];
163
+ if (mo) return mkDate(m[3], mo, m[1]);
164
+ }
165
+ if ((m = /^([A-Za-z]{3,9})\.?,?\s+(\d{4})$/.exec(t))) {
166
+ const mo = MONTHS[m[1].toLowerCase().replace(/\.$/, "")];
167
+ if (mo) return mkDate(m[2], mo);
168
+ }
169
+ if ((m = /^(\d{4})\s*年(?:度)?$/.exec(t))) return mkDate(m[1]);
170
+ if ((m = /^(?:fy)?(\d{4})$/i.exec(t))) return mkDate(m[1]);
171
+ return null;
172
+ } catch (e) { return null; }
173
+ }
174
+
175
+ /* ============================================================ SIMILARITY PRIMITIVES */
176
+
177
+ /**
178
+ * Guarantees: the same value as Python difflib.SequenceMatcher(None,a,b,autojunk=False).ratio().
179
+ * Ported rather than approximated, because the approximate/contradicted
180
+ * boundary is decided by this number.
181
+ */
182
+ function smRatio(a, b) {
183
+ const la = a.length, lb = b.length;
184
+ if (!la && !lb) return 1.0;
185
+ if (!la || !lb) return 0.0;
186
+ const b2j = new Map();
187
+ for (let j = 0; j < lb; j++) {
188
+ const c = b[j];
189
+ let arr = b2j.get(c);
190
+ if (!arr) { arr = []; b2j.set(c, arr); }
191
+ arr.push(j);
192
+ }
193
+ function longest(alo, ahi, blo, bhi) {
194
+ let besti = alo, bestj = blo, bestsize = 0;
195
+ let j2len = new Map();
196
+ for (let i = alo; i < ahi; i++) {
197
+ const newj2len = new Map();
198
+ const js = b2j.get(a[i]);
199
+ if (js) {
200
+ for (const j of js) {
201
+ if (j < blo) continue;
202
+ if (j >= bhi) break;
203
+ const k = (j2len.get(j - 1) || 0) + 1;
204
+ newj2len.set(j, k);
205
+ if (k > bestsize) { besti = i - k + 1; bestj = j - k + 1; bestsize = k; }
206
+ }
207
+ }
208
+ j2len = newj2len;
209
+ }
210
+ while (besti > alo && bestj > blo && a[besti - 1] === b[bestj - 1]) { besti--; bestj--; bestsize++; }
211
+ while (besti + bestsize < ahi && bestj + bestsize < bhi && a[besti + bestsize] === b[bestj + bestsize]) bestsize++;
212
+ return [besti, bestj, bestsize];
213
+ }
214
+ let matches = 0;
215
+ const queue = [[0, la, 0, lb]];
216
+ while (queue.length) {
217
+ const [alo, ahi, blo, bhi] = queue.pop();
218
+ const [i, j, k] = longest(alo, ahi, blo, bhi);
219
+ if (k) {
220
+ matches += k;
221
+ if (alo < i && blo < j) queue.push([alo, i, blo, j]);
222
+ if (i + k < ahi && j + k < bhi) queue.push([i + k, ahi, j + k, bhi]);
223
+ }
224
+ }
225
+ return (2.0 * matches) / (la + lb);
226
+ }
227
+
228
+ /** Guarantees: candidate window starts where needle and haystack share a character k-gram. */
229
+ function fuzzyAnchors(needle, haystack, maxAnchors = 96) {
230
+ const nl = needle.length;
231
+ // k must stay strictly below the needle length, and short identifiers need
232
+ // k=2: "第13条" against "第12条" shares no 3-gram, and that one-digit-off
233
+ // article number is exactly the fabrication this tool exists to catch.
234
+ let k = nl <= 6 ? 2 : Math.min(8, Math.max(4, Math.floor(nl / 3)));
235
+ k = Math.min(k, Math.max(2, nl - 1));
236
+ if (nl < 2) return haystack.includes(needle) ? [0] : [];
237
+ const offsets = [...new Set([0, Math.floor(nl / 4), Math.floor(nl / 2), Math.floor((3 * nl) / 4), Math.max(0, nl - k)])].sort((x, y) => x - y);
238
+ const anchors = [];
239
+ for (const off of offsets) {
240
+ const gram = needle.slice(off, off + k);
241
+ if (gram.length < k) continue;
242
+ let idx = haystack.indexOf(gram), hits = 0;
243
+ while (idx !== -1 && hits < 20 && anchors.length < maxAnchors) {
244
+ anchors.push(Math.max(0, idx - off));
245
+ hits++;
246
+ idx = haystack.indexOf(gram, idx + 1);
247
+ }
248
+ if (anchors.length >= maxAnchors) break;
249
+ }
250
+ return anchors;
251
+ }
252
+
253
+ /**
254
+ * Guarantees: (ratio, window) for an order-preserving match, with bounded work.
255
+ * No shared k-gram means no candidate windows and an immediate 0.0, which is
256
+ * what keeps a 20k context from becoming a full sliding-window scan.
257
+ */
258
+ export function bestFuzzy(needle, haystack, maxEvals = 240) {
259
+ try {
260
+ if (!needle || !haystack) return [0.0, ""];
261
+ const nl = needle.length;
262
+ const anchors = fuzzyAnchors(needle, haystack);
263
+ if (!anchors.length) return [0.0, ""];
264
+ const widths = [...new Set([Math.max(3, nl - 2), nl, Math.floor(nl * 1.25) + 3])].sort((a, b) => a - b);
265
+ let best = 0.0, bestTxt = "", evals = 0;
266
+ const seen = new Set();
267
+ for (let st of anchors) {
268
+ st = Math.max(0, Math.min(st, Math.max(0, haystack.length - 1)));
269
+ for (const w of widths) {
270
+ const key = st * 10000 + w;
271
+ if (seen.has(key)) continue;
272
+ seen.add(key);
273
+ const win = haystack.slice(st, st + w);
274
+ if (!win) continue;
275
+ evals++;
276
+ const r = smRatio(win, needle);
277
+ if (r > best) { best = r; bestTxt = win; }
278
+ if (best >= 0.995 || evals >= maxEvals) return [best, bestTxt];
279
+ }
280
+ }
281
+ return [best, bestTxt];
282
+ } catch (e) { return [0.0, ""]; }
283
+ }
284
+
285
+ // Rolling-hash longest common substring. difflib.find_longest_match was the
286
+ // original implementation and degenerates to O(n^2) on repetitive text: a 20k
287
+ // answer of one repeated character took 12.7s, a denial of service on a small box.
288
+ const LCS_MOD = 33554393, LCS_BASE = 257; // modulus kept under 2^25 so products stay exact in float64
289
+
290
+ function rkPositions(s, k) {
291
+ const out = new Map(), n = s.length;
292
+ if (k <= 0 || k > n) return out;
293
+ let h = 0;
294
+ for (let i = 0; i < k; i++) h = (h * LCS_BASE + s.charCodeAt(i)) % LCS_MOD;
295
+ out.set(h, 0);
296
+ let power = 1;
297
+ for (let i = 0; i < k; i++) power = (power * LCS_BASE) % LCS_MOD;
298
+ for (let i = k; i < n; i++) {
299
+ h = (h * LCS_BASE + s.charCodeAt(i) - s.charCodeAt(i - k) * power) % LCS_MOD;
300
+ if (h < 0) h += LCS_MOD;
301
+ if (!out.has(h)) out.set(h, i - k + 1);
302
+ }
303
+ return out;
304
+ }
305
+
306
+ /** Guarantees: the longest shared substring, O(n log n), bounded by `cap` characters. */
307
+ export function longestCommonFragment(a, b, cap = 3000) {
308
+ try {
309
+ a = (a || "").slice(0, cap); b = (b || "").slice(0, cap);
310
+ if (!a || !b) return "";
311
+ let lo = 1, hi = Math.min(a.length, b.length), best = "";
312
+ while (lo <= hi) {
313
+ const mid = (lo + hi) >> 1;
314
+ const ha = rkPositions(a, mid), hb = rkPositions(b, mid);
315
+ let found = null;
316
+ if (ha.size && hb.size) {
317
+ const [small, big, sa] = ha.size <= hb.size ? [ha, hb, a] : [hb, ha, b];
318
+ const other = small === ha ? b : a;
319
+ for (const [h, pos] of small) {
320
+ const j = big.get(h);
321
+ if (j === undefined) continue;
322
+ const cand = sa.slice(pos, pos + mid);
323
+ if (cand === other.slice(j, j + mid)) { found = cand; break; } // guard against collisions
324
+ }
325
+ }
326
+ if (found) { best = found; lo = mid + 1; } else hi = mid - 1;
327
+ }
328
+ return best;
329
+ } catch (e) { return ""; }
330
+ }
331
+
332
+ /* ============================================================== (B) CLAIM EXTRACTOR */
333
+
334
+ const URL_RE = /(?:https?:\/\/|ftp:\/\/|www\.)[^\s<>"'()\[\]{}、。,!?]+|\b[a-z0-9][a-z0-9\-]{0,62}\.(?:com|org|net|io|ai|co|jp|dev|app|gov|edu|info|me|cloud|xyz)(?:\/[^\s<>"'()\[\]{}、。,!?]*)?/gi;
335
+
336
+ const NUM_CORE = "(?:[0-9]{1,3}(?:,[0-9]{3})+(?:\\.[0-9]+)?|[0-9]+(?:\\.[0-9]+)?|\\.[0-9]+)";
337
+ const CUR_PAT = "(?:US\\$|USD|JPY|EUR|GBP|KRW|CNY|[$¥€£₩])";
338
+ const MAG_PAT = "(?:千|万|億|兆|[kKmMbB](?![A-Za-z0-9]))";
339
+ const PCT_PAT = "(?:%|パーセント|ポイント|割)";
340
+ const UNIT_PAT = "(?:円|ドル|ユーロ|元|ウォン|件|人|名|社|店|台|個|本|枚|冊|回|倍|点|位|票|室|席|品|語|文字|字|行|列|頁|ページ|年度|年間|年|ヶ月|か月|カ月|ヵ月|箇月|週間|日間|時間|分間|秒|分|時|日|歳|才|km|cm|mm|kg|mg|GB|MB|TB|KB|PB|Mbps|Gbps|bps|kHz|MHz|GHz|Hz|kW|MW|W|mL|L|m2|㎡|平方メートル|立方メートル|℃|°C|°F|m|g|t|V|A)";
341
+ // The whitespace sits INSIDE the currency group and the letter-lookahead INSIDE
342
+ // the unit group. Outside, the match could begin on a newline ("\n2", breaking
343
+ // list-marker suppression) and every ordinal ("25th") was silently dropped.
344
+ const NUM_RE = new RegExp(
345
+ "(?<![0-9A-Za-z._\\-])(?<sign>[-+▲△])?(?:(?<cur>" + CUR_PAT + ")\\s{0,2})?(?<num>" + NUM_CORE + ")" +
346
+ "(?<exp>[eE][-+]?[0-9]{1,3})?\\s?(?<mag>" + MAG_PAT + ")?\\s?(?<pct>" + PCT_PAT + ")?" +
347
+ "(?:\\s?(?<unit>" + UNIT_PAT + ")(?![A-Za-z]))?", "g");
348
+
349
+ const MON = "Jan|January|Feb|February|Mar|March|Apr|April|May|Jun|June|Jul|July|Aug|August|Sep|Sept|September|Oct|October|Nov|November|Dec|December";
350
+ const DATE_PATTERNS = [
351
+ /(?:令和|平成|昭和|大正|明治)\s*(?:\d{1,2}|元)\s*年(?:\s*\d{1,2}\s*月(?:\s*\d{1,2}\s*日)?)?/g,
352
+ /\d{4}\s*年\s*\d{1,2}\s*月\s*\d{1,2}\s*日/g,
353
+ /\d{4}\s*年\s*\d{1,2}\s*月/g,
354
+ // NOT \b: in "契約は2024-01-05に" both sides are word characters, so \b never
355
+ // fires and ISO dates inside Japanese text were invisible.
356
+ /(?<![0-9A-Za-z])\d{4}[-/.]\d{1,2}[-/.]\d{1,2}(?![0-9A-Za-z])/g,
357
+ /(?<![0-9A-Za-z])\d{4}[-/]\d{1,2}(?![-/.\d])/g,
358
+ new RegExp("(?<![0-9A-Za-z])(?:" + MON + ")\\.?\\s+\\d{1,2}(?:st|nd|rd|th)?,?\\s+\\d{4}(?![0-9A-Za-z])", "gi"),
359
+ new RegExp("(?<![0-9A-Za-z])\\d{1,2}(?:st|nd|rd|th)?\\s+(?:" + MON + ")\\.?,?\\s+\\d{4}(?![0-9A-Za-z])", "gi"),
360
+ new RegExp("(?<![0-9A-Za-z])(?:" + MON + ")\\.?,?\\s+\\d{4}(?![0-9A-Za-z])", "gi"),
361
+ /(?<![0-9A-Za-z])(?:FY)\s?\d{4}(?![0-9A-Za-z])/gi,
362
+ /\d{4}\s*年度/g,
363
+ /(?<![0-9])\d{4}\s*年(?![0-9度])/g,
364
+ ];
365
+
366
+ // Relative expressions are counted but never become claims: they cannot be
367
+ // resolved without knowing when the answer was generated.
368
+ const RELATIVE_DATE_RE = /(?:昨年|一昨年|来年|再来年|今年|本年|先月|今月|来月|先週|今週|来週|昨日|本日|今日|明日|明後日|現在|最近|近年|直近|先般|過去\d+年|今後\d+年|\byesterday\b|\btoday\b|\btomorrow\b|\blast\s+(?:year|month|week|quarter)\b|\bnext\s+(?:year|month|week|quarter)\b|\bthis\s+(?:year|month|week|quarter)\b|\brecently\b|\bcurrently\b|\bnowadays\b)/gi;
369
+
370
+ const QUOTE_BRACKET_RE = /[「『“"]([^「」『』“”"\n]{4,180})[」』”"]/g;
371
+ const QUOTE_MARKER_RE = /(?:によると|によれば|には|と記載(?:されて)?(?:い)?(?:ます|る)?|と明記|と述べ|と書かれ|と報告|と発表|states?\s+that|according\s+to|reported\s+that|said\s+that|notes?\s+that|writes?\s+that|claims?\s+that|indicates?\s+that)/gi;
372
+
373
+ const ENT_CAPWORDS_RE = /\b[A-Z][A-Za-z0-9&'’.\-]{1,30}(?:\s+(?:of|the|and|for|de|von|van)\s+|\s+)(?:[A-Z][A-Za-z0-9&'’.\-]{1,30})(?:\s+[A-Z][A-Za-z0-9&'’.\-]{1,30}){0,3}/g;
374
+ const ENT_MODEL_RE = /\b(?=[A-Za-z0-9\-]{4,24}\b)(?=[^\s]*\d)[A-Z]{1,6}[A-Za-z]{0,4}[-_]?\d{2,6}[A-Za-z0-9\-]{0,8}\b/g;
375
+ const ENT_ARTICLE_RE = /(?:第\s*\d+\s*条(?:\s*の\s*\d+)?(?:\s*第\s*\d+\s*項)?|Article\s+\d+(?:\.\d+)?|Sections?\s+\d+(?:\.\d+)?|§\s?\d+(?:\.\d+)?|ISO\s?\d{3,5}(?::\d{4})?|RFC\s?\d{3,5})/gi;
376
+ const ENT_ACRONYM_RE = /\b[A-Z]{3,8}\b/g;
377
+
378
+ // ISO currency codes are generic tokens, not entities. Left in, the acronym
379
+ // extractor claimed "USD" and its span blocked the number after it, so
380
+ // "USD 45,000" lost its number entirely.
381
+ const CURRENCY_CODES = new Set("usd jpy eur gbp krw cny chf aud cad hkd sgd inr brl rub sek nok dkk nzd mxn zar thb twd php idr vnd pln czk huf try ils aed sar qar myr clp cop ars".split(" "));
382
+
383
+ const ENTITY_STOPWORDS = new Set((`
384
+ the a an and or but if then than that this these those there here it its it's is are was were be been
385
+ i you we they he she him her his our your their us them my me not no yes of in on at to for from by with
386
+ as into over under about after before between during without within across per via such same other another
387
+ however therefore moreover furthermore additionally meanwhile although because since while when where which who whom
388
+ note notes please thank thanks hello hi ok okay yes no true false null none
389
+ january february march april may june july august september october november december
390
+ monday tuesday wednesday thursday friday saturday sunday
391
+ summary conclusion introduction overview background result results method methods discussion reference references
392
+ answer question context example examples figure table appendix section chapter page pages
393
+ company service product system data user users client customer report document file page site page
394
+ ai llm api url json html http https pdf csv xml sql cpu gpu ram ssd usb faq ceo cfo cto url uri id ids
395
+ note caution warning important warning info tip
396
+ `).trim().split(/\s+/).concat([...CURRENCY_CODES]));
397
+
398
+ const ABBREVIATIONS = new Set("mr mrs ms dr prof st vs etc e.g i.e fig no inc ltd co corp jr sr approx al u.s u.k cf vol pp est dept univ jan feb mar apr jun jul aug sep sept oct nov dec".split(" "));
399
+ const SENT_TERMINATORS = "。.!?!?\n";
400
+
401
+ class SpanMask {
402
+ constructor(n) { this.m = new Uint8Array(Math.max(0, n)); }
403
+ add(s, e) { if (e > s && s >= 0) this.m.fill(1, s, Math.min(e, this.m.length)); }
404
+ addAll(spans) { for (const sp of spans) this.add(sp[0], sp[1]); }
405
+ hits(s, e) { if (e <= s) return false; for (let i = Math.max(0, s); i < Math.min(e, this.m.length); i++) if (this.m[i]) return true; return false; }
406
+ contains(s, e) { if (e <= s) return false; if (e > this.m.length) return false; for (let i = s; i < e; i++) if (!this.m[i]) return false; return true; }
407
+ }
408
+
409
+ function overlaps(a, b) { return !(a[1] <= b[0] || b[1] <= a[0]); }
410
+
411
+ /** Guarantees: [start, end, text] per sentence, offsets exact; '.' between digits never splits. */
412
+ export function splitSentences(text, protectedSpans = []) {
413
+ const out = [];
414
+ if (!text) return out;
415
+ const n = text.length;
416
+ let start = 0, i = 0;
417
+ while (i < n) {
418
+ const ch = text[i];
419
+ if (SENT_TERMINATORS.includes(ch)) {
420
+ let inside = false;
421
+ for (const r of protectedSpans) if (r[0] <= i && i < r[1]) { inside = true; break; }
422
+ if (inside) { i++; continue; }
423
+ if (ch === "." || ch === ".") {
424
+ const prev = i > 0 ? text[i - 1] : "", nxt = i + 1 < n ? text[i + 1] : "";
425
+ if (/\d/.test(prev) && /\d/.test(nxt)) { i++; continue; }
426
+ const tail = text.slice(Math.max(0, i - 12), i).toLowerCase();
427
+ const parts = tail.split(/[^a-z.]/);
428
+ const word = parts.length ? parts[parts.length - 1] : "";
429
+ if (word && ABBREVIATIONS.has(word)) { i++; continue; }
430
+ }
431
+ let j = i + 1;
432
+ while (j < n && (SENT_TERMINATORS.includes(text[j]) || "」』”\")".includes(text[j]))) j++;
433
+ const seg = text.slice(start, j);
434
+ if (seg.trim()) out.push([start, j, seg]);
435
+ start = j; i = j; continue;
436
+ }
437
+ i++;
438
+ }
439
+ if (start < n) { const seg = text.slice(start, n); if (seg.trim()) out.push([start, n, seg]); }
440
+ return out;
441
+ }
442
+
443
+ function* iterUrls(text) {
444
+ URL_RE.lastIndex = 0;
445
+ let m;
446
+ while ((m = URL_RE.exec(text || "")) !== null) {
447
+ const trimmed = m[0].replace(/[.,;:!?)»”"']+$/, "");
448
+ yield [m.index, m.index + trimmed.length, trimmed];
449
+ }
450
+ }
451
+
452
+ /** Guarantees: a comparable URL form (lowercased, scheme/www/trailing-slash stripped). */
453
+ export function normUrl(u) {
454
+ try { return normText(u).replace(/^(?:https?:\/\/|ftp:\/\/)/, "").replace(/^www\./, "").replace(/\/+$/, "").split("#")[0]; }
455
+ catch (e) { return ""; }
456
+ }
457
+ function urlHost(u) { try { return normUrl(u).split("/")[0].split("?")[0]; } catch (e) { return ""; } }
458
+
459
+ function numericClaimFromMatch(m, whole) {
460
+ const g = m.groups || {};
461
+ const raw = m[0].trim();
462
+ if (!raw) return null;
463
+ let base = Number((g.num || "").replace(/,/g, "") + (g.exp || ""));
464
+ if (!Number.isFinite(base)) return null;
465
+ if (g.sign === "-" || g.sign === "▲" || g.sign === "△") base = -base;
466
+ if (g.mag && MAG_FACTORS[g.mag] !== undefined) base *= MAG_FACTORS[g.mag];
467
+ let kind = "number";
468
+ if (g.pct === "%" || g.pct === "パーセント") kind = "percent";
469
+ else if (g.pct === "割") { base *= 10; kind = "percent"; }
470
+ else if (g.pct === "ポイント") kind = "point";
471
+ else if (g.cur) kind = "currency";
472
+ else if (g.unit) kind = "unit";
473
+
474
+ let start = m.index;
475
+ let end = m.index + m[0].replace(/\s+$/, "").length;
476
+ // Parenthesised accounting negatives: "(1,200)" is -1,200 in financial tables.
477
+ if (!g.sign && start > 0 && whole[start - 1] === "(" && end < whole.length && whole[end] === ")") {
478
+ base = -base; start -= 1; end += 1;
479
+ }
480
+ return { type: "NUMERIC", value: raw, normalized: String(base), kind,
481
+ unit: (g.pct || g.unit || g.cur || g.mag || "").trim(), span: [start, end], _num: base };
482
+ }
483
+
484
+ /** Guarantees: true for enumeration markers like "1." or "2)" at a line start. */
485
+ function isListMarker(text, m) {
486
+ const s = m.index;
487
+ const lineStart = text.lastIndexOf("\n", Math.max(0, s - 1)) + 1;
488
+ const before = text.slice(lineStart, s).trim();
489
+ if (!["", "-", "*", "・", "(", "("].includes(before)) return false;
490
+ const after = text.slice(m.index + m[0].length, m.index + m[0].length + 2);
491
+ return [".", ")", "、", ".", ")", ""].includes(after.slice(0, 1));
492
+ }
493
+
494
+ /** Guarantees: typed, span-accurate claims; sentences yielding none are what lowers coverage. */
495
+ export function extractClaims(answer, options = {}) {
496
+ try {
497
+ if (!answer) return [];
498
+ const o = Object.assign({ enable_quote: true, enable_entity: true, enable_url: true,
499
+ enable_numeric: true, enable_date: true, max_claims: DEFAULTS.MAX_CLAIMS }, options);
500
+ const n = answer.length;
501
+ const urlSpans = [], urls = [];
502
+ for (const [s, e, u] of iterUrls(answer)) { urlSpans.push([s, e]); urls.push([s, e, u]); }
503
+ let sentences = splitSentences(answer, urlSpans);
504
+ if (!sentences.length) sentences = [[0, n, answer]];
505
+ const sentStarts = sentences.map((x) => x[0]);
506
+ const sentIndex = (pos) => {
507
+ let lo = 0, hi = sentStarts.length - 1, ans = 0;
508
+ while (lo <= hi) { const mid = (lo + hi) >> 1; if (sentStarts[mid] <= pos) { ans = mid; lo = mid + 1; } else hi = mid - 1; }
509
+ return Math.min(ans, sentences.length - 1);
510
+ };
511
+
512
+ const claims = [];
513
+ const taken = new SpanMask(n);
514
+ if (o.enable_url) for (const [s, e, u] of urls)
515
+ claims.push({ type: "URL", value: u, normalized: normUrl(u), span: [s, e], sentence_index: sentIndex(s) });
516
+ taken.addAll(urlSpans);
517
+
518
+ const quoteMask = new SpanMask(n);
519
+ if (o.enable_quote) {
520
+ QUOTE_BRACKET_RE.lastIndex = 0; let m;
521
+ while ((m = QUOTE_BRACKET_RE.exec(answer)) !== null) {
522
+ const s = m.index + m[0].indexOf(m[1]), e = s + m[1].length;
523
+ if (taken.contains(s, e)) continue;
524
+ const body = m[1].trim();
525
+ if (body.length < 4) continue;
526
+ quoteMask.add(s, e);
527
+ claims.push({ type: "QUOTE", value: body, normalized: normText(body), span: [s, e], sentence_index: sentIndex(s), source: "bracket" });
528
+ }
529
+ QUOTE_MARKER_RE.lastIndex = 0;
530
+ while ((m = QUOTE_MARKER_RE.exec(answer)) !== null) {
531
+ const s = m.index + m[0].length;
532
+ const si = sentIndex(s);
533
+ const sentEnd = si < sentences.length ? sentences[si][1] : n;
534
+ const raw = answer.slice(s, sentEnd);
535
+ const lead = raw.length - raw.replace(/^[  ::,、\n]+/, "").length;
536
+ const body = raw.trim().replace(/^[  ::,、。.\n"'「」『』“”]+|[  ::,、。.\n"'「」『』“”]+$/g, "");
537
+ if (body.length < 8) continue;
538
+ const bs = s + lead, be = bs + body.length;
539
+ if (quoteMask.hits(bs, be) || taken.contains(bs, be)) continue;
540
+ quoteMask.add(bs, be);
541
+ claims.push({ type: "QUOTE", value: body.slice(0, 200), normalized: normText(body.slice(0, 200)),
542
+ span: [bs, Math.min(be, bs + 200)], sentence_index: si, source: "marker" });
543
+ }
544
+ }
545
+
546
+ const dateSpans = [], dateMask = new SpanMask(n), urlMask = new SpanMask(n);
547
+ urlMask.addAll(urlSpans);
548
+ if (o.enable_date) {
549
+ for (const pat of DATE_PATTERNS) {
550
+ pat.lastIndex = 0; let m;
551
+ while ((m = pat.exec(answer)) !== null) {
552
+ const s = m.index, e = m.index + m[0].length;
553
+ if (m[0].length === 0) { pat.lastIndex++; continue; }
554
+ if (dateMask.hits(s, e) || urlMask.contains(s, e)) continue;
555
+ const iso = normDate(m[0]);
556
+ if (!iso) continue;
557
+ dateSpans.push([s, e]); dateMask.add(s, e);
558
+ claims.push({ type: "DATE", value: m[0], normalized: iso, span: [s, e], sentence_index: sentIndex(s) });
559
+ }
560
+ }
561
+ }
562
+ taken.addAll(dateSpans);
563
+
564
+ const entSpans = [], entMask = new SpanMask(n);
565
+ if (o.enable_entity) {
566
+ for (const [pat, sub] of [[ENT_ARTICLE_RE, "article"], [ENT_MODEL_RE, "model"]]) {
567
+ pat.lastIndex = 0; let m;
568
+ while ((m = pat.exec(answer)) !== null) {
569
+ const s = m.index, e = m.index + m[0].length;
570
+ if (taken.contains(s, e) || entMask.hits(s, e)) continue;
571
+ const v = m[0].trim();
572
+ const lead = /^[A-Za-z]+/.exec(v); // "USD45,000" is a price, not a part number
573
+ if (sub === "model" && lead && CURRENCY_CODES.has(lead[0].toLowerCase())) continue;
574
+ entSpans.push([s, e]); entMask.add(s, e);
575
+ claims.push({ type: "ENTITY", value: v, normalized: normText(v), span: [s, e], sentence_index: sentIndex(s), subtype: sub });
576
+ }
577
+ }
578
+ ENT_CAPWORDS_RE.lastIndex = 0; let m;
579
+ while ((m = ENT_CAPWORDS_RE.exec(answer)) !== null) {
580
+ let s = m.index, e = m.index + m[0].length;
581
+ if (taken.contains(s, e) || entMask.hits(s, e)) continue;
582
+ let v = m[0].trim();
583
+ const toks = normText(v).split(/\s+/).filter(Boolean);
584
+ if (!toks.length) continue;
585
+ if (toks.every((t) => ENTITY_STOPWORDS.has(t.replace(/^[.,'’-]+|[.,'’-]+$/g, "")))) continue;
586
+ // Trim leading/trailing function words so "The NASA report" compares as "NASA report".
587
+ let words = v.split(/\s+/);
588
+ while (words.length > 1 && ENTITY_STOPWORDS.has(normText(words[0]).replace(/^[.,'’-]+|[.,'’-]+$/g, ""))) { s += words[0].length + 1; words = words.slice(1); }
589
+ while (words.length > 1 && ENTITY_STOPWORDS.has(normText(words[words.length - 1]).replace(/^[.,'’-]+|[.,'’-]+$/g, ""))) { e -= words[words.length - 1].length + 1; words = words.slice(0, -1); }
590
+ v = words.join(" ");
591
+ if (v.length < 4 || e <= s) continue;
592
+ entSpans.push([s, e]); entMask.add(s, e);
593
+ claims.push({ type: "ENTITY", value: v, normalized: normText(v), span: [s, e], sentence_index: sentIndex(s), subtype: "name" });
594
+ }
595
+ ENT_ACRONYM_RE.lastIndex = 0;
596
+ while ((m = ENT_ACRONYM_RE.exec(answer)) !== null) {
597
+ const s = m.index, e = m.index + m[0].length;
598
+ if (taken.contains(s, e) || entMask.hits(s, e)) continue;
599
+ if (ENTITY_STOPWORDS.has(normText(m[0]))) continue;
600
+ entSpans.push([s, e]); entMask.add(s, e);
601
+ claims.push({ type: "ENTITY", value: m[0], normalized: normText(m[0]), span: [s, e], sentence_index: sentIndex(s), subtype: "acronym" });
602
+ }
603
+ }
604
+
605
+ if (o.enable_numeric) {
606
+ const block = new SpanMask(n);
607
+ block.addAll(urlSpans); block.addAll(dateSpans); block.addAll(entSpans);
608
+ NUM_RE.lastIndex = 0; let m;
609
+ while ((m = NUM_RE.exec(answer)) !== null) {
610
+ if (m[0].length === 0) { NUM_RE.lastIndex++; continue; }
611
+ const s = m.index, e = m.index + m[0].length;
612
+ if (block.hits(s, e)) continue;
613
+ if (isListMarker(answer, m)) continue;
614
+ const c = numericClaimFromMatch(m, answer);
615
+ if (!c) continue;
616
+ c.sentence_index = sentIndex(c.span[0]);
617
+ claims.push(c);
618
+ }
619
+ }
620
+
621
+ claims.sort((a, b) => a.span[0] - b.span[0] || a.span[1] - b.span[1]);
622
+ let dropped = 0;
623
+ if (claims.length > o.max_claims) { dropped = claims.length - o.max_claims; claims.length = o.max_claims; }
624
+ claims.forEach((c, i) => { c.id = "c" + String(i).padStart(4, "0"); });
625
+ if (dropped) claims._dropped = dropped;
626
+ return claims;
627
+ } catch (e) { return []; }
628
+ }
629
+
630
+ /* ==================================================================== (D) VERIFIER */
631
+
632
+ /** Guarantees: a stable 9-significant-digit key, so 0.1+0.2 and 0.3 collide. */
633
+ function fkey(v) {
634
+ const f = Number(v);
635
+ if (!Number.isFinite(f)) return "";
636
+ if (f === 0) return "0";
637
+ let s = String(Number(f.toPrecision(9)));
638
+ return s === "-0" ? "0" : s;
639
+ }
640
+
641
+ /**
642
+ * Guarantees: a bounded, time-budgeted map from a derived value to its formula.
643
+ * Built once per verify() call, so each numeric claim is an O(1) lookup.
644
+ * Coincidental hits are possible (2 and 3 in context make 6 "derivable"), which
645
+ * is why the formula is always returned as evidence for a human to reject.
646
+ */
647
+ class DerivationIndex {
648
+ constructor(numbers, maxTerms, budgetMs, maxNumbers) {
649
+ this.map = new Map(); this.truncated = false;
650
+ const t0 = Date.now(), budget = Math.max(5, budgetMs);
651
+ const seen = new Set(), uniq = [];
652
+ for (const pair of numbers) {
653
+ const k = fkey(pair[0]);
654
+ if (!k || seen.has(k)) continue;
655
+ seen.add(k); uniq.push(pair);
656
+ if (uniq.length >= maxNumbers) { this.truncated = true; break; }
657
+ }
658
+ if (uniq.length < 2) return;
659
+ const pairs = [];
660
+ for (let i = 0; i < uniq.length; i++) {
661
+ for (let j = 0; j < uniq.length; j++) {
662
+ if (i === j) continue;
663
+ for (const r of DerivationIndex.combine(uniq[i][0], uniq[i][1], uniq[j][0], uniq[j][1], false)) {
664
+ this.put(r[0], r[1]);
665
+ if (pairs.length < 800) pairs.push(r);
666
+ }
667
+ }
668
+ if (Date.now() - t0 > budget) { this.truncated = true; return; }
669
+ }
670
+ if (maxTerms >= 3 && uniq.length <= 20) {
671
+ for (const p of pairs) {
672
+ if (Date.now() - t0 > budget) { this.truncated = true; return; }
673
+ for (const c of uniq)
674
+ for (const r of DerivationIndex.combine(p[0], "(" + p[1] + ")", c[0], c[1], true)) this.put(r[0], r[1]);
675
+ }
676
+ }
677
+ }
678
+ static combine(a, ar, b, br, third) {
679
+ const out = [[a + b, ar + " + " + br], [a - b, ar + " - " + br], [a * b, ar + " * " + br]];
680
+ if (b !== 0) {
681
+ out.push([a / b, ar + " / " + br]);
682
+ if (!third) {
683
+ out.push([(a / b) * 100, ar + " / " + br + " * 100 (percentage)"]);
684
+ out.push([((a - b) / b) * 100, "(" + ar + " - " + br + ") / " + br + " * 100 (change rate)"]);
685
+ }
686
+ }
687
+ if (!third) out.push([(a * b) / 100, ar + " * " + br + "% (percent of)"]);
688
+ return out.filter((x) => Number.isFinite(x[0]));
689
+ }
690
+ put(v, ex) { if (Math.abs(v) < 1e-9) return; const k = fkey(v); if (k && !this.map.has(k)) this.map.set(k, ex); }
691
+ lookup(v) { return this.map.get(fkey(v)) || null; }
692
+ }
693
+
694
+ function contextNumbers(context) {
695
+ const out = [];
696
+ try {
697
+ context = context || "";
698
+ const mask = new SpanMask(context.length);
699
+ for (const u of iterUrls(context)) mask.add(u[0], u[1]);
700
+ for (const pat of DATE_PATTERNS) {
701
+ pat.lastIndex = 0; let m;
702
+ while ((m = pat.exec(context)) !== null) { if (!m[0].length) { pat.lastIndex++; continue; } mask.add(m.index, m.index + m[0].length); }
703
+ }
704
+ NUM_RE.lastIndex = 0; let m;
705
+ while ((m = NUM_RE.exec(context)) !== null) {
706
+ if (!m[0].length) { NUM_RE.lastIndex++; continue; }
707
+ if (mask.hits(m.index, m.index + m[0].length)) continue;
708
+ const c = numericClaimFromMatch(m, context);
709
+ if (c) out.push([c._num, c.value]);
710
+ }
711
+ } catch (e) {}
712
+ return out;
713
+ }
714
+
715
+ function contextDates(context) {
716
+ const out = new Set();
717
+ for (const pat of DATE_PATTERNS) {
718
+ pat.lastIndex = 0; let m;
719
+ while ((m = pat.exec(context || "")) !== null) {
720
+ if (!m[0].length) { pat.lastIndex++; continue; }
721
+ const iso = normDate(m[0]); if (iso) out.add(iso);
722
+ }
723
+ }
724
+ return out;
725
+ }
726
+
727
+ /** Guarantees: one pass over the context produces every lookup table the verifiers need. */
728
+ class ContextIndex {
729
+ constructor(context, opts) {
730
+ this.raw = context || "";
731
+ this.norm = normText(this.raw);
732
+ this.numbers = contextNumbers(this.raw);
733
+ this.numMap = new Map();
734
+ for (const p of this.numbers) { const k = fkey(p[0]); if (k && !this.numMap.has(k)) this.numMap.set(k, p[1]); }
735
+ this.dates = contextDates(this.raw);
736
+ // Years stated only as part of a context date still support a bare year in
737
+ // the answer; without this, context "FY2024" vs answer "2024" was unsupported.
738
+ this.dateYears = new Set(Array.from(this.dates).map((d) => parseInt(d.split("-")[0], 10)).filter(Number.isFinite));
739
+ this.urls = new Map();
740
+ for (const u of iterUrls(this.raw)) this.urls.set(normUrl(u[2]), u[2]);
741
+ this.hosts = new Set(Array.from(this.urls.values()).map(urlHost));
742
+ this.tokens = new Set(this.norm.split(/[^0-9a-z぀-ヿ一-鿿]+/).filter(Boolean));
743
+ this._deriv = null;
744
+ this._opts = opts || {};
745
+ const fb = this._opts.fuzzy_budget_ms;
746
+ this.fuzzyDeadline = Date.now() + (fb === undefined ? DEFAULTS.FUZZY_BUDGET_MS : fb);
747
+ this.fuzzyExhausted = false;
748
+ }
749
+ get deriv() {
750
+ if (!this._deriv) {
751
+ const o = this._opts;
752
+ this._deriv = new DerivationIndex(this.numbers,
753
+ o.derive_max_terms === undefined ? DEFAULTS.DERIVE_MAX_TERMS : o.derive_max_terms,
754
+ o.derive_budget_ms === undefined ? DEFAULTS.DERIVE_BUDGET_MS : o.derive_budget_ms,
755
+ o.derive_max_numbers === undefined ? DEFAULTS.DERIVE_MAX_NUMBERS : o.derive_max_numbers);
756
+ }
757
+ return this._deriv;
758
+ }
759
+ /** Guarantees: a bounded fuzzy match, or [0,""] once the request budget is spent. */
760
+ fuzzy(needle) {
761
+ if (Date.now() > this.fuzzyDeadline) { this.fuzzyExhausted = true; return [0.0, ""]; }
762
+ return bestFuzzy(needle, this.norm);
763
+ }
764
+ }
765
+
766
+ function nearestNumber(target, candidates) {
767
+ let best = null;
768
+ for (const p of candidates) {
769
+ const f = p[0];
770
+ const denom = Math.max(Math.abs(target), Math.abs(f), 1e-12);
771
+ const rel = Math.abs(target - f) / denom;
772
+ if (!best || rel < best[2]) best = [f, p[1], rel];
773
+ }
774
+ return best;
775
+ }
776
+
777
+ function orderOfMagnitudeOff(a, b) {
778
+ if (a === 0 || b === 0) return null;
779
+ const r = Math.abs(a) / Math.abs(b);
780
+ if (!(r > 0)) return null;
781
+ const lg = Math.log10(r), k = Math.round(lg);
782
+ return k !== 0 && Math.abs(lg - k) < 0.02 && Math.abs(k) <= 9 ? k : null;
783
+ }
784
+
785
+ /*
786
+ * NUMERIC. Misfires that can happen, concretely:
787
+ * 1) context "approx 1,200 items" vs answer "1200 items" -> supported after
788
+ * normalisation. But context "1,200" vs answer "1,250" is 4% off ->
789
+ * contradicted. To allow rounding raise numeric_tolerance, do NOT lower
790
+ * contradiction_rel.
791
+ * 2) context "revenue 100, cost 40" vs answer "profit 60" -> derived (100 - 40).
792
+ * 3) context "3.5%" vs answer "3.5 points" matches on value -> supported.
793
+ * Semantic unit errors are invisible to this tool.
794
+ */
795
+ function verifyNumeric(claim, ctx, o) {
796
+ let f = claim._num;
797
+ if (f === undefined || f === null) f = normNumber(claim.value);
798
+ if (f === null || !Number.isFinite(f)) { claim.status = "unsupported"; claim.evidence = null; claim.reason = "unparseable_number"; return claim; }
799
+ const tol = o.numeric_tolerance === undefined ? 0 : o.numeric_tolerance;
800
+ const contrRel = o.contradiction_rel === undefined ? DEFAULTS.CONTRADICTION_REL : o.contradiction_rel;
801
+
802
+ const k = fkey(f);
803
+ if (k && ctx.numMap.has(k)) { claim.status = "supported"; claim.evidence = "context literal: " + ctx.numMap.get(k); return claim; }
804
+ if (tol > 0) {
805
+ const near = nearestNumber(f, ctx.numbers);
806
+ if (near && near[2] <= tol) { claim.status = "supported"; claim.evidence = "context literal within tolerance " + tol + ": " + near[1]; return claim; }
807
+ }
808
+ if (claim.kind === "percent") {
809
+ const ak = fkey(f / 100);
810
+ if (ak && ctx.numMap.has(ak)) { claim.status = "derived"; claim.evidence = (f / 100) + " == " + f + "% (fraction form in context) -> " + ctx.numMap.get(ak); return claim; }
811
+ } else if (Math.abs(f) < 1) {
812
+ const ak = fkey(f * 100);
813
+ if (ak && ctx.numMap.has(ak)) { claim.status = "derived"; claim.evidence = f + " == " + (f * 100) + "% (percent form in context: " + ctx.numMap.get(ak) + ")"; return claim; }
814
+ }
815
+ // An exact sign flip is settled BEFORE derivation. Otherwise a loss reported
816
+ // as a profit gets excused by a coincidental 3-term formula and passes.
817
+ if (f !== 0) {
818
+ const flip = fkey(-f);
819
+ if (flip && ctx.numMap.has(flip)) {
820
+ claim.status = "contradicted";
821
+ claim.evidence = "context states " + ctx.numMap.get(flip) + ", the exact negation of this value (sign error)";
822
+ claim.nearest_context_value = ctx.numMap.get(flip);
823
+ return claim;
824
+ }
825
+ }
826
+ if (o.enable_derivation !== false) {
827
+ const expr = ctx.deriv.lookup(f);
828
+ if (expr) { claim.status = "derived"; claim.evidence = "derivable from context: " + expr + " = " + fkey(f); return claim; }
829
+ }
830
+ if (claim.kind === "number" && !claim.unit && Number.isInteger(f) && f >= 1900 && f <= 2100 && ctx.dateYears.has(f)) {
831
+ claim.status = "supported"; claim.evidence = "matches a year stated in the context (" + f + ")"; return claim;
832
+ }
833
+ const near = nearestNumber(f, ctx.numbers);
834
+ if (near) {
835
+ const nf = near[0], nraw = near[1], rel = near[2];
836
+ const oom = orderOfMagnitudeOff(f, nf);
837
+ // Bare integers 0-10 with no unit are near everything; never contradicted.
838
+ const bareSmall = claim.kind === "number" && Math.abs(f) <= 10 && Number.isInteger(f);
839
+ if (oom !== null && !bareSmall) {
840
+ claim.status = "contradicted";
841
+ claim.evidence = "closest context value " + nraw + " differs by 10^" + oom + " (order-of-magnitude error)";
842
+ claim.nearest_context_value = nraw; return claim;
843
+ }
844
+ if (!bareSmall && nf !== 0 && f === -nf) {
845
+ claim.status = "contradicted"; claim.evidence = "closest context value " + nraw + " has the opposite sign";
846
+ claim.nearest_context_value = nraw; return claim;
847
+ }
848
+ if (!bareSmall && rel <= contrRel) {
849
+ claim.status = "contradicted";
850
+ claim.evidence = "closest context value is " + nraw + " (relative difference " + (rel * 100).toFixed(1) + "%)";
851
+ claim.nearest_context_value = nraw; return claim;
852
+ }
853
+ claim.nearest_context_value = nraw;
854
+ }
855
+ claim.status = "unsupported"; claim.evidence = null; return claim;
856
+ }
857
+
858
+ /*
859
+ * DATE. Misfires that can happen, concretely:
860
+ * - context "2024-01" vs answer "2024-01-15" -> approximate: the model added
861
+ * precision the source never stated.
862
+ * - FY2024 and calendar 2024 are treated as the same year. Fiscal-year start
863
+ * months differ by country and company, so this is a known false positive.
864
+ */
865
+ function verifyDate(claim, ctx) {
866
+ const iso = claim.normalized || normDate(claim.value);
867
+ if (!iso) { claim.status = "unsupported"; claim.evidence = null; claim.reason = "unparseable_date"; return claim; }
868
+ if (ctx.dates.has(iso)) { claim.status = "supported"; claim.evidence = "context date: " + iso; return claim; }
869
+ for (const d of ctx.dates) if (d.startsWith(iso)) { claim.status = "supported"; claim.evidence = "context date " + d + " falls inside " + iso; return claim; }
870
+ for (const d of ctx.dates) if (iso.startsWith(d)) { claim.status = "approximate"; claim.evidence = "context only states " + d + "; the answer adds precision (" + iso + ")"; return claim; }
871
+ let best = null;
872
+ for (const d of ctx.dates) {
873
+ let common = 0;
874
+ const a = iso.split("-"), b = d.split("-");
875
+ for (let i = 0; i < Math.min(a.length, b.length); i++) { if (a[i] === b[i]) common++; else break; }
876
+ if (!best || common > best[1]) best = [d, common];
877
+ }
878
+ if (best && best[1] >= 1) { claim.status = "contradicted"; claim.evidence = "context has a near but different date: " + best[0]; claim.nearest_context_value = best[0]; return claim; }
879
+ claim.status = "unsupported"; claim.evidence = null; return claim;
880
+ }
881
+
882
+ /*
883
+ * QUOTE. Misfires that can happen, concretely:
884
+ * - A politeness-level paraphrase comes out approximate: the meaning survived,
885
+ * the quote did not. That is intended.
886
+ * - KNOWN LIMITATION: a one-character semantic reversal in Japanese
887
+ * (increase -> decrease, approve -> reject) keeps similarity above 0.9 and
888
+ * stays "approximate". A hard limit of character-level matching.
889
+ */
890
+ function verifyQuote(claim, ctx, o) {
891
+ const q = claim.normalized || normText(claim.value);
892
+ if (!q || q.length < 3) { claim.status = "unsupported"; claim.evidence = null; claim.reason = "quote_too_short"; return claim; }
893
+ if (ctx.norm.includes(q)) { claim.status = "supported"; claim.evidence = "exact (normalised) substring of the context"; return claim; }
894
+ const res = ctx.fuzzy(q), ratio = res[0], win = res[1];
895
+ const thr = o.approx_ratio === undefined ? DEFAULTS.APPROX_RATIO : o.approx_ratio;
896
+ claim.similarity = Math.round(ratio * 10000) / 10000;
897
+ if (ratio >= thr) { claim.status = "approximate"; claim.evidence = "closest context passage (similarity " + ratio.toFixed(2) + "): " + win.slice(0, 180); return claim; }
898
+ const floor = o.quote_contradiction_floor === undefined ? DEFAULTS.QUOTE_CONTRADICTION_FLOOR : o.quote_contradiction_floor;
899
+ if (ratio >= floor) {
900
+ claim.status = "contradicted";
901
+ claim.evidence = "a similar but materially different passage exists (similarity " + ratio.toFixed(2) + "): " + win.slice(0, 180);
902
+ claim.nearest_context_value = win.slice(0, 180); return claim;
903
+ }
904
+ claim.status = "unsupported"; claim.evidence = null; return claim;
905
+ }
906
+
907
+ /*
908
+ * ENTITY. Misfires that can happen, concretely:
909
+ * - Generic noun phrases like "Machine Learning" come out unsupported although
910
+ * nothing was hallucinated.
911
+ * - Acronym expansion is not handled: context "World Health Organization" vs
912
+ * answer "WHO" finds no match.
913
+ * ENTITY is the noisiest check. Disable it first.
914
+ */
915
+ function verifyEntity(claim, ctx) {
916
+ const v = claim.normalized || normText(claim.value);
917
+ if (!v) { claim.status = "unsupported"; claim.evidence = null; return claim; }
918
+ if (ENTITY_STOPWORDS.has(v)) { claim.status = "supported"; claim.evidence = "common word, not treated as a factual entity"; claim.suppressed = true; return claim; }
919
+ if (ctx.norm.includes(v)) { claim.status = "supported"; claim.evidence = "appears verbatim in the context"; return claim; }
920
+ const toks = v.split(/[^0-9a-z぀-ヿ一-鿿]+/).filter(Boolean);
921
+ const meaningful = toks.filter((t) => !ENTITY_STOPWORDS.has(t));
922
+ if (meaningful.length && meaningful.every((t) => ctx.tokens.has(t))) {
923
+ claim.status = "approximate"; claim.evidence = "all tokens appear in the context but not as this phrase"; return claim;
924
+ }
925
+ const res = ctx.fuzzy(v), ratio = res[0], win = res[1];
926
+ claim.similarity = Math.round(ratio * 10000) / 10000;
927
+ // Identifiers are exact-or-wrong: "Article 15" is not "approximately Article
928
+ // 12". This branch must precede the generic near-match branch below, or a
929
+ // one-digit-off article number is reported as a harmless approximation.
930
+ if ((claim.subtype === "model" || claim.subtype === "article") && ratio >= 0.72) {
931
+ claim.status = "contradicted"; claim.evidence = "a similar identifier exists in the context: " + win.slice(0, 120);
932
+ claim.nearest_context_value = win.slice(0, 120); return claim;
933
+ }
934
+ if (ratio >= 0.90 && v.length >= 6) { claim.status = "approximate"; claim.evidence = "near match in context (similarity " + ratio.toFixed(2) + "): " + win.slice(0, 120); return claim; }
935
+ claim.status = "unsupported"; claim.evidence = null; return claim;
936
+ }
937
+
938
+ /*
939
+ * URL. Misfires that can happen, concretely:
940
+ * - A tracking parameter makes an otherwise identical link fail exact matching.
941
+ * - Host matches but path differs -> contradicted. It may be a legitimate
942
+ * parent page rather than a fabricated one.
943
+ */
944
+ function verifyUrl(claim, ctx) {
945
+ const u = claim.normalized || normUrl(claim.value);
946
+ if (!u) { claim.status = "unsupported"; claim.evidence = null; return claim; }
947
+ if (ctx.urls.has(u)) { claim.status = "supported"; claim.evidence = "context URL: " + ctx.urls.get(u); return claim; }
948
+ for (const entry of ctx.urls) {
949
+ const cu = entry[0], orig = entry[1];
950
+ if (cu.startsWith(u) || u.startsWith(cu)) { claim.status = "approximate"; claim.evidence = "context has a related URL: " + orig; return claim; }
951
+ }
952
+ const h = urlHost(u);
953
+ if (h && ctx.hosts.has(h)) {
954
+ claim.status = "contradicted";
955
+ claim.evidence = "host " + h + " appears in the context but this exact path does not (possible fabricated link)";
956
+ claim.nearest_context_value = h; return claim;
957
+ }
958
+ claim.status = "unsupported"; claim.evidence = null; return claim;
959
+ }
960
+
961
+ const VERIFIERS = { NUMERIC: verifyNumeric, DATE: verifyDate, QUOTE: verifyQuote, ENTITY: verifyEntity, URL: verifyUrl };
962
+
963
+ function nowMs() { return (typeof performance !== "undefined" && performance.now) ? performance.now() : Date.now(); }
964
+
965
+ /** Guarantees: grounding_score AND coverage, separately, plus a status per claim. Never throws. */
966
+ export function verify(answer, context, options) {
967
+ const t0 = nowMs();
968
+ const o = Object.assign({}, options || {});
969
+ const maxChars = o.max_chars === undefined ? DEFAULTS.MAX_TEXT_CHARS : o.max_chars;
970
+ const aTrunc = (answer || "").length > maxChars, cTrunc = (context || "").length > maxChars;
971
+ answer = (answer || "").slice(0, maxChars);
972
+ context = (context || "").slice(0, maxChars);
973
+
974
+ const claims = extractClaims(answer, o);
975
+ const claimsDropped = claims._dropped || 0;
976
+ const ctx = new ContextIndex(context, o);
977
+ const urlSpans = [];
978
+ for (const u of iterUrls(answer)) urlSpans.push([u[0], u[1]]);
979
+ const sentences = splitSentences(answer, urlSpans);
980
+ const nSent = sentences.length;
981
+ const sentWith = new Set();
982
+
983
+ // Identical claim text gets the identical verdict against the same context,
984
+ // so a repetitive answer costs one verification per distinct value.
985
+ const memo = new Map();
986
+ const CARRY = ["status", "evidence", "reason", "similarity", "nearest_context_value", "suppressed"];
987
+ for (const c of claims) {
988
+ const key = [c.type, c.normalized, c.value, c.subtype].join(" |~| ");
989
+ const cached = memo.get(key);
990
+ if (cached) { for (const k of CARRY) if (k in cached) c[k] = cached[k]; }
991
+ else {
992
+ const fn = VERIFIERS[c.type];
993
+ try { if (!fn) { c.status = "unsupported"; c.evidence = null; } else fn(c, ctx, o); }
994
+ catch (e) { c.status = "unsupported"; c.evidence = null; c.reason = "verifier_error"; }
995
+ const snap = {}; for (const k of CARRY) if (k in c) snap[k] = c[k];
996
+ memo.set(key, snap);
997
+ }
998
+ if (!c.suppressed) sentWith.add(c.sentence_index);
999
+ }
1000
+
1001
+ const counts = {}; STATUSES.forEach((s) => (counts[s] = 0));
1002
+ const byType = {}; CLAIM_TYPES.forEach((t) => { byType[t] = {}; STATUSES.forEach((s) => (byType[t][s] = 0)); });
1003
+ const effective = [];
1004
+ for (const c of claims) {
1005
+ if (c.suppressed) continue;
1006
+ effective.push(c);
1007
+ counts[c.status] = (counts[c.status] || 0) + 1;
1008
+ if (byType[c.type]) byType[c.type][c.status] = (byType[c.type][c.status] || 0) + 1;
1009
+ }
1010
+ const total = effective.length;
1011
+ const good = counts.supported + counts.derived;
1012
+ const grounding = total ? good / total : 0;
1013
+ const coverage = nSent ? sentWith.size / nSent : 0;
1014
+ RELATIVE_DATE_RE.lastIndex = 0;
1015
+ const relDates = (answer.match(RELATIVE_DATE_RE) || []).length;
1016
+
1017
+ const notes = [
1018
+ "grounding_score is computed over verifiable claims only.",
1019
+ "coverage is the share of sentences that produced at least one verifiable claim; the rest were NOT checked.",
1020
+ ];
1021
+ if (claimsDropped) notes.push(claimsDropped + " further claims were dropped by the max_claims cap and were NOT checked.");
1022
+ if (ctx.fuzzyExhausted) notes.push("The fuzzy-matching budget ran out; later QUOTE/ENTITY claims were matched exactly only.");
1023
+
1024
+ const unverified = [];
1025
+ for (let i = 0; i < sentences.length && unverified.length < 50; i++)
1026
+ if (!sentWith.has(i)) unverified.push({ sentence_index: i, text: sentences[i][2].trim().slice(0, 200) });
1027
+
1028
+ return {
1029
+ ok: true,
1030
+ grounding_score: Math.round(grounding * 10000) / 10000,
1031
+ coverage: Math.round(coverage * 10000) / 10000,
1032
+ claims: effective.map((c) => { const d = {}; for (const k in c) if (!k.startsWith("_")) d[k] = c[k]; if (!("evidence" in d)) d.evidence = null; return d; }),
1033
+ counts: {
1034
+ claims_total: total, sentences_total: nSent, sentences_verified: sentWith.size,
1035
+ sentences_unverified: Math.max(0, nSent - sentWith.size), relative_date_mentions: relDates,
1036
+ by_status: counts, by_type: byType,
1037
+ context_numbers: ctx.numbers.length, context_dates: ctx.dates.size,
1038
+ derivation_entries: ctx._deriv ? ctx._deriv.map.size : 0,
1039
+ derivation_truncated: ctx._deriv ? ctx._deriv.truncated : false,
1040
+ claims_dropped_by_cap: claimsDropped, distinct_claims_verified: memo.size,
1041
+ fuzzy_budget_exhausted: ctx.fuzzyExhausted,
1042
+ },
1043
+ unverified_sentences: unverified,
1044
+ truncated: { answer: aTrunc, context: cTrunc },
1045
+ latency_ms: Math.round((nowMs() - t0) * 100) / 100,
1046
+ notes,
1047
+ };
1048
+ }
1049
+
1050
+ /* ============================================================== (E) LEAK DETECTOR */
1051
+
1052
+ function leakTokens(s) {
1053
+ const out = normText(s).match(/[0-9a-z]+|[぀-ヿ一-鿿가-힯]/g);
1054
+ return out || [];
1055
+ }
1056
+ function ngrams(tokens, n) {
1057
+ const out = new Set();
1058
+ if (n <= 0 || tokens.length < n) return out;
1059
+ for (let i = 0; i <= tokens.length - n; i++) out.add(tokens.slice(i, i + n).join(" "));
1060
+ return out;
1061
+ }
1062
+
1063
+ /**
1064
+ * Guarantees: the share of answer n-grams that echo the system prompt, plus the
1065
+ * longest shared fragment as evidence.
1066
+ * Misfire: boilerplate such as "answer concisely in Japanese" is shared
1067
+ * legitimately and raises overlap. Tune the threshold per deployment.
1068
+ */
1069
+ export function detectSystemPromptLeak(answer, systemPrompt, n, threshold) {
1070
+ try {
1071
+ n = n === undefined ? DEFAULTS.NGRAM_LEAK_N : Math.max(3, n);
1072
+ threshold = threshold === undefined ? DEFAULTS.LEAK_THRESHOLD : threshold;
1073
+ answer = (answer || "").slice(0, DEFAULTS.MAX_TEXT_CHARS);
1074
+ systemPrompt = (systemPrompt || "").slice(0, DEFAULTS.MAX_TEXT_CHARS);
1075
+ if (!answer || !systemPrompt)
1076
+ return { ok: true, leak: false, overlap: 0, n, threshold, matched_ngrams: 0, total_ngrams: 0, longest_fragment: "", fragment_len: 0, reason: "empty_input" };
1077
+ const aNg = ngrams(leakTokens(answer), n), sNg = ngrams(leakTokens(systemPrompt), n);
1078
+ let inter = 0;
1079
+ const sample = [];
1080
+ for (const g of aNg) if (sNg.has(g)) { inter++; if (sample.length < 5) sample.push(g); }
1081
+ const overlap = aNg.size ? inter / aNg.size : 0;
1082
+ const frag = longestCommonFragment(normText(answer), normText(systemPrompt));
1083
+ return { ok: true, leak: overlap >= threshold && inter > 0, overlap: Math.round(overlap * 10000) / 10000,
1084
+ n, threshold, matched_ngrams: inter, total_ngrams: aNg.size,
1085
+ longest_fragment: frag.slice(0, 400), fragment_len: frag.length, sample_ngrams: sample };
1086
+ } catch (e) { return { ok: true, leak: false, overlap: 0, n, threshold, matched_ngrams: 0, total_ngrams: 0, longest_fragment: "", fragment_len: 0 }; }
1087
+ }
1088
+
1089
+ const INSTRUCTION_PATTERNS = [
1090
+ /ignore\s+(?:all\s+)?(?:the\s+)?(?:previous|prior|above|earlier)\s+instructions?/gi,
1091
+ /disregard\s+(?:all\s+)?(?:the\s+)?(?:previous|prior|above)/gi,
1092
+ /(?:reveal|print|repeat|output|show|dump)\s+(?:me\s+)?(?:your|the)\s+(?:system\s+)?(?:prompt|instructions?|rules?)/gi,
1093
+ /you\s+are\s+now\b/gi, /\bact\s+as\b/gi, /\bdeveloper\s+mode\b/gi, /\bjailbreak\b/gi,
1094
+ /(?:これまでの|以前の|上記の)(?:指示|命令|ルール)(?:は|を)?(?:すべて)?(?:無視|忘れ)/g,
1095
+ /システム\s*プロンプト(?:を)?(?:表示|出力|教え|見せ)/g,
1096
+ /(?:あなたは|君は)今から/g,
1097
+ ];
1098
+
1099
+ /** Guarantees: reports whether imperative strings from the user input reappear in the answer. */
1100
+ export function detectInjectionEcho(answer, userInput) {
1101
+ try {
1102
+ answer = (answer || "").slice(0, DEFAULTS.MAX_TEXT_CHARS);
1103
+ userInput = (userInput || "").slice(0, DEFAULTS.MAX_TEXT_CHARS);
1104
+ if (!answer || !userInput) return { ok: true, echo: false, matches: [], reason: "empty_input" };
1105
+ const an = normText(answer);
1106
+ const found = [];
1107
+ for (const pat of INSTRUCTION_PATTERNS) {
1108
+ pat.lastIndex = 0; let m;
1109
+ while ((m = pat.exec(userInput)) !== null) {
1110
+ if (!m[0].length) { pat.lastIndex++; continue; }
1111
+ const phrase = m[0].trim(), pn = normText(phrase);
1112
+ if (!pn) continue;
1113
+ const exact = an.includes(pn);
1114
+ const ratio = exact ? 1 : bestFuzzy(pn, an)[0];
1115
+ found.push({ instruction: phrase.slice(0, 160), in_answer: exact || ratio >= 0.88, similarity: Math.round(ratio * 1000) / 1000, exact });
1116
+ }
1117
+ }
1118
+ const echoed = found.filter((f) => f.in_answer);
1119
+ return { ok: true, echo: echoed.length > 0, matches: found.slice(0, 20), echoed_count: echoed.length, instruction_like_in_input: found.length };
1120
+ } catch (e) { return { ok: true, echo: false, matches: [] }; }
1121
+ }
1122
+
1123
+ /* ================================================================ (F) OUTPUT SAFETY */
1124
+
1125
+ const CC_RE = /(?<![0-9])(?:[0-9]{4}[ \-]?){3}[0-9]{1,7}(?![0-9])/g;
1126
+ const JWT_RE = /\beyJ[A-Za-z0-9_\-]{6,}\.[A-Za-z0-9_\-]{6,}\.[A-Za-z0-9_\-]{0,600}/g;
1127
+ const EMAIL_RE = /\b[A-Za-z0-9._%+\-]+@[A-Za-z0-9.\-]+\.[A-Za-z]{2,24}\b/g;
1128
+ const PHONE_RE = /(?<![0-9])(?:\+\d{1,3}[ \-]?)?(?:\(0?\d{1,4}\)|0\d{1,4})[ \-]?\d{1,4}[ \-]?\d{3,4}(?![0-9])/g;
1129
+ const PRIVKEY_RE = /-----BEGIN\s+(?:RSA|DSA|EC|OPENSSH|PGP|ENCRYPTED)?\s*PRIVATE KEY(?:\s+BLOCK)?-----/g;
1130
+ const HIGH_ENTROPY_RE = new RegExp("[A-Za-z0-9+/=_\\-]{" + DEFAULTS.ENTROPY_MIN_LEN + ",200}", "g");
1131
+ const ENTROPY_ALLOW_RE = /^(?:[0-9a-f]{32}|[0-9a-f]{40}|[0-9a-f]{64})$/i;
1132
+
1133
+ const CRED_PREFIXES = [
1134
+ ["openai_key", /\bsk-(?:proj-|svcacct-)?[A-Za-z0-9_\-]{16,}/g],
1135
+ ["anthropic_key", /\bsk-ant-[A-Za-z0-9_\-]{16,}/g],
1136
+ ["github_token", /\b(?:ghp|gho|ghu|ghs|ghr)_[A-Za-z0-9]{20,}|\bgithub_pat_[A-Za-z0-9_]{20,}/g],
1137
+ ["aws_access_key", /\b(?:AKIA|ASIA|AGPA|AIDA|AROA|ANPA|ANVA)[A-Z0-9]{12,}/g],
1138
+ ["slack_token", /\bxox[abprs]-[A-Za-z0-9\-]{8,}/g],
1139
+ ["google_api_key", /\bAIza[A-Za-z0-9_\-]{30,}/g],
1140
+ ["google_oauth", /\bya29\.[A-Za-z0-9_\-]{20,}/g],
1141
+ ["gitlab_token", /\bglpat-[A-Za-z0-9_\-]{16,}/g],
1142
+ ["hf_token", /\bhf_[A-Za-z0-9]{20,}/g],
1143
+ ["npm_token", /\bnpm_[A-Za-z0-9]{30,}/g],
1144
+ ["sendgrid_key", /\bSG\.[A-Za-z0-9_\-]{16,}\.[A-Za-z0-9_\-]{16,}/g],
1145
+ ["stripe_key", /\b(?:sk|rk|pk)_(?:live|test)_[A-Za-z0-9]{16,}/g],
1146
+ ["digitalocean_token", /\bdop_v1_[a-f0-9]{32,}/g],
1147
+ ["twilio_sid", /\bAC[a-f0-9]{32}\b/g],
1148
+ ];
1149
+ const SAFETY_SEVERITY = {
1150
+ credit_card: "critical", private_key: "critical", jwt: "critical", openai_key: "critical",
1151
+ anthropic_key: "critical", github_token: "critical", aws_access_key: "critical", slack_token: "critical",
1152
+ google_api_key: "critical", google_oauth: "critical", gitlab_token: "critical", hf_token: "critical",
1153
+ npm_token: "critical", sendgrid_key: "critical", stripe_key: "critical", digitalocean_token: "critical",
1154
+ twilio_sid: "high", high_entropy_string: "medium", email: "low", phone: "low",
1155
+ };
1156
+
1157
+ /** Guarantees: Shannon entropy in bits/char, 0 for empty input. */
1158
+ export function shannonEntropy(s) {
1159
+ if (!s) return 0;
1160
+ const n = s.length, counts = new Map();
1161
+ for (const ch of s) counts.set(ch, (counts.get(ch) || 0) + 1);
1162
+ let acc = 0;
1163
+ for (const c of counts.values()) { const p = c / n; acc -= p * Math.log2(p); }
1164
+ return acc;
1165
+ }
1166
+
1167
+ /** Guarantees: true only if the digit string passes the Luhn mod-10 checksum. */
1168
+ export function luhnOk(digits) {
1169
+ const ds = String(digits).replace(/\D/g, "").split("").map(Number);
1170
+ if (ds.length < 13 || ds.length > 19) return false;
1171
+ let total = 0;
1172
+ const parity = ds.length % 2;
1173
+ for (let i = 0; i < ds.length; i++) {
1174
+ let d = ds[i];
1175
+ if (i % 2 === parity) { d *= 2; if (d > 9) d -= 9; }
1176
+ total += d;
1177
+ }
1178
+ return total % 10 === 0;
1179
+ }
1180
+
1181
+ function mask(s) {
1182
+ s = String(s);
1183
+ if (s.length <= 6) return "*".repeat(s.length);
1184
+ return s.slice(0, 3) + "*".repeat(Math.max(3, s.length - 5)) + s.slice(-2);
1185
+ }
1186
+
1187
+ /**
1188
+ * Guarantees: a list of findings (type/span/confidence/severity); never throws.
1189
+ * Credit-card candidates are reported ONLY after passing Luhn, because a bare
1190
+ * 16-digit run is far more often an order number than a card.
1191
+ */
1192
+ export function scanOutput(answer) {
1193
+ const findings = [];
1194
+ try {
1195
+ answer = (answer || "").slice(0, DEFAULTS.MAX_TEXT_CHARS);
1196
+ if (!answer) return findings;
1197
+ let m;
1198
+ PRIVKEY_RE.lastIndex = 0;
1199
+ while ((m = PRIVKEY_RE.exec(answer)) !== null)
1200
+ findings.push({ type: "private_key", value: m[0].slice(0, 30), span: [m.index, m.index + m[0].length], confidence: 0.99, severity: "critical" });
1201
+ for (const [name, pat] of CRED_PREFIXES) {
1202
+ pat.lastIndex = 0;
1203
+ while ((m = pat.exec(answer)) !== null)
1204
+ findings.push({ type: name, value: mask(m[0]), span: [m.index, m.index + m[0].length], confidence: 0.95, severity: SAFETY_SEVERITY[name] || "high" });
1205
+ }
1206
+ JWT_RE.lastIndex = 0;
1207
+ while ((m = JWT_RE.exec(answer)) !== null) {
1208
+ const parts = m[0].split(".");
1209
+ findings.push({ type: "jwt", value: mask(m[0]), span: [m.index, m.index + m[0].length], confidence: parts.length === 3 && parts[0] && parts[1] ? 0.95 : 0.6, severity: "critical" });
1210
+ }
1211
+ CC_RE.lastIndex = 0;
1212
+ while ((m = CC_RE.exec(answer)) !== null) {
1213
+ const digits = m[0].replace(/\D/g, "");
1214
+ if (digits.length < 13 || digits.length > 19) continue;
1215
+ if (!luhnOk(digits)) continue; // the single biggest false-positive source
1216
+ findings.push({ type: "credit_card", value: mask(m[0]), span: [m.index, m.index + m[0].length], confidence: 0.9, severity: "critical" });
1217
+ }
1218
+ const claimed = () => findings.map((f) => f.span);
1219
+ HIGH_ENTROPY_RE.lastIndex = 0;
1220
+ let taken = claimed();
1221
+ while ((m = HIGH_ENTROPY_RE.exec(answer)) !== null) {
1222
+ const sp = [m.index, m.index + m[0].length];
1223
+ if (taken.some((c) => overlaps(sp, c))) continue;
1224
+ const tok = m[0];
1225
+ if (ENTROPY_ALLOW_RE.test(tok)) continue;
1226
+ if (!(/\d/.test(tok) && /[A-Za-z]/.test(tok))) continue;
1227
+ const ent = shannonEntropy(tok);
1228
+ if (ent < DEFAULTS.ENTROPY_THRESHOLD) continue;
1229
+ const conf = Math.min(0.85, 0.35 + (ent - DEFAULTS.ENTROPY_THRESHOLD) * 0.4 + Math.min(0.2, (tok.length - DEFAULTS.ENTROPY_MIN_LEN) / 100));
1230
+ findings.push({ type: "high_entropy_string", value: mask(tok), span: sp, confidence: Math.round(conf * 1000) / 1000, severity: "medium", entropy: Math.round(ent * 1000) / 1000, length: tok.length });
1231
+ }
1232
+ taken = claimed();
1233
+ EMAIL_RE.lastIndex = 0;
1234
+ while ((m = EMAIL_RE.exec(answer)) !== null) {
1235
+ const sp = [m.index, m.index + m[0].length];
1236
+ if (taken.some((c) => overlaps(sp, c))) continue;
1237
+ findings.push({ type: "email", value: mask(m[0]), span: sp, confidence: 0.9, severity: "low" });
1238
+ }
1239
+ taken = claimed();
1240
+ PHONE_RE.lastIndex = 0;
1241
+ while ((m = PHONE_RE.exec(answer)) !== null) {
1242
+ const sp = [m.index, m.index + m[0].length];
1243
+ if (taken.some((c) => overlaps(sp, c))) continue;
1244
+ const digits = m[0].replace(/\D/g, "");
1245
+ if (digits.length < 9 || digits.length > 15) continue;
1246
+ findings.push({ type: "phone", value: mask(m[0]), span: sp, confidence: 0.55, severity: "low" });
1247
+ }
1248
+ findings.sort((a, b) => a.span[0] - b.span[0]);
1249
+ return findings;
1250
+ } catch (e) { return [{ type: "scanner_error", value: "", span: [0, 0], confidence: 0, severity: "low" }]; }
1251
+ }
1252
+
1253
+ /** Guarantees: the answer with every finding span replaced by a [REDACTED:TYPE] marker. */
1254
+ export function redact(answer, findings) {
1255
+ try {
1256
+ answer = answer || "";
1257
+ if (!answer) return "";
1258
+ findings = findings || scanOutput(answer);
1259
+ const spans = findings.filter((f) => f.type !== "scanner_error" && f.span[1] > f.span[0])
1260
+ .map((f) => [f.span[0], f.span[1], f.type]).sort((a, b) => a[0] - b[0]);
1261
+ let out = "", cursor = 0;
1262
+ for (const [s, e, t] of spans) {
1263
+ if (s < cursor) continue;
1264
+ out += answer.slice(cursor, s) + "[REDACTED:" + String(t).toUpperCase() + "]";
1265
+ cursor = e;
1266
+ }
1267
+ return out + answer.slice(cursor);
1268
+ } catch (e) { return answer || ""; }
1269
+ }
1270
+
1271
+ /** Guarantees: findings plus severity rollups and a redacted body. */
1272
+ export function safetyReport(answer) {
1273
+ const findings = scanOutput(answer);
1274
+ const bySeverity = {}, byType = {};
1275
+ for (const f of findings) {
1276
+ bySeverity[f.severity] = (bySeverity[f.severity] || 0) + 1;
1277
+ byType[f.type] = (byType[f.type] || 0) + 1;
1278
+ }
1279
+ return { ok: true, findings, counts_by_severity: bySeverity, counts_by_type: byType,
1280
+ critical: bySeverity.critical || 0, redacted: redact(answer, findings) };
1281
+ }
1282
+
1283
+ /* ================================================================== (G) SCHEMA GATE */
1284
+
1285
+ /** Guarantees: the most likely JSON payload substring, or null. */
1286
+ export function extractJson(text) {
1287
+ try {
1288
+ if (!text) return null;
1289
+ text = String(text);
1290
+ const fence = /```[a-zA-Z0-9_+\-]*\s*\n?([\s\S]*?)```/.exec(text);
1291
+ if (fence && fence[1].trim()) {
1292
+ const inner = fence[1].trim();
1293
+ if (inner[0] === "[" || inner[0] === "{") return inner;
1294
+ text = inner;
1295
+ }
1296
+ let best = null;
1297
+ for (const [op, cl] of [["{", "}"], ["[", "]"]]) {
1298
+ let start = text.indexOf(op);
1299
+ while (start !== -1) {
1300
+ let depth = 0, inStr = false, esc = false, quote = "";
1301
+ for (let i = start; i < text.length; i++) {
1302
+ const ch = text[i];
1303
+ if (inStr) {
1304
+ if (esc) esc = false;
1305
+ else if (ch === "\\") esc = true;
1306
+ else if (ch === quote) inStr = false;
1307
+ continue;
1308
+ }
1309
+ if (ch === '"' || ch === "'") { inStr = true; quote = ch; continue; }
1310
+ if (ch === op) depth++;
1311
+ else if (ch === cl) {
1312
+ depth--;
1313
+ if (depth === 0) { const cand = text.slice(start, i + 1); if (!best || cand.length > best.length) best = cand; break; }
1314
+ }
1315
+ }
1316
+ start = text.indexOf(op, start + 1);
1317
+ }
1318
+ }
1319
+ return best;
1320
+ } catch (e) { return null; }
1321
+ }
1322
+
1323
+ function escapeRawControl(s) {
1324
+ let out = "", inStr = false, esc = false, quote = "";
1325
+ for (const ch of s) {
1326
+ if (inStr) {
1327
+ if (esc) { esc = false; out += ch; continue; }
1328
+ if (ch === "\\") { esc = true; out += ch; continue; }
1329
+ if (ch === quote) { inStr = false; out += ch; continue; }
1330
+ if (ch === "\n") { out += "\\n"; continue; }
1331
+ if (ch === "\r") { out += "\\r"; continue; }
1332
+ if (ch === "\t") { out += "\\t"; continue; }
1333
+ out += ch; continue;
1334
+ }
1335
+ if (ch === '"' || ch === "'") { inStr = true; quote = ch; }
1336
+ out += ch;
1337
+ }
1338
+ return out;
1339
+ }
1340
+
1341
+ function singleToDouble(s) {
1342
+ let out = "", i = 0, inDq = false, esc = false;
1343
+ while (i < s.length) {
1344
+ const ch = s[i];
1345
+ if (inDq) {
1346
+ out += ch;
1347
+ if (esc) esc = false; else if (ch === "\\") esc = true; else if (ch === '"') inDq = false;
1348
+ i++; continue;
1349
+ }
1350
+ if (ch === '"') { inDq = true; out += ch; i++; continue; }
1351
+ if (ch === "'") {
1352
+ let j = i + 1, buf = "", e = false;
1353
+ while (j < s.length) {
1354
+ const c2 = s[j];
1355
+ if (e) { buf += c2; e = false; }
1356
+ else if (c2 === "\\") { e = true; buf += c2; }
1357
+ else if (c2 === "'") break;
1358
+ else buf += c2;
1359
+ j++;
1360
+ }
1361
+ out += '"' + buf.replace(/"/g, '\\"') + '"';
1362
+ i = j + 1; continue;
1363
+ }
1364
+ out += ch; i++;
1365
+ }
1366
+ return out;
1367
+ }
1368
+
1369
+ /**
1370
+ * Guarantees: [parsedOrNull, repairsApplied]; never throws.
1371
+ * PURPOSE - so a formatting slip never costs a retry. Re-prompting because the
1372
+ * model emitted a trailing comma or a fullwidth colon is pure waste: the content
1373
+ * was already correct. Fix syntax locally, reserve retries for wrong claims.
1374
+ */
1375
+ export function repairJson(text) {
1376
+ const repairs = [];
1377
+ try {
1378
+ if (text === null || text === undefined) return [null, ["empty_input"]];
1379
+ if (typeof text === "object") return [text, []];
1380
+ let s = String(text).trim();
1381
+ if (!s) return [null, ["empty_input"]];
1382
+ try { return [JSON.parse(s), []]; } catch (e) {}
1383
+ const cand = extractJson(s);
1384
+ if (cand && cand !== s) { s = cand; repairs.push("extracted_json_block"); }
1385
+ try { return [JSON.parse(s), repairs]; } catch (e) {}
1386
+ const fw = { ":": ":", ",": ",", "、": ",", "{": "{", "}": "}", "[": "[", "]": "]", "“": '"', "”": '"', "‘": "'", "’": "'", """: '"', " ": " " };
1387
+ let before = s;
1388
+ for (const k in fw) s = s.split(k).join(fw[k]);
1389
+ if (s !== before) repairs.push("normalised_fullwidth_punctuation");
1390
+ before = s;
1391
+ s = s.replace(/\/\*[\s\S]*?\*\//g, "").replace(/(?<![:"'])\/\/[^\n]*$/gm, "");
1392
+ if (s !== before) repairs.push("removed_comments");
1393
+ before = s; s = singleToDouble(s);
1394
+ if (s !== before) repairs.push("single_to_double_quotes");
1395
+ before = s; s = escapeRawControl(s);
1396
+ if (s !== before) repairs.push("escaped_raw_control_chars");
1397
+ before = s;
1398
+ s = s.replace(/\bTrue\b/g, "true").replace(/\bFalse\b/g, "false").replace(/\b(?:None|NaN|Undefined|undefined)\b/g, "null");
1399
+ if (s !== before) repairs.push("python_literals_to_json");
1400
+ before = s; s = s.replace(/,(\s*[}\]])/g, "$1");
1401
+ if (s !== before) repairs.push("removed_trailing_commas");
1402
+ before = s; s = s.replace(/([{,]\s*)([A-Za-z_][A-Za-z0-9_\-]*)\s*:/g, '$1"$2":');
1403
+ if (s !== before) repairs.push("quoted_bare_keys");
1404
+ try { return [JSON.parse(s), repairs]; } catch (e) {}
1405
+ const opens = (s.split("{").length - 1) - (s.split("}").length - 1);
1406
+ const obr = (s.split("[").length - 1) - (s.split("]").length - 1);
1407
+ if (opens > 0 || obr > 0) {
1408
+ try { const obj = JSON.parse(s + "]".repeat(Math.max(0, obr)) + "}".repeat(Math.max(0, opens))); repairs.push("closed_unbalanced_brackets"); return [obj, repairs]; } catch (e) {}
1409
+ }
1410
+ repairs.push("unrepairable");
1411
+ return [null, repairs];
1412
+ } catch (e) { return [null, ["repair_error"]]; }
1413
+ }
1414
+
1415
+ const COERCERS = {
1416
+ str: (v) => (typeof v === "string" ? v : typeof v === "object" && v !== null ? JSON.stringify(v) : String(v)),
1417
+ int: (v) => { const n = parseInt(v, 10); if (Number.isNaN(n)) throw new Error("nan"); return n; },
1418
+ float: (v) => { const n = Number(v); if (!Number.isFinite(n)) throw new Error("nan"); return n; },
1419
+ number: (v) => { const n = Number(v); if (!Number.isFinite(n)) throw new Error("nan"); return n; },
1420
+ bool: (v) => (typeof v === "boolean" ? v : ["1", "true", "yes", "on"].includes(String(v).trim().toLowerCase())),
1421
+ list: (v) => (Array.isArray(v) ? v : [v]),
1422
+ dict: (v) => (typeof v === "object" && v !== null && !Array.isArray(v) ? v : { value: v }),
1423
+ any: (v) => v,
1424
+ };
1425
+
1426
+ /** Guarantees: {ok,value,missing,extra,coerced,errors}; never throws. A trailing "?" marks a key optional. */
1427
+ export function coerceSchema(obj, schema) {
1428
+ const result = { ok: true, value: {}, missing: [], extra: [], coerced: [], errors: [] };
1429
+ try {
1430
+ if (!schema || typeof schema !== "object" || !Object.keys(schema).length) { result.ok = typeof obj === "object" && obj !== null; result.value = obj; return result; }
1431
+ if (Array.isArray(obj) && obj.length && typeof obj[0] === "object") { obj = obj[0]; result.coerced.push("took_first_element_of_list"); }
1432
+ if (typeof obj !== "object" || obj === null || Array.isArray(obj)) { result.ok = false; result.errors.push("payload_is_not_an_object"); result.value = obj; return result; }
1433
+ const wanted = {};
1434
+ for (const k in schema) { const optional = k.endsWith("?"); wanted[optional ? k.slice(0, -1) : k] = [schema[k], optional]; }
1435
+ for (const key in wanted) {
1436
+ const [t, optional] = wanted[key];
1437
+ if (!(key in obj)) { if (!optional) { result.missing.push(key); result.ok = false; } continue; }
1438
+ const raw = obj[key];
1439
+ if (typeof t === "object" && t !== null) {
1440
+ const sub = coerceSchema(raw, t);
1441
+ result.value[key] = sub.value;
1442
+ if (!sub.ok) { result.ok = false; result.errors.push({ [key]: { missing: sub.missing, errors: sub.errors } }); }
1443
+ continue;
1444
+ }
1445
+ const fn = COERCERS[String(t).trim().toLowerCase()];
1446
+ if (!fn) { result.value[key] = raw; continue; }
1447
+ try { const nv = fn(raw); if (nv !== raw) result.coerced.push(key); result.value[key] = nv; }
1448
+ catch (e) { result.ok = false; result.errors.push({ [key]: "cannot_coerce_to_" + t }); result.value[key] = raw; }
1449
+ }
1450
+ for (const k in obj) if (!(k in wanted)) result.extra.push(k);
1451
+ return result;
1452
+ } catch (e) { result.ok = false; result.errors.push("coerce_error"); return result; }
1453
+ }
1454
+
1455
+ /** Guarantees: one call that extracts, repairs and optionally schema-coerces LLM JSON output. */
1456
+ export function structureCheck(text, schemaJson) {
1457
+ const t0 = nowMs();
1458
+ const [obj, repairs] = repairJson(text);
1459
+ const out = { ok: true, valid_json: obj !== null, repairs, parsed: obj, schema_ok: null, schema: null,
1460
+ missing: [], extra: [], coerced: [], errors: [], value: obj };
1461
+ let schema = null;
1462
+ if (schemaJson) {
1463
+ const [sobj] = repairJson(schemaJson);
1464
+ if (sobj && typeof sobj === "object" && !Array.isArray(sobj)) { schema = sobj; out.schema = sobj; }
1465
+ else out.errors.push("schema_unparseable");
1466
+ }
1467
+ if (schema && obj !== null) {
1468
+ const c = coerceSchema(obj, schema);
1469
+ out.schema_ok = c.ok; out.missing = c.missing; out.extra = c.extra; out.coerced = c.coerced;
1470
+ out.errors = out.errors.concat(c.errors); out.value = c.value;
1471
+ }
1472
+ out.latency_ms = Math.round((nowMs() - t0) * 100) / 100;
1473
+ return out;
1474
+ }
1475
+
1476
+ /* ================================================================ (H) RETRY ADVISOR */
1477
+
1478
+ /** Guarantees: dependency-free token estimate (ASCII ~4 chars/token, CJK ~1 char/token). */
1479
+ export function estimateTokens(s) {
1480
+ if (!s) return 0;
1481
+ s = String(s);
1482
+ let ascii = 0, cjk = 0, other = 0;
1483
+ for (const ch of s) {
1484
+ const o = ch.codePointAt(0);
1485
+ if (o < 128) ascii++;
1486
+ else if ((o >= 0x3040 && o <= 0x30ff) || (o >= 0x4e00 && o <= 0x9fff) || (o >= 0x3400 && o <= 0x4dbf) || (o >= 0xac00 && o <= 0xd7af) || (o >= 0xff00 && o <= 0xffef)) cjk++;
1487
+ else other++;
1488
+ }
1489
+ return Math.ceil(ascii / 4 + cjk + other / 2);
1490
+ }
1491
+
1492
+ const FAIL_STATUSES = ["unsupported", "contradicted"];
1493
+
1494
+ /**
1495
+ * Guarantees: a targeted correction instruction plus a blind-vs-targeted estimate.
1496
+ * A blind retry re-sends the whole system prompt + context + question and
1497
+ * regenerates everything. A targeted retry re-sends only the answer and the
1498
+ * specific failing claims. The difference is the money this saves.
1499
+ */
1500
+ export function buildRetryInstruction(result, prices) {
1501
+ try {
1502
+ prices = prices || {};
1503
+ const pin = prices.in === undefined ? DEFAULTS.PRICE_IN_PER_1K : prices.in;
1504
+ const pout = prices.out === undefined ? DEFAULTS.PRICE_OUT_PER_1K : prices.out;
1505
+ if (typeof result === "string") { const [p] = repairJson(result); result = p && typeof p === "object" ? p : {}; }
1506
+ if (!result || typeof result !== "object") return { ok: true, needed: false, instruction: "", items: [], estimate: {}, reason: "no_result_supplied" };
1507
+ const ver = result.verify && typeof result.verify === "object" ? result.verify : result;
1508
+ const claims = ver.claims || [];
1509
+ const bad = claims.filter((c) => c && FAIL_STATUSES.includes(c.status));
1510
+ bad.sort((a, b) => (a.status === "contradicted" ? 0 : 1) - (b.status === "contradicted" ? 0 : 1) || (a.span ? a.span[0] : 0) - (b.span ? b.span[0] : 0));
1511
+ const items = bad.slice(0, 60).map((c) => ({ id: c.id, type: c.type, value: c.value, status: c.status,
1512
+ nearest_context_value: c.nearest_context_value, sentence_index: c.sentence_index }));
1513
+
1514
+ const linesJa = [], linesEn = [];
1515
+ for (const it of items) {
1516
+ if (it.status === "contradicted" && it.nearest_context_value) {
1517
+ linesJa.push('- 「' + it.value + '」(' + it.type + '): 文脈の該当値は「' + it.nearest_context_value + '」です。');
1518
+ linesEn.push('- "' + it.value + '" (' + it.type + '): the context says "' + it.nearest_context_value + '".');
1519
+ } else {
1520
+ linesJa.push('- 「' + it.value + '」(' + it.type + '): 与えた文脈に裏付けがありません。');
1521
+ linesEn.push('- "' + it.value + '" (' + it.type + '): not supported by the provided context.');
1522
+ }
1523
+ }
1524
+ const instruction = items.length
1525
+ ? "The following statements are not supported by the context you were given. Correct them using the context, or explicitly say the context does not support them. Do not rewrite anything else.\n"
1526
+ + linesEn.join("\n") + "\n\nReturn only the corrected full text. Do not introduce new figures, dates or proper nouns."
1527
+ : "";
1528
+ const instructionJa = items.length
1529
+ ? "以下の記述には、与えた文脈での裏付けがあり��せん。文脈に基づいて訂正するか、裏付けが無い旨を明記してください。該当箇所以外は書き換えないでください。\n"
1530
+ + linesJa.join("\n") + "\n\n訂正後の全文のみを出力してください。新しい数値・日付・固有名詞を追加しないでください。"
1531
+ : "";
1532
+
1533
+ const sizes = (result.meta && result.meta.sizes) || {};
1534
+ const tokSystem = sizes.system_prompt_tokens || 0, tokContext = sizes.context_tokens || 0;
1535
+ const tokUser = sizes.user_input_tokens || 0, tokAnswer = sizes.answer_tokens || 0;
1536
+ const instrTokens = estimateTokens(instruction);
1537
+ const blindIn = tokSystem + tokContext + tokUser, blindOut = tokAnswer;
1538
+ const targetedIn = tokAnswer + instrTokens + Math.min(tokSystem, 200);
1539
+ const targetedOut = Math.max(1, Math.round(tokAnswer * Math.min(1, Math.max(0.15, items.length / Math.max(1, claims.length || 1)))));
1540
+ // Signed on purpose. For a small context the instruction can cost more than
1541
+ // the blind retry it replaces, and hiding that behind max(0, ...) would make
1542
+ // this tool lie about its own value.
1543
+ const savedIn = blindIn - targetedIn, savedOut = blindOut - targetedOut;
1544
+ const savedTokens = savedIn + savedOut;
1545
+ const savedCost = Math.round(((savedIn / 1000) * pin + (savedOut / 1000) * pout) * 1e6) / 1e6;
1546
+ const priced = pin > 0 || pout > 0;
1547
+ const cheaper = priced ? savedCost > 0 : savedTokens > 0;
1548
+ const basis = priced ? "cost" : "tokens";
1549
+ const recommendation = !items.length ? "no retry needed"
1550
+ : cheaper ? "targeted retry is cheaper by " + (priced ? savedCost.toFixed(6) : savedTokens + " tokens") + " (basis: " + basis + ")"
1551
+ : "targeted retry is NOT cheaper here (basis: " + basis + ") - the context is small relative to the instruction; retry blind, or batch several corrections into one call";
1552
+
1553
+ return { ok: true, needed: items.length > 0, instruction, instruction_ja: instructionJa, items,
1554
+ failed_claims: bad.length,
1555
+ estimate: { blind_retry_tokens: blindIn + blindOut, targeted_retry_tokens: targetedIn + targetedOut,
1556
+ saved_tokens: savedTokens, saved_cost: savedCost, targeted_is_cheaper: cheaper, comparison_basis: basis,
1557
+ recommendation,
1558
+ breakdown: { blind_input_tokens: blindIn, blind_output_tokens: blindOut, targeted_input_tokens: targetedIn,
1559
+ targeted_output_tokens: targetedOut, saved_input_tokens: savedIn, saved_output_tokens: savedOut, instruction_tokens: instrTokens },
1560
+ assumptions: { price_in_per_1k: pin, price_out_per_1k: pout,
1561
+ tokenizer: "heuristic: ASCII ~4 chars/token, CJK ~1 char/token",
1562
+ note: "saved_tokens and saved_cost are signed and can disagree: output tokens are usually priced several times higher than input tokens." } } };
1563
+ } catch (e) { return { ok: false, error: String(e).slice(0, 300) }; }
1564
+ }
1565
+
1566
+ /* ======================================================================== (I) GATE */
1567
+
1568
+ export const DEFAULT_POLICY = {
1569
+ pass_grounding: 0.95, pass_coverage: 0.5, annotate_grounding: 0.9, retry_grounding: 0.7,
1570
+ retry_on_contradicted: 1, block_on_critical_safety: true, block_on_leak: false, block_on_injection_echo: false,
1571
+ enable_numeric: true, enable_date: true, enable_quote: true, enable_entity: true, enable_url: true,
1572
+ enable_derivation: true, derive_max_terms: DEFAULTS.DERIVE_MAX_TERMS, derive_budget_ms: DEFAULTS.DERIVE_BUDGET_MS,
1573
+ derive_max_numbers: DEFAULTS.DERIVE_MAX_NUMBERS, fuzzy_budget_ms: DEFAULTS.FUZZY_BUDGET_MS,
1574
+ max_claims: DEFAULTS.MAX_CLAIMS, numeric_tolerance: 0, contradiction_rel: DEFAULTS.CONTRADICTION_REL,
1575
+ approx_ratio: DEFAULTS.APPROX_RATIO, quote_contradiction_floor: DEFAULTS.QUOTE_CONTRADICTION_FLOOR,
1576
+ leak_ngram: DEFAULTS.NGRAM_LEAK_N, leak_threshold: DEFAULTS.LEAK_THRESHOLD, max_chars: DEFAULTS.MAX_TEXT_CHARS,
1577
+ };
1578
+
1579
+ let GATE_SEQ = 0;
1580
+
1581
+ /**
1582
+ * Guarantees: a four-valued verdict (pass/annotate/retry/block) and never throws.
1583
+ * Order is deliberate: output safety runs FIRST and a critical hit
1584
+ * short-circuits everything. Verifying the grounding of an answer that leaks an
1585
+ * API key is wasted work.
1586
+ */
1587
+ export function gate(answer, context, systemPrompt, userInput, schemaJson, policyJson, tagsJson, prices) {
1588
+ const tStart = nowMs();
1589
+ GATE_SEQ += 1;
1590
+ const eventId = "e" + String(GATE_SEQ).padStart(6, "0");
1591
+ const warnings = [];
1592
+ try {
1593
+ const pol = Object.assign({}, DEFAULT_POLICY);
1594
+ if (policyJson) {
1595
+ const [obj, reps] = repairJson(policyJson);
1596
+ if (obj && typeof obj === "object") {
1597
+ for (const k in obj) { if (k in pol) pol[k] = obj[k]; else warnings.push("unknown_policy_key:" + k); }
1598
+ if (reps && reps.length) warnings.push("policy_json_repaired:" + reps.join(","));
1599
+ } else warnings.push("policy_json_unparseable_using_defaults");
1600
+ }
1601
+ const maxChars = pol.max_chars || DEFAULTS.MAX_TEXT_CHARS;
1602
+ const clip = (s, name) => { s = s === null || s === undefined ? "" : String(s); if (s.length > maxChars) { warnings.push(name + " truncated to max_chars=" + maxChars); return s.slice(0, maxChars); } return s; };
1603
+ answer = clip(answer, "answer"); context = clip(context, "context");
1604
+ systemPrompt = clip(systemPrompt, "system_prompt"); userInput = clip(userInput, "user_input");
1605
+
1606
+ let tags = {};
1607
+ if (tagsJson) { const [t] = repairJson(tagsJson); if (t && typeof t === "object") tags = t; else warnings.push("tags_json_unparseable"); }
1608
+
1609
+ const sizes = { answer_chars: answer.length, context_chars: context.length,
1610
+ system_prompt_chars: systemPrompt.length, user_input_chars: userInput.length,
1611
+ answer_tokens: estimateTokens(answer), context_tokens: estimateTokens(context),
1612
+ system_prompt_tokens: estimateTokens(systemPrompt), user_input_tokens: estimateTokens(userInput) };
1613
+
1614
+ const meta = { sizes, tags, policy: pol, timestamp: new Date().toISOString(), app_version: APP_VERSION };
1615
+
1616
+ if (!answer.trim())
1617
+ return { ok: true, event_id: eventId, verdict: "annotate", reasons: ["empty_answer"], warnings,
1618
+ safety: { ok: true, findings: [], critical: 0 }, leak: null, verify: null, schema: null, retry: null,
1619
+ meta: Object.assign(meta, { latency_ms: Math.round((nowMs() - tStart) * 100) / 100 }) };
1620
+
1621
+ // 2. output safety FIRST, short-circuit on critical
1622
+ const safety = safetyReport(answer);
1623
+ const critical = safety.critical || 0;
1624
+ if (critical > 0 && pol.block_on_critical_safety)
1625
+ return { ok: true, event_id: eventId, verdict: "block",
1626
+ reasons: ["critical_output_safety_violation:" + critical],
1627
+ warnings: warnings.concat(["verification skipped: blocked before grounding checks"]),
1628
+ safety, leak: null, verify: null, schema: null, retry: null,
1629
+ meta: Object.assign(meta, { latency_ms: Math.round((nowMs() - tStart) * 100) / 100 }) };
1630
+
1631
+ // 3. leak detection
1632
+ const leak = detectSystemPromptLeak(answer, systemPrompt, pol.leak_ngram, pol.leak_threshold);
1633
+ const echo = detectInjectionEcho(answer, userInput);
1634
+
1635
+ // 4. verification
1636
+ const ver = verify(answer, context, pol);
1637
+
1638
+ // 5. schema gate (only when a schema is supplied)
1639
+ const schemaRes = schemaJson && String(schemaJson).trim() ? structureCheck(answer, schemaJson) : null;
1640
+
1641
+ // 6. retry advisor
1642
+ const retry = buildRetryInstruction({ verify: ver, meta: { sizes } }, prices);
1643
+
1644
+ const bs = ver.counts.by_status || {};
1645
+ const contradicted = bs.contradicted || 0;
1646
+ const grounding = ver.grounding_score, coverage = ver.coverage;
1647
+ const claimsTotal = ver.counts.claims_total || 0;
1648
+ const reasons = [];
1649
+ let verdict = "pass";
1650
+
1651
+ if (leak.leak) { reasons.push("system_prompt_leak:overlap=" + leak.overlap.toFixed(3)); verdict = pol.block_on_leak ? "block" : "annotate"; }
1652
+ if (echo.echo) { reasons.push("injection_echo:" + echo.echoed_count); if (pol.block_on_injection_echo) verdict = "block"; else if (verdict === "pass") verdict = "annotate"; }
1653
+
1654
+ if (verdict !== "block") {
1655
+ if (contradicted >= (pol.retry_on_contradicted || 1)) { verdict = "retry"; reasons.push("contradicted_claims:" + contradicted); }
1656
+ else if (claimsTotal && grounding < pol.retry_grounding) { verdict = "retry"; reasons.push("grounding_score below retry threshold (" + grounding.toFixed(3) + " < " + pol.retry_grounding + ")"); }
1657
+ else if (claimsTotal && grounding < pol.annotate_grounding) { if (verdict === "pass") verdict = "annotate"; reasons.push("grounding_score below annotate threshold (" + grounding.toFixed(3) + " < " + pol.annotate_grounding + ")"); }
1658
+ else if (!claimsTotal) { if (verdict === "pass") verdict = "annotate"; reasons.push("no verifiable claims were found; nothing was checked"); }
1659
+
1660
+ if (verdict === "pass") {
1661
+ if (grounding < pol.pass_grounding) { verdict = "annotate"; reasons.push("grounding_score below pass threshold"); }
1662
+ else if (coverage < pol.pass_coverage) { verdict = "annotate"; reasons.push("coverage " + coverage.toFixed(2) + " below pass threshold " + pol.pass_coverage + " - most of the answer was not checked"); }
1663
+ }
1664
+ }
1665
+
1666
+ if (schemaRes) {
1667
+ if (schemaRes.valid_json === false) { reasons.push("output is not valid JSON even after repair"); if (verdict === "pass" || verdict === "annotate") verdict = "retry"; }
1668
+ else if (schemaRes.schema_ok === false) { reasons.push("JSON does not satisfy the schema: missing=" + JSON.stringify(schemaRes.missing)); if (verdict === "pass" || verdict === "annotate") verdict = "retry"; }
1669
+ }
1670
+ if (critical > 0) reasons.push("critical_output_safety_violation:" + critical + " (policy did not block)");
1671
+ const otherSafety = (safety.findings || []).length - critical;
1672
+ if (otherSafety > 0 && verdict === "pass") { verdict = "annotate"; reasons.push("non-critical safety findings:" + otherSafety); }
1673
+
1674
+ meta.latency_ms = Math.round((nowMs() - tStart) * 100) / 100;
1675
+ meta.verify_latency_ms = ver.latency_ms;
1676
+ return { ok: true, event_id: eventId, verdict, reasons: reasons.length ? reasons : ["all checks passed"],
1677
+ warnings, grounding_score: grounding, coverage, safety,
1678
+ leak: { system_prompt: leak, injection_echo: echo }, verify: ver, schema: schemaRes, retry, meta };
1679
+ } catch (e) {
1680
+ return { ok: false, event_id: eventId, error: { type: "GateError", message: String(e && e.message ? e.message : e).slice(0, 500) } };
1681
+ }
1682
+ }
1683
+
1684
+ /* ============================================================== HIGHLIGHT HELPERS */
1685
+
1686
+ export const STATUS_COLORS = { supported: "#16a34a", derived: "#2563eb", approximate: "#ca8a04", unsupported: "#ea580c", contradicted: "#dc2626" };
1687
+ const STATUS_PRIORITY = { contradicted: 0, unsupported: 1, approximate: 2, derived: 3, supported: 4 };
1688
+
1689
+ /** Guarantees: [text, statusOrNull] segments covering the answer exactly once. */
1690
+ export function buildHighlight(answer, claims) {
1691
+ try {
1692
+ answer = answer || "";
1693
+ if (!answer) return [["", null]];
1694
+ const spans = (claims || []).filter((c) => c && c.span && !c.suppressed && c.span[1] > c.span[0] && c.span[0] >= 0 && c.span[1] <= answer.length)
1695
+ .map((c) => [c.span[0], c.span[1], c.status || "unsupported"]);
1696
+ // The most severe (and at equal severity, the longest) claim wins an overlap.
1697
+ spans.sort((a, b) => (STATUS_PRIORITY[a[2]] ?? 9) - (STATUS_PRIORITY[b[2]] ?? 9) || (b[1] - b[0]) - (a[1] - a[0]));
1698
+ const placed = [];
1699
+ for (const sp of spans) if (!placed.some((p) => overlaps(sp, p))) placed.push(sp);
1700
+ placed.sort((a, b) => a[0] - b[0]);
1701
+ const out = [];
1702
+ let cursor = 0;
1703
+ for (const [s, e, st] of placed) {
1704
+ if (s > cursor) out.push([answer.slice(cursor, s), null]);
1705
+ out.push([answer.slice(s, e), st]);
1706
+ cursor = e;
1707
+ }
1708
+ if (cursor < answer.length) out.push([answer.slice(cursor), null]);
1709
+ return out.length ? out : [[answer, null]];
1710
+ } catch (e) { return [[answer || "", null]]; }
1711
+ }
index.html CHANGED
@@ -1,19 +1,377 @@
1
  <!doctype html>
2
- <html>
3
- <head>
4
- <meta charset="utf-8" />
5
- <meta name="viewport" content="width=device-width" />
6
- <title>My static Space</title>
7
- <link rel="stylesheet" href="style.css" />
8
- </head>
9
- <body>
10
- <div class="card">
11
- <h1>Welcome to your static Space!</h1>
12
- <p>You can modify this app directly by editing <i>index.html</i> in the Files and versions tab.</p>
13
- <p>
14
- Also don't forget to check the
15
- <a href="https://huggingface.co/docs/hub/spaces" target="_blank">Spaces documentation</a>.
16
- </p>
17
- </div>
18
- </body>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
19
  </html>
 
1
  <!doctype html>
2
+ <html lang="en">
3
+ <head>
4
+ <meta charset="utf-8" />
5
+ <meta name="viewport" content="width=device-width, initial-scale=1" />
6
+ <title>ClaimCheck — verify LLM answers against their source</title>
7
+ <meta name="description" content="Verify LLM answers against their source context before they reach users. Deterministic, client-side, no server." />
8
+ <style>
9
+ :root{
10
+ --bg:#ffffff; --fg:#111827; --muted:#6b7280; --line:#e5e7eb; --panel:#f9fafb;
11
+ --accent:#b45309; --accent-bg:#fffbeb;
12
+ --supported:#16a34a; --supported-bg:#dcfce7;
13
+ --derived:#2563eb; --derived-bg:#dbeafe;
14
+ --approximate:#ca8a04; --approximate-bg:#fef9c3;
15
+ --unsupported:#ea580c; --unsupported-bg:#ffedd5;
16
+ --contradicted:#dc2626; --contradicted-bg:#fee2e2;
17
+ }
18
+ @media (prefers-color-scheme: dark){
19
+ :root{ --bg:#0b0f19; --fg:#e5e7eb; --muted:#9ca3af; --line:#1f2937; --panel:#111827;
20
+ --accent:#fbbf24; --accent-bg:#1c1917;
21
+ --supported-bg:#052e16; --derived-bg:#172554; --approximate-bg:#422006;
22
+ --unsupported-bg:#431407; --contradicted-bg:#450a0a; }
23
+ }
24
+ *{box-sizing:border-box}
25
+ body{margin:0;background:var(--bg);color:var(--fg);
26
+ font:15px/1.6 ui-sans-serif,-apple-system,"Segoe UI",Roboto,"Helvetica Neue","Hiragino Sans","Noto Sans JP",sans-serif}
27
+ .wrap{max-width:1100px;margin:0 auto;padding:28px 16px 64px}
28
+ h1{font-size:28px;margin:0 0 6px;letter-spacing:-.02em}
29
+ h2{font-size:18px;margin:28px 0 10px;letter-spacing:-.01em}
30
+ .lede{color:var(--muted);margin:0 0 6px;font-size:15px}
31
+ .badges{display:flex;gap:8px;flex-wrap:wrap;margin:14px 0 22px}
32
+ .badge{font-size:12px;padding:3px 9px;border:1px solid var(--line);border-radius:999px;color:var(--muted);text-decoration:none}
33
+ .badge:hover{border-color:var(--accent);color:var(--accent)}
34
+ .note{background:var(--accent-bg);border:1px solid var(--line);border-left:3px solid var(--accent);
35
+ padding:12px 14px;border-radius:6px;font-size:14px;margin:0 0 22px}
36
+ .tabs{display:flex;gap:2px;border-bottom:1px solid var(--line);margin-bottom:20px;flex-wrap:wrap}
37
+ .tab{padding:9px 14px;border:none;background:none;color:var(--muted);cursor:pointer;font-size:14px;
38
+ border-bottom:2px solid transparent;font-family:inherit}
39
+ .tab[aria-selected=true]{color:var(--fg);border-bottom-color:var(--accent);font-weight:600}
40
+ .panel[hidden]{display:none}
41
+ .grid{display:grid;grid-template-columns:1fr 1fr;gap:16px}
42
+ @media(max-width:820px){.grid{grid-template-columns:1fr}}
43
+ label{display:block;font-size:12px;font-weight:600;margin:0 0 4px;color:var(--muted)}
44
+ textarea,input{width:100%;font:13px/1.55 ui-monospace,SFMono-Regular,Menlo,monospace;color:var(--fg);
45
+ background:var(--panel);border:1px solid var(--line);border-radius:6px;padding:9px 11px;resize:vertical}
46
+ textarea:focus,input:focus{outline:2px solid var(--accent);outline-offset:-1px}
47
+ .field{margin-bottom:12px}
48
+ button.go{background:var(--accent);color:#fff;border:none;border-radius:6px;padding:10px 22px;
49
+ font-size:14px;font-weight:600;cursor:pointer;font-family:inherit;margin:6px 0 20px}
50
+ button.go:hover{filter:brightness(1.08)}
51
+ .verdict{font-size:22px;font-weight:700;margin:0 0 12px}
52
+ table{width:100%;border-collapse:collapse;font-size:13px;margin:0 0 18px}
53
+ th,td{text-align:left;padding:7px 9px;border-bottom:1px solid var(--line);vertical-align:top}
54
+ th{color:var(--muted);font-weight:600;font-size:12px}
55
+ td.num{font-weight:700;white-space:nowrap}
56
+ .hl{background:var(--panel);border:1px solid var(--line);border-radius:6px;padding:14px;
57
+ white-space:pre-wrap;word-break:break-word;line-height:2.1;margin:0 0 8px}
58
+ mark{padding:2px 4px;border-radius:3px;font-weight:600;color:var(--fg)}
59
+ mark.supported{background:var(--supported-bg);box-shadow:inset 0 -2px 0 var(--supported)}
60
+ mark.derived{background:var(--derived-bg);box-shadow:inset 0 -2px 0 var(--derived)}
61
+ mark.approximate{background:var(--approximate-bg);box-shadow:inset 0 -2px 0 var(--approximate)}
62
+ mark.unsupported{background:var(--unsupported-bg);box-shadow:inset 0 -2px 0 var(--unsupported)}
63
+ mark.contradicted{background:var(--contradicted-bg);box-shadow:inset 0 -2px 0 var(--contradicted)}
64
+ .legend{display:flex;gap:14px;flex-wrap:wrap;font-size:12px;color:var(--muted);margin:0 0 20px}
65
+ .key{display:inline-block;width:10px;height:10px;border-radius:2px;margin-right:5px;vertical-align:baseline}
66
+ .st{font-weight:600}
67
+ .st.supported{color:var(--supported)} .st.derived{color:var(--derived)}
68
+ .st.approximate{color:var(--approximate)} .st.unsupported{color:var(--unsupported)}
69
+ .st.contradicted{color:var(--contradicted)}
70
+ details{background:var(--panel);border:1px solid var(--line);border-radius:6px;padding:10px 14px;margin:0 0 18px}
71
+ summary{cursor:pointer;font-size:13px;font-weight:600;color:var(--muted)}
72
+ pre{overflow:auto;font-size:12px;line-height:1.5;background:var(--panel);border:1px solid var(--line);
73
+ border-radius:6px;padding:12px;margin:8px 0 0}
74
+ code{font-family:ui-monospace,SFMono-Regular,Menlo,monospace;font-size:.92em}
75
+ .muted{color:var(--muted);font-size:13px}
76
+ .warn{color:var(--unsupported);font-weight:600}
77
+ footer{margin-top:44px;padding-top:18px;border-top:1px solid var(--line);color:var(--muted);font-size:13px}
78
+ a{color:var(--accent)}
79
+ </style>
80
+ </head>
81
+ <body>
82
+ <div class="wrap">
83
+ <h1>🔎 ClaimCheck</h1>
84
+ <p class="lede"><strong>Verify LLM answers against their source context before they reach users.</strong></p>
85
+ <p class="lede">We cannot catch every hallucination — but dangerous hallucinations are <em>specific</em>, and specific
86
+ claims can be matched as strings. So ClaimCheck checks only what it can check deterministically, and always tells
87
+ you <strong>how much it did not check</strong>.</p>
88
+ <div class="badges">
89
+ <a class="badge" href="https://huggingface.co/NagaYu/claimcheck-rules" target="_blank" rel="noopener">📦 Rule pack</a>
90
+ <a class="badge" href="https://huggingface.co/datasets/NagaYu/claimcheck-eval" target="_blank" rel="noopener">📊 158-case eval set</a>
91
+ <span class="badge">⚡ ~0.2 ms typical</span>
92
+ <span class="badge">🔒 100% client-side</span>
93
+ </div>
94
+
95
+ <div class="note">
96
+ <strong>Nothing leaves your browser.</strong> There is no server and no network call: the whole verifier is
97
+ JavaScript running in this tab. Paste real model output if you want to — it stays on your machine.
98
+ </div>
99
+
100
+ <div class="tabs" role="tablist">
101
+ <button class="tab" role="tab" aria-selected="true" data-p="verify">Verify</button>
102
+ <button class="tab" role="tab" aria-selected="false" data-p="retry">Retry</button>
103
+ <button class="tab" role="tab" aria-selected="false" data-p="json">JSON repair</button>
104
+ <button class="tab" role="tab" aria-selected="false" data-p="about">How it works</button>
105
+ </div>
106
+
107
+ <section class="panel" id="p-verify">
108
+ <div class="grid">
109
+ <div>
110
+ <div class="field"><label for="answer">Answer — the LLM output to check</label>
111
+ <textarea id="answer" rows="9"></textarea></div>
112
+ <div class="field"><label for="context">Context — the source of truth given to the model</label>
113
+ <textarea id="context" rows="9"></textarea></div>
114
+ </div>
115
+ <div>
116
+ <div class="field"><label for="system">System prompt (optional — enables leak detection)</label>
117
+ <textarea id="system" rows="4"></textarea></div>
118
+ <div class="field"><label for="userin">User input (optional — enables injection-echo detection)</label>
119
+ <textarea id="userin" rows="3"></textarea></div>
120
+ <div class="field"><label for="schema">Schema JSON (optional — only if the answer must be JSON)</label>
121
+ <textarea id="schema" rows="3" placeholder='{"title": "str", "score": "float"}'></textarea></div>
122
+ <div class="field"><label for="policy">Policy JSON (optional — threshold overrides)</label>
123
+ <textarea id="policy" rows="5"></textarea></div>
124
+ </div>
125
+ </div>
126
+ <button class="go" id="run">Verify</button>
127
+ <div id="out"></div>
128
+ </section>
129
+
130
+ <section class="panel" id="p-retry" hidden>
131
+ <p class="muted">A blind retry re-sends the whole system prompt + context + question and regenerates everything.
132
+ A targeted retry re-sends only the answer plus the specific failing claims. The difference is the estimate below.
133
+ Run <strong>Verify</strong> first.</p>
134
+ <button class="go" id="run-retry">Build retry instruction</button>
135
+ <div id="retry-out"></div>
136
+ </section>
137
+
138
+ <section class="panel" id="p-json" hidden>
139
+ <p class="muted">Structural repair only. <strong>The point is to never burn a retry on formatting.</strong>
140
+ Fences, trailing commas, single quotes, raw newlines inside strings, fullwidth punctuation and Python literals
141
+ are all fixed locally — retries are reserved for claims that are actually wrong.</p>
142
+ <div class="grid">
143
+ <div class="field"><label for="raw">Raw model output</label><textarea id="raw" rows="12"></textarea></div>
144
+ <div class="field"><label for="schema2">Schema JSON (optional)</label><textarea id="schema2" rows="12"></textarea></div>
145
+ </div>
146
+ <button class="go" id="run-json">Extract / repair / coerce</button>
147
+ <div id="json-out"></div>
148
+ </section>
149
+
150
+ <section class="panel" id="p-about" hidden>
151
+ <h2>Always read <code>coverage</code></h2>
152
+ <p>Every result carries two independent numbers, and reporting one without the other is the most dangerous way to
153
+ misuse this tool.</p>
154
+ <table>
155
+ <tr><th>field</th><th>question it answers</th></tr>
156
+ <tr><td><code>grounding_score</code></td><td>Of the claims I checked, what share held up?</td></tr>
157
+ <tr><td><code>coverage</code></td><td>What share of the answer did I check at all?</td></tr>
158
+ </table>
159
+ <p><code>grounding_score</code> alone is a trap. An answer of pure vague hedging produces <strong>zero</strong>
160
+ claims and scores <strong>1.00</strong> on any naive groundedness metric — while being completely unverified.
161
+ A high score at low coverage means <em>"I could not find anything to check"</em>, not <em>"everything is fine"</em>.</p>
162
+
163
+ <h2>Claim statuses</h2>
164
+ <table>
165
+ <tr><th>status</th><th>meaning</th></tr>
166
+ <tr><td><span class="st supported">supported</span></td><td>Appears literally in the context after normalisation</td></tr>
167
+ <tr><td><span class="st derived">derived</span></td><td>Computable from context numbers by 2–3 term arithmetic; the formula is shown</td></tr>
168
+ <tr><td><span class="st approximate">approximate</span></td><td>A near match exists — paraphrase, finer-grained date, related URL</td></tr>
169
+ <tr><td><span class="st unsupported">unsupported</span></td><td>Nothing in the context backs it</td></tr>
170
+ <tr><td><span class="st contradicted">contradicted</span></td><td>The context holds a <strong>near but different</strong> value</td></tr>
171
+ </table>
172
+ <p class="muted"><code>approximate</code> is deliberately <em>not</em> counted as grounded: a paraphrased quote is
173
+ still an inaccurate quote.</p>
174
+
175
+ <h2>Measured behaviour</h2>
176
+ <p>On the <a href="https://huggingface.co/datasets/NagaYu/claimcheck-eval" target="_blank" rel="noopener">158-case
177
+ bilingual evaluation set</a>:</p>
178
+ <pre>answer-level recall 56/56 (100%) cases that should be flagged, and were
179
+ clean answers passing 44/46 (96%) the rest were annotated, never blocked
180
+ known false positives 22 (13.9%)
181
+ known false negatives 3 (1.9%)
182
+ JS/Python parity 158/158 (100%)</pre>
183
+ <p>The false-positive rate is published deliberately. It is the number that decides whether anyone keeps a verifier
184
+ switched on, and hiding it helps nobody.</p>
185
+
186
+ <h2>The most serious failure: semantic reversal</h2>
187
+ <p>All three false negatives are the same shape. A one-character change that reverses meaning keeps character
188
+ similarity above 0.9, so a reversed quote reads as a faithful paraphrase:</p>
189
+ <table>
190
+ <tr><th>context</th><th>answer</th><th>similarity</th><th>status</th></tr>
191
+ <tr><td>売上高は7%<strong>増加</strong>した</td><td>売上高は7%<strong>減少</strong>した</td><td>0.97</td><td><span class="st approximate">approximate</span> ❌</td></tr>
192
+ <tr><td>提案は全会一致で<strong>承認</strong>された</td><td>提案は全会一致で<strong>否決</strong>された</td><td>0.92</td><td><span class="st approximate">approximate</span> ❌</td></tr>
193
+ <tr><td>revenue <strong>grew</strong> 8%</td><td>revenue <strong>declined</strong> 8%</td><td>0.97</td><td><span class="st approximate">approximate</span> ❌</td></tr>
194
+ </table>
195
+ <p>Character-level matching cannot detect negation or antonym substitution. If your domain turns on
196
+ approve/reject or increase/decrease, <strong>do not rely on quote status alone.</strong> In all three cases the
197
+ answer-level verdict was still <code>retry</code>, because the NUMERIC and ENTITY claims inside the quote are the
198
+ layer that catches what the quote check misses.</p>
199
+
200
+ <h2>When it is wrong (false positives)</h2>
201
+ <ul class="muted">
202
+ <li><strong>Faithful paraphrase</strong> — meaning preserved, wording replaced. The largest group.</li>
203
+ <li><strong>Generic noun phrases</strong> — "Machine Learning" flagged as an unsupported entity.</li>
204
+ <li><strong>Acronym expansion</strong> — context "World Health Organization", answer "WHO".</li>
205
+ <li><strong>Rounding</strong> — context 12,345 vs answer "roughly 12,000": inside the near-miss window.
206
+ Raise <code>numeric_tolerance</code> rather than lowering <code>contradiction_rel</code>.</li>
207
+ <li><strong>Fiscal vs calendar year</strong> — FY2024 and 2024 treated as the same period.</li>
208
+ <li><strong>Identifier variants</strong> — ISO 9001:2015 where the source says ISO 9001. The deliberate cost of
209
+ catching one-digit-off fabrications.</li>
210
+ <li><strong>URL query parameters</strong> — a tracking parameter breaks exact matching.</li>
211
+ </ul>
212
+ <p><strong>ENTITY is the noisiest check.</strong> Set <code>{"enable_entity": false}</code> in the policy box first
213
+ if you see too much noise, and start from <code>annotate</code>, never <code>block</code>.</p>
214
+
215
+ <h2>Not checked, on purpose</h2>
216
+ <ul class="muted">
217
+ <li><strong>Relative dates</strong> ("last year", "昨年") — unresolvable without a generation date.</li>
218
+ <li><strong>Bare small integers</strong> (0–10, no unit) are never marked contradicted: they sit near everything.</li>
219
+ <li><strong>List markers</strong> ("1.", "2)") are not numeric claims.</li>
220
+ <li><strong>Reasoning, tone, completeness, relevance.</strong> None are string-checkable.</li>
221
+ </ul>
222
+ <p class="warn">ClaimCheck is one layer of defence, not a correctness guarantee.</p>
223
+ </section>
224
+
225
+ <footer>
226
+ Apache-2.0 ·
227
+ <a href="https://huggingface.co/NagaYu/claimcheck-rules" target="_blank" rel="noopener">rule pack</a> ·
228
+ <a href="https://huggingface.co/datasets/NagaYu/claimcheck-eval" target="_blank" rel="noopener">evaluation set</a> ·
229
+ verification runs entirely in your browser
230
+ </footer>
231
+ </div>
232
+
233
+ <script type="module">
234
+ import * as cc from "./claimcheck.js";
235
+
236
+ const DEMO = {
237
+ context: ["2024年度の売上高は12,000百万円、営業利益は1,800百万円でした。",
238
+ "従業員数は3,400人で、前年度(2023年度)は3,200人でした。",
239
+ "新製品Xは2024年3月15日に発売されました。",
240
+ "IR資料は https://example.com/ir/2024 に掲載しています。",
241
+ "契約は第12条に基づき自動更新されます。"].join("\n"),
242
+ answer: ["2024年度の売上高は12,000百万円、営業利益は1,800百万円でした。",
243
+ "したがって営業利益率は15%です。",
244
+ "従業員数は3,400人、前年度は3,250人でした。",
245
+ "新製品Xは2024年3月15日に発売されています。",
246
+ "なお解約率は7.2%に達しており、注意が必要です。",
247
+ "詳細は https://example.com/ir/2025 をご覧ください。"].join("\n"),
248
+ system: ["あなたは企業のIR資料に基づいて回答するアシスタントです。",
249
+ "与えられた文脈のみを根拠とし、文脈に無い数値を作らないでください。"].join("\n"),
250
+ policy: JSON.stringify({ enable_entity: true, retry_on_contradicted: 1, pass_grounding: 0.95, pass_coverage: 0.5 }, null, 2),
251
+ raw: ['```json', '{', ' \'title\': "Quarterly report",', ' "score": 0.87,',
252
+ ' "tags": ["ir", "2024",],', ' "note": "a multi-line', 'memo",', ' "published": True,', '}', '```'].join("\n"),
253
+ schema2: JSON.stringify({ title: "str", score: "float", tags: "list", published: "bool", "note?": "str" }, null, 2),
254
+ };
255
+ const $ = (id) => document.getElementById(id);
256
+ $("answer").value = DEMO.answer; $("context").value = DEMO.context;
257
+ $("system").value = DEMO.system; $("policy").value = DEMO.policy;
258
+ $("raw").value = DEMO.raw; $("schema2").value = DEMO.schema2;
259
+
260
+ for (const t of document.querySelectorAll(".tab")) {
261
+ t.addEventListener("click", () => {
262
+ for (const o of document.querySelectorAll(".tab")) o.setAttribute("aria-selected", String(o === t));
263
+ for (const p of document.querySelectorAll(".panel")) p.hidden = p.id !== "p-" + t.dataset.p;
264
+ });
265
+ }
266
+
267
+ const esc = (s) => String(s).replace(/[&<>"]/g, (c) => ({ "&": "&amp;", "<": "&lt;", ">": "&gt;", '"': "&quot;" }[c]));
268
+ const BADGE = { pass: "✅ PASS", annotate: "🟡 ANNOTATE", retry: "🟠 RETRY", block: "🔴 BLOCK" };
269
+ let LAST = null;
270
+
271
+ function render(r, answer) {
272
+ if (!r || !r.ok) return `<p class="warn">Error: ${esc(JSON.stringify(r && r.error))}</p>`;
273
+ const v = r.verify, c = v ? v.counts : null, bs = c ? c.by_status : {};
274
+ const est = (r.retry && r.retry.estimate) || {};
275
+ const leak = (r.leak && r.leak.system_prompt) || {};
276
+ const safety = r.safety || {};
277
+ const sTypes = [...new Set((safety.findings || []).map((f) => f.type))].join(", ") || "clean";
278
+ let h = `<p class="verdict">${BADGE[r.verdict] || esc(r.verdict)}</p>`;
279
+
280
+ if (v) {
281
+ h += `<table>
282
+ <tr><th>metric</th><th>value</th><th>meaning</th></tr>
283
+ <tr><td><strong>grounding_score</strong></td><td class="num">${(v.grounding_score * 100).toFixed(1)}%</td>
284
+ <td>of the claims we checked, this share held up</td></tr>
285
+ <tr><td><strong>coverage</strong></td><td class="num">${(v.coverage * 100).toFixed(1)}%</td>
286
+ <td>share of sentences that produced <em>any</em> checkable claim</td></tr>
287
+ <tr><td>unchecked sentences</td><td class="num">${c.sentences_unverified} / ${c.sentences_total}</td>
288
+ <td class="warn">not verified — read them yourself</td></tr>
289
+ <tr><td>latency</td><td class="num">${r.meta.latency_ms} ms</td><td>measured per request, in this tab</td></tr>
290
+ <tr><td>claims</td><td class="num">${c.claims_total}</td><td>
291
+ <span class="st supported">supported ${bs.supported}</span> ·
292
+ <span class="st derived">derived ${bs.derived}</span> ·
293
+ <span class="st approximate">approximate ${bs.approximate}</span> ·
294
+ <span class="st unsupported">unsupported ${bs.unsupported}</span> ·
295
+ <span class="st contradicted">contradicted ${bs.contradicted}</span></td></tr>
296
+ <tr><td>safety</td><td class="num">${(safety.findings || []).length} / ${safety.critical || 0} critical</td><td>${esc(sTypes)}</td></tr>
297
+ <tr><td>system-prompt leak</td><td class="num">${leak.leak ? "YES" : "no"}</td>
298
+ <td>${(leak.overlap || 0).toFixed(3)} overlap vs threshold ${leak.threshold ?? "—"}</td></tr>
299
+ <tr><td>retry saving</td><td class="num">${est.saved_tokens ?? 0} tok</td><td>${esc(est.recommendation || "—")}</td></tr>
300
+ </table>`;
301
+ h += `<p class="muted"><strong>Reasons:</strong> ${esc((r.reasons || []).join("; "))}</p>`;
302
+ if (v.coverage < 0.5)
303
+ h += `<div class="note"><strong>⚠️ coverage is low.</strong> A high grounding_score here means very little —
304
+ most of this answer was never checked.</div>`;
305
+
306
+ h += `<div class="legend">` + ["supported", "derived", "approximate", "unsupported", "contradicted"]
307
+ .map((s) => `<span><i class="key" style="background:var(--${s})"></i>${s}</span>`).join("") + `</div>`;
308
+ h += `<div class="hl">` + cc.buildHighlight(answer, v.claims)
309
+ .map(([t, st]) => (st ? `<mark class="${st}">${esc(t)}</mark>` : esc(t))).join("") + `</div>`;
310
+
311
+ if (v.claims.length) {
312
+ h += `<table><tr><th>type</th><th>value</th><th>status</th><th>evidence</th></tr>` +
313
+ v.claims.map((cl) => `<tr><td class="muted">${cl.type}</td><td><code>${esc(cl.value)}</code></td>
314
+ <td><span class="st ${cl.status}">${cl.status}</span></td>
315
+ <td class="muted">${esc(cl.evidence || "—")}</td></tr>`).join("") + `</table>`;
316
+ }
317
+ if (v.unverified_sentences.length) {
318
+ h += `<details><summary>${v.unverified_sentences.length} sentence(s) were NOT checked — read these yourself</summary>`
319
+ + v.unverified_sentences.map((u) => `<p class="muted">${esc(u.text)}</p>`).join("") + `</details>`;
320
+ }
321
+ } else {
322
+ h += `<p class="muted"><strong>Reasons:</strong> ${esc((r.reasons || []).join("; "))}</p>`;
323
+ h += `<div class="note">Verification was skipped: a critical output-safety violation short-circuits the gate.
324
+ Verifying the grounding of an answer that leaks a credential is wasted work.</div>`;
325
+ if ((safety.findings || []).length)
326
+ h += `<table><tr><th>type</th><th>severity</th><th>masked</th></tr>` + safety.findings
327
+ .map((f) => `<tr><td>${esc(f.type)}</td><td class="warn">${esc(f.severity)}</td><td><code>${esc(f.value)}</code></td></tr>`).join("") + `</table>`;
328
+ }
329
+ if ((r.warnings || []).length) h += `<p class="muted"><strong>Warnings:</strong> ${esc(r.warnings.join("; "))}</p>`;
330
+ h += `<details><summary>Full result JSON</summary><pre>${esc(JSON.stringify(r, null, 2))}</pre></details>`;
331
+ return h;
332
+ }
333
+
334
+ $("run").addEventListener("click", () => {
335
+ const a = $("answer").value;
336
+ LAST = cc.gate(a, $("context").value, $("system").value, $("userin").value, $("schema").value, $("policy").value, "");
337
+ LAST._answer = a;
338
+ $("out").innerHTML = render(LAST, a);
339
+ });
340
+
341
+ $("run-retry").addEventListener("click", () => {
342
+ if (!LAST || !LAST.verify) { $("retry-out").innerHTML = `<p class="muted">Run <strong>Verify</strong> first.</p>`; return; }
343
+ const adv = cc.buildRetryInstruction({ verify: LAST.verify, meta: { sizes: LAST.meta.sizes } });
344
+ if (!adv.needed) { $("retry-out").innerHTML = `<p class="muted">No unsupported or contradicted claims — a retry is not warranted.</p>`; return; }
345
+ const e = adv.estimate;
346
+ $("retry-out").innerHTML =
347
+ `<table><tr><th>metric</th><th>value</th></tr>
348
+ <tr><td>failing claims</td><td class="num">${adv.failed_claims}</td></tr>
349
+ <tr><td>blind retry</td><td class="num">${e.blind_retry_tokens} tokens</td></tr>
350
+ <tr><td>targeted retry</td><td class="num">${e.targeted_retry_tokens} tokens</td></tr>
351
+ <tr><td>saved</td><td class="num">${e.saved_tokens} tokens</td></tr>
352
+ </table>
353
+ <p class="muted">${esc(e.recommendation)}</p>
354
+ <p class="muted"><code>saved_tokens</code> is signed on purpose: with a small context the instruction can cost more
355
+ than the blind retry it replaces, and hiding that would make this tool lie about its own value.</p>
356
+ <label>Instruction — append this to the next request</label>
357
+ <pre>${esc(adv.instruction)}</pre>
358
+ <details><summary>日本語版</summary><pre>${esc(adv.instruction_ja)}</pre></details>
359
+ <details><summary>Full advice JSON</summary><pre>${esc(JSON.stringify(adv, null, 2))}</pre></details>`;
360
+ });
361
+
362
+ $("run-json").addEventListener("click", () => {
363
+ const r = cc.structureCheck($("raw").value, $("schema2").value);
364
+ $("json-out").innerHTML =
365
+ `<table><tr><th>check</th><th>result</th></tr>
366
+ <tr><td>valid JSON after repair</td><td class="num">${r.valid_json ? "✅ yes" : "❌ no"}</td></tr>
367
+ <tr><td>satisfies schema</td><td class="num">${r.schema_ok === null ? "—" : r.schema_ok ? "✅ yes" : "❌ no"}</td></tr>
368
+ <tr><td>repairs applied</td><td>${r.repairs.length ? r.repairs.map((x) => `<code>${esc(x)}</code>`).join(" ") : "none needed"}</td></tr>
369
+ <tr><td>missing keys</td><td>${r.missing.length ? esc(r.missing.join(", ")) : "none"}</td></tr>
370
+ </table>
371
+ <label>Repaired value</label><pre>${esc(JSON.stringify(r.value, null, 2))}</pre>`;
372
+ });
373
+
374
+ $("run").click();
375
+ </script>
376
+ </body>
377
  </html>