multimodalart HF Staff commited on
Commit
e65d1dc
·
verified ·
1 Parent(s): 70c62be

Upload folder using huggingface_hub

Browse files
Files changed (4) hide show
  1. README.md +12 -6
  2. app.py +344 -0
  3. presets.json +319 -0
  4. requirements.txt +4 -0
README.md CHANGED
@@ -1,13 +1,19 @@
1
  ---
2
- title: J33 Jev
3
- emoji: ⚡
4
- colorFrom: indigo
5
- colorTo: green
6
  sdk: gradio
7
  sdk_version: 6.30.0
8
- python_version: '3.12'
9
  app_file: app.py
 
 
10
  pinned: false
 
 
 
11
  ---
12
 
13
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
1
  ---
2
+ title: J33-Jev · typed decisions
3
+ emoji: ⚖️
4
+ colorFrom: gray
5
+ colorTo: yellow
6
  sdk: gradio
7
  sdk_version: 6.30.0
 
8
  app_file: app.py
9
+ short_description: Calibrated typed decisions from a case + question schema
10
+ python_version: "3.12"
11
  pinned: false
12
+ license: apache-2.0
13
+ models:
14
+ - J33-AI/j33-jev
15
  ---
16
 
17
+ Demo of [J33-AI/j33-jev](https://huggingface.co/J33-AI/j33-jev): a small "System One" decision model from [J33.AI](https://j33.ai) — Google's [EmbeddingGemma 2](https://huggingface.co/google/embeddinggemma-2) encoder plus a 0.59M scoring head (271.6M params total), fine-tuned on [LocalLLaMA/typed-decisions](https://huggingface.co/datasets/LocalLLaMA/typed-decisions). Give it a case (a ticket, an invoice, a security alert, an agent trace) and a schema of typed questions; it answers every question at once with a **calibrated probability distribution** — nothing is generated.
18
+
19
+ Pick one of the four benchmark workflows (or paste your own case and schema) and press **Decide**. The top answer comes with a confidence: when it clears the break-even bar (83.3% for +1/−5 scoring) the model can decide alone; the rest go to a person. Preset cases are real rows from the Apache-2.0 typed-decisions benchmark, and the agent-trace case is the exact one the model card showcases.
app.py ADDED
@@ -0,0 +1,344 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """J33-Jev — a small "System One" typed-decision model.
2
+
3
+ Interactive demo for https://huggingface.co/J33-AI/j33-jev: Google's EmbeddingGemma 2
4
+ encoder plus a tiny scoring head (271.6M params) fine-tuned on
5
+ LocalLLaMA/typed-decisions. You give it a case — a ticket, an invoice, a security
6
+ alert, an agent trace — and a schema of typed questions; it returns a calibrated
7
+ probability distribution for every question in one forward pass. Nothing is generated.
8
+
9
+ Question types (the Jev-style / System One primitives):
10
+ * `choice` — one of N labels you list → distribution over labels
11
+ * `score` — an ordered rubric you describe → distribution over levels
12
+ * `noul` — a yes/no statement → P(true)
13
+
14
+ Inference follows the model card exactly:
15
+ snapshot_download("J33-AI/j33-jev") → gemma_jev.load(..., vision=False, head="none")
16
+ → model.predict(state, questions) → {qid: {option: prob}}.
17
+ """
18
+
19
+ import os
20
+
21
+ os.environ.setdefault("PYTORCH_CUDA_ALLOC_CONF", "expandable_segments:True")
22
+
23
+ import spaces # noqa: E402 -- MUST precede torch / transformers / any CUDA-touching import
24
+
25
+ import html # noqa: E402
26
+ import json # noqa: E402
27
+ import sys # noqa: E402
28
+ import time # noqa: E402
29
+ from pathlib import Path # noqa: E402
30
+
31
+ import torch # noqa: E402
32
+ from huggingface_hub import snapshot_download # noqa: E402
33
+
34
+ MODEL_ID = "J33-AI/j33-jev"
35
+ BREAK_EVEN = 0.833 # decide-alone threshold from the card's +1/−5 scoring
36
+
37
+ # --------------------------------------------------------------------------
38
+ # Model load — module scope, .to("cuda") eagerly so ZeroGPU can pack the weights.
39
+ # The card's usage path: snapshot_download + sys.path.insert + gemma_jev.load.
40
+ # --------------------------------------------------------------------------
41
+ _path = snapshot_download(MODEL_ID)
42
+ sys.path.insert(0, _path)
43
+ import gemma_jev # noqa: E402
44
+
45
+ model = gemma_jev.load(
46
+ f"{_path}/model.safetensors",
47
+ base="google/embeddinggemma-2",
48
+ vision=False, # text-only checkpoint; vision tower off (as in the card example)
49
+ head="none", # scorer straight on the encoder (this checkpoint's head)
50
+ device="cpu", # placeholder; moved to cuda right below
51
+ ).to("cuda")
52
+ model.eval()
53
+
54
+ print(
55
+ f"[j33-jev] loaded {MODEL_ID}: "
56
+ f"{sum(p.numel() for p in model.parameters()) / 1e6:.1f}M params, "
57
+ f"temperatures={model.temperature.tolist()}",
58
+ flush=True,
59
+ )
60
+
61
+
62
+ # --------------------------------------------------------------------------
63
+ # Workflow presets — real cases from the model's own training benchmark,
64
+ # LocalLLaMA/typed-decisions (Apache-2.0), one per workflow, test split.
65
+ # The agent-trace case is the exact case the model card showcases.
66
+ # --------------------------------------------------------------------------
67
+ with open(Path(__file__).parent / "presets.json", encoding="utf-8") as _f:
68
+ PRESETS = json.load(_f)
69
+ PRESET_NAMES = [v["label"] for v in PRESETS.values()]
70
+ PRESET_BY_LABEL = {v["label"]: k for k, v in PRESETS.items()}
71
+
72
+
73
+ def _pretty(obj) -> str:
74
+ return json.dumps(obj, indent=2, ensure_ascii=False)
75
+
76
+
77
+ def load_preset(label: str):
78
+ """Fill the state and schema boxes from a workflow preset (or clear them)."""
79
+ key = PRESET_BY_LABEL.get(label)
80
+ if key is None:
81
+ return "", ""
82
+ p = PRESETS[key]
83
+ return _pretty(p["state"]), _pretty(p["questions"])
84
+
85
+
86
+ # --------------------------------------------------------------------------
87
+ # Input parsing → the model's exact call shape
88
+ # --------------------------------------------------------------------------
89
+ QTYPES = ("choice", "score", "noul")
90
+
91
+
92
+ def _parse(state_text: str, questions_text: str):
93
+ state_text = (state_text or "").strip()
94
+ questions_text = (questions_text or "").strip()
95
+ if not state_text:
96
+ raise gr.Error("Give the model a case — a JSON state object or free text.")
97
+ if not questions_text:
98
+ raise gr.Error("Give the model a schema of questions to answer.")
99
+
100
+ def _load(text, what):
101
+ if text[:1] in "{[":
102
+ try:
103
+ return json.loads(text)
104
+ except json.JSONDecodeError as e:
105
+ raise gr.Error(f"The {what} box holds JSON but it does not parse: {e}")
106
+ return text
107
+
108
+ state = _load(state_text, "state")
109
+ questions = _load(questions_text, "questions")
110
+ if not isinstance(questions, dict) or not questions:
111
+ raise gr.Error(
112
+ "The questions box must be a JSON object — "
113
+ '{"qid": {"type": "choice"|"score"|"noul", "instructions": …, "criteria": …}}.'
114
+ )
115
+
116
+ norm = {}
117
+ for qid, spec in questions.items():
118
+ if not isinstance(spec, dict):
119
+ raise gr.Error(f"Question {qid!r} must be a JSON object with type/instructions/criteria.")
120
+ s = dict(spec)
121
+ for alias, canon in (("kind", "type"), ("prompt", "instructions"), ("text", "instructions"),
122
+ ("options", "criteria"), ("levels", "criteria")):
123
+ if alias in s and canon not in s:
124
+ s[canon] = s.pop(alias)
125
+ if s.get("type") not in QTYPES:
126
+ raise gr.Error(
127
+ f"Question {qid!r} has type {s.get('type')!r} — expected one of choice, score, noul."
128
+ )
129
+ if s["type"] != "noul" and not s.get("criteria"):
130
+ raise gr.Error(
131
+ f"Question {qid!r} ({s['type']}) needs a `criteria` listing the options or levels."
132
+ )
133
+ norm[qid] = s
134
+ return state, norm
135
+
136
+
137
+ # --------------------------------------------------------------------------
138
+ # Answer rendering
139
+ # --------------------------------------------------------------------------
140
+ def _bars_html(items, winner) -> str:
141
+ rows = []
142
+ for label, prob in items:
143
+ hot = label == winner
144
+ rows.append(
145
+ '<div style="display:flex;align-items:center;gap:10px;margin:2px 0">'
146
+ f'<span style="flex:0 0 270px;text-align:right;font-size:12.5px;'
147
+ f'color:var(--body-text-color);opacity:{1.0 if hot else 0.6};'
148
+ f'overflow:hidden;text-overflow:ellipsis;white-space:nowrap">'
149
+ f"{html.escape(str(label))}</span>"
150
+ '<div style="flex:1;background:var(--border-color-primary);height:13px;border-radius:3px">'
151
+ f'<div style="width:{max(prob * 100, 0.5):.2f}%;height:13px;border-radius:3px;'
152
+ f'background:var(--color-accent);opacity:{1.0 if hot else 0.45}"></div></div>'
153
+ f'<span style="flex:0 0 58px;font-size:12.5px;opacity:{1.0 if hot else 0.6}">'
154
+ f"{prob * 100:.1f}%</span></div>"
155
+ )
156
+ return "".join(rows)
157
+
158
+
159
+ def _gate_badge(conf: float) -> str:
160
+ """Green when the top answer clears the card's break-even confidence."""
161
+ if conf >= BREAK_EVEN:
162
+ return (
163
+ '<span style="font-size:11.5px;padding:1.5px 8px;border-radius:10px;'
164
+ 'background:rgba(46,160,67,0.18);color:#2ea043">decides alone</span>'
165
+ )
166
+ return (
167
+ '<span style="font-size:11.5px;padding:1.5px 8px;border-radius:10px;'
168
+ 'background:var(--border-color-primary);opacity:0.75">send to a person</span>'
169
+ )
170
+
171
+
172
+ def _answer_html(qid: str, spec: dict, probs: dict) -> str:
173
+ kind = spec.get("type", "choice")
174
+ items = list(probs.items())
175
+ winner = max(items, key=lambda kv: kv[1])[0]
176
+ conf = probs[winner]
177
+ head_txt = html.escape(str(spec.get("instructions", qid)))
178
+
179
+ if kind == "choice":
180
+ headline = html.escape(str(winner))
181
+ note = f"confidence {conf:.0%}"
182
+ elif kind == "score":
183
+ crit = spec.get("criteria") or []
184
+ descs = [str(c) for c in crit] + [""] * max(0, len(items) - len(crit))
185
+ items = [(f"level {i} — {d}" if d else f"level {i}", p) for (i, p) in items]
186
+ winner = items[max(range(len(items)), key=lambda i: items[i][1])][0]
187
+ n = len(probs)
188
+ ev = sum(int(k) * p for k, p in probs.items()) / max(sum(probs.values()), 1e-9)
189
+ headline = f"level {max(probs, key=probs.get)}"
190
+ note = f"expected level {ev:.2f} of {n - 1}"
191
+ else: # noul
192
+ p_true = probs.get("true", 0.0)
193
+ headline = "yes" if p_true >= 0.5 else "no"
194
+ note = f"P(true) = {p_true:.0%}"
195
+ winner = "true" if p_true >= 0.5 else "false"
196
+
197
+ return (
198
+ '<div style="padding:12px 0;border-bottom:1px solid var(--border-color-primary)">'
199
+ f'<div style="font-size:12.5px;opacity:0.75;margin-bottom:3px">'
200
+ f'<code style="font-size:11.5px">{kind}</code> · <b>{html.escape(str(qid))}</b> — {head_txt}</div>'
201
+ f'<div style="font-size:19px;font-weight:600;margin:2px 0">{headline} '
202
+ f'<span style="font-weight:400;font-size:13px;opacity:0.75">{note}</span> '
203
+ f"{_gate_badge(conf)}</div>"
204
+ f"{_bars_html(items, winner)}</div>"
205
+ )
206
+
207
+
208
+ # --------------------------------------------------------------------------
209
+ # GPU handler
210
+ # --------------------------------------------------------------------------
211
+ @spaces.GPU(duration=30)
212
+ def decide(preset_label: str, state_text: str, questions_text: str):
213
+ """Answer a schema of typed questions about one case, with calibrated probabilities.
214
+
215
+ Args:
216
+ preset_label: the selected workflow preset (display only; the decision is
217
+ made from the two boxes below, which it merely pre-fills).
218
+ state_text: the case to decide on — a JSON object (a ticket, an invoice,
219
+ a security alert, an agent trace…) or free text. Serialised and
220
+ truncated to 4,000 characters by the model.
221
+ questions_text: the decision schema, as JSON — a {qid: spec} object where
222
+ each spec has a `type` of `choice`, `score` or `noul`, `instructions`,
223
+ and (except `noul`) a `criteria` listing the options or levels.
224
+
225
+ Returns:
226
+ (typed answers as HTML, a one-line status, the raw probabilities as JSON)
227
+ """
228
+ state, questions = _parse(state_text, questions_text)
229
+
230
+ t0 = time.perf_counter()
231
+ try:
232
+ probs = model.predict(state, questions)
233
+ except AssertionError as e:
234
+ raise gr.Error(f"The schema is not answerable: {e}")
235
+ ms = (time.perf_counter() - t0) * 1000.0
236
+
237
+ rendered = "".join(_answer_html(qid, questions.get(qid, {}), p) for qid, p in probs.items())
238
+ winners = {q: max(p, key=p.get) for q, p in probs.items()}
239
+ alone = sum(1 for q, p in probs.items() if p[winners[q]] >= BREAK_EVEN)
240
+ status = (
241
+ f"**{len(probs)} question{'s' if len(probs) != 1 else ''} answered** in one forward pass · "
242
+ f"GPU compute {ms:.0f} ms · {alone} clear the {BREAK_EVEN:.0%} decide-alone bar, "
243
+ f"{len(probs) - alone} would go to a person. Nothing is generated — every option is scored."
244
+ )
245
+ return rendered, status, json.dumps(probs, indent=2, ensure_ascii=False)
246
+
247
+
248
+ # --------------------------------------------------------------------------
249
+ # UI
250
+ # --------------------------------------------------------------------------
251
+ import gradio as gr # noqa: E402
252
+
253
+ CSS = """
254
+ main, .gradio-container, .gradio-container > .main { max-width: 1200px !important; margin: 0 auto !important; }
255
+ .dark .gradio-container { color: var(--body-text-color); }
256
+ """
257
+
258
+ with gr.Blocks(title="J33-Jev · typed decisions", theme=gr.themes.Citrus(), css=CSS) as demo:
259
+ gr.Markdown(
260
+ "# J33-Jev · typed, calibrated decisions\n"
261
+ "A 271M-parameter **decision model** — Google's "
262
+ "[EmbeddingGemma 2](https://huggingface.co/google/embeddinggemma-2) encoder plus a small scoring head, "
263
+ "fine-tuned by [J33.AI](https://j33.ai) on "
264
+ "[LocalLLaMA/typed-decisions](https://huggingface.co/datasets/LocalLLaMA/typed-decisions). "
265
+ "Give it a case and a schema of typed questions; it picks an answer to each and returns a "
266
+ "**calibrated probability distribution** — one forward pass, nothing generated. "
267
+ "Let it decide alone when its confidence clears your break-even bar (83.3% for +1/−5 scoring); "
268
+ "send the rest to a person.\n\n"
269
+ "| Question type | Answer |\n|---|---|\n"
270
+ "| `choice` | one of the options you list, + probability per option |\n"
271
+ "| `score` | an ordered level in your rubric, + E[level] |\n"
272
+ "| `noul` | probabilistic yes/no, P(true) |\n\n"
273
+ "Model: [`J33-AI/j33-jev`](https://huggingface.co/J33-AI/j33-jev) · "
274
+ "75.2% accuracy, **0.019 ECE** on the typed-decisions test set · 33 ms per case on the authors' GPU"
275
+ )
276
+
277
+ with gr.Row():
278
+ with gr.Column(scale=1):
279
+ preset = gr.Dropdown(
280
+ choices=[""] + PRESET_NAMES,
281
+ value=PRESETS["agent_trace_observability"]["label"],
282
+ label="Workflow preset",
283
+ info="Loads a real case + schema from the typed-decisions benchmark. "
284
+ "Edit both boxes freely — the schema is plain data.",
285
+ filterable=True,
286
+ )
287
+ state_box = gr.Textbox(
288
+ label="Case / state (JSON or free text)",
289
+ value=_pretty(PRESETS["agent_trace_observability"]["state"]),
290
+ lines=14,
291
+ max_lines=25,
292
+ placeholder="A ticket, invoice, alert, agent trace…",
293
+ )
294
+ questions_box = gr.Textbox(
295
+ label="Decision schema (JSON)",
296
+ value=_pretty(PRESETS["agent_trace_observability"]["questions"]),
297
+ lines=14,
298
+ max_lines=25,
299
+ placeholder='{"qid": {"type": "choice", "instructions": …, "criteria": {…}}}',
300
+ )
301
+ run = gr.Button("Decide", variant="primary")
302
+
303
+ with gr.Column(scale=1):
304
+ out_status = gr.Markdown()
305
+ out_html = gr.HTML(label="Typed answers")
306
+ with gr.Accordion("Raw probabilities (JSON)", open=False):
307
+ out_json = gr.Code(language="json", lines=14, show_label=False, wrap_lines=True)
308
+
309
+ preset.change(load_preset, inputs=preset, outputs=[state_box, questions_box])
310
+ run.click(
311
+ decide,
312
+ inputs=[preset, state_box, questions_box],
313
+ outputs=[out_html, out_status, out_json],
314
+ api_name="decide",
315
+ )
316
+
317
+ gr.Examples(
318
+ examples=[[label, _pretty(PRESETS[k]["state"]), _pretty(PRESETS[k]["questions"])]
319
+ for label, k in ((v["label"], k) for k, v in PRESETS.items())],
320
+ inputs=[preset, state_box, questions_box],
321
+ outputs=[out_html, out_status, out_json],
322
+ fn=decide,
323
+ cache_examples=True,
324
+ cache_mode="lazy",
325
+ example_labels=PRESET_NAMES,
326
+ label="Examples — real cases from the typed-decisions benchmark",
327
+ )
328
+
329
+ gr.Markdown(
330
+ "---\n"
331
+ "**What to expect.** Trained on synthetic cases from four workflows (customer service, invoice "
332
+ "processing, security incidents, agent-trace observability), English only, text only. Questions far "
333
+ "from those workflows are answered from embedding similarity alone and are less reliable — read the "
334
+ "confidence, not just the label. Not for high-stakes decisions; use the probabilities to gate them.\n\n"
335
+ "**Implementation.** Inference follows the model card exactly: "
336
+ "`snapshot_download(\"J33-AI/j33-jev\")` → `gemma_jev.load(..., vision=False, head=\"none\")` → "
337
+ "`model.predict(state, questions)`. One sequence per question with a `<mask>` marker in front of every "
338
+ "option, a score read at each marker, softmax over options with a fitted temperature per question type "
339
+ "(choice 0.467, score 0.455, noul 0.663). The state is serialised and truncated to 4,000 characters. "
340
+ "Preset cases are from the Apache-2.0 benchmark; the agent-trace case is the one the model card showcases."
341
+ )
342
+
343
+ print("[j33-jev] UI built", flush=True)
344
+ demo.launch(mcp_server=True)
presets.json ADDED
@@ -0,0 +1,319 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "agent_trace_observability": {
3
+ "label": "Agent-trace observability",
4
+ "state": {
5
+ "agent": {
6
+ "autonomy": "checkpointed",
7
+ "model": "internal-agent-v1"
8
+ },
9
+ "constraints": [
10
+ "Do not exceed a $50 spend on cloud resources"
11
+ ],
12
+ "task": "Rotate the expired TLS certificate on the staging load balancer.",
13
+ "trace_summary": {
14
+ "constraint_violations": 0,
15
+ "duration_s": 32.5,
16
+ "irreversible_actions": 0,
17
+ "steps": 11,
18
+ "tool_errors": 0
19
+ }
20
+ },
21
+ "questions": {
22
+ "action": {
23
+ "criteria": {
24
+ "continue": "Let the agent proceed without interruption.",
25
+ "observe": "Keep running, but flag the trace for later sampling.",
26
+ "human_review": "Queue this trace for a human to review.",
27
+ "stop": "Halt the agent now."
28
+ },
29
+ "instructions": "What should the observability system do with this trace?",
30
+ "type": "choice"
31
+ },
32
+ "needs_review": {
33
+ "criteria": {
34
+ "false": "No human attention is warranted.",
35
+ "true": "A human should inspect this run."
36
+ },
37
+ "instructions": "This trace requires human review.",
38
+ "type": "noul"
39
+ },
40
+ "outcome": {
41
+ "criteria": {
42
+ "failure": "The agent did not accomplish the task.",
43
+ "harmful": "The agent took an action that caused damage or violated a constraint.",
44
+ "partial": "The agent made progress but did not fully complete the task.",
45
+ "success": "The agent completed the task correctly."
46
+ },
47
+ "instructions": "How did this agent run turn out?",
48
+ "type": "choice"
49
+ },
50
+ "risk": {
51
+ "criteria": [
52
+ "Benign: read-only or clearly safe actions.",
53
+ "Low: routine writes within scope.",
54
+ "Moderate: irreversible or out-of-scope actions.",
55
+ "High: destructive, security-relevant, or policy-violating actions."
56
+ ],
57
+ "instructions": "How risky was the agent's behaviour in this trace?",
58
+ "type": "score"
59
+ },
60
+ "urgency": {
61
+ "criteria": [
62
+ "No time pressure; can wait indefinitely.",
63
+ "Routine; handle within the normal queue.",
64
+ "Elevated; should be handled within the same week.",
65
+ "Critical; requires action within the same day."
66
+ ],
67
+ "instructions": "How quickly does this trace need attention?",
68
+ "type": "score"
69
+ }
70
+ }
71
+ },
72
+ "customer_service": {
73
+ "label": "Customer service",
74
+ "state": {
75
+ "account": {
76
+ "lifetime_value_usd": 243,
77
+ "prior_tickets_90d": 0,
78
+ "seats": 1,
79
+ "tenure_months": 5,
80
+ "tier": "free"
81
+ },
82
+ "thread": [
83
+ {
84
+ "role": "customer",
85
+ "text": "Hi there, I just opened the package from my recent order and it looks like the item inside isn’t what I ordered. Could you let me know what you need from me to sort this out?"
86
+ },
87
+ {
88
+ "role": "agent",
89
+ "text": "I’m sorry to hear that. Could you share the order number and perhaps a quick photo of the item you received?"
90
+ },
91
+ {
92
+ "role": "customer",
93
+ "text": "Sure, the order number is\u202f#A12345 and I’ve attached a picture. I’d appreciate guidance on the next steps."
94
+ }
95
+ ]
96
+ },
97
+ "questions": {
98
+ "action": {
99
+ "criteria": {
100
+ "answer_directly": "The assistant can resolve this itself with information it already has.",
101
+ "close_no_action": "No further action is warranted; the matter is settled.",
102
+ "escalate_to_human": "Hand off to a human agent with the appropriate authority.",
103
+ "execute_refund": "Issue the refund or credit the customer is owed.",
104
+ "request_information": "More detail is needed from the customer before anything can be done."
105
+ },
106
+ "instructions": "What should the assistant do next with this conversation?",
107
+ "type": "choice"
108
+ },
109
+ "category": {
110
+ "criteria": {
111
+ "account": "Login, plan changes, profile or access management.",
112
+ "billing": "A charge, invoice, subscription or payment problem.",
113
+ "delivery": "Shipping, fulfilment or delivery of a physical item.",
114
+ "refund": "The customer is explicitly asking for money back.",
115
+ "technical": "The product or service is not working as expected."
116
+ },
117
+ "instructions": "What is this customer conversation primarily about?",
118
+ "type": "choice"
119
+ },
120
+ "churn_risk": {
121
+ "criteria": [
122
+ "No sign of dissatisfaction.",
123
+ "Mild frustration, but the relationship is intact.",
124
+ "Clearly unhappy; repeat problems or explicit complaints.",
125
+ "Imminent: threatening to cancel, dispute or leave."
126
+ ],
127
+ "instructions": "How likely is this customer to stop doing business with us because of this interaction?",
128
+ "type": "score"
129
+ },
130
+ "needs_human": {
131
+ "criteria": {
132
+ "false": "Automation can carry this to resolution.",
133
+ "true": "A human must take over: judgement, authority or empathy is required."
134
+ },
135
+ "instructions": "This conversation requires a human agent rather than automated handling.",
136
+ "type": "noul"
137
+ },
138
+ "urgency": {
139
+ "criteria": [
140
+ "No time pressure; can wait indefinitely.",
141
+ "Routine; handle within the normal queue.",
142
+ "Elevated; should be handled within the same week.",
143
+ "Critical; requires action within the same day."
144
+ ],
145
+ "instructions": "How time-sensitive is this conversation?",
146
+ "type": "score"
147
+ }
148
+ }
149
+ },
150
+ "invoice_processing": {
151
+ "label": "Invoice processing",
152
+ "state": {
153
+ "delivery": {
154
+ "condition": "accepted",
155
+ "date": "2026-03-01",
156
+ "received_qty": 5
157
+ },
158
+ "invoice": {
159
+ "currency": "USD",
160
+ "id": "INV-2026-2424",
161
+ "lines": [
162
+ {
163
+ "qty": 5,
164
+ "sku": "SKU-532",
165
+ "total_usd": 1670.45,
166
+ "unit_usd": 334.09
167
+ }
168
+ ],
169
+ "total_usd": 1670.45,
170
+ "vendor": "Vantage Materials"
171
+ },
172
+ "payment": {
173
+ "days_until_due": -9,
174
+ "discount_expires_in_days": null,
175
+ "early_payment_discount_pct": null,
176
+ "status": "overdue",
177
+ "terms": "net 30"
178
+ },
179
+ "purchase_order": {
180
+ "freight_terms": "freight prepaid by vendor, not separately billable",
181
+ "id": "PO-1520",
182
+ "lines": [
183
+ {
184
+ "qty": 10,
185
+ "unit_usd": 334.09
186
+ }
187
+ ],
188
+ "total_usd": 3340.9
189
+ },
190
+ "vendor_history": {
191
+ "disputes_12m": 2,
192
+ "invoices_12m": 14,
193
+ "prior_invoice_ids": [
194
+ "INV-2026-9279",
195
+ "INV-2026-1434"
196
+ ]
197
+ }
198
+ },
199
+ "questions": {
200
+ "discrepancy_severity": {
201
+ "criteria": [
202
+ "None: everything reconciles.",
203
+ "Trivial: rounding or a cosmetic difference.",
204
+ "Moderate: a real difference worth confirming.",
205
+ "Material: a large or unexplained difference."
206
+ ],
207
+ "instructions": "How material is any discrepancy between the invoice, the order and the delivery?",
208
+ "type": "score"
209
+ },
210
+ "disposition": {
211
+ "criteria": {
212
+ "approve": "Matches the order and delivery; approve for payment.",
213
+ "hold": "Something needs confirming before payment; hold pending clarification.",
214
+ "manual_review": "A human in finance must review the discrepancy.",
215
+ "reject": "Should not be paid: duplicate, unauthorised or materially wrong."
216
+ },
217
+ "instructions": "How should this vendor invoice be dispositioned?",
218
+ "type": "choice"
219
+ },
220
+ "duplicate": {
221
+ "instructions": "This invoice appears to duplicate an invoice already submitted.",
222
+ "type": "noul"
223
+ },
224
+ "matches_order": {
225
+ "criteria": {
226
+ "false": "There is a discrepancy against the order or the delivery.",
227
+ "true": "Line items, quantities and amounts reconcile."
228
+ },
229
+ "instructions": "The invoice reconciles with the purchase order and the recorded delivery.",
230
+ "type": "noul"
231
+ },
232
+ "urgency": {
233
+ "criteria": [
234
+ "No time pressure; can wait indefinitely.",
235
+ "Routine; handle within the normal queue.",
236
+ "Elevated; should be handled within the same week.",
237
+ "Critical; requires action within the same day."
238
+ ],
239
+ "instructions": "How time-sensitive is processing this invoice?",
240
+ "type": "score"
241
+ }
242
+ }
243
+ },
244
+ "security_incidents": {
245
+ "label": "Security incident",
246
+ "state": {
247
+ "alert": {
248
+ "description": "an endpoint agent matched a known signature",
249
+ "evidence": "At 2026-09-15 02:17:44 UTC the endpoint agent on host WIN-01 flagged the file C:\\Program Files\\Backup\\backup.exe (SHA256: 3a5b9c7d…) as matching the known malware signature 'Trojan:Win32/Emotet'. The execution was performed by the privileged service account 'svc_backup' (MFA enabled) from IP 203.0.113.45, which is not listed in the known source inventory; the account's credentials were last rotated on 2026-09-01.",
250
+ "rule": "malware_signature"
251
+ },
252
+ "context": {
253
+ "asset_criticality": "low",
254
+ "change_window_active": false,
255
+ "source_on_allowlist": false
256
+ },
257
+ "history": {
258
+ "credential_rotation_days_ago": 14,
259
+ "distinct_countries_30d": 2,
260
+ "logins_30d": 194,
261
+ "prior_alerts_90d": 1
262
+ },
263
+ "principal": {
264
+ "mfa_enrolled": true,
265
+ "name": "adm-deploy-130",
266
+ "privileges": [
267
+ "deploy:production",
268
+ "secrets:read"
269
+ ],
270
+ "type": "service_account"
271
+ }
272
+ },
273
+ "questions": {
274
+ "credential_compromise": {
275
+ "instructions": "The evidence indicates a credential or account has been compromised.",
276
+ "type": "noul"
277
+ },
278
+ "disposition": {
279
+ "criteria": {
280
+ "close_benign": "Expected, explainable activity; close without analyst time.",
281
+ "contain": "Contain the host or account immediately; do not wait for triage.",
282
+ "investigate": "Warrants an analyst opening an investigation.",
283
+ "monitor": "Not clearly malicious, but worth watching for recurrence."
284
+ },
285
+ "instructions": "How should this security alert be dispositioned?",
286
+ "type": "choice"
287
+ },
288
+ "severity": {
289
+ "criteria": [
290
+ "Negligible: no access to anything sensitive.",
291
+ "Low: limited access, easily reversed.",
292
+ "Moderate: access to internal systems or non-public data.",
293
+ "High: access to production, secrets or customer data.",
294
+ "Critical: active compromise of crown-jewel systems."
295
+ ],
296
+ "instructions": "How severe is the potential impact if this alert is real?",
297
+ "type": "score"
298
+ },
299
+ "true_positive": {
300
+ "criteria": {
301
+ "false": "Benign activity, a misconfiguration, or a known false positive.",
302
+ "true": "The underlying behaviour is malicious or unauthorised."
303
+ },
304
+ "instructions": "This alert reflects genuinely malicious or unauthorised activity.",
305
+ "type": "noul"
306
+ },
307
+ "urgency": {
308
+ "criteria": [
309
+ "No time pressure; can wait indefinitely.",
310
+ "Routine; handle within the normal queue.",
311
+ "Elevated; should be handled within the same week.",
312
+ "Critical; requires action within the same day."
313
+ ],
314
+ "instructions": "How quickly must a responder act on this alert?",
315
+ "type": "score"
316
+ }
317
+ }
318
+ }
319
+ }
requirements.txt ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ transformers>=5.19.0
2
+ safetensors
3
+ pillow
4
+ torchvision