# deg

deg reads a context, takes a JSON Schema, and returns a JSON object that is valid
against that schema. Most fields are **decided** by classification heads or **extracted**
as spans copied from the context. New tokens are generated only for fields marked
`x-generate: true`. The result costs far fewer generated tokens than writing the whole
JSON out, and everything outside the generated fields is deterministic.

Base model: Qwen3.5-0.8B. The full contract is in [SPEC.md](SPEC.md).

## The one flag

```json
{"type": "object",
 "properties": {
   "department": {"enum": ["billing", "technical", "sales"]},
   "urgent":     {"type": "boolean"},
   "order_id":   {"type": "string"},
   "reply":      {"type": "string", "x-generate": true, "maxLength": 400}},
 "required": ["department", "urgent", "order_id", "reply"]}
```

- `department` is decided (choice head + probabilities).
- `urgent` is decided (noul head).
- `order_id` is copied from the context (span pointer).
- `reply` is the only field that generates tokens, bounded by `maxLength`.

The mode follows only from the JSON type. There is no router, and `x-generate` is valid
only on `string` or on an array of `string`.

## Status

What is implemented:

- schema validation and compilation into a deterministic fill plan (`deg.schema`);
- response assembly and validation (`deg.response`);
- deterministic span parsers (`deg.parsers`);
- `/v1/fill` request parsing (`deg.request`).

- the model: Qwen3.5-0.8B trunk + decision heads + span pointer + LM head (`deg.model`);
- `fill(model, context, schema)` (`deg.infer`), training (`deg.train`) and evaluation
  (`deg.evaluate`); design in SPEC.md §12.

- head-to-head bench vs structured-output baselines (`deg.bench`; results in
  `results/deg/bench-v1/summary.md`):
  `python -m deg.bench.run --system {deg,constrained,constrained-llamacpp,nuextract,clef,frontier} --set datagen-test`
  (extra `bench`).
- JevBench (Benchmark Heaven, MIT; the 231 public typed decisions, scored by its own v1.3.0
  rules, `deg.bench.jevbench`; results in `results/deg/jevbench-v1/`):
  `python -m deg.bench.jevbench --system {deg,systemone} ...` then `--report`.
  EVAL ONLY: `deg.train` refuses JevBench paths and drops any record or jev example
  containing a JevBench state.
- **deg-suite v1** — the checkpoint/config selection metric (`deg.bench.suite`; results
  and design in `results/deg/suite-v1/`): one frozen held-out suite whose headline is the
  mean of DECIDE, EXTRACT and GENERATE, each the mean of its sources (datagen-test,
  JevBench, Banking77, SQuAD v2, ExtractBench, DialogSum + an open-judge, and a mixed set of
  real v3 val-split excerpts); invalid output scores 0 on all its fields:
  `CUDA_VISIBLE_DEVICES=0 python -m deg.bench.suite run --system deg --ckpt <ckpt> --label <name>`
  (or `results/deg/suite-v1/run_suite.sh <ckpt> <label>`), then `python -m deg.bench.suite report`.
  EVAL ONLY: `deg.train` refuses suite paths and drops any training record or jev example
  containing a suite context.

- **decide-v1** datagen (`deg.datagen.decide`): decide-heavy SystemOne-style records that
  teach adequacy judging, policy compliance, routing, ordinal grading, multi-hop, traps,
  trade-offs, under-specified cases, adversarial states and date/number thresholds. Every
  state and question is new, written by an open teacher; the skill list is a taxonomy only and
  no benchmark item is shown to a teacher. Each record has a primary question, a yes/no
  question with its exact negation (yes/no labels are 50/50 by construction), and optionally a
  verbatim reference (extract) and a rationale (x-generate). Targets are balanced: noul
  polarity, choice gold slot (options permuted) and score level. Labels are kept only where
  the blind voters (teachers' first-token logprobs over both option orders, plus Clef-flash)
  agree with the authored gold; `decide_probs` is the voters' mean distribution. Output:
  `/data/deg/datagen/decide-v1` (test = held-out domains, val 4%):
  `python -m deg.datagen.decide --out /data/deg/datagen/decide-v1 --spark-role voter
  --teacher url=http://127.0.0.1:8780/v1,model=Qwen3.8-27B-INT4,concurrency=32`.

- **gen-v1** datagen (`deg.datagen.gen`, corpus `deg.datagen.gen_sources`): generation
  records on SHORT-to-medium REAL text (<= 10k chars: v3 train-split excerpts, CNN/DM news,
  OASST1 chats, StackExchange threads, Enron email, Taskmaster-2 / MultiWOZ dialogues,
  QMSum AMI/ICSI meetings; training splits only, every excerpt checked by the suite guard;
  licences in the corpus `manifest.json`). 1-3 x-generate fields per request (summary, title,
  rationale, reply, action items, explanation, answer, note, next step; maxLength and most
  with x-max-tokens) plus 0-2 decide / 0-3 extract fields. The local 27B writes the schema and
  fills it (reasoning low); the OTHER model judges every generated field with the suite's
  open-judge rubric and votes blind on every decide field; only faithful 2 / responsive 2
  texts and agreeing decisions are kept. Output `/data/deg/datagen/gen-v1` (split by source
  document: test 2%, val 2%).

- **extract-real-v1** datagen (`deg.datagen.extract_real`, passages from
  `deg.datagen.extract_real_sources`): extraction records on REAL, permissively licensed
  text (v3 train-split EDGAR/arXiv excerpts, Wikipedia, CC-BY news, CUAD contracts,
  regulations.gov notices, Foodista posts, CORD receipts, arXiv abstracts; license and URL
  on every record; SQuAD v2 dev articles, suite contexts, JevBench states and ExtractBench
  text excluded and audited with `--audit`). The 27B writes a request schema per passage
  (question-style fields, typed dates/amounts/ids, arrays of objects, ~1/3 unanswerable
  nullable fields); fill A and an independent fill B (same open model, shuffled fields,
  different instructions) must agree per field after normalization, the span must be
  verbatim, and agreed nulls are re-checked; disagreeing fields are dropped. Short passages
  (<= 3,800 tokens, index-mode arrays) and long ones (4,300-5,600 tokens, anchored):
  `python -m deg.datagen.extract_real_sources --out /data/deg/external/extract-real-v1`,
  then `python -m deg.datagen.extract_real --out /data/deg/datagen/extract-real-v1
  --teacher url=http://127.0.0.1:8780/v1,model=Qwen3.8-27B-INT4,concurrency=24`.

- the HTTP server (`deg.server`): OpenAI Decisions API (`/v1/decisions`), TypeSafe Jev
  SystemOne (`/v1/systemone`) and `/v1/fill`, with request batching and OpenTelemetry;
  a stdlib Python SDK (`deg.client`). A TypeScript SDK is not written yet (deferred).

```python
from deg.model import load_model
from deg.infer import fill

model = load_model("/data/deg/runs/smoke-1/final")   # or None: base model, untrained heads
response = fill(model, "Order AB-1234 was charged twice. Refund today!", schema)
```

Training (one GPU; see `deg.train` for multi-GPU):

```bash
CUDA_VISIBLE_DEVICES=3 uv run --package deg --extra model python -m deg.train \
    --out /data/deg/runs/smoke-1 --steps 300
```

```python
from deg import compile_schema

plan = compile_schema(schema)
for field in plan.fields:
    print(field.pointer, field.mode)   # /department decide.choice, ... /reply generate
```

## HTTP API

```bash
CUDA_VISIBLE_DEVICES=0 uv run --package deg --extra model --extra server \
    python -m deg.server --ckpt /data/deg/runs/run-1/step-0009000 --port 8790
```

One server, three request shapes; each request is one fill (the input is read once). Set
`DEG_API_KEY` to require `Authorization: Bearer <key>`. Shapes and error bodies are in
`src/deg/api.py`; golden fixtures in `tests/fixtures/`.

| Route | Compatible with | Question types |
|---|---|---|
| `POST /v1/decisions` | OpenAI Decisions API (`openai` >= 3.26, `client.decisions.create`) | `predicate`, `choice`, `score` + deg `extract`, `generate` |
| `POST /v1/systemone` | TypeSafe Jev / SystemOne, Clef | `noul`, `choice`, `score` + deg `extract`, `generate` |
| `POST /v1/fill` | deg (SPEC.md §9) | any JSON Schema, `x-generate` |
| `GET /v1/models`, `GET /health` | | |

`predicate` = `noul` (boolean), `choice` = `enum`, `score` = integer levels `0..L-1`. Clients
may keep sending `gpt-6-luna` or `jev-latest` / `clef-flash`; deg answers them and reports
its own model id (`--no-compat-models` accepts only deg names).

### OpenAI Decisions API, stock SDK

```python
from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8790/v1", api_key="unused")
decision = client.decisions.create(
    model="gpt-6-luna",
    input="I was charged twice for my order.",
    questions=[{"type": "choice", "name": "department",
                "instructions": "Which department should handle this complaint?",
                "choices": [{"value": "billing", "description": "Payments, invoices, and refunds."},
                            {"value": "technical", "description": "Problems using the product."}]}],
)
decision.answers[0].choice        # "billing"; .probabilities, .confidence as in OpenAI's API
```

### The `generate` extension: event dispatch

Decide the event, copy its order id and write a note for the agent in one request. The note
is the only generated text; it is written after the other answers and sees them.

```python
from deg.client import Deg

client = Deg("http://127.0.0.1:8790")
decision = client.decisions.create(
    model="deg-latest",
    input="Hi, order AB-1234 arrived with a cracked screen on 2026-10-01. "
          "I want a refund, not a replacement.",
    questions=[
        {"type": "choice", "name": "event", "instructions": "Which event should be dispatched?",
         "choices": [{"value": "refund"}, {"value": "replacement"}, {"value": "other"}]},
        {"type": "extract", "name": "order_id", "instructions": "The order id."},
        {"type": "extract", "name": "delivered", "instructions": "Delivery date.",
         "value_type": "date"},
        {"type": "generate", "name": "note", "instructions": "A one-line note for the agent.",
         "max_length": 120, "max_tokens": 64},
    ],
)
# answers (step-9000, GPU smoke 2026-10-09):
# {"type": "choice", "name": "event", "choice": "refund", "probabilities": [...], "confidence": 0.82}
# {"type": "extract", "name": "order_id", "value": "AB-1234", "span": [10, 17], "confidence": 0.998}
# {"type": "extract", "name": "delivered", "value": "2026-10-01", "span": [51, 61], ...}
# {"type": "generate", "name": "note", "text": "Customer requested a refund for defective
#   order AB-1234 with a cracked screen on 2026-10-01, not a replacement.", "output_tokens": 35}
```

- `extract`: `value_type` is `string` (default), `number`, `integer`, `date`, `date-time`,
  `email`, `uri` or `uuid`; optional `pattern`, `min_length`, `max_length` (strings). The
  answer is `{value, span, confidence}`; `value` is `null` with a `null_reason` when the input
  does not contain it.
- `generate`: optional `max_tokens`, `min_length`, `max_length`, `pattern`; greedy. All
  `generate` questions of one request must share `max_tokens`.
- Request `options` (`temperature` + `seed`, `date_order`, `default_offset`); with the stock
  SDK pass them as `extra_body={"options": {...}}`.
- The stock `openai` SDK sends and parses the extensions too. It has no class for them, so
  an extension answer comes back as an object whose `.type` is `"extract"`/`"generate"` and
  whose fields (`.value`, `.span`, `.text`) are readable attributes.

### TypeSafe Jev / SystemOne

```python
client.systemone(
    state="I was charged twice for order AB-1234. Fix it today!",
    questions={
        "urgent": {"type": "noul", "instructions": "Is the customer asking for urgent action?"},
        "anger": {"type": "score", "instructions": "How angry is the customer?",
                  "criteria": ["calm", "annoyed", "angry", "furious"]},
        "order_id": {"type": "extract", "instructions": "The order id."},
        "reply": {"type": "generate", "instructions": "A short apology.", "max_length": 200},
    },
)
# {"model": ..., "answers": {"urgent": {"type": "noul", "noul": ...},
#   "anger": {"type": "score", "score": ..., "confidence": ..., "legend": {...}, "probabilities": {...}},
#   "order_id": {"type": "extract", "value": "AB-1234", ...}, "reply": {"type": "generate", "text": ...}},
#  "usage": {"input_tokens": ..., "output_tokens": ...}}
```

Existing Jev and Clef clients work unchanged (point them at `http://host:8790`).

### deg fill

```python
client.fill(context="Order AB-1234 was charged twice. Refund today!", schema=schema)
# {"model": ..., "value": {...}, "meta": {"/department": {"mode": "decide.choice", ...}, ...},
#  "usage": {"prompt_tokens": ..., "generated_tokens": ...}, "plan": "..."}
```

## Tests

```bash
cd packages/deg && uv run --package deg pytest
```