Blame for source/SPEC.md

# deg contract v0.1

Status: contract and schema compiler implemented (`src/deg/schema.py`, `response.py`,
`request.py`, `parsers.py`); model, trainer and in-process runtime implemented
(`model.py`, `train.py`, `infer.py`, `evaluate.py`, design in §12); HTTP server and Python
SDK implemented (`api.py`, `server.py`, `client.py`, §9).

## 1. Model

- Base model: **Qwen3.5-0.8B** (default; LFM2.5-1.2B-Instruct is the alternative
  backbone, §12.8), with three kinds of heads on one trunk:
  - decision heads: `choice`, `score`, `noul` (as in jev v1);
  - a span pointer (start/end over context tokens, plus a null outcome);
  - the LM head, used only for `x-generate` fields.
- Model id: `deg-qwen3.5-0.8b-0.1` (alias `deg-latest`).

## 2. The one flag (and two annotations)

`x-generate: true` is the only extension that changes a field's mode.

- It is allowed only on `type: string` or on `type: array` whose `items` is a plain
  `type: string` (no `enum`, no `format`). On an array, it applies to every item.
- `x-generate: false` means the same as leaving it out.

Two annotations refine a field without changing its mode:

- `x-max-tokens: N` (integer >= 1, only where `x-generate` applies; on an array it applies
  to every item): that field's own decode budget, overriding `options.max_new_tokens`, so
  one request can mix budgets.
- `x-levels: {"<level>": "label", ...}` (only on an integer score field, keys among its
  admissible levels, labels non-empty strings): each labeled level's option line becomes
  `<level>: <label>` (the layout the score head was trained on from jev level criteria);
  unlabeled levels keep the bare number. Use it instead of listing level meanings in the
  description.

Any other `x-*` keyword is an error, so a typo like `x-genrate` fails loudly.

There is **no router head**. A field's mode follows only from its JSON type and keywords
(§3). The model never chooses between deciding, extracting and generating.

## 3. Type → mechanism

| Schema (no `x-generate`) | Mode | Output |
|---|---|---|
| `enum` (any scalar types, 2..255 values; `null` allowed as a value) | `decide.choice` | the value, its probability, the full distribution, confidence |
| `oneOf`/`anyOf` whose branches are all `const`/`enum` | `decide.choice` | the same; branch `description`s become option descriptions |
| `boolean` | `decide.noul` | `p(true)`; the value is `p >= 0.5` |
| `["boolean","null"]` | `decide.choice` over `[true, false, null]` | as choice |
| `integer` whose bounds admit 2..11 values (after `multipleOf`) | `decide.score` | the most probable level, the distribution, the expected level |
| every other number: any `type: number`, or an `integer` with a missing bound or more than 11 admissible values | `extract.number` | a span plus a deterministic number parser; bounds only validate |
| `string` with `format: date` / `date-time` | `extract.date` | a span plus a deterministic date parser, giving an ISO 8601 / RFC 3339 value |
| `string` with `format: email` / `uri` / `uuid` | `extract.span` | a verbatim span that must pass the format check |
| `string` | `extract.span` | a verbatim span |
| `array` | per item | the length is a runtime count; each item uses the item schema's mode |
| `object` | recursive | the properties, in schema order |

| Schema with `x-generate: true` | Mode |
|---|---|
| `string` (optional `minLength`, `maxLength`, `pattern`) | `generate` |
| `array` of `string` | `generate` per item |

Rules of thumb that follow from the table:

- **Number rule.** Only a small integer scale is a judgment: an `integer` whose bounds give
  at most 11 possible values (`1..5`, `0..10`, `0..100` step 25) is decided. Every other
  number is a quantity copied from the context. Bounds (`minimum`, `maximum`, exclusive
  bounds, `multipleOf`) never turn a quantity into a decision; they only validate the
  extracted value. There is no expected-value path for numbers: deg never computes a
  number that is not written in the context. To get a judgment on a finer or non-integer
  scale, use an integer scale or an `enum`.
- Extracted values are always copied from the context. Parsers only normalize them; they
  never invent digits or dates.
- **x-generate fields are the only source of new tokens.**

## 4. Null, optional and constraints

- Nullable means `type: [X, "null"]`, `anyOf`/`oneOf: [X, {type: null}]`, or an `enum`
  that lists `null`. Nullable objects and arrays are not supported.
- Decide fields are always filled, whether optional or not. Nullable score fields get an
  extra null outcome, and nullable booleans become a 3-way choice.
- When an extract field finds nothing, or its span fails the parser or a constraint, the
  plan's `on_null` decides what happens:
  - `null`: the field is nullable, so the value is `null`;
  - `omit`: the field is optional and not nullable, so the key is left out;
  - `forbid`: the field is required and not nullable, so the extractor must return its
    best valid span. If it can't, that is a runtime error (`FillError`, HTTP 500), never a
    made-up value. This is deliberate: authors who want a fallback mark the field
    nullable (`type: [X, "null"]`) and get `null` with a `null_reason` instead.
    **Exception: inside an array item** (anchored or index-addressed, §12.6) a `forbid`
    leaf with no valid value means the item does not exist: the item is dropped (not an
    error), the array's meta says `"dropped": n`, and a `minItems` the drops break is
    handled as in §12.6 (drop the enclosing item, else a `FillError` at the top level).
- `meta.null_reason` is one of `not_found`, `unparsable`, or the name of the violated
  keyword.
- Constraints are enforced as follows:
  - Spans are filtered by `minLength`, `maxLength`, `pattern` (searched, as in JSON
    Schema) and `format`.
  - Numbers are filtered by `minimum`, `maximum`, `exclusiveMinimum`, `exclusiveMaximum`
    and `multipleOf`.
  - Generated text must satisfy `minLength`, `maxLength` and `pattern`. Output that
    violates them is a runtime error (`FillError`), never a silent truncation or repair.
    The generator uses **constrained decoding** (a length budget plus a regex/grammar
    automaton for `pattern`; EOS masked until `minLength`), so a valid output exists by
    construction and this error should not occur in practice; it is a guard, not a code
    path.
- Arrays: the length is bounded by `minItems` and `maxItems`. When `maxItems` is absent,
  the cap is `options.max_items` (default `null` = no cap: an anchored array has as many
  items as the context has, §12.6; an index-addressed array stops at 32, the count head's
  trained range). `uniqueItems` is checked when the response is validated.
- Objects: `additionalProperties` may be absent, `true` or `false`. deg never emits
  undeclared keys. A schema-valued `additionalProperties` is an error, because the key
  names would be unknown.

## 5. Validation errors (HTTP 422, every problem listed as `{loc, msg}`)

- `x-generate` on anything other than `string` or `array` of `string`. This includes
  `enum`, `boolean`, `integer`, `number`, `object`, arrays of non-strings, and strings
  that have `enum` or `format`.
- **Unsupported keywords:** `allOf`, `not`, `if`/`then`/`else`, `const` (a single value
  has nothing to decide), `dependentSchemas`, `dependentRequired`, `dependencies`,
  `patternProperties`, `propertyNames`, `unevaluatedProperties`, `minProperties`,
  `maxProperties`, `unevaluatedItems`, `prefixItems`, `additionalItems`, tuple `items:
  [...]`, `contains`, `minContains`, `maxContains`, `$dynamicRef`, `$dynamicAnchor`,
  `$recursiveRef`, `$recursiveAnchor`, `contentEncoding`, `contentMediaType`,
  `contentSchema`.
- **Limited keywords:**
  - `anyOf`/`oneOf` are allowed only in the nullable form or the const/enum form.
  - `$ref` must be local (`#` or `#/...`) and not recursive. Its siblings may only be
    annotations, which override the target's.
  - Union types are allowed only as `[X, "null"]`.
  - `format` must be one of `date`, `date-time`, `email`, `uri` or `uuid`.
- Also errors:
  - unknown keywords, and keywords that don't apply to the type (e.g. `maxLength` on
    `integer`);
  - an `enum` with a duplicate value, a value of the wrong type, or fewer than 2 or more
    than 255 values;
  - integer bounds that admit 0 or 1 values;
  - number bounds that admit no value (e.g. `minimum` > `maximum`);
  - an invalid `pattern`;
  - a `required` name that isn't in `properties`;
  - a root that isn't `type: object`;
  - any document that isn't a valid JSON Schema (draft 2020-12 by default, or the draft
    named in `$schema`).
- Allowed annotations: `title`, `description`, `default`, `examples`, `$comment`,
  `$schema`, `$id`, `$anchor`, `$defs`, `definitions`, `readOnly`, `writeOnly`,
  `deprecated`. A leaf's `title` and `description` are its instructions to the model.

## 6. Fill plan

`compile_schema(schema) -> FillPlan` is a pure function. The plan holds:

- `fields`: the leaves in schema order (depth-first). Each leaf has:
  - its path: property names, with `*` meaning each item;
  - its mode and JSON type;
  - `options`/`option_descriptions` (choice) or `levels` (score);
  - `null_outcome`, `format`, `constraints`;
  - `required`, `nullable`, `on_null`, `title`, `description`.
- `arrays`: the array nodes with `min_items`, `max_items` and `unique_items`.
- `FieldPlan.fingerprint`: a sha256 of the leaf's canonical spec. It is identical wherever
  that leaf appears, whatever its siblings or their order.
- `FillPlan.fingerprint`: a hash of the **set** of parallel-phase fingerprints plus the
  **ordered list** of generate fingerprints.
  - Reordering decide/extract properties keeps the hash.
  - Reordering generate fields changes it.

## 7. Execution order

1. **Read once.** The context is rendered once and encoded once:
   - strings are used verbatim;
   - JSON is serialized with `", "`/`": "` separators, the given key order and
     `ensure_ascii=false`.
2. **Parallel phase: all decide.\* and extract.\* fields.** Each field's outcome is a
   function of exactly the context encoding, the field's key path, the field's own
   annotations, and its options/levels/constraints. It is never a function of other fields
   or their order.
   - So outcomes are **bitwise-reproducible** when fields are reordered, added or removed,
     or when the schema is split into several requests or merged into a bigger one.
   - Array items are first enumerated as item anchors (the count). Each item's leaves are
     then conditioned on the context plus that item's anchor.
3. **Generate phase.** The `generate` fields run after the parallel phase. Each one is
   conditioned on the context and all decided/extracted values (one shared KNOWN VALUES
   block), never on other generate fields: generate fields are independent of each other
   and decode together.
   - Decoding is greedy by default.
   - `options.temperature > 0` requires `options.seed`; each field samples from its own
     stream seeded by `(seed, pointer)`.
   - Decoding is bounded by `maxLength`, `pattern` and the field's `x-max-tokens`, else
     `options.max_new_tokens` (default 256 per field).
   - Not implemented: generate-to-generate dependencies. If added, they must be declared
     (e.g. `"x-generate": {"after": "/summary"}`) and the dependent field must continue
     from the parent row's final state after its generated tokens (forked), never
     re-prefill the parent's text; unannotated schemas keep the independence above.

## 8. Response

```json
{
  "model": "deg-qwen3.5-0.8b-0.1",
  "value": {"department": "billing", "urgent": true, "reply": "..."},
  "meta": {
    "/department": {"mode": "decide.choice", "probability": 0.8, "confidence": 0.7,
                     "probabilities": [0.8, 0.15, 0.05]},
    "/urgent":     {"mode": "decide.noul", "probability": 0.91},
    "/anger":      {"mode": "decide.score", "probability": 0.6, "confidence": 0.5,
                     "probabilities": [0, 0.1, 0.6, 0.3, 0], "expected": 2.2},
    "/order_id":   {"mode": "extract.span", "confidence": 0.88, "span": [32, 39]},
    "/due":        {"mode": "extract.date", "confidence": 0.4, "span": null,
                     "null_reason": "not_found"},
    "/line_items": {"mode": "array", "items": 2},
    "/reply":      {"mode": "generate", "generated_tokens": 11}
  },
  "usage": {"prompt_tokens": 42, "generated_tokens": 11},
  "plan": "<FillPlan.fingerprint>"
}
```

- `value` is always valid against the input schema. `assemble` checks this with
  `jsonschema` and deg's own format checks, and raises `FillError` (HTTP 500) otherwise.
- `meta` is keyed by concrete RFC 6901 pointer, with integer array indices. An array
  that dropped items whose required values were not in the context (§12.6) has
  `"dropped": n` beside `"items"`.
- `probabilities` follow the schema's option or level order; for score, the null outcome
  comes last.
- `span` gives `[start, end)` code-point offsets into the rendered context.
  `assumed_offset: true` marks a date-time whose offset came from
  `options.default_offset`.
- `confidence = (count * peak - 1) / (count - 1)` is the jev v1 formula. Ties go to the
  first maximum.
- `usage.generated_tokens` is the sum of the per-field `generated_tokens`. It is 0 for a
  schema with no `x-generate`.
- On a multimodal context an extract field whose pointer lands in a media item has
  `"span": null` and says where instead: `"media": "m0"` plus `"box": [x0, y0, x1, y1]`
  (pixels of the source image, `x1`/`y1` exclusive) or `"time": [s, e]` (seconds of the
  source audio), `"tokens": [i, j]` (the item's soft tokens the pointer chose) and
  `"read": "crop"` (the value was read from that region, §12.10). No schema keyword asks
  for it, as none asks for `span`: the value and the schema are unchanged, and the
  location is meta. A located value that cannot be read is null with
  `null_reason: "media_unread"`.

## 9. API

`POST /v1/fill`

```json
{"context": "string | object | array",
 "schema": {"type": "object", "...": "..."},
 "model": "deg-latest",
 "options": {"max_new_tokens": 256, "max_items": null, "date_order": null,
             "default_offset": "Z", "temperature": 0, "seed": null,
             "array_decode": "count_peaks", "null_margin": null}}
```

- `array_decode` (`"threshold"` | `"count_peaks"` | `"hybrid"`) picks how anchored
  arrays find their items (§12.6); also accepted in the decision APIs' `options`.
- `null_margin` (finite number, nats, or `null`) biases nullable/optional extract leaves
  toward null: such a leaf is null when its best valid span's log-prob is below the null
  log-prob + `null_margin` (§12.1). `null` = the server's `--null-margin` (default 0 =
  plain argmax). > 0 trades made-up answers for omissions (deg-2b held-out choice: 2.6,
  deg-0.8b: 1.4; `deg.bench.null_margin`, results/deg/null-margin-v1). It touches no
  decide field, no `forbid` (required non-nullable) leaf and no count/anchor decision; it
  applies to leaves inside index-addressed and anchored items the same way (a nullable
  item leaf becomes null, an optional one is omitted). Also accepted in the decision APIs'
  `options`.

- `date_order` is `"MDY"`, `"DMY"` or `null`. With `null`, a numeric date like
  `03/05/2026` parses only if exactly one reading is a valid date; otherwise the field is
  null or omitted.
- Unknown request fields and unknown options are 422 errors.

`GET /v1/models` lists the accepted model names; `GET /health` reports the batcher.

### 9.1 Decision APIs (`src/deg/api.py`, `server.py`)

`python -m deg.server --ckpt DIR --port 8790` serves three routes on one model. Each
request becomes **one** fill: every question is a property of one schema (key = question
name / id), so the input is read once, decisions/extractions are independent fields, and
`generate` questions run last, conditioned on the other answers.

| Question (OpenAI / Jev) | Field schema | Mode |
|---|---|---|
| `predicate` / `noul` | `boolean`; Jev `criteria.true/false` appended to the description as `Yes means:` / `No means:` lines (the jev training format) | decide.noul |
| `choice` | `oneOf` of `const` + `description` (or `enum` when no option has a description); OpenAI values are typed (`"true"` != `true`) | decide.choice |
| `score` | `integer`, `minimum 0`, `maximum L-1`, description = the instructions; level labels (and descriptions) as `x-levels: {"i": "label[: description]"}`, so each is the option line `i: label` the score head was trained on | decide.score |
| `extract` (deg) | `[string\|number\|integer, null]` + `format` from `value_type`, `pattern`, `minLength`, `maxLength` | extract.* (nullable: a miss is `null`) |
| `generate` (deg) | `string`, `x-generate: true`, `minLength`, `maxLength`, `pattern`; `max_tokens` -> that field's `x-max-tokens` (questions in one request may use different budgets) | generate |

A one-option choice or one-level score (accepted by live Jev) is answered with probability 1
without the model.

- `POST /v1/decisions` mirrors the OpenAI Decisions API (openai-python 3.26.1
  `types/decision.py`, `types/decision_create_params.py`; guide
  https://developers.openai.com/api/docs/guides/decisions): request `model`, `input`
  (string or user messages of `input_text` parts; `input_image` is refused with 400 because
  this model reads text only), `questions`, `safety_identifier`; response `{answers, model,
  usage}` with answers in question order: `predicate {type, name, probability}`, `choice
  {type, name, choice, probabilities: [{value, probability}], confidence}`, `score {type,
  name, score, probabilities: [{value, label, probability}], confidence}` (`score` = the
  expected level index), usage `{input_tokens, input_tokens_details{cache_write_tokens,
  cached_tokens}, output_tokens, output_tokens_details{reasoning_tokens}, total_tokens}`.
  `name` is optional (echoed, `null` when absent) and must be unique. Errors: 400
  `{"error": {message, type: "invalid_request_error", param, code}}`. deg never returns a
  `refusal` answer.
- `POST /v1/systemone` mirrors live TypeSafe Jev (`jev-1.13.0`, probed 2026-10-09;
  `tests/fixtures/jev_systemone_live.json`): answers keyed by id in request order, `noul
  {type, noul}`, `choice {type, choice, confidence, probabilities{option: p}}`, `score {type,
  score, confidence, legend{"i": level}, probabilities{"i": p}}`, usage `{input_tokens,
  output_tokens}`. Limits and errors as live Jev: 1..255 options (`{"detail": "Too many
  choices. Must have at most 255 choices."}`), 1..10 levels, unknown type / unknown
  top-level field -> 400 `{"detail": {"error_type": "api_usage_error", "message": "Invalid
  request."}}`, unknown model -> 400 `Unknown model: X`, missing field / empty questions ->
  422 `{"detail": [{type, loc, msg, input}]}`, missing key -> 403. Unknown question keys are
  ignored, as live Jev does. Unlike live Jev, probabilities are not rounded to 2 decimals.
- Extensions on both: `extract` -> `{type, value, span, confidence[, null_reason]}`,
  `generate` -> `{type, text, output_tokens}`; request `options` (`temperature`, `seed`,
  `date_order`, `default_offset`). Usage output tokens are the generated tokens (0 without
  `generate`).
- Model names: deg aliases, plus `gpt-6-luna` (decisions) and `jev-latest`,
  `jev-preview`, `clef-flash` (systemone) so existing clients work unchanged; responses
  carry deg's model id.
- Batching: one GPU worker; requests arriving within the batch window (default 5 ms) are
  taken together (default 16) and handed to `deg.infer.fill_many(model, jobs)` when the
  runtime defines it, else run back to back. A failing job fails only its own request;
  more than `--max-pending` queued jobs -> 429.

## 10. Determinism guarantees

- **Plan:** the same schema always gives the same plan and fingerprints. Compilation never
  mutates the input.
- **Parallel phase:** greedy and sampled requests behave identically here, since decide
  and extract involve no sampling. On the same build and hardware, outcomes are bitwise
  identical across:
  - repeated runs;
  - field reordering;
  - adding or removing other fields;
  - splitting or merging schemas.

  - batching with other requests (`fill_many`).

  The runtime batches per-field queries with batch-invariant kernels so that a field's
  numerics don't depend on the batch composition (§12.2).
- **Generate phase:** given the same parallel outcomes, a generate field's text is
  bitwise reproducible (greedy, or sampled with `seed`) and independent of the other
  generate fields in the schema and of other requests decoding with it.
- **Post-processing** (argmax, tie-breaks, parsing, assembly) is pure Python over
  the head outputs, with no clock, locale or randomness.

## 11. What is measured against the constrained-decoding baseline

The baseline is the same base model (Qwen3.5-0.8B, at the same quantization) emitting the
whole JSON object under grammar-constrained decoding (a JSON-Schema-derived grammar),
greedy, from the same prompt.

The same eval set of (context, schema, gold JSON) is measured on both:

- **Tokens:** generated tokens per response, prompt tokens, and their total.
- **Latency:** end-to-end on one pinned CPU core (the deploy target), plus GPU throughput.
- **Accuracy per mode:**
  - choice: exact match;
  - score: MAE;
  - noul: accuracy;
  - span: exact match and token F1;
  - number and date: exact match after parsing;
  - null: precision and recall.
- **Calibration:** the ECE of decide probabilities and of extract confidences.
- **Validity:** the rate of schema-valid responses. deg must hit 100%, by construction.
- **Determinism:** bitwise equality across 10 repeats, 5 field permutations, and
  split-vs-merged schemas.
- **jev gates:** the jev v1 eval gates on `/v1/systemone`.

## 12. Model (v1 implementation)

Code: `src/deg/model.py` (trunk, forks, heads), `infer.py` (`fill`), `train.py`,
`evaluate.py`. Tests: `tests/test_model.py`.

### 12.1 Encoding: one trunk, one fork per question

- **Trunk** = a fixed system/user prefix, the rendered context, `"\n"`. It contains the
  context only, never the schema, so it is shared by every field and every request on
  the same context.
- **Fork** = one continuation of a *copy* of the trunk state. Qwen3.5 is a hybrid: Gated
  DeltaNet layers carry a recurrent state plus a 3-token causal-conv tail, attention layers
  carry K/V. A fork starts from exactly that state, so it reads the whole context and
  nothing else. Fork kinds:
  - decide/extract leaf: `FIELD <concrete pointer>`, `ITEM i of <array>` per enclosing
    array (index mode; anchored items use `ITEM <array> AT: "..."`, §12.6), `TITLE`, `DESCRIPTION`, `KIND`, extract
    `CONSTRAINTS`, then `OPTIONS:` with one `- <option>` line per outcome, then `ANSWER:`.
    Outcome text follows the plan order (choice: value + option description; score:
    level, plus a null line for nullable scales; noul: no option lines).
  - array count: `ARRAY <pointer>`, item anchors of enclosing arrays, the array's
    title/description and item-field pointers, `ANSWER:`.
  - generate: `KNOWN VALUES:` (every decided/extracted value, one `pointer = json` line
    each, in leaf order; generate values are never included), `WRITE <pointer>`,
    annotations, `LIMITS`, `VALUE:`, then the text and `<|im_end|>`. At inference the
    KNOWN VALUES block is prefilled ONCE on the trunk and every generate field forks that
    state with only its own header (no per-field restatement of the values).
- **Why forks and not an attention mask:** DeltaNet layers are recurrent, so a mask cannot
  hide one field's tokens from the next; separate continuations can. The backbone is a
  functional re-implementation of the HF Qwen3.5 stack over explicit state
  (`Backbone.forward(ids, lengths, state)`); it matches the HF forward (fp32 relative error
  < 1e-3 on GPU, `assert_close` on CPU) and lets gradients flow from forks into the trunk.
- **Padding is exact**: padded positions get `beta = 0` and `g = 0` in the delta rule (no
  write, no decay), their K/V are masked, and the conv tail is gathered at each row's real
  length, so a right-padded row gives the same state as an unpadded one.
- **Long contexts stream; there is no context cap.** Inputs up to `DENSE_PREFILL_MAX`
  (8192) tokens are read in one shot (dense causal mask). Longer inputs prefill in
  `PREFILL_CHUNK` (4096) token chunks, carrying each DeltaNet layer's recurrent state and
  conv tail and appending each attention layer's K/V; on CUDA the chunk attends the K/V
  buffer through the flash-style fork kernel (no mask tensor at all), elsewhere through
  SDPA with a `[B, chunk, P+chunk]` mask. Nothing is O(L^2): peak memory is the K/V
  (6 attention layers x 2 KV heads x 256 x bf16 = 12 KiB/token) + hidden states
  (2 KiB/token) + one chunk's work (~0.46 GiB at 4096, independent of L). Span-pointer
  keys are projected in 8192-row blocks (bitwise equal to one `bi_linear`), and span
  candidates come from an `[T, MAX_SPAN_TOKENS]` band, never a T x T grid. Chunked
  outputs equal the one-shot read within bf16 noise; at most 8192 tokens they are bitwise
  the one-shot path. A context whose K/V does not fit the device fails with a CUDA OOM
  (reported, never truncated); serving those needs K/V offload to host memory.
- **Nothing scales with fields x context on the device.** Fork batches are sized from the
  memory free right now (`fork_token_budget`: `FORK_MEM_FRACTION` of free + cached-unused
  memory at `FORK_BYTES_PER_TOKEN` ~128 KiB per padded fork token, capped at
  `FORK_TOKENS`; OOM split stays as a backstop); after a fork pass only the rows the heads
  read (answer + option tokens) are kept. Span logits for a long context or a big schema
  are computed in answer-row micro-batches of at most `HEAD_LOGIT_BYTES` and context-row
  blocks of `HEAD_ROWS` (keys re-projected per micro-batch, no `[T, 256]` copy), then wait
  in host memory until decoding (bitwise the one-shot values). After field decoding the
  raw head outputs and the trunk hidden states are released, and the KNOWN VALUES block
  extends the context state in place (`forward(consume=True)`), so a context's K/V is
  never held twice. Peak device memory = weights + context K/V + hidden + one fork batch.

### 12.2 Independence (bitwise) and the one-pass runtime

Inference runs every query of a request (array counts, array items speculated up to 8
per array, every decide/extract field) as fork rows of ONE batch
(`DegModel.answer_many`; chunked only above 16k padded tokens, for memory), and every
x-generate field of a request as rows of one decode batch. `fill_many` puts the rows of
several requests in the same batches. A row's numerics never depend on the other rows
because every op in the fork path is batch-invariant (`src/deg/kernels.py`):

- **Prefixes by reference.** A fork row reads its prefix (context trunk, or trunk +
  KNOWN VALUES) in place: attention reads the prefix K/V at per-row offsets, the
  recurrent kernel starts from the prefix DeltaNet state by index. No per-row copy of
  the context exists; per-row state is the fixed-size recurrent state, conv tail and the
  row's own K/V.
- **GEMMs**: cuBLAS picks kernels (tiles, split-K) by shape, so a row's result changes
  with the number of rows (measured on the L4: rows differ for many M). `bi_linear` is a
  Triton GEMM with one fixed tile config and no split-K; `bi_linear_small` (decode steps
  and the LM head) uses 16-row tiles and a split-K factor fixed by the weight shape, with
  partials summed in a fixed order. Each call site always uses the same config, so a
  row's output depends only on that row. Projections sharing an input are fused into one
  GEMM (weights concatenated once; parameters become views).
- **Attention** (`fork_attention`): one program per (row, head, query block), prefix
  keys then own causal keys in a fixed block order; fully masked blocks are exact no-ops
  of the online softmax.
- **DeltaNet** (`fork_delta_rule`): FLA's fused recurrent gated delta rule, one program
  per (row, head, value block), tokens in order; padded positions have `beta = g = 0`.
- **Norms**: fixed-config Triton row kernels; elementwise ops run eagerly.
  `torch.compile` is not used in the fork path: it specializes size-1 dims, so one row
  alone and the same row in a batch would run different generated kernels.
- **Heads** run batched in fp32 through `bi_linear` (IEEE fp32 dot).

`tests/test_model.py` checks with `torch.equal` that every field's raw head outputs are
identical alone, reordered, inside bigger schemas and in split vs merged schemas; that a
generate field's text is identical alone and next to another generate field; and that
`fill_many` equals `fill` per request, on CPU (tiny random Qwen3.5; the CPU reference runs
each fork row alone at a 64-token width grid because torch's CPU SIMD/scalar-tail split
depends on tensor size) and on GPU (the real model). Caveat: the trunk prefill (batch 1)
still uses the FLA chunk kernel and torch.compile'd ops with autotuning, so equality is
guaranteed within one process; across processes it holds when the autotune cache
matches (same build, hardware and cache).

**Decode step.** Each generate decode step (all rows x 1 token, LM head, argmax) is
captured once as a CUDA graph over static buffers (`DecodeGraph`: prefixes up to a
power-of-2 token capacity, rows bucketed 1/2/4/..., own-token capacity) and replayed.
Greedy takes the argmax when it is admissible and only otherwise walks the top 256 in
logit order (the same rule as the constraint walk). Measured on one L4 (bf16): ~7.3-7.9 ms
per step at batch 1 vs 36 ms before; the floor is weight bandwidth (~1.5 GB/step incl. the
LM head).

### 12.3 Heads (fp32)

`a` = hidden state at `ANSWER:`, `o_i` = hidden state at the end of option line `i`,
`c_t` = trunk hidden state at context token `t`.

- `choice`: `q(a) · k(o_i) / sqrt(256) + u · o_i`, softmax over options (the jev v1
  pointer head). The option is encoded as text in the fork, so any enum works zero-shot.
  Training shuffles option order (target permuted) with p = 0.5.
- `score`: a separate pointer head of the same form over the level lines.
- `noul`: `w · a + b` → logits `[l, 0]`, i.e. `[p_true, p_false]`.
- `extract`: start and end pointers `qs(a) · ks(c_t)`, `qe(a) · ke(c_t)` over the context
  tokens only (prefix, suffix and query tokens are not addressable), plus a null logit from
  `a` as one extra outcome. Decoding:
  - numbers, dates, and email/uri/uuid strings: every candidate span the deterministic
    grammar finds in the context (the same candidate regexes datagen used to locate gold
    spans) is scored by `log p_start + log p_end` of its covering tokens; the best one
    that parses and meets the constraints wins, unless null scores higher.
  - plain strings: the top 32 token spans (`i <= j < i + 128`), mapped to code points
    and trimmed of whitespace and boundary punctuation a token carried in (`trim_span`;
    98.7% of gold spans are recovered exactly by trimming the covering tokens).
  - `options.null_margin` m (default 0) shifts that comparison: null wins when
    `log p(best valid span) < log p(null) + m`. Decoding stays deterministic and
    batch-invariant (the margin is per request, applied after the shared head pass).
  - `forbid` fields ignore null and take the best valid span; none at all is a
    `FillError` (inside an array item: the item is dropped, §12.6).
  - confidence = `exp(log p_start + log p_end)` of the chosen outcome.
- `count`: linear over 0..64 items, masked to `[minItems, min(maxItems or
  options.max_items or 32, 64)]`. Index-addressed arrays only (§12.6). Arrays resolve
  level by level (a nested array's count fork needs its parent's item anchor).
- `anchor`: per context token `t`, an independent logit `qa(a) · ka(c_t) / sqrt(256) +
  ua · c_t + b` that an item of the array starts at `t` (sigmoid; `b` starts at -6),
  masked to the array's region (§12.6).
- anchored `extract`: inside an anchored item, the start/end logits of context token
  `t` get a learned bias `B[start|end, inside, bucket(t - anchor)]`, where `inside` means
  `anchor <= t < region end` and the buckets edge at -256, -64, -16, -4, 0, 1, 2, 4, ...,
  256 tokens (zero-initialized).
- `generate`: the base LM head (tied embeddings, frozen in training), greedy by default,
  seeded multinomial (one stream per field) when `temperature > 0`. Constrained decoding: candidates are tried in
  logit (or sampled) order among the top 256; a token is admissible if the decoded text
  stays within `maxLength` and is a partial match of `(?s:.*?)(?:pattern)(?s:.*)`
  (`regex` partial matching, i.e. JSON Schema search semantics); EOS is admissible only
  when `minLength` and the pattern are met. `generated_tokens` counts EOS.
- Calibration: one temperature per head kind (`choice, score, noul, extract, count`), fit
  by NLL on datagen `val` only (`deg.evaluate calibrate --source datagen-val`, default;
  jev dev is reported separately with `--report-jev`, or pooled with `--source
  datagen-val+jev-dev`), stored in `heads.safetensors`; `count` is not fit. Decide heads
  fit the one-hot filled value (`--target hard`, default), the label accuracy and ECE
  are judged against; the soft training targets (`--target soft`) are less confident
  than the gold and over-soften (results/deg/bench-v2/calibration-v2.md).

### 12.4 Training (`deg.train`)

- Data: datagen records (`split == "train"`; val/test, including the held-out test
  domains, are never trained on) plus jev v1 `train.jsonl` decisions (noul/choice/score
  with teacher soft labels), mapped to one-field forks (`FIELD /answer`, the instructions
  and criteria as text, score levels as `i: criterion` options). A micro-batch draws
  records at random (jev with probability `--jev-frac`, default 0.3) up to a token budget.
  `--reload-every N` re-reads the growing datagen files.
- Sources: `--records PATH[:WEIGHT]` is repeatable (default: the v2 `full-off` file,
  weight 1). Each source reads its own `split == "train"` records (its own val/test labels
  are kept out) into its own pool, and every reload re-reads all sources. Each deg
  micro-batch picks one source with probability proportional to its weight among the
  **active** sources, those with at least `--source-min-train` (default 50, summed over
  ranks) train records, so a small, slowly growing source (e.g.
  `/data/deg/datagen/v3-long/records.jsonl:0.25` next to v2 at 1.0 → 20%) gets its fixed
  share as soon as it reaches the threshold. The jev mix stays per record inside every
  micro-batch. End-of-run calibration fits on the val records of every source (contexts
  within `--max-context-tokens`); the end-of-run eval uses the first source.
- Long contexts: a record whose context exceeds `--max-context-tokens` (default 6000)
  is kept whole and trains a **span-aware window** at every draw (inference is unchanged:
  it streams the whole context). The window is `max-context-tokens` minus the template
  tokens, placed uniformly among the positions that contain one gold span picked at
  random (spans and alternatives), so windows differ across epochs and cover the gold.
  A cropped window trains only what its text shows: extract fields with a gold span (or
  alternative) inside the window, and anchored arrays with at least one gold item start
  inside it (the items starting outside are left out with their subtrees). Absent-gold
  (null) extract fields, decide and generate fields (their evidence has no span) and
  count heads train only on a whole context; nothing is ever trained as null or empty
  because of cropping. Train logs report `windows`, `window_targets`, `window_dropped`.
- Decide cap per field (`--decide-forks-per-field K`, default 0 = off; `cap_decide_forks`,
  applied first in every draw): at most K decide forks (choice/score/noul) per schema field
  (array indices folded to `*`), a fresh random subset each draw. v3-long filings carry
  ~135 choice forks per record, 99.8% one repeated in-array enum per table cell (`unit`,
  `scale`, `statement_type`; 68% of their booleans are false); uncapped they are most of
  the decide forks of every step that draws one. run-2b, which trained them uncapped, lost
  decide accuracy on short inputs (datagen val top-level choice .939 -> .873 at step 4500;
  JevBench choice 94 -> 74-92 of 139 across steps 500-6000, noisy; results/deg/regress-2b).
- Fork cap per draw (`cap_forks`): every drawn example (whole context, window, filler)
  trains a fresh random subset of its forks when they exceed its room: `--record-fork-tokens`
  when set, else the token budget beside its trunk and at least half the budget however
  long the context. Whole-array forks (anchor, and anchored arrays' count) are always
  kept; the other forks are taken in random order while they fit, the first one whatever
  its length (so a generate fork longer than the room still trains). A record therefore
  costs one trunk pass plus about the room in fork tokens per step, whatever its field
  count: unsubsampled, a long filing (1,153 field rows, 176 generate forks each repeating
  its ~16k-token known-value block) was 2.9M fork tokens and 260 micro-batches, a 20+ min
  step. Loss terms stay means over the forks drawn (a subsample; forks longer than the
  room are drawn less often). Train logs report `subsampled_records`, `subsampled_forks`.
- Training trunks have no 8192 cap: a trunk longer than `DENSE_PREFILL_MAX` is read in
  `PREFILL_CHUNK` chunks with every layer of every chunk gradient-checkpointed
  (`Backbone._forward_chunked_ckpt`; the mask is built inside the checkpoint, earlier
  chunks' K/V are passed in as they are), so nothing O(L^2) is saved and one layer's
  recompute is O(chunk x L). It equals the dense checkpointed read in hidden states,
  state, fork outputs and gradients (`tests/test_train_long.py`). The memory planner
  models the chunked trunk separately (per-chunk recurrent state, the concatenated K/V
  copy, no saved mask, chunk-sized recompute).
- Row chunks share one trunk pass: a record too big for one micro-batch is split by the
  memory planner into field-row chunks (`MemPlanner.split_rows`); `run_step` runs them on
  ONE trunk forward (`run_chunks`): the trunk outputs (final hidden, recurrent state, conv
  tails, K/V) are detached into leaves, each chunk's forks + heads backpropagate into the
  leaves, and the trunk backpropagates once from their accumulated gradient (equal to the
  unsplit record's gradient, dense and chunked trunks alike: `tests/test_train_mem.py`).
  An out-of-memory error in such a run drops its gradients and reruns the chunks one by
  one with their own trunk passes (`--no-share-trunk` always does). Logs: `trunk_shared`.
- Losses, each a mean over its items:
  - `decide`: KL(t ‖ q). `t` is the record's `decide_probs` (Clef probabilities where
    Clef's argmax matched the filled value), else one-hot of the filled value; jev uses
    `probs`, else `hard`.
  - `extract`: start and end CE where every `span_alternatives` entry is a positive:
    with `span_probs` the target is that soft distribution; otherwise the loss is the
    marginal `-log Σ_pos p`. A field absent from the filled value targets null.
  - `count`: CE of the array length (index-addressed arrays).
  - `anchor`: per anchored array instance, BCE of every in-region token against the gold
    item starts, summed and divided by 1 + the number of items (§12.6).
  - `lm`: next-token CE on x-generate text + EOS, teacher-forced on gold values; the
    prompt is the inference layout (KNOWN VALUES = decided/extracted values only).
  - `anchor_count` (`--anchor-count-weight W`, default 0 = off): with W > 0 every
    anchored array of a whole context (never a cropped window: a count needs the whole
    array) also gets the index-mode count fork, CE of its gold item count over
    `[minItems, min(maxItems | MAX_COUNT, MAX_COUNT)]` with the target capped at the top
    class (which then means "at least that many"), so `count_peaks` decoding has a
    trained count.
  - total = `decide + extract + 0.5 count + anchor + W anchor_count + L lm`
    (`--anchor-weight`, `--anchor-count-weight`, `--lm-weight L`, default 1).
- Filler augmentation (`--filler-frac P`, `--filler-corpus`, default
  `/data/deg/external/v3/docs.jsonl`, `--filler-max-tokens N`, default 16384): the
  anchor head under-fires on long documents when it has only seen short ones. With
  probability P a drawn short datagen record is rebuilt with its document at a random
  offset inside unrelated text (`pre + "\n\n" + doc + "\n\n" + post`), total length
  log-uniform from its own length + 64 to `min(N, --max-context-tokens)`; every gold
  span shifts with it and every target is the plain example's (the filler is unrelated
  text). A `--filler-negative-frac` share (default 0.25) is filler only: the record's
  top-level anchored arrays with minItems 0 train their anchor fork with no item start
  (every token negative) and count 0; nothing else trains on it. Filler comes only from
  v3 **train-split** documents (`datagen.v3.split_of`), never the record's own document
  and never ExtractBench (refused by path). Draws are placeholders sorted at their planned
  length and materialized when the packing reaches them, like long-document windows, so
  the packing and the memory planner see real examples; a rebuilt example that would
  exceed `--max-context-tokens` falls back to the plain one. Logs: `filler`,
  `filler_negative`, `filler_tokens`.
- Optimization: bf16 weights with fp32 master copies (AdamW, lr 2e-5 trunk / 1e-3 heads,
  warmup then cosine to 10%), embeddings and the tied LM head frozen, per-layer gradient
  checkpointing, elementwise ops fused with `torch.compile`. Forks are length-grouped
  (≤25% padding). Multi-GPU: `torchrun`, one data shard per rank, explicit gradient
  all-reduce. Logs: losses, tokens/s (real and padded), GPU utilization and memory.

### 12.5 Smoke run and full-run launch

Smoke (2026-10-08, one L4, `/data/deg/runs/smoke-1`, 300 steps, budget 14k tokens,
≤32 records/step, jev 20k-record subset at 30%): total loss 13.0 → ~1.5 (decide 1.54 →
0.15, extract 8.2 → ~0.5, lm 1.83 → ~0.5, count 2.7 → ~0.8); ~4.0k real tokens/s
(4.7k padded), 85-92% GPU utilization, 14.9 GiB peak. Eval through the full `fill` path
after calibration:

| split | records | valid | decide acc / ECE | extract EM | count acc | generated / full-JSON tokens |
|---|---|---|---|---|---|---|
| val | 29 | 100% | 0.93 / 0.097 | 0.79 | 0.73 | 0.33 |
| test (held-out domains) | 60 | 100% | 0.90 / 0.071 | 0.85 | 0.70 | 0.29 |
| jev dev (1312 questions) | | | 0.81 / 0.024 | | | |

Full run (4 GPUs; not launched by default — the orchestrator allocates GPUs):

```bash
systemd-run --user --unit deg-train-full --property=MemoryMax=80G \
  --setenv=CUDA_VISIBLE_DEVICES=0,1,2,3 --setenv=PYTHONUNBUFFERED=1 \
  --setenv=PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True \
  --working-directory=/shared/rwkv/packages/deg \
  /bin/bash -c 'exec /shared/rwkv/.venv/bin/torchrun --standalone --nproc-per-node 4 \
    -m deg.train --out /data/deg/runs/full-1 --steps 20000 --budget 14000 \
    --max-records 32 --jev-frac 0.3 --reload-every 500 --save-every 1000 \
    --eval-records 200 --eval-jev 1000 > /data/deg/runs/full-1.log 2>&1'
```

### 12.6 Position-anchored array items (`array_mode = "anchor"`, run-2 default)

Run-1 addressed array item `i` by index ("ITEM i of /arr") and sized arrays with a count
head over 0..64. On real documents that failed: array-item leaves were 10% correct vs 45%
for scalars, forcing the gold counts did not help, and nested counts copied the parent's
index (results/deg/extractbench-v1/DIAGNOSIS.md, cause d). Anchor mode addresses an
item by **where it is**, not by its ordinal:

- **Which arrays.** An array whose item subtree has an extract leaf is anchored. Arrays of
  only decide/generate leaves (no span can say where an item is) keep the count head and
  index addressing. The mode is static per schema and checkpoint
  (`model.meta["array_mode"]`; a checkpoint without anchor heads, i.e. run-1, loads as
  `index` and behaves exactly as before; `deg.train --array-mode anchor|index`, default
  `anchor`).
- **Routing by document length** (`model.meta["anchor_min_tokens"]`, default 0 = off;
  `deg.train --anchor-min-tokens N`, saved in `deg.json`). An anchor-mode model fills a
  document of fewer than N context tokens exactly as index mode (count fork + `ITEM i`
  forks), and trains its whole-context records the same way; documents of N or more
  tokens, filler-augmented records that reach N, and every cropped window (its document is
  longer than any window) anchor. Why: on short datagen records run-2b step-4500 in index
  mode beats its own anchor mode (test array-item EM .844 vs .769 count_peaks, extract EM
  .896 vs .866; run-1 index .891 / .922), while anchors win on long documents
  (ExtractBench; results/deg/regress-2b).
- **Anchor fork.** One fork per array instance: `ARRAY <ptr>`, the enclosing items'
  locators, title/description, `ITEM FIELDS`, `KIND: mark where each item starts in the
  context`, `ANSWER:`. The anchor head scores every context token of the array's
  **region**; tokens with p > 0.5 are marked, each run of adjacent marked tokens is one
  item (its best token). Fewer than `minItems`: the best remaining non-adjacent in-region
  tokens are added (none left: `FillError`); more than `maxItems` (or `options.max_items`
  when given): the highest are kept. **No other cap**: the count is the number of
  anchors, and item forks run in the usual memory-bounded fork micro-batches.
- **Decode modes** (`options.array_decode`, default `count_peaks`). `threshold`: as above.
  `count_peaks`: the array also runs its count fork (the index-mode count query, trained
  on anchored arrays by `anchor_count`, §12.4); N = its argmax over `[minItems,
  min(maxItems | max_items | MAX_COUNT, MAX_COUNT)]` (at the saturated top class, the
  threshold count when larger), capped by maxItems; the items are the N best local
  maxima of the anchor logits with non-maximum suppression (`PEAK_RADIUS` = 4 tokens:
  one peak per item), then the best remaining tokens outside every radius if too few
  peaks exist. `hybrid`: threshold, unless no token passes p > 0.5 and the count says
  N > 0, then count_peaks. The count fork is one more independent fork and peak picking
  is a pure function of the logits, so every mode is deterministic and batch-invariant.
  Default chosen on datagen val only (`results/deg/array-decode-v1`, run-2b step-3000,
  131 records / 153 anchored arrays, paired): aligned item-leaf EM threshold .609,
  count_peaks .553, hybrid .659 (count exact .575 / .784 / .641); with 4k tokens of
  filler before the record: .428 / .482 / .572 (count .342 / .704 / .513).
  `PEAK_RADIUS` 2/4/8 changes count_peaks/hybrid leaf EM by at most .03.
  Default re-chosen at run-2b step-4500 (count head trained by `anchor_count` since step
  4000; `results/deg/regress-2b`): count_peaks vs hybrid vs threshold, datagen val count
  .846 / .577 / .558, extract EM .828 / .818 / .816; datagen test (200) count .850 / .740 /
  .690, extract EM .866 / .847 / .842; ExtractBench normalized 239 vs 221 (hybrid),
  aligned 339 vs 338, count_ok 132 vs 111, valid docs 28 vs 27.
- **Regions.** A top-level array's region is the whole context. Item `k`'s region runs
  from its anchor to the next sibling anchor (the last item's to its parent's region
  end). A nested array searches anchors only inside its parent item's region, so a child
  count cannot copy the parent's count or index.
- **Item forks.** Every fork inside an anchored item renders anchored indices as `*`
  (`FIELD /results/*/time`) and one locator line per enclosing anchored item:
  `ITEM /results AT: "<context text from the anchor, at most 24 tokens, cut at the region
  end>"`. Extract forks also get the anchored span bias (§12.3) from the innermost
  anchor and region. There is no ordinal anywhere in an anchored item's forks.
- **Items that do not exist.** An anchor only says where an item would start; the item
  exists only if every required non-nullable (`on_null: forbid`) extract leaf in it has a
  valid value. When one has none (`forbid` already ignores null and searches every valid
  candidate in the context, so this means no span parses/meets the constraints at all —
  e.g. a `format: email` string item anchored in a résumé with no address), the
  innermost anchored item holding that leaf is **dropped**: never a `FillError`, never a
  made-up value. Survivors keep their anchors, regions and fork outputs (no fork sees an
  anchored item's index, so nothing is re-run and their values are bitwise unchanged)
  and are renumbered in document order; the array's meta gets `"dropped": n`; generate
  fields of a dropped item are not decoded and its values are not in KNOWN VALUES. If the
  drops leave an array below `minItems`, its enclosing anchored item cannot be
  represented and is dropped in turn; a top-level array left below `minItems` is a
  `FillError` (`minItems N: only k items have their required values`), like an anchor
  fork with too few positions. There is no "replacement anchor" fallback: for typed
  leaves (numbers, dates, email/uri/uuid) the valid candidate set is the whole context
  and does not depend on the anchor, so another anchor cannot succeed where this one
  failed. Dropping is a pure function of each item's own fork outputs, so it is
  deterministic and batch-invariant. It does not depend on `array_decode` (which only
  picks the anchors). **Index-addressed items** (index-mode models, documents routed to
  index mode below `anchor_min_tokens`, arrays without extract leaves) follow the same
  rule: the innermost enclosing item — anchored or index-addressed — is dropped and the
  survivors renumbered with their fork outputs unchanged (their `ITEM i of` forks are
  not re-run); generate forks of survivors render the new index. A required
  non-nullable value outside every array item keeps the §4 `FillError`.
- **Rounds.** Anchored arrays add one fork round per nesting level (anchors first, then
  the items' forks and nested anchor forks); index arrays keep the speculative one-pass
  layout.
- **Order.** Items come out in document order (anchor order). `deg.evaluate` reports
  positional `extract_em` and order-insensitive `extract_em_aligned`;
  `deg.extractbench.score_record(..., align=True)` and the ExtractBench eval report
  `normalized_aligned_rate` next to the positional rates.
- **Independence.** An anchor fork depends only on the context and its own query (array
  pointer, enclosing locators, annotations); an item's forks depend only on the context,
  their own query, and the item's anchor and region. Both are bitwise identical alone,
  inside a bigger schema and reordered (`tests/test_anchors.py`).
- **Training gold.** From the datagen spans (`spans`, else the first `span_alternatives`
  entry): an item's anchor is the first token of its earliest span that no sibling item
  shares (a shared span such as a table column header does not locate an item), else of
  its earliest span. An anchored array instance whose items do not all get distinct
  anchors inside the region is left out of training (counted as
  `anchor_skipped_arrays` in the data log); an empty gold array trains an anchor fork with
  no positives (the hard negative for nested arrays).

### 12.7 Who writes x-generate text: hybrid and LoRA (deg-suite v1 follow-up)

deg-suite v1 found the untrained base Qwen3.5-0.8B writes better x-generate text than
every fine-tuned checkpoint (judge faithful 0.94 vs 0.27 on DialogSum, 1.59 vs 0.43 on
mixed, out of 2; lengths similar): full fine-tuning made generated text unfaithful.
Three ways to keep the base model's generation, all deterministic and batch-invariant:

* **Two-copy hybrid** (`load_model(..., generate="base")`, `--generate base` on the server
  and the suite): a frozen base body sits beside the fine-tuned one (token embeddings =
  tied LM head shared when equal, +~1 GB bf16). Decide/extract are unchanged; for
  x-generate the base body re-reads trunk + KNOWN VALUES in one pass (one more context
  read) and decodes the header of :func:`deg.infer.base_generate_header`, which closes
  the user turn with a plain instruction and opens the assistant turn (thinking off).
* **LoRA** (`deg.train --lora-rank R`): the backbone stays the base; named low-rank
  adapters on every attention / linear-attention / MLP projection (`lora_A` [n*r, in],
  `lora_B` [out, n*r], B = 0 at start) train with the heads. A per-row selector picks
  each fork row's adapter (`--adapters shared` or `decide,extract`: by fork kind; choice,
  score, noul -> decide; extract, count, anchor -> extract), so rows with different
  adapters share one batch and stay bitwise independent (the delta goes through the same
  bi_linear GEMMs). x-generate rows run adapter-off = the base model, never trained.
  `--lora-context base` (default): the context is read ONCE by the base weights and
  every fork (adapted decide/extract rows, adapter-off generate rows) continues it, so the
  one-pass structure holds and the trunk has no gradient in training.
  `--lora-context <adapter>`: that adapter also reads the context; generation re-reads it
  adapter-off (2x prefill for requests with x-generate fields).
  Checkpoints keep the base `backbone.safetensors` and add `adapters.safetensors` +
  `deg.json["lora"]`.
* **GENERATE LoRA on the hybrid's base copy** (`--generate base+lora:<dir>`;
  `load_model(..., generate="base+lora:<dir>")`): the two-copy hybrid, with one low-rank
  adapter trained by `deg.gentrain` merged into the frozen base copy at load
  (`W += alpha/r * B @ A`, fp32 on the CPU, cast to the body dtype; deterministic).
  Deg's own backbone, heads and the one-pass flow are unchanged; generate rows stay
  batch-invariant (the merged copy is an ordinary frozen body); a zero adapter is the
  plain hybrid. A generator dir holds `generator.json` (`format`
  `deg-generator-lora-v1`, rank, alpha, training meta) + `lora.safetensors`
  (`<module>.lora_A` [r, in], `<module>.lora_B` [out, r], named as in the HF
  `Qwen3_5ForCausalLM`). Training (`python -m deg.gentrain`): SFT on x-generate fields
  in exactly the hybrid's layout (context + KNOWN VALUES + `base_generate_header`, target
  = value + EOS; `deg_example(..., base_layout=True)`), the adapter on every
  `LORA_TARGETS` projection reading the whole sequence; data = gen-v1 `train` (suite
  guard; open-weight writer + judge only, `check_open_teacher`; fields the judge kept at
  faithful 2 / responsive 2; targets over the field's decode budget dropped, never cut);
  `val` token NLL selects the checkpoint (early stopping); `--kl-base W` adds
  W * KL(base || adapted) on the target tokens.
* **Anti-forgetting for full fine-tunes** (`--lm-kl-base W`): KL(frozen base || deg) on
  the x-generate target tokens, computed by the frozen base on the identical trunk + fork
  tokens (no grad).

### 12.8 Pluggable backbones (`--backbone`, `deg.json["backbone"]`)

The trunk/fork runtime, the heads, training and the memory planner do not depend on the
architecture beyond one `Family` of layer functions (`model.py`): every supported base is
a hybrid of **state layers** and **attention layers**. `State.lin[i]` = (recurrent state,
conv tail) per state layer, `State.kv[i]` = K/V per attention layer; `ForkState` reads the
prefix K/V by reference and copies only the fixed-size per-row state, so prefill once,
fork-by-reference, chunked prefill (inference and checkpointed training), the CUDA-graph
decode step and the two-copy hybrid are shared code. A family supplies: the trunk/training
state layer (`rec_layer`), attention projections and output (`attn_qkv`, `attn_out`), the
batch-invariant fork versions of both, per-row state shapes, the final norm, the rotary
call and the LoRA targets.

| `--backbone` | layers | per state layer | per attention layer | tokens |
|---|---|---|---|---|
| `qwen3.5-0.8b` (default) | 18 Gated DeltaNet + 6 attention, hidden 1024 | recurrent [16,128,128] fp32 + 3-token conv tail | K/V 2 heads x 256 | ChatML, `<think></think>` gen turn |
| `qwen3.5-2b` (`--base-model Qwen/Qwen3.5-2B`) | same 18 + 6 layout, hidden 2048, MLP 6144 | as the 0.8B | as the 0.8B | as the 0.8B |
| `lfm2.5-1.2b` (LiquidAI LFM2.5-1.2B-Instruct) | 10 gated short-conv + 6 GQA attention (32 q / 8 kv heads x 64), hidden 2048, MLP 8192, tied embeddings, vocab 65,536 | none (empty `[B,0]`) + 2-token tail of `B*x` | K/V 8 heads x 64 | `<\|startoftext\|>` BOS + ChatML, pad `<\|pad\|>`, gen turn without a thinking block |

LFM2 conv layer: `B, C, x = in_proj(norm(h))`, `y = C * conv3(B * x)` (depthwise, no
activation), `h += out_proj(y)`; attention has per-head q/k RMSNorm and plain 1-D RoPE;
norms are `w * norm(x)`. Both families have K/V = 12 KiB/token (6 layers x 2 KiB).

- **Determinism (LFM).** The fork path uses the same kernels as Qwen3.5: `bi_linear` /
  `bi_linear_small` for every projection (fused q/k/v and w1/w3), `fork_attention` (head
  dim 64, GQA group 4), a fixed-config Triton row kernel for the LFM norm
  (`kernels.rms_rows_w`), and an eager elementwise short conv accumulated in fp32 in a
  fixed tap order (per element, so batch-invariant; no conv kernel is needed). No fallback
  path was required. `tests/test_lfm_backbone.py` checks (tiny random LFM2 + the real
  tokenizer, CPU) the functional stack against HF `Lfm2ForCausalLM`, every fork row
  against an independent full forward of context + its query and bitwise alone vs
  batched, keep/decode continuation, chunked prefill = one-shot (inference, padded rows,
  prefix state, checkpointed training incl. gradients), and runs the `test_model` /
  `test_hybrid_generate` independence and hybrid-generate contracts against the LFM
  fixtures; on CUDA (real model) bitwise alone / reordered / bigger schema / fill_many,
  generate rows next to other generate rows, HF match, chunked prefill match.
- **Checkpoints** record `backbone`; a checkpoint loads its own base (pre-§12.8
  checkpoints are Qwen3.5-0.8B) and refuses another `--backbone`; `--generate base`
  attaches that backbone's frozen base. Token budgets (`--max-context-tokens`,
  `--anchor-min-tokens`, `--budget`) count the backbone's own tokens.
- **Memory.** `MemCoef.for_backbone` gives analytic coefficients per backbone (calibrated
  online by the measured scale as before). LFM2.5-1.2B full fine-tuning keeps ~15.7 GiB
  resident (bf16 weights 2.2 GiB + fp32 master/AdamW 12 B and bf16 grads 2 B per trainable
  parameter, 1.04B trainable with the embeddings frozen) vs ~6.8 GiB for Qwen3.5-0.8B, so
  it needs a higher `--mem-target-gb` on a 24 GB L4.
- **Qwen3.5 is size-generic.** `--backbone` / `--base-model` accept `qwen3.5-<size>` or
  the HF id (`Qwen/Qwen3.5-2B`); every Qwen3.5 size is the one `QWEN35` family with the
  same tokenizer, chat layout and gen turn, and the backbone and heads read layer types,
  hidden/MLP widths and head counts from the snapshot's config. `qwen3.5-2b` has its own
  memory coefficients (2x saved hidden per token, ~1.9x wider layer recompute; same
  recurrent state, K/V and vocabulary). `tests/test_qwen2b_backbone.py` runs the model,
  hybrid-generate and long-read contracts on a tiny 2B-shaped Qwen3.5 (hidden 2x, MLP
  3x hidden), checks the real 2B config sizes backbone and heads (meta device), that a
  2B checkpoint refuses other backbones and `--generate base` loads the frozen 2B, and on
  CUDA the real 2B's bitwise independence and HF match.

### 12.9 Data-parallel training: ZeRO-2 over several GPUs (`torchrun`, `deg.zero`)

`torchrun --nproc-per-node N -m deg.train ...` trains one model on N GPUs. Each rank
reads its own uid shard of every source (`zlib.crc32(uid) % N`), draws its own
micro-batches with the per-rank `--budget` and memory plan, and the step's gradient is the
gradient of the **global** step loss: each loss term's item counts are all-reduced before
the forward (`global_counts`), each rank's term is its share of the global mean, and the
ranks' gradients are **summed** (not averaged; a rank with more decide items carries more
of the decide gradient, exactly as one batch of all ranks' records). One global batch is N
per-rank micro-batch sets (N x `--budget` tokens). Gradient clipping uses the global norm.
A rank that cannot finish its step (minimal unit OOM) makes all ranks redraw
(`all_ranks_ok`), so every optimizer step is complete on every rank.

**ZeRO-2** (`deg.zero.ZeroMaster`, default with N > 1; `--no-zero` = whole masters per
rank). bf16 compute weights stay whole on every rank (the functional backbone calls its
layers' weights directly). The fp32 master weights, both AdamW moments and the fp32
gradient are sharded: the trainable parameters form one flat vector cut into 32M-element
buckets; rank r owns the r-th of N pieces of each bucket. A step's bf16 gradients stay on
the parameters until the step's last part (several-part steps sum them in a pinned host
fp32 vector, so the device never holds two gradient copies), then a bucketed fp32
reduce-scatter (SUM) gives each rank its shard's gradient; AdamW steps the shard (one flat
parameter: element-wise the same arithmetic and decay as the unsharded trainer); each
rank rounds its shard to bf16 and an all-gather refreshes every rank's compute weights
(bitwise `p.copy_(master)`). Per trainable parameter a rank keeps 2 B (bf16 grad) +
16/N B instead of 18 B: Qwen3.5-2B (1.37B trainable, embeddings frozen) on 3 L4s holds
~3.8 GiB weights + 2.6 GiB grads + 6.8 GiB shard state = ~13 GiB static vs ~26 GiB
unsharded. The memory planner uses this static (`sharded_static_bytes`), then the
measured resident memory as before.

- **Start/resume.** Heads (fresh, per-rank seeds) and backbone are broadcast from rank 0
  at start. Every rank keeps the whole bf16 weights, so rank 0 saves an ordinary deg
  checkpoint (`step-N/`, same format as one GPU; no sharded files, no conversion).
  `--init <run>/step-N --start-step N` resumes: weights from the checkpoint, the cosine
  schedule continues at step N, optimizer moments start fresh with a `--warmup` re-warm
  (as on one GPU), the data order is re-drawn (the rng mixes in N). The process group
  timeout is 4 h (rank 0 alone calibrates, evaluates and saves at the end).
- **Smoke on one GPU.** `--zero-sim-world N` shards as rank 0 of N ranks with no
  collectives: the real per-rank memory and compute time on one GPU (other shards are not
  updated; never a real run).
- `tests/test_zero.py` (CPU, gloo, 3 processes): sharded masters + AdamW + global clip
  equal one process stepping whole fp32 masters on the summed gradient, for one-part
  (device bf16 grads) and two-part (host fp32 sum) steps, replicas bitwise equal; the
  flat layout covers every element once; global counts make two ranks' gradients and
  losses sum to the one-batch ones.


### 12.10 Grounded extraction on media (`deg.ground`)

A text extract field copies a span and `meta.span` says where it came from. On a media
context the same extract field may point into an image or audio item; then meta gives a
box or a time window (§8). There is no annotation: the mode follows from the field's type
(§3), and text vs media follows from where the pointer lands.

- **Pointer.** The extract fork and span head are unchanged; they already score every
  context token, the media **soft tokens** included. `landing`: when the start pointer's
  probability mass on the soft tokens beats both the text tokens' and null's, the field
  is located in media (`decode_ground`: the best pair inside one item, logits
  renormalized over soft tokens + null); otherwise the text path runs exactly as before
  with the soft tokens masked. No parameters are added and a context without media never
  takes this path, so text checkpoints load unchanged and text fills are bitwise
  identical.
- **Image geometry.** Gemma 4 resizes an image (aspect-preserving) to multiples of 48 px
  with at most `max_soft_tokens * 9` 16-px patches and average-pools 3x3 patches: soft
  token `k` is cell `(k mod gw, k div gw)` of a `gw x gh` grid in raster order
  (`ground.image_grid`, tested against HF's resize and the tower's position ids).
  `start` = the box's top-left cell, `end` = its bottom-right cell: `start <= end` in
  raster order, the span head's own order. Valid pairs have `col(end) >= col(start)` and
  `row(end) >= row(start)`; cells map back to source pixels (`cells_to_box`).
- **Audio geometry.** One soft token per 40 ms (`audio_ms_per_token`): `start`/`end` are
  the window's first and last token, `[start * 0.04, (end + 1) * 0.04]` s, clipped.
- **Value from the region: crop re-read.** A media item has no text to copy, so the
  located region (padded 15 % per side, at least 48 px; audio: the window +-0.5 s) is cut
  from the source and filled as the only media item of a one-field x-generate schema
  (`ground.read_schema`: the leaf's key, title/description, maxLength); the text is then
  parsed by the leaf's mode like a copied span (`response.extract`: number/date parsing,
  constraints). All reads of a `fill_many` batch run as one nested batch, before the
  KNOWN VALUES block needs them. Chosen over a region-masked attention bias because (1) at
  a receipt's 280-token budget one soft token covers ~48x48 resized px, often more than a
  printed word; a crop at the same budget reads the region at several times the
  resolution, which masking cannot add; (2) it reuses the trunk's media path unchanged: no
  change to the fork attention kernels, the CUDA decode graphs or the text path; (3) its
  training data are ordinary media records. Media given as embeddings only (`Media` parts,
  no source file) can be located but not read (`null_reason: "media_unread"`).
- **Training.** A media record's `grounds` (concrete pointer -> `{"media", "box"}` /
  `{"media", "time"}`, or null = not in the media) gives an extract leaf's pointer target:
  the gold corner soft tokens (`ground_target`; null -> the null slot), in the normal
  `extract` term (`extract_media` logs its share). A present value with neither a text
  span nor a ground entry trains no pointer (never null). Crop-read records
  (`records.read.jsonl`, family `mm-read-v1`) are the gold region cropped exactly as
  inference crops (same `crop_sha` key, so the media cache serves both) with the leaf's
  read schema and gold value.
- **Gold.** `python -m deg.datagen.mm_ground --set cord-v2`: CORD-v2 word boxes
  (`ocr_boxes`, with their line category) matched to the gold values (items
  name/quantity/price, totals); a matched field turns from x-generate into a plain string
  (an extract field) in `records.ground.jsonl` and the `.ground` eval files. A value with
  no exact same-category word run gets no entry. invoices-donut (no boxes),
  RICO-Screen2Words (no view hierarchy, no localized field), MINDS-14 and AMI (no word
  timings in these sources) stay value-only.
- **Eval.** `python -m deg.bench.mm_eval run --ground` fills `<set>.<split>.ground.jsonl`
  and reports, beside (never in) the set score: box IoU, IoU >= .5 rate, containment
  (share of the gold box inside the predicted one), gold-centre hit, time IoU and
  containment, and null accuracy for gold-null entries. A field not located scores 0.
- `tests/test_ground.py` (CPU, tiny Gemma 4 hub): geometry against HF and the towers,
  corner decoding, landing and null, CORD matching and crop-read records, a training step
  with image and audio pointer targets, fills that locate and read values (and the text
  path when the pointer lands in text), and the metrics.