DEG-5B-E2B-1019 (v0.2)

deg fills a JSON Schema from a context in one model. The context can be text, images, audio, or a mix. deg uses three task-routed expert sets:

• Decide (D): classification heads choose enum, boolean and score fields, and report their probabilities.
• Extract (E): a span pointer copies string fields verbatim from the context.
• Generate (G): new tokens are written only for fields marked x-generate: true, or for values that have to be read from an image or audio clip.

The output is always valid against the schema. Only generated fields cost decode tokens: about 15 per text record.

v0.2 adds two things over v0.1:

• Multimodal contexts: receipts, invoices, UI screenshots and speech. They go through Gemma 4's own frozen vision and audio encoders.
• A better decide path: a D-expert fine-tune on new holistic-judgement data (rubric scores and document section/period on real documents), with D-path soft targets relabelled by Qwen3.8-27B and pplx-decider-v1.1.

Results

deg-suite v1 (text)

deg-suite v1 is a frozen, held-out suite.

• Headline = mean of the DECIDE, EXTRACT and GENERATE scores. Each of those is the mean of its sources.
• Validity gate: an invalid record scores 0 on every field.
• GENERATE judge: Qwen3.8-Flash-Next.

system	active params	headline	DECIDE	EXTRACT	GENERATE	valid
DEG-5B-E2B-1019 (v0.2)	2.3B	0.767	0.785	0.644	0.873	0.999
DEG-5B-E2B-1018 (v0.1)	2.3B	0.764	0.774	0.644	0.874	0.999
gemma-4-E4B-it (Q8_0)	~4B	0.752	0.781	0.574	0.902	0.989
Qwen3.5-4B	4.7B	0.749	0.791	0.576	0.881	0.981
Qwen3.5-9B (Q8_0)	9.7B	0.748	0.814	0.553	0.876	0.981
Qwen3.8-27B (INT4), upper bound	27B	0.817	0.879	0.644	0.929	0.982

DECIDE per source, v0.2 vs v0.1: datagen-test 0.924 (vs 0.929), JevBench 0.648 (0.640), Banking77 0.740 (0.714), mixed 0.829 (0.815).

Speed: schema routing vs writing the whole JSON

Same A100-80GB, batch 1, the same 260 deg-suite requests. The baseline is v0.2's own base model, Gemma-4-E2B-it, writing the full JSON object under a JSON-schema grammar (HF transformers + xgrammar, greedy). Both produced 100% valid output.

	DEG v0.2	Gemma-4-E2B-it, full JSON	speedup
median latency	0.40 s	3.74 s	9.3×
p95 latency	1.15 s	9.70 s	8.4×
throughput	2.13 req/s	0.25 req/s	8.5×
tokens decoded per request	33	111	3.4× fewer
decide/extract-only requests (median)	0.11 s	1.25 s	11.6×
requests with generated fields (median)	0.66 s	5.30 s	8.0×

Per source (median): datagen-test 0.61 vs 5.56 s; mixed (real documents) 0.73 vs 5.46 s; SQuAD2 0.11 vs 1.53 s; DialogSum 0.41 vs 1.65 s; JevBench 0.09 vs 0.24 s.

The speedup comes from three places. Decide and extract fields decode nothing: they are answered by heads and a span pointer in one query pass. Generated fields decode only the value text. The JSON structure itself (keys, braces, enums, numbers) is computed, not sampled.

mm-eval-v1 (images and audio)

mm-eval-v1 is held out: each source dataset's own val/test split is never used in training. The score is per set, as the mean of decide accuracy, count accuracy and generate ROUGE-L. Numbers are on the test split.

set	kind	DEG v0.2	Gemma-4-E2B towers, deg zero-shot	key fields (v0.2)
CORD-v2 receipts	image	0.809	0.767	decide 0.975, items count 0.74, totals ROUGE-L ~0.85
invoices (Donut)	image	0.932	0.932	decide 1.0, line-item count 1.0
RICO screen2words	image	0.321	0.287	app category 0.51, summary ROUGE-L 0.13
MINDS-14	audio	0.922	0.903	intent 0.988, transcript WER 0.27
AMI meetings	audio	0.597	0.421	speaker count 0.42 (zero-shot 0.07), transcript WER 0.32
mean		0.716	0.662

Decision accuracy vs LiquidAI d1 (decide fields only)

d1 models answer the same field questions deg receives. The metric is field accuracy on the same records.

source	DEG v0.2	d1-3B	d1-omni-600M
datagen-test	0.924	0.892	0.733
JevBench (public 231)	0.740	0.749	0.580
Banking77	0.740	0.882	0.784
mixed (real docs)	0.829	0.801	0.596
CORD-v2 (image)	0.980	0.835	0.520
invoices (image)	0.980	0.940	0.300
RICO category (image)	0.500	0.442	0.049
MINDS-14 intent (audio)	0.989	–	0.155
AMI speaker count (audio)	0.548	–	0.057

d1 also answers zero-token decisions only. deg additionally extracts spans and generates text fields in the same pass.

Model

Base	google/gemma-4-E2B-it @ 3e22461, including its vision and audio encoders, which stay frozen
Architecture	Gemma-4-E2B trunk with a D/E/G task MoE in every layer's FFN. Attention, embeddings and PLE are shared.
Routing	By task. Decide → D, context + extract → E, generate → G.
Size	5.39B on GPU, plus the 2.35B frozen PLE table and the frozen towers, which come from the base. 2.28B active per token.
Media	An image is about 260 soft tokens; audio is about 25 tokens/s, up to 30 s. They are placed in the context with Gemma 4's own placeholders.
Weights	Only the D experts and the decide heads changed from v0.1. The trunk, E and G are identical: backbone.safetensors is byte-identical.

Usage

The runtime is in source/ (pip install -e "./source[model]"). The PLE table, towers and tokenizer are read from the base model in your HF cache (DEG_HF_HUB).

from deg.model import load_model
from deg.infer import fill
model = load_model("path/to/DEG-5B-E2B-1019", generate="moe")
schema = {"type": "object", "properties": {
    "payment_method": {"enum": ["cash", "card", "e-wallet", "other"]},
    "total_price": {"type": "string", "x-generate": True},
}, "required": ["payment_method", "total_price"]}
receipt = {"context": "<media:m0>", "media": [{"kind": "image", "path": "receipt.png"}]}
print(fill(model, receipt, schema))  # audio: {"kind": "audio", "path": "clip.wav"} (16 kHz mono)

Text contexts work exactly as in v0.1. See source/SPEC.md for the request contract, including media parts. source/ also contains a grounding prototype that returns an image box or audio time window in response meta for extract fields. It is not enabled by default in this checkpoint.

Training (v0.2 delta)

• Init: v0.1 lineage (G reset to the base FFN), then:
  • a D-only fine-tune with a fresh LR schedule (500 steps, lr 2e-5, decide heads at 1e-4; trunk/E/G frozen);
  • the multimodal round: the same recipe with media contexts upweighted 4×.
• New data:
  • Holistic decide fields on real train-split passages: Qwen3.8-27B wrote rubric 1–5/1–10 score fields and section/period/type choices, then labelled them blind from first-token option log-probs over both option orders. Scores are rebalanced by level. Booleans are excluded, because generated yes/no questions are biased towards "yes".
  • Relabelled soft targets: pplx-decider-v1.1-27b for choice/score fields on existing datagen sets, and Qwen3.8-27B option log-probs for score and section/period fields. JevBench dataset rows are 50/50 pplx/gold.
  • Multimodal sets: CORD-v2, Donut invoices, RICO-screen2words, MINDS-14, AMI (dataset gold labels).
• Suite guards: training refuses every suite and JevBench path, and drops records whose context appears in the suite.

Limitations

• Long-document extraction is still weak (ExtractBench 0.110).
• Banking77-style 77-way intents trail d1-3B and clef-flash.
• Generated values from images and audio are E2B zero-shot, because G was not trained.
  • Invoice amounts are re-formatted (22,49 → 22.49), so exact-match on totals and VAT is low.
  • RICO summaries are longer than the terse gold.
  • Training G on multimodal data fixed formatting but broke transcription in our runs, so v0.2 keeps G frozen.
• AMI speaker counting remains hard (0.42).
• Grounding (box / time window) is a prototype: center-hit 0.82 on CORD, IoU 0.18.

Acknowledgments

Thanks to Shadeform for the compute: this model was trained and evaluated on 8×A100-80GB instances rented through Shadeform.

License

Apache-2.0, following the base model (see the Gemma 4 license). Teacher labels come from Apache-2.0 open models (Qwen3.8-27B, pplx-decider-v1.1-27b). LiquidAI d1 models were used for evaluation only.