DEG-5B-E2B-1019 / source /src /deg /bench /__init__.py
gcoderw's picture
DEG-5B-E2B-1019 v0.2 (multimodal)
f8dbde1 verified
Raw History Blame Contribute Delete
496 Bytes
"""Head-to-head bench: deg vs structured-output baselines on the same records and scoring.
* :mod:`deg.bench.sets` - pluggable eval sets with fixed uid manifests;
* :mod:`deg.bench.systems` - deg and the baselines (same-model constrained decoding on HF
+ xgrammar and on llama.cpp + GBNF, NuExtract3, Clef-flash decide, frontier references);
* :mod:`deg.bench.score` - one scoring rule for all systems;
* :mod:`deg.bench.run` - run one system over sets; :mod:`deg.bench.summary` - tables.
"""