jex / README.md
J-I-B's picture
feat: Jex 1.0 model card (no weights) + config + assets
4d56cc1 verified
|
Raw History Blame Contribute Delete
5.72 kB
---
license: apache-2.0
base_model: google/gemma-4-31B-it
tags:
- decision
- text-classification
- decision-index
- frozen
- jex
---
<p align="center">
<img src="assets/hero.svg" alt="Jex 1.0 by Bixoto β€” a frozen decision engine. Public index 60.29 on Decision Index 0.3." width="100%">
</p>
# Jex 1.0
**This repository contains no weights.**
Jex 1.0 serves the frozen `google/gemma-4-31B-it` checkpoint at revision
`842da3794eaa0b77d5f08bae87a17459d91ff475` (bf16) through our own deterministic,
batch-invariant inference engine. The weights are not copied here: fetch them from
[`google/gemma-4-31B-it`](https://huggingface.co/google/gemma-4-31B-it) at that revision.
We did **not** train, fine-tune, quantize, or add a head or LoRA β€” the checkpoint is
untouched, and the whole difference is the engine around it.
Jex 1.0 is a **decision engine, not a language model**. Given a state and a set of typed
questions, it returns one answer per question with a probability for every allowed
option β€” in a single forward pass, no free-form text, no output parsing. It cannot answer
with something that was not on the list.
## Headline
| field | value |
|---|---:|
| **Decision Index 0.3 public index** | **60.29** |
| balanced skill / raw | 60.29 / 69.88 |
| breadth skill | 59.25 |
| requests | **140,178 / 140,178 ok** |
| unsupported Β· error | 0 Β· 0 |
| `complete` | `true` |
| base model | `google/gemma-4-31B-it` @ `842da3794eaa0b77d5f08bae87a17459d91ff475` (bf16, frozen) |
The board's next-best frozen entry, **deck31b**, runs the **same checkpoint** and lands at
58.19 public. Jex 1.0 lands at 60.29 β€” a **+2.10** difference that is entirely engine work:
the readout, the batching, the decision rules, and four disclosed deterministic
post-processing levers.
> `60.29` is a **public index** β€” the chance-corrected score over the 37-benchmark public
> panel (5 areas). It is **not** the board's Full score, which adds private tests only the
> maintainers can run. No Full-score or rank is claimed here.
## Quick links
- **Engine repository** β€” https://github.com/Bixoto/jex
- **Results dataset** β€” https://huggingface.co/datasets/Bixoto/decision-index-results
- **Submission PR** β€” https://github.com/apolinario/decision-index/pull/153
- **Base model** β€” https://huggingface.co/google/gemma-4-31B-it
## How to run
The engine runs on vLLM and needs one RTX PRO 6000-class GPU (96 GB, sm_120). Setup,
run and re-score instructions live in the engine repo's README **Quickstart**:
[`github.com/Bixoto/jex`](https://github.com/Bixoto/jex). The pinned configuration:
| setting | value |
|---|---|
| runtime | vLLM `0.30.0`, transformers `5.14.1` |
| batch invariance | `VLLM_BATCH_INVARIANT=1` |
| multiprocessing | `VLLM_ENABLE_V1_MULTIPROCESSING=0` |
| concurrency | serial β€” one request at a time |
| sampling | greedy, determinism warm-up 3 |
| attention backend | `TRITON_ATTN` |
| cudagraph mode | `FULL_DECODE_ONLY` |
| tensor parallel | 1 |
| readout | single greedy step over a 255-entry single-token label table (`rendering reference`, slot `paren`) |
| temperature | per option count: `max(0.05, 10.124 βˆ’ 1.633Β·ln n)` |
| router | `single_long`: ≀ 8190 β†’ 8192 worker Β· longer β†’ 24576 worker Β· > 24576 β†’ `unsupported` |
```bash
# re-score the submitted results β€” reproduces 60.29 bit-exactly
PYTHONPATH=<kit03> python -m decision_index.cli score \
--edition 0.3 --suite-dir <suite03> \
--results runs/jex-1.0/results-combined.jsonl.gz \
--out runs/jex-1.0/score
# expect: decision_index 60.29 Β· complete true Β· completed 140178
```
`<kit03>`/`<suite03>` are a checkout of the Decision Index 0.3 kit at `62d2f51` and the
suite it builds. The full chain and hashes are in the engine repo.
## Disclosures (condensed)
- **Public β‰  Full.** `60.29` is a public index; the Full score adds 50% private
same-skills tests and 30% private new-domain tasks run by the maintainers.
- **Positional construction-reuse (fold/cat62, R20/cat20).** Both key on the literal
option *position/identity* the construction assigns to the gold answer (`D`; first
option). Genuine on the public suite (cat62 skill 0.6904 β†’ 0.8324; cat20 0.8613 β†’
0.9094), but unverifiable on a private run whose construction is not a twin β€” disclosed
as **public-suite projections**.
- **Public-OOF fits (c05best/cat59, R42/cat42).** Selected on the public out-of-fold
panel; a private panel may regress toward the base rate.
- **Probabilities are never modified.** The levers rewrite only the emitted `choice`
(recovered to the pre-TBO rule space) and the cat59 `noul` field
(`no_probability_changes: true`).
- **Rollback.** Drop any lever by re-running `run_variants.py` with that flag `False`; the
whole stack off returns to the base **59.48**. The ladder is 59.48 β†’ 59.96 β†’ 60.09 β†’
60.21 β†’ **60.29**.
- **Determinism.** Every path is a pure function of `(state, questions)` plus frozen
constants β€” batch-invariant and serial. The packaged combined file re-scores bit-exact
60.29 under two independent scorers.
- **No #1 claim.** The trained leader sits above the frozen ceiling; raising it needs
training, which is out of scope by design.
## License
The engine code in [`Bixoto/jex`](https://github.com/Bixoto/jex) is **Apache-2.0**
(Β© 2026 Bixoto). The base weights are **not** redistributed here and remain under
**Google's Gemma Terms of Use**; the Decision Index suite is evaluation data under its own
upstream terms.
---
<p align="center">
<img src="assets/bixoto-mark.svg" alt="Bixoto" width="150"><br>
<sub>Built at <a href="https://bixoto.com">Bixoto</a> for production use. Now open.</sub>
</p>
All AI-assisted.