|
Download README.md from Bixoto/jex: direct link, hf CLI and curl.
- Browser
- Download file 5.72 kB
-
https://huggingface.co/Bixoto/jex/resolve/main/README.md
- Command line
-
hf download hf://Bixoto/jex/README.md
-
curl -L -o README.md https://huggingface.co/Bixoto/jex/resolve/main/README.md
5.72 kB
| license: apache-2.0 | |
| base_model: google/gemma-4-31B-it | |
| tags: | |
| - decision | |
| - text-classification | |
| - decision-index | |
| - frozen | |
| - jex | |
| <p align="center"> | |
| <img src="assets/hero.svg" alt="Jex 1.0 by Bixoto β a frozen decision engine. Public index 60.29 on Decision Index 0.3." width="100%"> | |
| </p> | |
| # Jex 1.0 | |
| **This repository contains no weights.** | |
| Jex 1.0 serves the frozen `google/gemma-4-31B-it` checkpoint at revision | |
| `842da3794eaa0b77d5f08bae87a17459d91ff475` (bf16) through our own deterministic, | |
| batch-invariant inference engine. The weights are not copied here: fetch them from | |
| [`google/gemma-4-31B-it`](https://huggingface.co/google/gemma-4-31B-it) at that revision. | |
| We did **not** train, fine-tune, quantize, or add a head or LoRA β the checkpoint is | |
| untouched, and the whole difference is the engine around it. | |
| Jex 1.0 is a **decision engine, not a language model**. Given a state and a set of typed | |
| questions, it returns one answer per question with a probability for every allowed | |
| option β in a single forward pass, no free-form text, no output parsing. It cannot answer | |
| with something that was not on the list. | |
| ## Headline | |
| | field | value | | |
| |---|---:| | |
| | **Decision Index 0.3 public index** | **60.29** | | |
| | balanced skill / raw | 60.29 / 69.88 | | |
| | breadth skill | 59.25 | | |
| | requests | **140,178 / 140,178 ok** | | |
| | unsupported Β· error | 0 Β· 0 | | |
| | `complete` | `true` | | |
| | base model | `google/gemma-4-31B-it` @ `842da3794eaa0b77d5f08bae87a17459d91ff475` (bf16, frozen) | | |
| The board's next-best frozen entry, **deck31b**, runs the **same checkpoint** and lands at | |
| 58.19 public. Jex 1.0 lands at 60.29 β a **+2.10** difference that is entirely engine work: | |
| the readout, the batching, the decision rules, and four disclosed deterministic | |
| post-processing levers. | |
| > `60.29` is a **public index** β the chance-corrected score over the 37-benchmark public | |
| > panel (5 areas). It is **not** the board's Full score, which adds private tests only the | |
| > maintainers can run. No Full-score or rank is claimed here. | |
| ## Quick links | |
| - **Engine repository** β https://github.com/Bixoto/jex | |
| - **Results dataset** β https://huggingface.co/datasets/Bixoto/decision-index-results | |
| - **Submission PR** β https://github.com/apolinario/decision-index/pull/153 | |
| - **Base model** β https://huggingface.co/google/gemma-4-31B-it | |
| ## How to run | |
| The engine runs on vLLM and needs one RTX PRO 6000-class GPU (96 GB, sm_120). Setup, | |
| run and re-score instructions live in the engine repo's README **Quickstart**: | |
| [`github.com/Bixoto/jex`](https://github.com/Bixoto/jex). The pinned configuration: | |
| | setting | value | | |
| |---|---| | |
| | runtime | vLLM `0.30.0`, transformers `5.14.1` | | |
| | batch invariance | `VLLM_BATCH_INVARIANT=1` | | |
| | multiprocessing | `VLLM_ENABLE_V1_MULTIPROCESSING=0` | | |
| | concurrency | serial β one request at a time | | |
| | sampling | greedy, determinism warm-up 3 | | |
| | attention backend | `TRITON_ATTN` | | |
| | cudagraph mode | `FULL_DECODE_ONLY` | | |
| | tensor parallel | 1 | | |
| | readout | single greedy step over a 255-entry single-token label table (`rendering reference`, slot `paren`) | | |
| | temperature | per option count: `max(0.05, 10.124 β 1.633Β·ln n)` | | |
| | router | `single_long`: β€ 8190 β 8192 worker Β· longer β 24576 worker Β· > 24576 β `unsupported` | | |
| ```bash | |
| # re-score the submitted results β reproduces 60.29 bit-exactly | |
| PYTHONPATH=<kit03> python -m decision_index.cli score \ | |
| --edition 0.3 --suite-dir <suite03> \ | |
| --results runs/jex-1.0/results-combined.jsonl.gz \ | |
| --out runs/jex-1.0/score | |
| # expect: decision_index 60.29 Β· complete true Β· completed 140178 | |
| ``` | |
| `<kit03>`/`<suite03>` are a checkout of the Decision Index 0.3 kit at `62d2f51` and the | |
| suite it builds. The full chain and hashes are in the engine repo. | |
| ## Disclosures (condensed) | |
| - **Public β Full.** `60.29` is a public index; the Full score adds 50% private | |
| same-skills tests and 30% private new-domain tasks run by the maintainers. | |
| - **Positional construction-reuse (fold/cat62, R20/cat20).** Both key on the literal | |
| option *position/identity* the construction assigns to the gold answer (`D`; first | |
| option). Genuine on the public suite (cat62 skill 0.6904 β 0.8324; cat20 0.8613 β | |
| 0.9094), but unverifiable on a private run whose construction is not a twin β disclosed | |
| as **public-suite projections**. | |
| - **Public-OOF fits (c05best/cat59, R42/cat42).** Selected on the public out-of-fold | |
| panel; a private panel may regress toward the base rate. | |
| - **Probabilities are never modified.** The levers rewrite only the emitted `choice` | |
| (recovered to the pre-TBO rule space) and the cat59 `noul` field | |
| (`no_probability_changes: true`). | |
| - **Rollback.** Drop any lever by re-running `run_variants.py` with that flag `False`; the | |
| whole stack off returns to the base **59.48**. The ladder is 59.48 β 59.96 β 60.09 β | |
| 60.21 β **60.29**. | |
| - **Determinism.** Every path is a pure function of `(state, questions)` plus frozen | |
| constants β batch-invariant and serial. The packaged combined file re-scores bit-exact | |
| 60.29 under two independent scorers. | |
| - **No #1 claim.** The trained leader sits above the frozen ceiling; raising it needs | |
| training, which is out of scope by design. | |
| ## License | |
| The engine code in [`Bixoto/jex`](https://github.com/Bixoto/jex) is **Apache-2.0** | |
| (Β© 2026 Bixoto). The base weights are **not** redistributed here and remain under | |
| **Google's Gemma Terms of Use**; the Decision Index suite is evaluation data under its own | |
| upstream terms. | |
| --- | |
| <p align="center"> | |
| <img src="assets/bixoto-mark.svg" alt="Bixoto" width="150"><br> | |
| <sub>Built at <a href="https://bixoto.com">Bixoto</a> for production use. Now open.</sub> | |
| </p> | |
| All AI-assisted. | |