| --- |
| license: apache-2.0 |
| language: |
| - en |
| tags: |
| - decision-model |
| - reax |
| --- |
| |
| # REAX 1.3 |
|
|
| REAX is a decentralised lab on Bittensor subnet 92: miners train decision models in an open competition, and our releases build on the best of them. |
|
|
| REAX 1.3 is **a decision model with two parts that run in parallel on a single GPU** (reference: one RTX PRO 6000). Each part gives a probability for every option of every request, and the model combines these probabilities into an answer. It returns probabilities over typed decision options through a Jev-compatible interface. |
|
|
| The two parts are: |
|
|
| - **Part G:** miner model uid 18 ([CharlieBT/gx31-decider](https://huggingface.co/CharlieBT/gx31-decider), a fine-tune trained by a REAX miner; Gemma line, derived from tattrongvu/Jeffan-31b-1710), blended with PelaAI/KnowLine-31B-Gen1 (Gemma-4-31B). |
| - **Part Q:** miner model uid 83 ([gurujustin/sn92-gen-p5](https://huggingface.co/gurujustin/sn92-gen-p5), a fine-tune trained by a REAX miner; Qwen3.5-27B lineage via baselabs/mammoth-1), blended with vllm-sr/d3-ultra (27B, Qwen lineage). |
|
|
| Credits, licences and notices for all upstream contributors, including the competition miners, are in |
| [ATTRIBUTION.md](ATTRIBUTION.md), [NOTICE](NOTICE) and `licenses/`. |
|
|
| ## Run with the REAX Engine |
|
|
| The inference code is the `reax13` mode of the open-source [REAX Engine](https://github.com/reaxlabs/reax-engine) (MIT License), |
| directory `reax13/` at commit `ff8f595f149abdd50ea914455eadc74cf97a277f`. The `engine/` folder of this repository holds the same files, |
| byte for byte. Every request goes to both parts; their per-option probabilities are averaged 50/50 and every supplied option is returned. |
|
|
| **You need:** Linux, one NVIDIA RTX PRO 6000 Blackwell 96 GB (driver with CUDA 13 / SM120), Docker with the NVIDIA Container Toolkit, |
| at least 400 GB free disk and 64 GiB RAM (128 GiB recommended while the FP8 weights load). |
|
|
| ```bash |
| # 1. Engine (REAX Engine, reax13 mode) and the Decision Index kit it builds on |
| git clone https://github.com/reaxlabs/reax-engine.git |
| git -C reax-engine checkout ff8f595f149abdd50ea914455eadc74cf97a277f |
| cd reax-engine/reax13 |
| docker build -t reax13-base -f Dockerfile . |
| docker build -t reax13-runtime -f Dockerfile.conv . |
| git clone https://github.com/apolinario/decision-index.git decision-index |
| git -C decision-index checkout 25c6ec7b6b5f44a216c63c3a700de3a9c6d6699d |
| |
| # 2. Weights: both parts from this revision (about 115 GB) |
| python3 -m venv .hf-cli |
| source .hf-cli/bin/activate |
| pip install -U "huggingface_hub[cli]" |
| hf download decisionmodels/REAX-1.3 --include 'part-g/*' --include 'part-q/*' --local-dir model-repo |
| |
| # 3. Serve (local files only): GET /healthz, POST /v1/systemone on 127.0.0.1:8080 |
| docker run --rm --name reax13-server -p 127.0.0.1:8080:8080 \ |
| --gpus device=0 --read-only --user "$(id -u):$(id -g)" \ |
| --cap-drop ALL --security-opt no-new-privileges --shm-size 8g --tmpfs /tmp:rw,exec,size=8g \ |
| -v "$PWD/model-repo:/model:ro" -v "$PWD/decision-index/decision_index:/kit/decision_index:ro" \ |
| --entrypoint env reax13-runtime -i \ |
| PATH=/usr/local/bin:/usr/bin:/bin HOME=/tmp PYTHONPATH=/engine:/kit \ |
| DUO_ISOLATED=1 HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 HF_HUB_DISABLE_TELEMETRY=1 HF_HOME=/tmp/hf \ |
| KERNELS_CACHE=/opt/kernels LOCAL_KERNELS=kernels-community/finegrained-fp8=/opt/fp8-kernel \ |
| D3_STATE_ONCE=0 OMP_NUM_THREADS=4 OPENBLAS_NUM_THREADS=4 \ |
| PYTORCH_ALLOC_CONF=expandable_segments:True TRITON_CACHE_DIR=/tmp/triton \ |
| python /engine/serve.py --host 0.0.0.0 --port 8080 --model /model --config /engine/reax13.json |
| |
| # 4. Ask a question (first answer after loading and warmup, a few minutes) |
| curl -s 127.0.0.1:8080/v1/systemone -H 'content-type: application/json' -d '{"state":"A parcel arrived late.","questions":{"route":{"type":"choice","instructions":"Which team should handle the complaint?","criteria":{"delivery":"Delivery support","billing":"Billing support"}}}}' |
| ``` |
|
|
| To score with the Decision Index kit instead of HTTP, use the `python -m decision_index run --engine reax13_engine:Reax13Engine ...` |
| command in [engine/README.md](engine/README.md). Requests above 32,000 input tokens (size proxy) are refused as unsupported, never |
| truncated; long requests can take minutes, so allow a client timeout of at least 1,800 s. |
|
|
| Hardware: both parts are loaded in FP8 (block-128) on that one GPU; peak reserved memory about 88.5 GiB. |
| "Jev-compatible" describes the request/response interface only; it does not imply endorsement or identical outputs. |
|
|
| ## Measurements (our own, internal; not official leaderboard scores) |
|
|
| | Measure | Result | Basis | |
| | --- | --- | --- | |
| | Public Decision Index 0.3, full public suite, native index | 73.71 | Run reax13-public-0.3-20261103 (results in decisionmodels/REAX-1.3-decision-index), all 140,178 scoreable requests answered, official scorer, FP8 engine. Counting the one request above the released engine's 32,000-token cap as refused gives 73.70 | |
| | Latency median / mean / p80 | 207 / 655 / 741 ms | Released engine, uniform 1,000-call sample of the public suite, one RTX PRO 6000, one request at a time. The latency in the run's `scores.json` comes from the parallel run | |
|
|
| Long multi-question requests take far longer than the median; some take several minutes, so allow a client timeout of 1,800 s. |
|
|
| ### Comparison on public items not used in our recipe selection |
|
|
| Own measurement, not official scores, and not comparable with the 73.71 above. Native index with the published Decision Index 0.3 scorer on the |
| 3,478 of 3,513 drawn public Hugging Face Decision Index items that all eight models answered (two fixed-seed draws of 1,000 and 2,513 items, |
| 41 of 43 benchmarks). The second draw was added after a partial result on it (REAX ahead of KnowLine and d3-ultra) was known. Default settings; |
| 95% CIs reflect item sampling only. |
|
|
| | Model | Index | 95% CI | REAX 1.3 minus this model (95% CI) | How it ran | |
| | --- | --- | --- | --- | --- | |
| | REAX 1.3 | 69.5 | [67.4, 71.4] | | Run by us, FP8 | |
| | d3-ultra | 68.8 | [66.5, 70.8] | +0.7 [-0.8, +2.2], within noise | Run by us | |
| | KnowLine-31B-Gen1 | 67.4 | [65.1, 69.3] | +2.1 [+0.6, +3.8] | Run by us | |
| | d1-ultra | 64.7 | [62.0, 66.6] | +4.8 [+2.9, +7.1] | Vendor API | |
| | Clef | 60.1 | [57.2, 62.2] | +9.4 [+7.1, +12.1] | Author's server software on our rented GPU | |
| | Jev 1.14 | 59.5 | [56.9, 61.4] | +10.0 [+8.1, +12.1] | Vendor API | |
| | Microsoft-Decision-1 | 55.9 | [53.1, 57.9] | +13.6 [+11.5, +16.1] | Vendor API, our Foundry deployment | |
| | OpenAI Decisions (gpt-6-luna) | 45.1 | [42.7, 47.4] | +24.4 [+22.1, +26.6] | Vendor API | |
|
|
| This is a manual selection, not a ranking; other models are not shown. The public Hugging Face Decision Index ranks by its Full score, which includes private |
| test sets we cannot measure, so these numbers do not say where REAX 1.3 would rank. |
|
|
| ## Limitations and licences |
|
|
| Accuracy and calibration vary by task and domain. Two model parts need more GPU memory and compute than either part alone. |
|
|
| The Apache-2.0 root licence covers our blended work, subject to the preserved upstream licences and notices (KnowLine weights Apache-2.0, |
| KnowLine serving code MIT, Gemma 4 Apache-2.0, gx31-decider Apache-2.0, d3-ultra Apache-2.0, sn92-gen-p5 MIT). See `licenses/` and |
| `ATTRIBUTION.md`; model metadata does not replace those component terms. |
|
|
