OpenJevX v0.5.0: model folder (config.json temperatures, tokenizer.json), clean-gate card, Python helper reads config.json

README.md

@@ -13,9 +13,10 @@ one container, the model inside, your requests never leave your machine.
 - Server, releases, Docker image, recipes: [github.com/muthuishere/openjevx](https://github.com/muthuishere/openjevx)
 - CLI: [jevx](https://github.com/muthuishere/jevx)
 
-**Use `openjevx.w8.onnx`** (8-bit weight-only, 598 MB; activations stay float, so answers don't depend on
-what else is in the request). The other files are for fine-tuning further: `model.safetensors`, `encoder/`,
-`tokenizer/`, `rl_agent_config.json`. Older versions: git revision `v0.4.0`.
+**The model is a folder:** `openjevx.w8.onnx` (8-bit weight-only, 598 MB; activations stay float, so answers
+don't depend on what else is in the request) + `config.json` (this model's own calibration temperatures) +
+`tokenizer.json`. The other files are the trainable checkpoint for fine-tuning further: `model.safetensors`,
+`encoder/`, `tokenizer/`, `rl_agent_config.json`. Older versions: git revision `v0.4.0`.
 
 ## Run it in a container (recommended)
 
@@ -58,7 +59,7 @@ path = hf_hub_download("muthuishere/openjevx", "openjevx.py")
 spec = importlib.util.spec_from_file_location("openjevx", path); openjevx = importlib.util.module_from_spec(spec)
 spec.loader.exec_module(openjevx)
 
-jev = openjevx.OpenJevX()   # downloads openjevx.w8.onnx + tokenizer once
+jev = openjevx.OpenJevX()   # downloads openjevx.w8.onnx + tokenizer.json + config.json once
 print(jev.decide({"age": 17}, {
     "adult": {"type": "noul", "instructions": "Is this person an adult (18 or older)?"},
 }))
@@ -68,61 +69,37 @@ print(jev.decide({"age": 17}, {
 `openjevx.py` is a single file (also in the GitHub repo under `python/`); copy it into your project if you
 prefer. Request and answer shapes match `POST /v1/systemone`.
 
-## Release gate (v0.5.0, the exact 8-bit file on the Go server)
+## Release gate (v0.5.0)
 
-"Right and confident" means right AND past jevx's default thresholds (yes >= 0.8, no <= 0.2, choice/score >= 0.6).
-Gate questions are worded differently from anything in training.
+The model folder on the Go server, on CPU. The gate sets share **no state** with training.
+"Right and confident" = right AND past jevx's default thresholds (yes >= 0.8, no <= 0.2, choice/score >= 0.6).
 
-| Gate set | Right and confident | Confidently wrong | Limit |
+| Gate set | Accuracy | Right and confident | Confidently wrong |
 |---|---|---|---|
-| Everyday basics (469 questions, 58 rules) | 98.9% | 0.6% | pass |
-| Rules with conditions (300 questions) | 94.3% | **3.7%** | **miss** (limit 2%) |
+| Everyday basics (463) | 96.3% | 96.3% | 3.0% |
+| Rule-checking basics (300) | 94.7% | 94.3% | 3.7% |
 
-The conditions set misses its confidently-wrong limit; it was released anyway and fixing it is planned for v0.6.
-Through the jevx CLI, 11 of 13 everyday fundamentals are answered correctly, 1 is unsure and 0 are wrong.
+**Known miss:** confidently wrong is 3.0–3.7% on the basics gates, against our 2% target. It was accepted for this
+release; the fix is planned for v0.5.1. The rest of the gate (held-out test sets, logs gate, 13 fundamentals) is
+still being scored and will be added here.
 
-## How it compares
+**Benchmarks:** coming in a follow-up.
 
-Measured on 2026-09-29. Every model got the same questions, one question per request, 100 questions per set
-(so each number is within about ±9 points). "Confident" means the model gave its answer at least 80%.
-Jeff-0.8B ([mstrasser/Jeff-Qwen3.5-0.8B](https://huggingface.co/mstrasser/Jeff-Qwen3.5-0.8B)) is the closest
-open model in size and API; it ran on Apple silicon (MLX), OpenJevX on CPU.
+## What v0.5.0 was trained on
 
-**Everyday rules and logs** (the work OpenJevX is built for)
+734,795 decisions (408,505 rows). Labels come from evaluating rules or from real outcomes, never from another model's guesses.
 
-| Set | OpenJevX: confident and right | OpenJevX: confident and wrong | Jeff-0.8B: confident and right | Jeff-0.8B: confident and wrong |
-|---|---|---|---|---|
-| Everyday rules | 100% | 0% | 86% | 0% |
-| Rules with conditions | 91% | 5% | 82% | 0% |
-| Log triage | **93%** | 7% | 38% | 8% |
-| Real public postmortems | 50% | **18%** | 71% | 8% |
-| All four | **83.5%** | 7.5% | 69.2% | 4.0% |
-
-**General benchmarks** (Jeff's public panel: BBH, Financial PhraseBank, JudgeBench, RAGTruth, WinoGrande)
-
-| | OpenJevX | Jeff-0.8B |
-|---|---|---|
-| Accuracy | 60.4% | **75.4%** |
-| Confident on | 11% | 65% |
-| Confident and wrong | 1.0% | 9.6% |
-
-**Size and speed**
-
-| | OpenJevX | Jeff-0.8B |
+| Area | Decisions | Share |
 |---|---|---|
-| Model | 421M, 8-bit ONNX, 598 MB | 0.8B, 16-bit, 1.7 GB |
-| Runs on | any CPU, one binary, no Python | Python + PyTorch, or MLX on Apple silicon |
-| Median time per question | 450 ms (CPU, shared with another job) | 34 ms (Apple GPU) |
-
-**Where it stands**
-
-- Use it for rule-shaped decisions and log triage: it commits to an answer on 91% of those and is right 92% of
-  the time when it does.
-- It is not a general reasoner. On the public benchmarks it is mostly unsure (confident on 11%), so it rarely
-  misleads there but rarely helps either. Larger models, and hosted Jev, do much better on reasoning.
-- Known miss: on real incident postmortems it is confidently wrong 18% of the time. Check those answers.
-- The rules set shares facts with the training data (the questions are reworded), so its 100% overstates
-  performance on new rules.
+| Rule-checking across 38 business domains (retail pricing, recruiting, real estate, CI/CD, insurance claims, pharmacy stock, HR payroll, gaming, ...) | 200,139 | 27.2% |
+| Software-work roles (developers, testers, tech leads, managers, operations, everyone, agent checks) | 183,543 | 25.0% |
+| tasksource decision corpus | 146,567 | 19.9% |
+| Public sets with real labels (CVE fixes from bigvul, defect detection, code search, ms_marco relevance, HDFS and BGL operator log alerts) | 104,990 | 14.3% |
+| Log triage: application (Java/Python/Node/Go/nginx), database (Postgres/MySQL, real SQLSTATE codes), frontend (browser/Sentry) | 60,001 | 8.2% |
+| Everyday basics (driving licence age, store hours, parcel late, discount thresholds, fever, bag weight, ...) | 30,405 | 4.1% |
+| Public typed-decisions | 9,150 | 1.2% |
+
+Run: one RTX 4090 on Vast.ai, 734,145 decisions after packaging, one full pass in 2.4 h at ~88 items/s, about $1.10 in total. Leakage-checked (22,592 test questions removed); every row checked against the trainer before renting.
 
 ## Using it with jevx
 

config.json

@@ -0,0 +1,20 @@
+{
+  "name": "openjevx",
+  "version": "0.5.0",
+  "temperature": {
+    "choice": 1.220432996749878,
+    "score": 1.161743402481079,
+    "noul": 1.2535336017608643
+  },
+  "max_len": 1024,
+  "head_max": 256,
+  "special_ids": {
+    "cls": 50281,
+    "sep": 50282,
+    "pad": 50283,
+    "mask": 50284
+  },
+  "quantization": "8-bit weight-only",
+  "base_model": "convaiinnovations/laya",
+  "sha256": "109b2b1d3717ff9038da811d84f538262fc9d9bc1f958967d437e837f507e294"
+}

openjevx.int8.onnx

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:02ec28639b86bc9891f396c2a86909234df080336b59ef13addb0cec8eec4b93
-size 598012474

openjevx.py

@@ -23,7 +23,6 @@ from tokenizers import Tokenizer
 REPO = "muthuishere/openjevx"
 CLS, SEP, MASK = 50281, 50282, 50284
 MAX_LEN, HEAD_MAX = 1024, 256
-TEMPERATURE = {0: 1.02553391456604, 1: 1.034013271331787, 2: 1.147902488708496}  # choice, score, noul
 
 
 def _criterion(v):
@@ -48,9 +47,12 @@ def _parse(q):
 
 
 class OpenJevX:
-    def __init__(self, model=None, tokenizer=None, providers=None):
+    def __init__(self, model=None, tokenizer=None, config=None, providers=None):
         model = model or hf_hub_download(REPO, "openjevx.w8.onnx")
-        tokenizer = tokenizer or hf_hub_download(REPO, "tokenizer/tokenizer.json")
+        tokenizer = tokenizer or hf_hub_download(REPO, "tokenizer.json")
+        config = config or hf_hub_download(REPO, "config.json")  # this model's own calibration temperatures
+        t = json.load(open(config))["temperature"]
+        self.temperature = {0: t["choice"], 1: t["score"], 2: t["noul"]}
         self.tok = Tokenizer.from_file(tokenizer)
         self.session = ort.InferenceSession(model, providers=providers or ["CPUExecutionProvider"])
 
@@ -93,7 +95,7 @@ class OpenJevX:
         out = {}
         for r, qid in enumerate(qids):
             _, markers, qtype, keys = items[r]
-            z = [float(x) / TEMPERATURE[qtype] for x in logits[r, :len(markers)]]
+            z = [float(x) / self.temperature[qtype] for x in logits[r, :len(markers)]]
             m = max(z); p = [math.exp(x - m) for x in z]; s = sum(p); p = [x / s for x in p]
             best = max(range(len(p)), key=p.__getitem__)
             ans = {"type": questions[qid]["type"], "probabilities": {k: round(p[j], 4) for j, k in enumerate(keys)},