jevhome L
A typed-decision model (cross-encoder, ~396M parameters, fp32) for jevhome: give it a state and
yes/no (noul), pick-one (choice) or graded (score) questions, and it returns calibrated probabilities in one
forward pass, on a plain CPU. Same API as Jev (POST /v1/systemone).
The order of the options of a choice question does not change the answer: all options get the same position ids, so the network reads them as a set, not a sequence, and a permutation gives the same probabilities up to floating-point rounding (int8 rounding can rarely change a close call).
pytorch/ holds the PyTorch weights (backbone model.safetensors + decision head head.safetensors) for GPU or research use; jevhome only needs the ONNX files (--exclude 'pytorch/*' skips them).
Run it
git clone https://github.com/Jibril-Frej/jev-at-home && cd jev-at-home && cargo build --release # the jevhome binary (needs Rust)
hf download jevhome/jevhome-L --local-dir models/jevhome-L --exclude 'pytorch/*'
./target/release/jevhome serve models/jevhome-L
Numbers
| JevBench Easy | Standard | Hard | ECE (Easy+Standard, lower is better) | External mean (10 sets) | Latency p50 | p90 | Peak RAM |
|---|---|---|---|---|---|---|---|
| 100 | 93 | 38 | 0.126 | 68.9 | 195 ms | 561 ms | 2.1 GB |
Accuracy in %, measured with the jevhome binary. Latency: one decision at a time, 4 threads on 4 cores of an AMD EPYC 9654.
The full tables (every set, Jev, Laya and the teacher), the training recipe and the data are in the jevhome README.
Files
model.onnx: the network, probabilities come from calibrated logits.decision_config.json,tokenizer*.json,config.json: input format and tokenizer.calibration_u.json: one temperature per question type and number of options.
Licence
CC BY-NC 4.0 (non-commercial): several training sources are non-commercial or share-alike. Backbone: answerdotai/ModernBERT-Large-Instruct.
- Downloads last month
- 9