README.md

---
license: mit
base_model: Qwen/Qwen3.5-4B
language:
- en
- zh
tags:
- openjev
- decision-making
- open-vocabulary
- reinforcement-learning
- safetensors
---

<div align="center">
<h1>OpenJev-4B</h1>
<p style="color:#627f92;font-size:18px"><strong>Learning Open-Vocabulary Probabilistic Decision Models from Pretrained Language Models</strong></p>
<p>
  <a href="https://github.com/ejhshen/OpenJev"><img src="https://img.shields.io/badge/GitHub-OpenJev-181717.svg?logo=github&amp;logoColor=white" alt="GitHub repository"></a>
</p>
</div>

## 1. Introduction

**OpenJev-4B** is a decision model built from the text backbone of [Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B). Given a **state**, a **question**, and a request-defined set of **natural-language options**, it directly predicts a probability distribution over those options. It supports selection, binary judgments, and rubric-based scoring through the same probabilistic interface.

The model combines a language-model backbone with a lightweight **Option Set Interactor** and a shared decision head. Its output dimension follows the candidate set supplied with each request: option names and optional criteria are inputs, rather than classes fixed in the model parameters. Decision probabilities are computed without autoregressive answer generation.

OpenJev-4B is post-trained on [OpenJevData-140k](https://huggingface.co/datasets/shenjunhao/OpenJevData-140k) in two stages: **supervised fine-tuning (SFT)** followed by **REINFORCE-Analysis (REINFORCE-A)**. SFT establishes decision behavior using predominantly hard labels. RL increases the share of soft-target contexts and refines probability quality through sampled outcome feedback.

## 2. Model Summary

OpenJev-4B brings together the following architecture and training features:

- **Open-Vocabulary Decisions**: Accepts request-defined options and criteria, returning explicit probabilities for selection, binary judgments and rubric-based scoring.
- **Permutation Equivariance**: Shared option encoders and an order-independent decision head make predictions follow the options when they are reordered, up to numerical precision.
- **Cross-Option Understanding**: A lightweight **Option Set Interactor** models semantic relationships across candidates, including overlap, competition and references to other options.
- **Direct, Parallel Inference**: Computes decision probabilities without autoregressive answer generation, with parallel option branches and shared-prefix reuse for attention caches and recurrent states.
- **Two-Stage Post-Training**: Full-parameter **SFT → REINFORCE-Analysis** combines predominantly hard-label supervision with a larger share of soft-target contexts during RL to refine probability quality.
- **Efficient Outcome-Based RL**: **REINFORCE-Analysis** samples 16 candidate actions with 10% uniform exploration and learns from binary outcome feedback. Analytic computation, importance correction and a leave-one-out baseline reduce reliance on sampled estimates.
- **Replay and Reference Regularization**: A fixed 25% of SFT contexts are replayed with the same RL objective. A frozen SFT reference provides KL regularization to help retain learned behavior.

Training data: [OpenJevData-140k](https://huggingface.co/datasets/shenjunhao/OpenJevData-140k).

## 3. Evaluation Results

1. **Accuracy (↑):** the percentage of hard-label examples for which the highest-probability option matches the gold answer. Higher is better; unsupported or failed predictions count as incorrect.
2. **Brier (↓):** the average sum of squared differences between predicted and gold option probabilities. Lower is better, with **0** indicating a perfect match. For hard labels, the gold distribution is one-hot; for soft labels, it is the provided target distribution.

We compare **OpenJev-4B** with **Jev, Decider 4B, JevK5 4B, and Intern-Decision-4B** on [JevBench Public](https://github.com/fstandhartinger/jevbench).

<style>
.openjev-eval th,.openjev-eval td{box-sizing:border-box}
.openjev-eval td{font-variant-numeric:tabular-nums}
.openjev-eval sup{font-size:9px;line-height:0;vertical-align:super;margin-left:2px}
.openjev-eval a{color:inherit;text-decoration:none}
.openjev-eval a:hover{text-decoration:underline}
.openjev-eval .oj-muted{color:#738895;font-size:11px;font-weight:400}
.openjev-eval .oj-separator{color:#9baeb9;padding:0 2px}
@media(prefers-color-scheme:dark){
.openjev-eval{color:#dce5eb!important}
.openjev-eval th{color:#bfd1de!important;border-color:#7894a5!important}
.openjev-eval .oj-band{background:#344650!important;color:#d9e5ed!important}
.openjev-eval .oj-own{background:rgba(137,163,184,.13)!important}
.openjev-eval .oj-total{background:rgba(137,163,184,.16)!important}
.openjev-eval .oj-muted{color:#a3b4c0!important}
}
</style>
<div style="overflow-x:auto;margin:18px 0 10px">
<table class="openjev-eval" style="display:table!important;width:100%;min-width:700px;table-layout:fixed;border-collapse:collapse;font-family:-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif;font-size:12px;color:#374f60;line-height:1.4">
<thead><tr>
<th style="width:16%;padding:10px 6px;overflow-wrap:break-word;text-align:left;border-bottom:2px solid #7894a5;color:#486779;font-weight:600">Task type</th>
<th class="oj-own" style="width:16.8%;padding:10px 4px;text-align:center;border-bottom:2px solid #7894a5;color:#486779;background:rgba(137,163,184,.11);"><a href="https://huggingface.co/shenjunhao/OpenJev-4B">OpenJev-4B</a></th>
<th class="" style="width:16.8%;padding:10px 4px;text-align:center;border-bottom:2px solid #7894a5;color:#486779;"><a href="https://docs.typesafe.ai/introduction/quickstart">Jev</a><sup><a href="#openjev-note-1">1</a></sup></th>
<th class="" style="width:16.8%;padding:10px 4px;text-align:center;border-bottom:2px solid #7894a5;color:#486779;"><a href="https://huggingface.co/Mapika/decider-4b">Decider 4B</a><sup><a href="#openjev-note-3">3</a></sup></th>
<th class="" style="width:16.8%;padding:10px 4px;text-align:center;border-bottom:2px solid #7894a5;color:#486779;"><a href="https://huggingface.co/alibiserikbay/JevK5">JevK5 4B</a><sup><a href="#openjev-note-3">3</a></sup></th>
<th class="" style="width:16.8%;padding:10px 4px;text-align:center;border-bottom:2px solid #7894a5;color:#486779;"><a href="https://huggingface.co/internlm/Intern-Decision-4B">Intern-<wbr>Decision-4B</a><sup><a href="#openjev-note-2">2</a></sup></th>
</tr></thead>
<tbody>
<tr><td colspan="6" class="oj-band " style="padding:9px 6px;background:#dfe8ed;color:#486779;font-weight:600;letter-spacing:.3px">JevBench · Public · Accuracy (%) ↑ / Brier ↓<sup><a href="#openjev-note-4">4</a></sup></td></tr>
<tr class="" style="background:transparent;border-top:none"><td style="padding:8px 6px;overflow-wrap:break-word;border-bottom:1px solid rgba(118,147,164,.20);font-weight:500">Original<br><span class="oj-muted">n = 72</span></td>
<td class="oj-own" style="padding:8px 3px;text-align:center;white-space:nowrap;border-bottom:1px solid rgba(118,147,164,.20);background:rgba(137,163,184,.11);"><strong>100.00</strong><span class="oj-separator">/</span>0.0312</td>
<td class="" style="padding:8px 3px;text-align:center;white-space:nowrap;border-bottom:1px solid rgba(118,147,164,.20);">98.61<span class="oj-separator">/</span><strong>0.0283</strong></td>
<td class="" style="padding:8px 3px;text-align:center;white-space:nowrap;border-bottom:1px solid rgba(118,147,164,.20);">98.61<span class="oj-separator">/</span>0.0574</td>
<td class="" style="padding:8px 3px;text-align:center;white-space:nowrap;border-bottom:1px solid rgba(118,147,164,.20);">94.44<span class="oj-separator">/</span>0.1075</td>
<td class="" style="padding:8px 3px;text-align:center;white-space:nowrap;border-bottom:1px solid rgba(118,147,164,.20);">98.61<span class="oj-separator">/</span>0.0340</td>
</tr>
<tr class="" style="background:transparent;border-top:none"><td style="padding:8px 6px;overflow-wrap:break-word;border-bottom:1px solid rgba(118,147,164,.20);font-weight:500">Easy<br><span class="oj-muted">n = 48</span></td>
<td class="oj-own" style="padding:8px 3px;text-align:center;white-space:nowrap;border-bottom:1px solid rgba(118,147,164,.20);background:rgba(137,163,184,.11);"><strong>100.00</strong><span class="oj-separator">/</span><strong>0.000011</strong></td>
<td class="" style="padding:8px 3px;text-align:center;white-space:nowrap;border-bottom:1px solid rgba(118,147,164,.20);"><strong>100.00</strong><span class="oj-separator">/</span>0.000196</td>
<td class="" style="padding:8px 3px;text-align:center;white-space:nowrap;border-bottom:1px solid rgba(118,147,164,.20);"><strong>100.00</strong><span class="oj-separator">/</span>0.000045</td>
<td class="" style="padding:8px 3px;text-align:center;white-space:nowrap;border-bottom:1px solid rgba(118,147,164,.20);"><strong>100.00</strong><span class="oj-separator">/</span>0.002509</td>
<td class="" style="padding:8px 3px;text-align:center;white-space:nowrap;border-bottom:1px solid rgba(118,147,164,.20);"><strong>100.00</strong><span class="oj-separator">/</span>0.001119</td>
</tr>
<tr class="" style="background:transparent;border-top:none"><td style="padding:8px 6px;overflow-wrap:break-word;border-bottom:1px solid rgba(118,147,164,.20);font-weight:500">Hard<br><span class="oj-muted">n = 111</span></td>
<td class="oj-own" style="padding:8px 3px;text-align:center;white-space:nowrap;border-bottom:1px solid rgba(118,147,164,.20);background:rgba(137,163,184,.11);">75.68<span class="oj-separator">/</span>0.3469</td>
<td class="" style="padding:8px 3px;text-align:center;white-space:nowrap;border-bottom:1px solid rgba(118,147,164,.20);">72.07<span class="oj-separator">/</span>0.3594</td>
<td class="" style="padding:8px 3px;text-align:center;white-space:nowrap;border-bottom:1px solid rgba(118,147,164,.20);">64.86<span class="oj-separator">/</span>0.4686</td>
<td class="" style="padding:8px 3px;text-align:center;white-space:nowrap;border-bottom:1px solid rgba(118,147,164,.20);"><strong>77.48</strong><span class="oj-separator">/</span><strong>0.3118</strong></td>
<td class="" style="padding:8px 3px;text-align:center;white-space:nowrap;border-bottom:1px solid rgba(118,147,164,.20);">73.87<span class="oj-separator">/</span>0.3439</td>
</tr>
<tr class="oj-total" style="background:#eaf0f3;border-top:2px solid #afc0cb"><td style="padding:8px 6px;overflow-wrap:break-word;border-bottom:2px solid #afc0cb;font-weight:650">Total<br><span class="oj-muted">n = 231</span></td>
<td class="oj-own" style="padding:8px 3px;text-align:center;white-space:nowrap;border-bottom:2px solid #afc0cb;background:rgba(137,163,184,.11);"><strong>88.31</strong><span class="oj-separator">/</span>0.1764</td>
<td class="" style="padding:8px 3px;text-align:center;white-space:nowrap;border-bottom:2px solid #afc0cb;">86.15<span class="oj-separator">/</span>0.1816</td>
<td class="" style="padding:8px 3px;text-align:center;white-space:nowrap;border-bottom:2px solid #afc0cb;">82.68<span class="oj-separator">/</span>0.2431</td>
<td class="" style="padding:8px 3px;text-align:center;white-space:nowrap;border-bottom:2px solid #afc0cb;">87.45<span class="oj-separator">/</span>0.1839</td>
<td class="" style="padding:8px 3px;text-align:center;white-space:nowrap;border-bottom:2px solid #afc0cb;">87.01<span class="oj-separator">/</span><strong>0.1761</strong></td>
</tr>
</tbody>
</table>
</div>

<ol style="font-size:12px;line-height:1.65;color:#627f92">
<li id="openjev-note-1"><strong>Jev:</strong> official API, evaluated as Jev 1.13.0.</li>
<li id="openjev-note-2"><strong>Intern-Decision-4B:</strong> official inference code and temperature 1.99241824, BF16, SDPA, no input truncation, in the existing Torch 2.11.0 / Transformers 5.17.0 environment.</li>
<li id="openjev-note-3"><strong>External model versions:</strong> Decider 4B v2.1 and JevK5 4B v0.3. Their returned decision distributions are used for probability metrics. OpenJev-4B is our final release model; earlier OpenJev checkpoints are omitted.</li>
<li id="openjev-note-4"><strong>JevBench Public:</strong> local evaluations of the v1.3.0 public suite: Original (72), Easy (48), Hard (111). Accuracy and Brier follow its gold-label scoring protocol; all five models return valid predictions for all 231 examples. “Hard” names a difficulty cohort here, rather than a target type. These results are separate from the official private leaderboard. Easy Brier is shown to six decimal places to retain the small differences. Bold marks the best value for each metric; zero probabilities are not clipped.</li>
<li id="openjev-note-5"><strong>Evaluation context:</strong> OpenJev training includes newly constructed scenarios informed by public decision-task patterns. The table reports performance on this fixed public evaluation collection.</li>
</ol>

Machine-readable values and evaluation counts are included in [evaluation.json](evaluation.json).

## 4. Model Weights and Code

The complete inference and two-stage training implementation is available at [**ejhshen/OpenJev**](https://github.com/ejhshen/OpenJev). This model repository contains the text backbone, decision head, tokenizer, model configuration and calibration settings. Load them together using the OpenJev runtime.

### Run a decision

Install the [GitHub repository](https://github.com/ejhshen/OpenJev#4-inference) in a compatible CUDA environment, then run:

```python
from openjev.runtime.engine import DecisionEngine

engine = DecisionEngine("shenjunhao/OpenJev-4B", execution_mode="expanded")
result = engine.predict({
    "state": "A customer recognizes a purchase but reports being charged twice.",
    "questions": {
        "route": {
            "type": "choice",
            "instructions": "Which team should handle this request?",
            "criteria": {
                "Duplicate payment": "The same purchase was charged more than once.",
                "Unrecognized payment": "The customer does not recognize the purchase."
            }
        }
    }
})
print(result["answers"]["route"]["choice"])
print(result["answers"]["route"]["probabilities"])
```

The model accepts a shared `state` and a `questions` object. For `choice`, each question supplies instructions and a mapping from candidate names to criteria; candidate names are defined by the request. It returns the selected candidate and its full probability distribution. The same interface supports `noul` for binary judgments and `score` for expected rubric-level scores. See the [input/output format and HTTP example](https://github.com/ejhshen/OpenJev/blob/main/docs/INFERENCE.md) for details.

The runtime downloads the complete artifact from this repository or accepts a local directory. The release uses **temperature 1.0**, matching the evaluation results. Decision outputs are computed directly, without generating explanation text.

### Train SFT and REINFORCE-A

Training data is available at [**shenjunhao/OpenJevData-140k**](https://huggingface.co/datasets/shenjunhao/OpenJevData-140k). The repository includes data preparation, the optimized distributed training engine, sample replay, resumable caches, checkpointing and model export. To fine-tune OpenJev-4B, set up the official verl/SGLang environment described in the [training guide](https://github.com/ejhshen/OpenJev/blob/main/docs/TRAINING.md), then load the complete released model:

```bash
hf download shenjunhao/OpenJev-4B --local-dir work/base-model
python -m openjev.cli validate-artifact work/base-model
python -m openjev.training.prepare_data \
  --dataset shenjunhao/OpenJevData-140k \
  --mandatory-replay training/mandatory-replay.json --output work/data
NPROC=8 bash training/run_two_stage.sh
```

The pipeline runs SFT, exports the SFT artifact, caches frozen reference probabilities, runs REINFORCE-A and exports the final decision model. Both stages use global batch 1,024 by default; their step counts are calculated from the prepared sample counts. Individual stage launchers and resume instructions are included in the training guide.