Simplify model summary and remove mathematical notation

README.md

@@ -15,7 +15,6 @@ tags:
 <div align="center">
 <h1>OpenJev-4B</h1>
 <p style="color:#627f92;font-size:18px"><strong>Open-vocabulary decisions. Explicit probabilities.</strong></p>
-<p><a href="https://huggingface.co/shenjunhao/OpenJev-4B/tree/main">Model weights</a> · <a href="https://huggingface.co/datasets/shenjunhao/mmdm">MMDM benchmark</a> · <a href="#4-evaluation-results">Evaluation results</a></p>
 </div>
 
 ## 1. Introduction
@@ -26,195 +25,15 @@ The model combines a language-model backbone with a lightweight **Option Set Int
 
 OpenJev-4B is post-trained in two stages: **supervised fine-tuning (SFT)** followed by **REINFORCE-Analysis (REINFORCE-A)**. SFT establishes decision behavior using predominantly hard labels. RL increases the share of soft-target contexts and refines probability quality through sampled outcome feedback. A fixed subset of SFT contexts is replayed using the same RL objective.
 
-## 2. Jevification: From Language Model to Decision Model
-
-Jevification is our conversion of a pretrained language model into an open-vocabulary decision model. The implementation below describes OpenJev's architecture.
+## 2. Model Summary
 
 ![OpenJev architecture: independent option branches, a two-layer Option Set Interactor, and question-option compatibility scoring](assets/openjev-architecture.png)
 
-### 2.1 Input and probability interface
-
-Let the input be
-
-$$
-x=(S,Q,\mathcal O),\qquad
-\mathcal O=(O_1,\ldots,O_K),\qquad O_i=(n_i,\kappa_i),
-$$
-
-where $S$ is the state, $Q$ is the question, $n_i$ is the option name, and $\kappa_i$ is optional natural-language criteria. Option IDs are used to associate outputs with inputs. The model returns
-
-$$
-\mathbf p=(p_1,\ldots,p_K),\qquad p_i\geq 0,\qquad\sum_{i=1}^{K}p_i=1.
-$$
-
-**Choice** selects the highest-probability option. **Noul** expresses a binary judgment through two candidate outcomes. **Score** associates each candidate with a numeric value $v_i$ and returns an expected score $\sum_i v_i p_i$, alongside the full distribution.
-
-For example, an option may be named `Platform Reliability`, with criteria describing production outages, latency, and capacity failures. A different request can introduce an entirely new option name or criterion without resizing or retraining the decision head. This is the model's **open-vocabulary option interface**.
-
-### 2.2 Independent branch encoding and set interaction
-
-We retain the text backbone and replace vocabulary-level answer prediction with a newly initialized decision head. The backbone produces endpoint representations for the question and each option:
-
-$$
-\mathbf h_Q=f_\theta(S,Q)\in\mathbb R^H,\qquad
-\mathbf h_i=f_\theta(S,Q,O_i)\in\mathbb R^H.
-$$
-
-Each option is encoded on its own branch under the same state and question. A branch does not include preceding options. The decision head then projects the endpoint representations into a shared decision space:
-
-$$
-\mathbf u=P_Q\mathbf h_Q+\mathbf b_Q,\qquad
-\mathbf r_i=P_O\mathbf h_i+\mathbf b_O,
-$$
-
-$$
-(\mathbf z_1,\ldots,\mathbf z_K)
-=\Phi_\psi(\mathbf r_1+\mathbf u,\ldots,\mathbf r_K+\mathbf u).
-$$
-
-Here $H=2560$, the decision width is $d=512$, and $\Phi_\psi$ is a **two-layer, pre-norm Transformer over option representations**, with eight attention heads and a $512\rightarrow2048\rightarrow512$ GELU feed-forward network. It has no causal mask or option-position embedding; a padding mask excludes padded candidates. Question conditioning is added to every option before interaction.
-
-The final compatibility scores and probabilities are
-
-$$
-\ell_i=\frac{(U_Q\mathbf u)^\top(U_O\mathbf z_i)}{\sqrt d},
-\qquad
-p_i=\frac{\exp(\ell_i/\tau)}{\sum_{j=1}^{K}\exp(\ell_j/\tau)}.
-$$
-
-The endpoint projections include biases; the compatibility maps $U_Q,U_O$ do not. The released model uses $\tau=1$. All options share the same branch encoder and option-side head parameters.
-
-In the figure, the backbone endpoint labeled **q** corresponds to $\mathbf h_Q$ above, while **e** denotes option endpoints. Throughout the training equations below, **$\mathbf q$ denotes only the target probability distribution**; it is not a question embedding. $\kappa_i$ denotes option criteria, reserving $c_g$ for RL feedback.
-
-This construction is **permutation equivariant**: reordering the same option objects reorders their output probabilities, up to numerical execution effects. Set interaction lets each option's representation depend on the other candidates while avoiding a fixed class vocabulary or an ordering-dependent chain of option tokens. Changing an option's text or criteria is a different input, rather than a permutation.
-
-### 2.3 Shared-prefix execution
-
-Inference can reuse the state prefix across questions and the question prefix across option branches. For Qwen3.5, this reusable state includes both full-attention KV caches and the convolution/recurrent states of its Gated DeltaNet layers. Branches fork these states before continuing their own tokens.
-
-Training uses batched, expanded state–question–option branches through the differentiable backbone. This keeps the gradient path compatible with distributed training and the existing Qwen kernels. Both the backbone and decision head are updated during SFT and RL.
-
-<div style="overflow-x:auto">
-
-| Model property | OpenJev-4B |
-| --- | --- |
-| Backbone | Qwen3.5-4B text model |
-| Total parameters | 4.215B |
-| Decision head | 9.452M parameters; two set-interaction layers |
-| Weight precision | BF16 backbone; FP32 decision head |
-| Output | A distribution over the current candidate set |
-| Shipped input budgets | 8,192 tokens per expanded branch; 65,536 logical tokens per decision |
-
-</div>
-
-The logical token budget counts the shared state/question prefix once plus all option suffixes. These are the limits stored in the released configuration; they are distinct from the backbone's architectural maximum context length.
-
-## 3. Post-Training
-
-$$
-\boxed{\text{SFT}\;\longrightarrow\;\text{REINFORCE-Analysis}}
-$$
-
-### 3.1 Supervised fine-tuning
-
-For a target distribution $\mathbf q=(q_1,\ldots,q_K)$, SFT minimizes
+**Architecture.** OpenJev-4B is designed to combine language understanding with direct, efficient decisions over an open-vocabulary candidate set. It encodes each option independently under the same state and question, then brings the resulting representations together in a lightweight, two-layer **Option Set Interactor**. This lets the model consider semantic relationships between candidates—such as overlap, competition, and statements that refer to other options—before assigning probabilities through a shared decision head. The interactor has no option-position embeddings or causal ordering, and all option branches share their parameters. As a result, the architecture is **permutation equivariant**: reordering the same option objects reorders their output probabilities, up to numerical execution effects. Candidate names and criteria are supplied with each request, so the model is not restricted to a fixed vocabulary of labels. For efficient inference, it computes decision probabilities directly instead of generating answer tokens, supports parallel option-branch computation, and can reuse shared state and question prefixes, including both attention caches and recurrent states in the Qwen3.5 backbone. These choices reduce repeated prefix computation and avoid autoregressive answer decoding while preserving interaction across options.
 
-$$
-\boxed{\mathcal L_{\mathrm{SFT}}=-\sum_{i=1}^{K}q_i\log p_i.}
-$$
+**Training.** We train both the backbone and the decision head in two stages: **supervised fine-tuning (SFT)** followed by **REINFORCE-Analysis (REINFORCE-A)**. SFT learns the decision distribution from complete targets, with predominantly hard labels establishing broad decision accuracy; RL uses a larger share of soft-target contexts to refine probability quality. Unlike token-generation RL with sequence-level rewards, our RL samples candidate decisions directly from the predicted distribution, mixed with 10% uniform exploration. Each episode draws 16 actions with replacement and receives binary feedback indicating whether each action matches the same hidden outcome. For soft-target examples, that outcome is drawn from the target distribution by the environment; the actor's update uses sampled feedback rather than the complete target. REINFORCE-A combines importance correction and a leave-one-out baseline with an exactly computed probability-based term: only the outcome-dependent part requires sampling. This aims to reduce estimation noise and sampling overhead while improving both decision accuracy and distribution quality. A KL penalty to the frozen SFT reference helps retain learned behavior, without PPO clipping or a learned value head. We also select a fixed 25% of the SFT contexts for replay during RL; these contexts use the same RL objective as all other RL examples. The final stages cover 100,420 unique SFT contexts and 71,498 unique RL contexts including replay, with the share of soft-target contexts rising from 14.3% to 29.8%.
 
-A hard label is the one-hot case $\mathbf q=\mathbf e_y$, giving $\mathcal L_{\mathrm{SFT}}=-\log p_y$. Soft targets supervise the complete distribution.
-
-The SFT stage makes one shuffled pass through **100,420 decisions**, with **86,096 hard** and **14,324 soft** examples. Hard labels account for **85.7%** of this stage and establish broad decision accuracy, while the soft examples introduce distributional supervision.
-
-### 3.2 REINFORCE-Analysis
-
-The RL actor receives the context, candidate options, and sampled correctness feedback. It does not receive the full target distribution in its optimization batch. For soft-target contexts, the environment draws outcomes according to the target distribution using a pre-sampled, reproducible outcome tape. Each episode has one hidden outcome $Y$, shared by all action queries in that episode.
-
-**Exploration and feedback.** Construct an $\epsilon$-greedy behavior distribution:
-
-$$
-\boxed{\mu_i=(1-\epsilon)p_i+\frac{\epsilon}{K},\qquad\epsilon=0.1.}
-$$
-
-Sample $G=16$ actions independently **with replacement** and obtain binary feedback:
-
-$$
-a_g\sim\boldsymbol\mu,\qquad
-c_g=\mathbf 1[a_g=Y],\qquad g=1,\ldots,G.
-$$
-
-**Importance correction and baseline.** With $\alpha=0.5$, define
-
-$$
-D_g=\frac{(\alpha p_{a_g}+1-\alpha)c_g}{\mu_{a_g}},
-\qquad
-w_g=\frac{p_{a_g}}{\mu_{a_g}},
-\qquad
-b_{-g}=\frac{1}{G-1}\sum_{h\ne g}D_h.
-$$
-
-The importance-weighted leave-one-out baseline uses the other sampled actions. Its $w_g$ factor accounts for sampling from $\boldsymbol\mu$ while optimizing $\mathbf p$.
-
-**REINFORCE-A objective.** The implemented gradient surrogate is
-
-$$
-\boxed{
-\widehat{\mathcal L}_{\mathrm{RF\text{-}A}}
-=\frac{\alpha}{2}\sum_{i=1}^{K}p_i^2
--\frac{1}{G}\sum_{g=1}^{G}
-\operatorname{sg}\!\left(D_g-w_gb_{-g}\right)\log p_{a_g}.
-}
-$$
-
-Here $\operatorname{sg}$ denotes stop-gradient. The behavior probabilities, feedback, importance weights, and baseline coefficients are detached in the sampled term; gradients flow through $\log p_{a_g}$ and through the analytic quadratic term.
-
-The **Analysis** step computes $\frac{\alpha}{2}\sum_i p_i^2$ exactly. Only the part that depends on hidden outcome feedback is estimated with REINFORCE. Under the outcome model $Y\sim\mathbf q$, its expected gradient matches the gradient of
-
-$$
-\mathcal J(\mathbf p,\mathbf q)
-=(1-\alpha)\operatorname{CE}(\mathbf q,\mathbf p)
-+\frac{\alpha}{2}\lVert\mathbf p-\mathbf q\rVert_2^2.
-$$
-
-This connects the update to both log-loss and probability squared error. It is a statement about the **expected gradient**: the numerical value of the sampled surrogate is not itself a calibrated performance metric or the value of $\mathcal J$.
-
-**Reference regularization.** The complete RL objective also includes a forward KL penalty to the frozen SFT reference distribution $\mathbf p_{\mathrm{ref}}$:
-
-$$
-\mathcal L_{\mathrm{RL}}
-=\widehat{\mathcal L}_{\mathrm{RF\text{-}A}}
-+\beta\sum_{i=1}^{K}p_i\log\frac{p_i}{p_{\mathrm{ref},i}},
-\qquad\beta=0.01.
-$$
-
-The release uses REINFORCE-A with this KL term, without a PPO clipping objective or a learned value head.
-
-### 3.3 Data replay and training configuration
-
-The primary RL pool contains **46,393 contexts**, including **17,726 soft-target contexts** (38.2%). We select a fixed **25% of the SFT dataset**—**25,105 contexts**—without replacement and include them in RL. Selection is stratified by source, task, and supervision type.
-
-**Replay refers to reusing contexts with the RL objective.** Replayed SFT contexts receive sampled outcome feedback and optimize REINFORCE-A plus KL, exactly like the primary RL pool. No supervised cross-entropy replay term is added. The 25% refers to the fraction selected from SFT, not the fraction of each RL batch.
-
-The combined RL set has **71,498 unique contexts**, comprising **50,201 hard-target** and **21,297 soft-target** contexts. Soft-target coverage therefore rises from **14.3% in SFT to 29.8% in RL**. Hard targets remain useful for retaining deterministic decision behavior.
-
-<div style="overflow-x:auto">
-
-| Configuration | SFT | REINFORCE-A |
-| --- | --- | --- |
-| Training hardware | 8 × H200 | 8 × H200 |
-| Global decision batch | 1,024 | 1,024 |
-| Unique training contexts | 100,420 | 71,498, including replay |
-| Backbone learning rate | $4\times10^{-6}$ | $7.5\times10^{-7}$ |
-| Decision-head learning rate | $4\times10^{-5}$ | $7.5\times10^{-6}$ |
-| Warmup steps | 10 | 5 |
-| Parameters updated | Backbone and decision head | Backbone and decision head |
-
-</div>
-
-The stages use a shuffled full-pass schedule. SFT masks final distributed padding from the loss; RL pads the final rank-local batches with a small number of repeated draws. The RL coefficients are $\alpha=0.5$, $\epsilon=0.1$, $G=16$, and $\beta=0.01$.
-
-## 4. Evaluation Results
+## 3. Evaluation Results
 
 We compare **OpenJev-4B** with **Jev, Decider 4B, JevK5 4B, and Intern-Decision-4B** on [MMDM](https://huggingface.co/datasets/shenjunhao/mmdm) and [JevBench Public](https://github.com/fstandhartinger/jevbench). The OpenJev column contains only the final release model. MMDM is grouped by hard/soft target type and task category; JevBench Public is grouped into Original, Easy, and Hard cohorts. Each hard/Public cell shows **Accuracy (%) ↑ / Brier ↓**, while soft-target cells show **Brier ↓**.
 
@@ -466,17 +285,11 @@ We compare **OpenJev-4B** with **Jev, Decider 4B, JevK5 4B, and Intern-Decision-
 <li id="openjev-note-6"><strong>Evaluation context:</strong> the published MMDM split combines the previously separate development and test sets, so these numbers are not a new blind evaluation. OpenJev training includes newly constructed scenarios informed by public decision-task patterns. The tables report performance on these fixed evaluation collections.</li>
 </ol>
 
-For hard labels, the Brier target is the one-hot gold outcome. For MMDM soft targets, it is the full supplied distribution:
-
-$$
-\operatorname{Brier}
-=\frac{1}{N}\sum_{j=1}^{N}\sum_{i=1}^{K_j}
-\left(p_i^{(j)}-q_i^{(j)}\right)^2.
-$$
+Brier measures the sum of squared differences between predicted and target probabilities, averaged over examples. Hard-label targets place all probability on the gold outcome; MMDM soft targets use the full supplied distribution. Lower is better.
 
 Machine-readable values and evaluation counts are included in [evaluation.json](evaluation.json).
 
-## 5. Model Weights and Code
+## 4. Model Weights and Code
 
 The repository contains the complete OpenJev-4B decision artifact: