Arbiter v0.5: replace v0.4 (merged model, new GGUFs, new model card)

README.md

@@ -14,35 +14,68 @@ tags:
 - wealth-management
 - sec-filings
 - news
+- central-banks
 - calibration
 - lora
 - jev
 ---
 
-# Arbiter-8B
+# Arbiter-8B (v0.5)
 
-Arbiter-8B is a Jev-inspired local decision model built on Alibaba's Qwen3-8B, for wealth and asset management. It reads verbatim news articles and SEC Form 10-K passages and answers typed questions about a named company, such as event type, implied outlook, how settled an event is, risk category, or filing flags. It returns a **calibrated probability for every option** in a single forward pass, with no generated text.
+Arbiter-8B is a Jev-inspired decision model built on Alibaba's Qwen3-8B, for wealth and asset management. Arbiter is aimed to be run on-prem at low cost. It reads financial text (news articles, headlines, SEC Form 10-K passages, central-bank statements) and answers a typed question about it, such as the type of corporate event, the implied outlook for a company, the sentiment toward it, or the policy stance of a sentence. It returns a **calibrated probability for every option** in a single forward pass, with no generated text.
+
+This repository holds **Arbiter v0.5**. It replaces v0.4, which is still available in the repository history.
+
+## Results at a glance
+
+Same items, same question and same option definitions for every system. EDT and FOMC are scored on their held-out test splits; SEntFiN was never trained on.
+
+| Benchmark | Arbiter v0.5 | Qwen3-8B (base) | Jev 1.14 | OpenAI Decisions API |
+|---|---|---|---|---|
+| **EDT**, corporate event detection, 1,933 articles, accuracy | **94.2%** | 89.3% | 88.3% | 90.8% |
+| **FOMC**, policy stance, 496 sentences, accuracy | **67.5%** | 55.0% | 66.7% | 67.3% |
+| **SEntFiN**, entity sentiment, 1,000-headline sample, accuracy | 75.7% | 75.4% | **81.6%** | 78.0% |
+| SEntFiN, signals caught (positive/negative labels called correctly) | **92.0%** | 82.8% | 91.0% | 91.7% |
+| SEntFiN, signal precision (positive/negative calls that are right) | 72.2% | 76.3% | **78.8%** | 74.8% |
+
+- **EDT:** Arbiter leads all three systems; every gap is significant (paired McNemar, z ≥ 7.5).
+- **FOMC:** Arbiter, Jev and OpenAI are level (z ≤ 0.4); all three are far ahead of the base model (z 4.8).
+- **SEntFiN:** Arbiter catches the most positive and negative signals but is less precise than Jev and OpenAI; it labels too few headlines neutral. 
+
+
+## What changed from v0.4
+
+v0.5 was trained to follow the option definitions it is given, to separate genuine corporate events from look-alikes, and to read central-bank language.
+
+| | v0.4 | v0.5 |
+|---|---|---|
+| EDT accuracy | 83.5% | **94.2%** |
+| FOMC accuracy | 60.1% | **67.5%** |
+| SEntFiN accuracy (full benchmark, 14,329 headlines, zero-shot) | 76.0% | **76.4%** |
+| EDT `new_contract` precision | 38.3% | **86.7%** |
+| EDT `none` recall | 81.2% | **95.2%** |
+| Answers that change when options are shuffled, FOMC | 15.5% | **4.8%** |
+| Calibration error (ECE), EDT / FOMC | 0.099 / 0.127 | **0.012 / 0.067** |
+| Human-labelled dev set (151, news + 10-K) | **85.4%** | 84.1% |
+| Human-labelled test set (56, 10-K) | **83.9%** | 82.1% |
+
+The two human-labelled sets moved by one or two items, which is within noise: the new data neither helped nor hurt the original news and 10-K tasks. One clear regression: **EDT clinical-trial recall fell from 87.1% to 54.8%** (precision held at 79.1%); missed trials mostly go to `none`.
 
 ## Model details
 
 - **Model type:** decision model (single-pass scoring of a fixed set of options), built on the Qwen3 causal language model (`Qwen3ForCausalLM`, 8B parameters)
 - **Base model:** [Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B), fine-tuned with LoRA (rank 32) and a decision head
 - **Language:** English
-- **Domain:** financial news and SEC Form 10-K filings, for wealth and asset management
-- **License:** Apache-2.0, the same as Qwen3-8B
+- **Domain:** financial news, headlines, SEC Form 10-K filings and central-bank communication, for wealth and asset management
 - **Precision:** fp16 (the fine-tuning is merged into the weights)
 - **Method:** follows "Releasing the Inner Jev" (Geuter et al., 2026)
 
 ### How it works
 
-You give Arbiter a text, a question and a set of options, each with a short definition. It scores every option in one forward pass and returns the probabilities, calibrated per question type with the temperatures in `calibration.json`. It never generates text.
-
-### Versions in this repository
+You give Arbiter a text, a question and a set of options, each with a short definition. It scores every option in one forward pass and returns the probabilities, calibrated per question type with the temperatures in `calibration.json`.
 
-- **Full model** (default): the weights at the top level load with `transformers`, with no separate download of the base model.
-- **4-bit adapter** (`adapter/`): the unmerged LoRA for Qwen/Qwen3-8B, as trained with 4-bit QLoRA. It needs about 6.5 GB of GPU memory.
 
-The full model was built by merging the LoRA into the dequantized 4-bit base weights it was trained against, saved in fp16. It reproduces the trained model: it gives the same answer on 730 of 731 evaluation decisions, with probabilities within 0.004.
+For llama.cpp, Ollama and LM Studio, use [ternary/Arbiter-8B-GGUF](https://huggingface.co/ternary/Arbiter-8B-GGUF).
 
 ## Quick start
 
@@ -54,7 +87,7 @@ cd Arbiter-8B
 
 ```python
 from arbiter import Arbiter
-from questions import OUTLOOK
+from questions import OUTLOOK, EDT_TIGHT, FOMC_ORIGINAL
 
 arbiter = Arbiter.from_pretrained(".")                    # full model (fp16)
 # arbiter = Arbiter.from_pretrained(".", adapter=True)    # LoRA on Qwen/Qwen3-8B in 4-bit, as trained
@@ -62,9 +95,12 @@ arbiter = Arbiter.from_pretrained(".")                    # full model (fp16)
 passage = ("Revenue increased 15% to $8,449 million, reflecting higher shipments and higher "
            "revenue per ton. Cost of sales increased 14%.")
 answer = arbiter.decide(state=passage, question=OUTLOOK, focus="Constellium")
-
 print(answer["choice"])          # e.g. "improved"
 print(answer["probabilities"])   # {"improved": ..., "worsened": ..., "unchanged": ..., "unclear": ...}
+
+answer = arbiter.decide(state="The Committee judges that inflation remains elevated.",
+                        question=FOMC_ORIGINAL, focus="Federal Reserve (FOMC)")
+print(answer["choice"])          # e.g. "hawkish"
 ```
 
 `decide()` returns a typed answer: `{"type": "choice", "choice": ..., "probabilities": {...}}` for choice questions, `{"type": "noul", "noul": P(true), ...}` for yes/no questions, and `{"type": "score", "score": level, "probabilities": {...}}` for ordered scales. If a state is too long, it is shortened from the end (never the options), and the answer reports `state_truncated: true`. `arbiter.fallback_counts` counts every fallback taken.
@@ -85,55 +121,67 @@ Import the exact definitions from `questions.py`; Arbiter is most reliable with
 | Constant | Type | Options |
 |---|---|---|
 | `EVENT_TYPE` | choice | corporate_action, financial_results, company_guidance, product_operations, analyst_action, analyst_opinion, market_move, legal_safety, industry_macro, insufficient_evidence |
-| `OUTLOOK` | choice | improved, worsened, unchanged, unclear (conflicting signals → the stronger, more material signal; `unclear` = no outlook signal at all) |
+| `OUTLOOK` | choice | improved, worsened, unchanged, unclear |
 | `CERTAINTY` | score | 0 rumoured … 3 realised |
 | `MAIN_SUBJECT` | yes/no | is the company the main subject of the event? |
 | `ROLE` | choice | acquirer, target, regulator, mentioned_only |
 | `FILING_RISK` | choice | ten risk categories for an Item 1A passage |
 | `FILING_LEGAL` | score | 0 possible … 3 resolved |
 | `GOING_CONCERN`, `CUSTOMER_CONCENTRATION`, `CYBER_INCIDENT` | yes/no | 10-K flags |
+| `EDT_ORIGINAL`, `EDT_TIGHT` (new) | choice | acquisition, clinical_trial, regular_dividend, dividend_cut, dividend_increase, guidance_change, new_contract, reverse_stock_split, special_dividend, stock_repurchase, stock_split, none |
+| `FOMC_ORIGINAL`, `FOMC_TIGHT` (new) | choice | hawkish, neutral, dovish |
+| `SENTIMENT_ORIGINAL`, `SENTIMENT_IMPLIED`, `SENTIMENT_STATED` (new) | choice | positive, neutral, negative |
+
+The `_TIGHT`, `_IMPLIED` and `_STATED` variants spell out a labelling convention in the definitions, for example that a partnership is not a new contract. v0.5 was trained on both wordings of each question so that it follows whichever definition it is given; tighter definitions usually score higher (SEntFiN, F16 GGUF: 81.9% tightened vs 76.8% original).
 
 You can pass your own question in the same shape (`{"type", "instructions", "criteria"}`, at most 16 options). The base model's general decision ability carries over, but quality on questions outside this list has not been measured.
 
 ## Evaluation
 
-All results are on companies held out from training. Labels and limits are described under each table.
+### Public benchmarks
 
-**2,000 news decisions** (stratified estimate). Labels come from blind adjudication by Claude Opus 5.5, which saw no system's answer, of all 323 disagreements between Arbiter v0.2 and Jev 1.14 plus 201 randomly sampled agreements. The judge applied the project's labelling conventions; on the 207 human-labelled decisions it agreed with the human label 84.5% of the time (90.3% on news).
+| | SEntFiN (14,329) | EDT (1,933) | FOMC (496) |
+|---|---|---|---|
+| **Arbiter v0.5, full model (this repository)** | 76.4% | **94.2%** | **67.5%** |
+| Arbiter v0.5, F16 GGUF | 76.8% | 93.7% | 66.3% |
+| Arbiter v0.5, Q4_K_M GGUF (imatrix) | **77.4%** | 93.6% | 67.3% |
+| Arbiter v0.4, full model | 76.0% | 83.5% | 60.1% |
+| Qwen3-8B, F16 GGUF | 75.0% | 89.3% | 55.0% |
+| Majority class | 38.1% | 77.2% | 49.8% |
 
-| | Accuracy (95% CI) | Test companies | Dev companies | Calibration (100 = perfect) |
-|---|---|---|---|---|
-| **Arbiter v0.4 (this full model)** | **85.8% ± 3.3** | **89.4%** | **82.8%** | **98.2** |
-| Arbiter v0.4 (4-bit adapter) | 85.7% ± 3.3 | 89.3% | 82.8% | 98.2 |
-| Qwen3.8-27B (teacher) | 85.6% ± 3.4 | 89.2% | 82.6% | 95.1 |
-| Arbiter v0.3 | 83.2% ± 3.5 | 85.3% | 81.4% | 98.0 |
-| Jev 1.14 (TypeSafe API) | 79.3% ± 3.5 | 79.9% | 78.8% | 93.8 |
-
-Calibration is 100 × (1 − expected calibration error), 10 bins.
+95% confidence half-widths: SEntFiN ±0.7, EDT ±1.1, FOMC ±4.2 points. Original definitions, options in the published order. EDT and FOMC are scored on their held-out test splits. SEntFiN was never trained on. Paired McNemar tests against base Qwen3-8B on the same items: FOMC +11.3 points (z 4.9), EDT +4.4 (z 9.1), SEntFiN +1.9 (z 5.9), all p < 0.001.
 
-Per question on the same decisions: event type 87.0% (Jev 72.8%), outlook 84.8% (85.6%), certainty 74.4% (79.3%), main subject 92.1% (87.7%), role 95.2% (85.5%; few decisions).
+**Calibration.** Expected calibration error of the full model: SEntFiN 0.078, EDT 0.012, FOMC 0.067. When Arbiter v0.5 is more than 99% confident it is right 91.5–92.6% of the time; base Qwen3-8B gives such answers on 85–98% of items and is right on 59.9% (FOMC) to 90.2% (EDT) of them.
 
-Part of the lead over Jev comes from the labelling conventions, such as "earnings previews are analyst_opinion", which Jev never sees. When event-type disagreements were re-judged on the option definitions alone, Arbiter v0.2's lead over Jev fell from 4.0 to 1.4 points (not significant). This ablation was not repeated for v0.4.
+**Option order.** Share of answers that change when the options are shuffled: FOMC 4.8%, SEntFiN 6.3%, EDT 1.0%.
 
-**Human-labelled decisions** (single human adjudicator):
+### Human-labelled decisions
 
 | | Dev, 151 (news + 10-K) | Test, 56 (10-K) |
 |---|---|---|
-| **Arbiter v0.4** (full model and adapter alike) | **85.4%** | **83.9%** |
+| **Arbiter v0.5** | 84.1% | 82.1% |
+| Arbiter v0.4 | 85.4% | 83.9% |
 | Jev 1.14 | 84.8% | 82.1% |
 | Qwen3-8B zero-shot | 74.2% | 76.8% |
 
-These sets are small: a difference of one or two points is within noise. Calibration temperatures were fitted on the dev set.
+Companies held out from training, labelled by a single human adjudicator. These sets are small: a difference of one or two points is within noise. Calibration temperatures were fitted on the dev set.
 
 ## Training
 
-- **Data:** about 120k decisions. 100,000 news decisions, 11,752 SEC 10-K decisions and 8,000 news-outlook decisions were labelled by a local Qwen3.8-27B (one-pass letter readout, soft probability targets). The rest are 106 human-reviewed 10-K decisions. No human-labelled news rows; no output of Jev or of any Claude model was used for training. The training data is not released: the news text is licensed and not redistributable.
-- **Recipe:** 4-bit QLoRA (NF4, double quantization), rank 32, alpha 64, on all attention and MLP projections, trained jointly with the head. 2,000 steps × 16 decisions, sampled per question so that weaker questions (certainty, outlook) get more draws. Options are shuffled on every presentation. Cross-entropy against the teacher's soft targets. Details in `metadata.json`.
+- **Data:** 140,705 training decisions.
+  - 100,000 news, 11,752 SEC 10-K, 8,000 news-outlook and 10,300 headline-sentiment decisions labelled by a local Qwen3.8-27B teacher (one-pass letter readout, soft probability targets);
+  - 12,869 corporate-event and policy-stance decisions with published labels, under original and tightened definitions, plus same-article contrast pairs and hard `none` examples;
+  - 106 human-reviewed 10-K decisions.
+  - SEntFiN was never trained on, and no benchmark test item was trained on. No output of Jev, OpenAI or any Claude model was used for training. The training data is not released: the news article text is licensed and not redistributable.
+- **Recipe:** 4-bit QLoRA (NF4, double quantization), rank 32, alpha 64, on all attention and MLP projections, trained jointly with the decision head. 2,400 steps × 16 decisions, sampled per question family. Options are shuffled on every presentation. Cross-entropy against soft targets, with label smoothing 0.05 on hard labels. Details in `metadata.json`.
 
 ## Limitations
 
-- English, US companies, financial news and 10-K text only. Not investment advice; use it as decision support with human review.
-- Weakest on **certainty** (how settled an event is): there it trails both its teacher and Jev, and the teacher is the ceiling.
+- English, mostly US companies. Not investment advice; use it as decision support with human review.
+- **EDT clinical trials:** recall fell to 54.8% in v0.5. Dividend cuts are found only 47.6% of the time under the original definition (95.2% with `EDT_TIGHT`).
+- **SEntFiN neutral:** Arbiter under-calls neutral headlines (neutral recall 52.0%), so it raises more positive/negative signals than the labels contain.
+- **FOMC dovish:** dovish recall is 41.5%; Arbiter leans hawkish or neutral.
+- **Certainty** (how settled an event is) remains the weakest of the original news questions.
 - Labels follow this project's definitions and conventions. A desk with other conventions should re-evaluate on its own labelled examples before relying on the probabilities.
 - The model sees only the supplied text; it has no knowledge of later events.
 
@@ -142,9 +190,13 @@ These sets are small: a difference of one or two points is within noise. Calibra
 | File | Purpose |
 |---|---|
 | `model-*.safetensors`, `config.json`, tokenizer files, `chat_template.jinja` | full model: Qwen3-8B with the LoRA merged in, fp16 |
-| `adapter/` | the unmerged LoRA for Qwen/Qwen3-8B (4-bit use), with its own `calibration.json` |
+| `adapter/` | the unmerged LoRA for Qwen/Qwen3-8B (4-bit use), with its `calibration.json` |
 | `decision_head.safetensors`, `head_config.json` | decision head weights and the letter token ids it expects |
-| `calibration.json` | per-type temperatures for the full model, fitted on the dev split |
+| `calibration.json` | per-type temperatures, fitted on the dev split |
 | `arbiter.py`, `model.py` | loading and inference code |
 | `questions.py` | the exact question definitions used in training |
 | `metadata.json` | training settings and counts |
+
+## License
+
+Apache-2.0, like the base model Qwen3-8B.

adapter/adapter_config.json

@@ -37,11 +37,11 @@
   "target_modules": [
     "up_proj",
     "down_proj",
-    "o_proj",
+    "q_proj",
     "v_proj",
     "k_proj",
-    "q_proj",
-    "gate_proj"
+    "gate_proj",
+    "o_proj"
   ],
   "target_parameters": null,
   "task_type": null,

adapter/adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:863d4b581ad5a25c19121980e68c30cfc7957a9471c26e5b42b8fce1ff6110d2
+oid sha256:8f1a1e236537bb3d9b28d0f3fc485b46f5fbeae039cf317fd449896201a51a10
 size 349240728

adapter/calibration.json

@@ -1,9 +1,9 @@
 {
   "split": "dev",
   "temperatures": {
-    "choice": 1.087543781730068,
+    "choice": 1.056737703565464,
     "noul": 1.0,
-    "score": 1.0
+    "score": 1.0000000000000013
   },
   "counts": {
     "choice": 83,
@@ -11,14 +11,14 @@
     "score": 29
   },
   "nll_raw": {
-    "choice": 0.5404467355611896,
-    "noul": 0.1435942221737603,
-    "score": 0.4154224320345729
+    "choice": 0.5215100010627519,
+    "noul": 0.1584453682220628,
+    "score": 0.4403726302605306
   },
   "nll_calibrated": {
-    "choice": 0.5377780521547135,
-    "noul": 0.1435942221737603,
-    "score": 0.4154224320345729
+    "choice": 0.5202457346238334,
+    "noul": 0.1584453682220628,
+    "score": 0.4403726302605305
   },
   "fallback_types_without_data": []
 }

calibration.json

@@ -1,9 +1,9 @@
 {
   "split": "dev",
   "temperatures": {
-    "choice": 1.087248976537121,
+    "choice": 1.056737703565464,
     "noul": 1.0,
-    "score": 1.0
+    "score": 1.0000000000000013
   },
   "counts": {
     "choice": 83,
@@ -11,14 +11,14 @@
     "score": 29
   },
   "nll_raw": {
-    "choice": 0.5398788178387731,
-    "noul": 0.14350210398196886,
-    "score": 0.41513550974218305
+    "choice": 0.5215100010627519,
+    "noul": 0.1584453682220628,
+    "score": 0.4403726302605306
   },
   "nll_calibrated": {
-    "choice": 0.5372287788193248,
-    "noul": 0.14350210398196886,
-    "score": 0.41513550974218305
+    "choice": 0.5202457346238334,
+    "noul": 0.1584453682220628,
+    "score": 0.4403726302605305
   },
   "fallback_types_without_data": []
 }

decision_head.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f2f2ea1e8b3cde4fbc92553e1ff93ffa17714df26018b5a1953a153f5b3e2a62
+oid sha256:06b4559ea3feeff270fda81d9db822f2282507d834a51723a06976750d81392e
 size 557632

metadata.json

@@ -3,22 +3,21 @@
  "quantization": "4bit",
  "max_length": 4096,
  "softcap": null,
- "steps": 2000,
- "train_rows": 119858,
- "dev_zero_shot": {
-  "accuracy": 0.7218543046357616,
-  "nll": 3.117936041840196,
-  "count": 151
- },
- "dev_final": {
-  "accuracy": 0.8543046357615894,
-  "nll": 0.41393711996655635,
-  "count": 151
- },
  "recipe": {
+  "data": [
+   "pilot_decisions.jsonl",
+   "sentinel_decisions.jsonl",
+   "filing_decisions.jsonl",
+   "teacher_decisions.jsonl",
+   "filing_teacher_decisions.jsonl",
+   "teacher_outlook_decisions.jsonl",
+   "v5_edt_decisions.jsonl",
+   "v5_fomc_decisions.jsonl",
+   "v5_sentiment_decisions.jsonl"
+  ],
   "base_model": "Qwen/Qwen3-8B",
   "include_teacher": true,
-  "mix": "news:type=0.22,news:certainty=0.148,news:outlook=0.12,news:main=0.08,news:role=0.03,filing:risk=0.12,filing:outlook=0.09,filing:legal=0.08,filing:going_concern=0.012,filing:customer=0.02,filing:cyber=0.03,human=0.04",
+  "mix": "news:type=0.150,news:certainty=0.100,news:outlook=0.082,news:main=0.054,news:role=0.020,filing:risk=0.082,filing:outlook=0.061,filing:legal=0.054,filing:going_concern=0.008,filing:customer=0.014,filing:cyber=0.020,human=0.040,news:edt_orig=0.045,news:edt_tight=0.045,news:edt_contrast=0.030,news:edt_hardnone=0.030,news:fomc_orig=0.030,news:fomc_tight=0.030,news:senti_single_implied=0.025,news:senti_single_stated=0.030,news:senti_multi_implied=0.025,news:senti_multi_stated=0.025",
   "mix_by": "question",
   "quantization": "4bit",
   "minimum_train": 100,
@@ -30,17 +29,88 @@
   "micro_batch": 1,
   "grad_accum": 16,
   "epochs": 3.0,
-  "steps": 2000,
+  "steps": 2400,
   "max_length": 4096,
   "max_grad_norm": 1.0,
   "stats_rows": 512,
   "log_every": 5,
-  "eval_every": 250,
-  "seed": 1
+  "eval_every": 400,
+  "seed": 1,
+  "label_smoothing": 0.05
+ },
+ "steps": 2400,
+ "train_rows": 140705,
+ "dev_rows": 151,
+ "dev_zero_shot": {
+  "accuracy": 0.7218543046357616,
+  "nll": 3.117936041840196,
+  "count": 151
+ },
+ "dev_final": {
+  "accuracy": 0.8410596026490066,
+  "nll": 0.4121556690861687,
+  "count": 151
+ },
+ "fallback_counts": {
+  "excluded_train_provisional": 401,
+  "train_teacher_rows_used": 140599,
+  "excluded_dev_provisional": 13,
+  "excluded_dev_rejected": 1,
+  "excluded_dev_teacher": 2322,
+  "float16_no_bf16_device": 1,
+  "flash_sdp_disabled_pre_ampere_gpu": 1,
+  "dev_state_truncated": 0,
+  "mix_news:type_passes_started": 1,
+  "mix_news:type_draws": 5733,
+  "soft_target_presentations": 29233,
+  "mix_news:fomc_tight_passes_started": 1,
+  "mix_news:fomc_tight_draws": 1140,
+  "label_smoothed_presentations": 9167,
+  "mix_news:edt_orig_passes_started": 1,
+  "mix_news:edt_orig_draws": 1771,
+  "mix_filing:risk_passes_started": 1,
+  "mix_filing:risk_draws": 3211,
+  "mix_news:outlook_passes_started": 1,
+  "mix_news:outlook_draws": 3199,
+  "mix_filing:outlook_passes_started": 1,
+  "mix_filing:outlook_draws": 2358,
+  "mix_news:certainty_passes_started": 1,
+  "mix_news:certainty_draws": 3879,
+  "mix_news:fomc_orig_passes_started": 1,
+  "mix_news:fomc_orig_draws": 1186,
+  "mix_news:edt_contrast_passes_started": 2,
+  "mix_news:edt_contrast_draws": 1123,
+  "mix_news:senti_single_implied_passes_started": 1,
+  "mix_news:senti_single_implied_draws": 941,
+  "mix_filing:legal_passes_started": 2,
+  "mix_filing:legal_draws": 2049,
+  "mix_news:main_passes_started": 1,
+  "mix_news:main_draws": 1995,
+  "mix_news:edt_tight_passes_started": 1,
+  "mix_news:edt_tight_draws": 1754,
+  "mix_filing:cyber_passes_started": 1,
+  "mix_filing:cyber_draws": 706,
+  "mix_news:senti_single_stated_passes_started": 1,
+  "mix_news:senti_single_stated_draws": 1138,
+  "mix_human_passes_started": 15,
+  "mix_human_draws": 1511,
+  "mix_news:senti_multi_stated_passes_started": 1,
+  "mix_news:senti_multi_stated_draws": 935,
+  "mix_news:role_passes_started": 1,
+  "mix_news:role_draws": 741,
+  "mix_news:senti_multi_implied_passes_started": 1,
+  "mix_news:senti_multi_implied_draws": 970,
+  "mix_filing:customer_passes_started": 2,
+  "mix_filing:customer_draws": 532,
+  "mix_filing:going_concern_passes_started": 2,
+  "mix_filing:going_concern_draws": 306,
+  "mix_news:edt_hardnone_passes_started": 1,
+  "mix_news:edt_hardnone_draws": 1222
  },
- "training_data": "100,000 news + 11,752 SEC 10-K + 8,000 news-outlook decisions labelled by a local Qwen3.8-27B teacher, plus 106 human-reviewed 10-K decisions",
+ "version": "Arbiter v0.5",
+ "training_data": "100,000 news + 11,752 SEC 10-K + 8,000 news-outlook decisions labelled by a local Qwen3.8-27B teacher; 10,300 headline-sentiment decisions labelled by the same teacher; 9,033 EDT and 3,836 FOMC training-split decisions with their published labels (original and tightened definitions, contrast pairs, hard 'none' examples); 106 human-reviewed 10-K decisions. SEntFiN was never trained on.",
  "layout": {
-  "full_model": "root (LoRA merged into Qwen3-8B, bf16 safetensors)",
+  "full_model": "root (LoRA merged into Qwen3-8B with its decoder weights round-tripped through NF4, as in training; fp16 safetensors)",
   "adapter": "adapter/ (LoRA for Qwen/Qwen3-8B, trained with 4-bit QLoRA)"
  }
 }

model-00001-of-00005.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d3ddd3e4d3c9921fe1cb4fb42d91752935888ebd577e1025374896fe35182206
+oid sha256:e01f1b2636864c70e18b02b91fb23af6ad72df7b1bfbc838e4a37bfed84a878b
 size 3990952856

model-00002-of-00005.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8c0722450e49dfe88b3dcf0993b0dbcb0732c3b5853ab1b3d4a56356ce6ebd9f
+oid sha256:7dcaf015c6807df8379d8b816a6f9c7b10129ebf5a0469faf230f5df268ec364
 size 3900892760

model-00003-of-00005.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2c688c5443a38aad06b0b4d3d59dce9256ad6802df29dacb785d080bc74ce951
+oid sha256:58d204278b49e04212ba28eabcf96ccf146cd2f2444f7e9d8168f681fa09fc93
 size 3959604680

model-00004-of-00005.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5ed8790155fea4f0995eac3679771b8b7cf80734877d88a3429fc2fcd2fa8b37
+oid sha256:73f2d980802617a76efae9329906b8dcaa95f4050784282d6b57279489a8dde9
 size 3959604672

model-00005-of-00005.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f27dcf543b31a708884254a4569b333d8dd4fffb3cd32e9629fada4271f7861e
+oid sha256:7219e7a9592bd7fbe6b178e74e4ec80be1c14a1e22ca23b522c82dc9a1250bea
 size 570461408

questions.py

@@ -153,3 +153,100 @@ FILING_QUESTIONS = {
     "customer": ("filing_flags", CUSTOMER_CONCENTRATION),
     "cyber": ("filing_flags", CYBER_INCIDENT),
 }
+
+# --- v5 (next-stage.md): definition-following data ---------------------------------------------------
+# The *_ORIGINAL questions are word for word the ones in the Arbiter-Test benchmark converters
+# (edt_to_jsonl.py, fomc_to_jsonl.py, sentfin_to_jsonl.py); the *_TIGHT ones are the rewritten
+# definitions that state each benchmark's labelling convention. Benchmarks are scored under both;
+# the gap shows whether the model follows the definition it is given. Option ids never change.
+
+EDT_ENTITY = "the company that issued the news article"
+EDT_INSTRUCTIONS = f"What type of corporate event does the news article announce? Focus entity: {EDT_ENTITY}."
+EDT_ORIGINAL = {
+    "type": "choice",
+    "instructions": EDT_INSTRUCTIONS,
+    "criteria": {
+        "acquisition": "the company agrees to acquire, or completes the acquisition of, another company, business or asset",
+        "clinical_trial": "the company reports the start, progress or results of a clinical trial",
+        "regular_dividend": "the company declares its regular, unchanged dividend",
+        "dividend_cut": "the company cuts, suspends or eliminates its dividend",
+        "dividend_increase": "the company raises its dividend",
+        "guidance_change": "the company raises or changes its financial guidance or outlook",
+        "new_contract": "the company wins or signs a new contract, order or customer agreement",
+        "reverse_stock_split": "the company announces a reverse stock split",
+        "special_dividend": "the company declares a special or one-time dividend",
+        "stock_repurchase": "the company announces or expands a share buyback programme",
+        "stock_split": "the company announces a (forward) stock split",
+        "none": "the article announces none of the events above",
+    },
+}
+EDT_TIGHT = {
+    "type": "choice",
+    "instructions": EDT_INSTRUCTIONS,
+    "criteria": {
+        **EDT_ORIGINAL["criteria"],
+        "acquisition": "the company agrees to acquire, or completes the acquisition of, another company, business or asset; minority investments, financings, joint ventures and sales of a business or asset (divestitures) are not acquisitions",
+        "clinical_trial": "the company reports the start, progress or results of a clinical trial, or a regulatory approval, clearance or authorisation of a drug, device or test",
+        "dividend_cut": "the company cuts, suspends or eliminates its dividend or another shareholder payout, including suspending a share buyback",
+        "new_contract": "the company wins or signs a new contract, order or customer agreement; partnerships, collaborations, sponsorships, and distribution or licensing agreements are not new contracts",
+        "stock_repurchase": "the company announces or expands a share buyback programme (suspending one is a dividend_cut)",
+        "none": "the article announces none of the events above, for example a partnership, sponsorship, minority investment, financing, divestiture, patent award, product launch or appointment",
+    },
+}
+# Contrast definitions: one option changed so the same article's answer changes.
+EDT_STRICT_TRIAL = {**EDT_ORIGINAL, "criteria": {
+    **EDT_ORIGINAL["criteria"],
+    "clinical_trial": "the company reports the start, progress or results of a clinical trial; a regulatory approval, clearance or authorisation is not a clinical-trial event",
+}}
+EDT_INCLUSIVE_CONTRACT = {**EDT_ORIGINAL, "criteria": {
+    **EDT_ORIGINAL["criteria"],
+    "new_contract": "the company wins or signs a new contract, order or customer agreement, including a partnership, collaboration, sponsorship, alliance, or distribution or licensing agreement",
+}}
+
+FOMC_INSTRUCTIONS = "What monetary policy stance does the sentence express? Focus entity: Federal Reserve (FOMC)."
+FOMC_ORIGINAL = {
+    "type": "choice",
+    "instructions": FOMC_INSTRUCTIONS,
+    "criteria": {
+        "hawkish": "points towards tighter monetary policy, such as higher rates or concern about inflation",
+        "neutral": "no clear lean towards tighter or looser policy",
+        "dovish": "points towards looser monetary policy, such as lower rates or concern about growth and employment",
+    },
+}
+FOMC_TIGHT = {
+    "type": "choice",
+    "instructions": FOMC_INSTRUCTIONS,
+    "criteria": {
+        "hawkish": "points towards tighter monetary policy: higher rates or less accommodation, inflation described as high, rising or a risk, or growth and employment described as strong or overheating",
+        "neutral": "no lean towards tighter or looser policy: a procedural or factual statement, or a mention of inflation or the economy that describes neither upward nor downward pressure",
+        "dovish": "points towards looser monetary policy: lower rates or more accommodation, inflation described as subdued, muted, low or below target, or growth and employment described as weak or at risk",
+    },
+}
+
+SENTIMENT_ORIGINAL = {
+    "type": "choice",
+    "instructions": "What sentiment toward the company does the headline express?",
+    "criteria": {
+        "positive": "the headline is favourable for the entity",
+        "neutral": "the headline is neither favourable nor unfavourable for the entity",
+        "negative": "the headline is unfavourable for the entity",
+    },
+}
+SENTIMENT_IMPLIED = {
+    "type": "choice",
+    "instructions": "What do the facts in the headline imply for the focus entity? Judge what is reported, not only the wording: a profit, an approval, a new order or a dividend is favourable even when the headline states no direction.",
+    "criteria": {
+        "positive": "the reported facts are favourable for the entity",
+        "neutral": "the reported facts imply nothing favourable or unfavourable for the entity",
+        "negative": "the reported facts are unfavourable for the entity",
+    },
+}
+SENTIMENT_STATED = {
+    "type": "choice",
+    "instructions": "What sentiment toward the focus entity does the headline itself state? Count only a direction the headline expresses for that entity, such as a rise or fall, a beat or miss, a comparison with an earlier period, an upgrade or downgrade, a win or loss, praise or criticism.",
+    "criteria": {
+        "positive": "the headline states a favourable direction for the entity",
+        "neutral": "the headline states no direction for the entity, for example a factual report of a corporate action or a figure without a comparison: a profit figure, a dividend or bonus declaration, an appointment, an IPO filing or a regulatory approval",
+        "negative": "the headline states an unfavourable direction for the entity",
+    },
+}