Noah 2.5, standard export
Beinex AI · 2026-10-11
A plain ModernBertForSequenceClassification (no custom code) distilled from Noah 2.5, Beinex's research model (not public). It takes (premise, hypothesis) and returns P(supported) in label 0. Use it where only standard Transformers models are allowed (leaderboard scorers, inference servers). It has no number audit, no evidence head and no chunking; for those, use the full model.
Open weights, non-commercial (CC BY-NC 4.0). Free for non-commercial use; for commercial use, contact Beinex (beinex.ai). The full Noah 2.5, with the number audit, evidence and certified escalation, is available from Beinex.
Nutrient grounding leaderboard: 6th of 15; first independent entry; above every permissively-licensed open model; open weights under CC BY-NC 4.0.
Usage
from transformers import pipeline
clf = pipeline("text-classification", model="beinexai/noah-v2.5-research-std", top_k=None) # gated: accept the terms on the model page once, then log in (huggingface-cli login)
premise = "Item | FY2023 | FY2022\nNet sales | $4,213M | $3,987M"
out = clf(dict(text=premise, text_pair="Net sales were $4.2 billion in FY2023."))
p_supported = next(d["score"] for d in out if d["label"] == "supported")
Batch scoring, exactly as the Nutrient benchmark scorer does it:
import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification
tok = AutoTokenizer.from_pretrained("beinexai/noah-v2.5-research-std")
model = AutoModelForSequenceClassification.from_pretrained("beinexai/noah-v2.5-research-std").eval()
premises = ["Item | FY2023 | FY2022\nNet sales | $4,213M | $3,987M"]
hypotheses = ["Net sales were $4.2 billion in FY2023."]
enc = tok(premises, hypotheses, truncation=True, padding=True, max_length=1024, return_tensors="pt")
with torch.no_grad():
p_supported = model(**enc).logits.softmax(-1)[:, 0]
Results
Nutrient grounding benchmark (en, 5,000 rows), official unmodified score.py: 0.8145 overall / 0.7041 number. Independently reproduced: the leaderboard maintainers reran the scorer on their hardware before merging (12 Oct 2026) and obtained the same numbers. Everything else in this table is self-reported.
| Standard export | Full Noah 2.5 (own pipeline) | |
|---|---|---|
Nutrient (en) ROC-AUC overall, official score.py |
0.8145 | 0.800 (Noah-native scoring) |
| Nutrient number AUC | 0.7041 | 0.755 |
| Sources not in our training mix (3,551 rows), overall | 0.819 (DeBERTa-v3-large zero-shot: 0.840) | — |
| Nutrient table / prose / date / string AUC (official) | 0.803 / 0.908 / 0.965 / 0.846 | 0.787 / 0.922 / 0.935 / 0.754 |
| Agreement with the full model (held-out training items, decision at 0.5) | 0.966 | — |
| Correlation with the full model's calibrated score | 0.979 | — |
| Dev ECE | 0.022 | — |
How it was made. The encoder is copied from Noah 2.5, then the whole model is fine-tuned on 60,000 of Noah 2.5's own training items (filtered against the benchmark by exact hash and 20-word shingles), in the official input format, to match Noah 2.5's calibrated probability (soft targets) plus the gold label (weight 0.3). One epoch, lr 1e-05. The number channel cannot be carried into a standard model, so its gains are only partly retained.
Training data. Public NLI, fact-verification and table sources plus our own synthetic tables. Three LLM-generated sources — MiniCheck (MIT), WANLI (CC BY 4.0), SummEdits (CC BY 4.0) — are under provenance review; RAGTruth is MIT; the base model, answerdotai/ModernBERT-large, is Apache-2.0.
Licence: CC BY-NC 4.0: free for non-commercial use with attribution. Commercial licences: beinex.ai.
- Downloads last month
- 25
Model tree for beinexai/noah-v2.5-research-std
Base model
answerdotai/ModernBERT-largeSpace using beinexai/noah-v2.5-research-std 1
Evaluation results
- ROC-AUC (overall) on Nutrient grounding benchmark (en), official score.pytest set Beinex, with the benchmark's unmodified score.py; independently reproduced by the leaderboard maintainers (2026-10-12)0.815
- ROC-AUC (number) on Nutrient grounding benchmark (en), official score.pytest set Beinex, with the benchmark's unmodified score.py; independently reproduced by the leaderboard maintainers (2026-10-12)0.704