Beinex AI

Beinex is a data, analytics and AI company headquartered in Dubai. We work with finance, risk and audit teams, and we build our own models for the problems we see every day.

Noah: small, specialised models

Noah models are small enough to run on your own hardware, explain their answers, and say when they are unsure.

Latest results (2026-10-12)

Nutrient grounding benchmark (en, 5,000 rows), scored with the benchmark's own, unmodified score.py. The leaderboard maintainers pulled the public repo and reran the scorer on their hardware before merging: 0.8145 overall / 0.7041 number, identical to our run — the result is independently reproduced.

Nutrient grounding leaderboard: 6th of 15; first independent entry; above every permissively-licensed open model; open weights under CC BY-NC 4.0.

• Sources we did not train on (3,551 rows): Noah 0.819 overall · DeBERTa 0.840 (our re-run of the same scorer). Noah's lead comes from the source families whose training splits we used; on the others DeBERTa zero-shot is ahead.
• The full Noah 2.5, with its number-checking channel and chunked reading, scores 0.755 number / 0.800 overall in our own pipeline. The channel adds +0.065 number AUC in a same-data, same-seed ablation. It needs custom code, so the official scorer can't run it.
• Guardrails AI hallucination index data (BUMP, 1,386 news summaries; self-run, as the index has no submission process; report only, no ranking claim):
  • Our harness reproduces Vectara HHEM's published 0.632 exactly, as F1 with "faithful" as the positive class.
  • On that metric, Noah 2.5 scores 0.673 (95% CI 0.641–0.700).
  • A constant "everything is faithful" baseline scores 0.667 on it, above the published Microsoft (0.648) and Vectara (0.632) scores, so we make no ranking claim.
  • On threshold-free ROC-AUC, Noah scores 0.689 vs HHEM's 0.604.
• Misses we are working on:
  • LLM-AggreFact fell to 70.6 (our previous model: 72.7);
  • our stretch goal of beating gpt-6-luna was not met;
  • the calibration target was met only after post-hoc recalibration;
  • on sources we did not train on, DeBERTa zero-shot is still ahead (see above).
• Self-run, single training seed, except where independent reproduction is stated.

How we work

• Pre-registered evaluation. We write the pass/fail bars before training and report misses as well as wins.
• Data provenance. We list every training source and licence. Three LLM-generated sources — WANLI (CC BY 4.0), MiniCheck (MIT), SummEdits (CC BY 4.0) — are under provenance review; the base model, ModernBERT-large, is Apache-2.0.
• Open where we can. The standard Noah 2.5 export has open weights for non-commercial use (CC BY-NC 4.0). The full models are research previews; contact us for evaluation or commercial access.

📫 beinex.ai