BHDAI: same-event matcher for Turkish news

BHDAI («BültenHub Doğrulama AI») estimates whether a written story reports a different event than its source. In BültenHub, stories below the threshold (0.1) pass without an extra verification call; the rest are checked by a larger model. It is retrained every night and replaces the production version only if it passes the same exam.

Version

Version	bhdai-20261112-c03
Trained	2026-11-12
Method	logistic regression on 10 deterministic overlap features (kos_baslik, kos_baslik_tum, kos_tum, ozel_oran, ozel_az, kaynak_baslik_kapsam, metin_baslik_dayanak, sayi_oran, sayi_yok, uzunluk_orani)
Training data	120,000 stories, 300,064 pairs (window 21 days, C 0.3)
Replaced	bhdai-20261111

Exam (the metrics the nightly gate uses)

Last 2 days, 7,096 held-out stories. Same-event passed: share of correct (source, story) pairs that pass without a call (higher is better). Hard negative: a story about a different event from the same category and day, the typical small-model mistake (lower is better).

Threshold	Same-event passed	Hard negative passed	Random negative passed	Previous: same-event	Previous: hard negative
0.05	0.509	0.013	0.000	0.514	0.013
0.1 (production)	0.692	0.031	0.000	0.697	0.032
0.2	0.838	0.059	0.000	0.842	0.061
0.3	0.898	0.080	0.000	0.901	0.082

Gate decision: promoted — at threshold 0.1: same-event pairs passed without a call 69.2% (was 69.7%), hard negatives wrongly passed 3.14% (was 3.22%).

Usage (Türkçe örnek)

# pip install huggingface_hub numpy
import sys
from huggingface_hub import snapshot_download

yol = snapshot_download("BultenHub/bhdai")
sys.path.insert(0, yol)
from bhdai import Bhdai

model = Bhdai(yol)
kaynak = ("Kurgukent'te iki mahallede perşembe su kesintisi",
          "Kurgukent Belediyesi'ne göre perşembe 09.00-17.00 arasında Çınarlı ve Yeşiltepe'de su verilmeyecek.")
ayni = model.farkli_olasiligi(*kaynak, "Çınarlı ve Yeşiltepe'de perşembe su yok",
                              "Kurgukent'te Çınarlı ve Yeşiltepe mahallelerinde perşembe 09.00-17.00 su kesilecek.")
farkli = model.farkli_olasiligi(*kaynak, "Kurgukent'te kütüphane açıldı",
                                "Eski itfaiye binası 24 saat açık kent kütüphanesine dönüştürüldü.")
print(round(ayni, 3), round(farkli, 3))   # düşük = aynı olay

Files: agirlik.npz (weights and feature scaling), sozluk.json (vocabulary), bilgi.json (version, exam), bhdai.py (numpy inference).

Limitations

• A fast pre-filter, not a fact checker: it measures overlap of names, numbers and words, not truth.
• Paraphrases with few shared names or numbers can look like a different event (they then get the full check).
• Trained on Turkish news only.

Changelog

• 2026-11-12 bhdai-20261112-c03: at threshold 0.1: same-event pairs passed without a call 69.2% (was 69.7%), hard negatives wrongly passed 3.14% (was 3.22%)
• 2026-11-11 bhdai-20261111: at threshold 0.1: same-event pairs passed without a call 69.1% (was 68.9%), hard negatives wrongly passed 2.61% (was 2.61%)
• 2026-11-10 bhdai-20261110-adet120k: at threshold 0.1: same-event pairs passed without a call 67.0% (was 68.5%), hard negatives wrongly passed 2.39% (was 2.60%)
• 2026-11-05 bhdai-20261105: at threshold 0.1: same-event pairs passed without a call 67.0% (was 67.9%), hard negatives wrongly passed 2.59% (was 2.73%)
• 2026-11-04 bhdai-20261104: at threshold 0.1: same-event pairs passed without a call 70.0% (was 69.5%), hard negatives wrongly passed 2.69% (was 2.69%)
• 2026-11-03 bhdai-20261103: at threshold 0.1: same-event pairs passed without a call 68.6% (was 68.0%), hard negatives wrongly passed 2.90% (was 2.94%)

License

CC BY-NC 4.0 (non-commercial). The base model, if any, keeps its own license. For commercial use, contact https://bultenhub.com.