AI & ML interests

NLP, Responsible AI, Hallucination detection, AI verification

Recent Activity

AI-that-worksย  updated a dataset 13 days ago
groundlens/human-confabulation-benchmark
AI-that-worksย  updated a Space 13 days ago
groundlens/README
AI-that-worksย  updated a Space 13 days ago
groundlens/groundlens-mcp
View all activity

Organization Card

Groundlens

Deterministic grounding verification for LLM pipelines. The cheap first stage that decides what your expensive check has to look at, and says when it cannot decide.

What we do

We check LLM grounding with embedding geometry: is this answer drawn from its source, or did it disengage? It is the deterministic first stage of a two-stage pipeline. Cheap, reproducible, auditable, and bounded. When it cannot settle a case, it says so and hands off to a verifier: an entailment model, a source lookup, or a judge. We do not claim geometry replaces judgment. We claim it makes judgment affordable.

  • SGI (Semantic Grounding Index) โ€” measures whether a response actually used the source material it was given. Built for RAG pipeline verification. Use it whenever you have the source.
  • DGI (Directional Grounding Index) โ€” context-free. A coarse triage signal with a measured ceiling (see below). Use it only when there is no source.

Both run a single embedding call. No model in the scoring path.

The wall, and the ceiling

Bin confabulations by how far they sit from the register of a correct answer, and every distributional and embedding-similarity detector, ours included, declines toward chance as the answer moves into register: same vocabulary, same phrasing, one wrong fact. At the in-register end classic encoders reach AUROC 0.62 to 0.68 and raw cosine 0.595.

A detector that appears to beat this is usually reading who wrote the text, not whether it is grounded. Hold authorship constant and the skill collapses:

Detector Uncontrolled Authorship matched
Large instruction-tuned embedder โ‰ˆ 0.99 shortcut
Logistic probe 0.932 0.660
MLP 0.935 0.675
Directional score (DGI) high 0.606

With authorship matched, the best supervised decoder over these embeddings sits in the high 0.6s. That is the ceiling of the whole class, not our shortfall. Entailment does not decline: across the same bins it holds 0.836, 0.786, 0.837, 0.719, 0.887, and it is strongest exactly where geometry is weakest. Entailment is the recommended second stage.

We are retracting the earlier figures (0.90 to 0.99 with domain calibration, DGI 0.958, 87.8% detection, NLI at chance). All of them rested on evaluations where the grounded and the confabulated text had different authors.

Research

  1. Semantic Grounding Index (arXiv:2512.13771) โ€” ratio-based grounding verification for RAG systems.
  2. A Geometric Taxonomy of Hallucinations (arXiv:2602.13224) โ€” three-type hallucination classification with a confabulation benchmark.
  3. Rotational Dynamics of Factual Constraint Processing (arXiv:2603.13259) โ€” transformers reject wrong answers via rotation, not rescaling. Phase transition at 1.6B parameters.

Use groundlens

How What
Python library pip install groundlens โ€” GitHub ยท Docs
MCP server Remote: groundlens-mcp Space ยท local: pip install groundlens-mcp โ€” GitHub
REST API groundlens-api โ€” Swagger docs at /docs
Interactive demo demo โ€” geometry and entailment side by side, so you can see where geometry fails

Philosophy

groundlens is triage, not truth detection. It tells you which responses were drawn from their source and which need a closer look. It does not check facts.

We publish our AUROC numbers even when they are unflattering, and when a published number turns out to have measured the wrong thing, we withdraw it in public and say what it was measuring instead. We document what we cannot detect as a number, not as a footnote. The rule that killed our own numbers is the one we now ship: no benchmark figure without the authorship and length controls.

Links

models 0

None public yet