Blame for README.md

---
library_name: sklearn
license: mit
pipeline_tag: text-classification
tags:
- artefactual
- skops
- sklearn
- hallucination-detection
- uncertainty-estimation
---


# Model description

**Target model:** [`microsoft/phi-4`](https://huggingface.co/microsoft/phi-4) -- this detector scores responses produced by that model. It is not a fine-tune of it and contains none of its weights.

A calibrated EPR hallucination detector for responses generated by [`microsoft/phi-4`](https://huggingface.co/microsoft/phi-4).

**EPR (Entropy Production Rate)** pools every rank of the token distribution into a single number: the truncated entropy `-sum_k p_k ln p_k`, averaged over the response's tokens. One feature, one coefficient.

The artifact is the fitted `LogisticRegression` alone. The feature extraction that feeds it -- parsing top-`15` log-probabilities out of a completion response and reducing them to entropy features -- lives in the [artefactual](https://github.com/artefactory/artefactual) library, so this file contains no custom classes and loads with an empty `trusted` list.

Introduced in [Learned Hallucination Detection in Black-Box LLMs Using Token-Level Entropy Production Rate](https://doi.org/10.1007/978-3-032-21289-4_8) (ECIR 2026); the preprint is [arXiv:2509.04492](https://arxiv.org/abs/2509.04492).

## Intended uses & limitations

Scores a response on `[0, 1]`, where 1 is the hallucination class.

- **Tied to `microsoft/phi-4`.** The coefficients are fit against that model's output distribution. Scoring another model's responses with them is not meaningful, even though nothing in the file prevents it.
- **Fixed at k=15.** Responses must be generated with `logprobs=True` and `top_logprobs=15`. Fewer ranks are rejected rather than zero-filled, because the missing ranks are unfetched rather than absent and padding them would score the response as more confident than it was.
- **No published operating point.** The paper reports ROC-AUC and PR-AUC, both threshold-free, so no decision threshold is published. Choose one on your own labelled data.

## Evaluation Results

See [the paper](https://arxiv.org/abs/2509.04492). It reports ROC-AUC and PR-AUC across the evaluated models; no figures are restated here so that this card cannot drift from the published results.

# How to Get Started with the Model

```python
from artefactual.scoring import EPR

detector = EPR.from_pretrained("artefactory/epr-phi4")
scores = detector.predict_proba(response)[:, 1]
```

`response` is an OpenAI-compatible chat completion or responses payload carrying `top_logprobs=15`.

Requires `artefactual>=2026.9`, where the detector is the `EPR` class. Up to 2026.08.1 the same weights were loaded with the lowercase `epr()` factory.

# Model Card Authors

Artefact Research Center

# Model Card Contact

https://github.com/artefactory/artefactual/issues

# Citation

```bibtex
@inproceedings{moslonka2026learned,
  title     = {Learned Hallucination Detection in Black-Box LLMs Using Token-Level Entropy Production Rate},
  author    = {Moslonka, Charles and Randrianarivo, Hicham and Garnier, Arthur and Malherbe, Emmanuel},
  booktitle = {Advances in Information Retrieval},
  series    = {Lecture Notes in Computer Science},
  volume    = {16483},
  pages     = {115--130},
  publisher = {Springer, Cham},
  year      = {2026},
  doi       = {10.1007/978-3-032-21289-4_8},
}
```