---
license: cc-by-sa-4.0
language:
- en
library_name: transformers
pipeline_tag: text-classification
base_model: answerdotai/ModernBERT-large
datasets:
- tals/vitaminc
- nyu-mll/multi_nli
- alisawuffles/WANLI
tags:
- natural-language-inference
- nli
- fact-verification
- fact-checking
- cross-encoder
- modernbert
- rag
- evidence
metrics:
- accuracy
model-index:
- name: ease-delta-reader
  results:
  - task:
      type: text-classification
      name: Fact verification
    dataset:
      name: VitaminC
      type: tals/vitaminc
      split: test
    metrics:
    - type: accuracy
      value: 91.53
      name: Accuracy
  - task:
      type: text-classification
      name: Natural language inference
    dataset:
      name: MultiNLI (mismatched)
      type: nyu-mll/multi_nli
      split: validation_mismatched
    metrics:
    - type: accuracy
      value: 90.21
      name: Accuracy
  - task:
      type: text-classification
      name: Natural language inference
    dataset:
      name: WANLI
      type: alisawuffles/WANLI
      split: test
    metrics:
    - type: accuracy
      value: 77.42
      name: Accuracy
  - task:
      type: text-classification
      name: Natural language inference
    dataset:
      name: SNLI
      type: stanfordnlp/snli
      split: test
    metrics:
    - type: accuracy
      value: 84.39
      name: Accuracy
---

# EASE-Delta Reader (large)

**Does this passage support the claim, refute it, or not settle it at all?**

A 395M-parameter cross-encoder (`answerdotai/ModernBERT-large` fine-tuned) that reads one claim against one passage and answers `SUPPORTS`, `REFUTES` or `NOT_ENOUGH_INFO`. It was trained to say `NOT_ENOUGH_INFO` when a passage is about something else, which is where general NLI models most often guess: on passages taken from unrelated documents it gives a decisive answer 0.58% of the time; a widely used public NLI model does so 45.07% of the time.

It is the reader inside [EASE-Delta](https://github.com/jithinsaireddy/ease-delta), a system that keeps a task's decisions current as its messages and documents change. Try both in the [demo](https://huggingface.co/spaces/jithinpothireddy21/ease-delta-demo).

## Use it

Claim first, evidence second.

```python
from transformers import pipeline

reader = pipeline("text-classification", model="jithinpothireddy21/ease-delta-reader", top_k=None)
claim = "The client has approved the final design."
passages = [
    "Email from the client: we approve the final design, please go ahead.",
    "Email from the client: we cannot approve the design yet; the colours are wrong.",
    "The office is closed on the first Monday of the month for staff training.",
]
for passage in passages:
    print(reader({"text": claim, "text_pair": passage}, truncation=True)[0])

# {'label': 'SUPPORTS', 'score': 0.935}
# {'label': 'REFUTES', 'score': 0.991}
# {'label': 'NOT_ENOUGH_INFO', 'score': 0.999}
```

With sentence-transformers, continuing from above:

```python
from sentence_transformers import CrossEncoder

model = CrossEncoder("jithinpothireddy21/ease-delta-reader")
probs = model.predict([(claim, passage) for passage in passages], apply_softmax=True)
# one row per pair; columns SUPPORTS, REFUTES, NOT_ENOUGH_INFO
```

Inputs were at most 256 tokens in training (`truncation=True` cuts there). Split long documents into passages and read each one; say who is speaking in the passage itself ("Email from the client: ..."), because the reader sees only the text.

Probabilities are best calibrated after dividing the logits by **1.203** (fitted on development data) before the softmax.

## Results

Test sets, accuracy. One evaluation harness for all three models; brackets and paired tests are in the [results report](https://github.com/jithinsaireddy/ease-delta/blob/main/docs/RESULTS.md).

| | **This model** (large) | base reader | `tasksource/ModernBERT-base-nli` |
|---|---:|---:|---:|
| Passages from unrelated documents read as decisive (lower is better) | 0.58% | 0.82% | 45.07% |
| VitaminC: claims against Wikipedia revisions | 91.53% | 90.20% | 68.30% |
| MultiNLI, mismatched genres | 90.21% | 88.56% | 89.84% |
| WANLI | 77.42% | 74.74% | 66.34% |
| SNLI (not trained on) | 84.39% | 80.33% | 89.15% |
| ANLI round 1 (not trained on) | 53.70% | 46.40% | 63.40% |
| ANLI round 2 (not trained on) | 38.10% | 33.30% | 48.20% |
| ANLI round 3 (not trained on) | 36.00% | 34.75% | 42.25% |

`tasksource/ModernBERT-base-nli` is a base-size multi-task NLI model whose training includes SNLI and ANLI; it is the better choice for adversarial NLI of the ANLI kind. This reader is built for evidence: contrastive Wikipedia revisions (VitaminC), and knowing when a passage does not bear on the claim.

On VitaminC, real revisions: 89.90%; synthetic: 94.25%.

## Labels

| Label | Meaning | NLI equivalent |
|---|---|---|
| `SUPPORTS` | the passage establishes the claim | entailment |
| `REFUTES` | the passage establishes that the claim is false | contradiction |
| `NOT_ENOUGH_INFO` | the passage does not settle the claim, including when it is about something else | neutral |

## Training

900,000 examples streamed from the Hugging Face Hub: VitaminC (cc-by-sa-3.0), MultiNLI and WANLI (cc-by-4.0), plus 20% synthetic unrelated pairs (the claim of one example with the evidence of another from a different topic, labelled `NOT_ENOUGH_INFO`). SNLI was left out because its annotators labelled unrelated content as contradiction; ANLI because its licence is non-commercial. One run, on one Apple M4 Max. The evaluation plan was written and hashed before training: [`docs/PREREGISTRATION_LARGE.md`](https://github.com/jithinsaireddy/ease-delta/blob/main/docs/PREREGISTRATION_LARGE.md).

## Limitations

- English only.
- Passages on the claim's own subject that do not settle it are still read as decisive about one time in five (VitaminC test, not-enough-info items).
- Conflicts that follow only from a consequence ("broke a leg" against "cycles to work") are mostly missed.
- Numbers and dates are read as text; compare them with code when they matter.
- Not a judge of truth: it reports what the passage says, not whether the passage is right.

## How this checkpoint was made

Converted exactly from the EASE-Delta edge model (`ease/export.py` in the code repository explains why the conversion is exact). On 1,200 development pairs the largest logit difference between the two was 9.5e-07 and the predicted label never differed (0 disagreements). The full system, with its calibrated rules and regression suites, is [`jithinpothireddy21/ease-delta`](https://huggingface.co/jithinpothireddy21/ease-delta).

Other size: [`jithinpothireddy21/ease-delta-reader-base`](https://huggingface.co/jithinpothireddy21/ease-delta-reader-base).

## Licence

CC BY-SA 4.0 for the weights, because VitaminC is share-alike (Creative Commons' guidance on AI training describes this as the cautious course). The code is Apache-2.0. Credit for the training data: VitaminC (Schuster, Fisch and Barzilay, NAACL 2021), MultiNLI (Williams, Nangia and Bowman, NAACL 2018), WANLI (Liu, Swayamdipta, Smith and Choi, EMNLP 2022). Backbone: ModernBERT (Warner et al., 2024).

## Citation

```bibtex
@software{pothireddy2026easedelta,
  author = {Pothireddy, Jithin},
  title = {EASE-Delta: revision-aware decision computation},
  year = {2026},
  url = {https://github.com/jithinsaireddy/ease-delta}
}
```