FDSpotter DeBERTa V3 Large

FDSpotter checks whether an atomic fact or a discourse relation is supported by a source text. It is the verifier introduced in Structured Discourse Representation for Factual Consistency Verification (Findings of ACL 2025).

The model takes a pair consisting of a source text and a structured information unit, and predicts entailment, neutral, or contradiction. The FDSpotter evaluation pipeline uses the entailment probability to score extracted information. Extraction and score aggregation are implemented in the code repository.

Results

The paper reports the following accuracy scores (%) on the DiscInfer test set (Table 3):

Model	Temporal	Contingency	Comparison
DeBERTa without DiscInfer training	58.2	48.3	23.1
DeBERTa with DiscInfer training	76.4	84.3	59.6
GPT-4 zero-shot	73.0	53.8	48.1

DiscInfer contains adversarial discourse-relation examples. Training on it improves the verifier's ability to recognise changes in the relations between facts.

Usage

Install the dependencies:

pip install torch transformers sentencepiece

Pass the source text first and the information unit second. This example runs on an NVIDIA GPU with CUDA-enabled PyTorch:

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

model_id = "Inria-CEDAR/FDSpotter-DeBERTa-V3-Large"
device = torch.device("cuda")
tokenizer = AutoTokenizer.from_pretrained(model_id, use_fast=False)
model = AutoModelForSequenceClassification.from_pretrained(model_id).to(device)
model.eval()

source = "A birthday present was given to Kate by Amy last Monday."
unit = "Amy | gave | a birthday gift | to Kate | last Monday"
inputs = tokenizer(
    source,
    unit,
    return_tensors="pt",
    truncation="longest_first",
    max_length=384,
).to(device)

with torch.inference_mode():
    probabilities = model(**inputs).logits.softmax(dim=-1)[0]

scores = {
    model.config.id2label[label_id]: probability.item()
    for label_id, probability in enumerate(probabilities)
}
print(scores)
print("Entailment score:", scores["entailment"])

Inputs and outputs

Atomic facts use the following fields:

Subject | Predicate | Direct object | Indirect object | Short adverbial and complement

Discourse relations connect two facts:

Fact 1 | Connective | Fact 2

The output label order is:

Index	Label	Meaning
0	entailment	The source supports the information unit.
1	neutral	The source does not establish or contradict the information unit.
2	contradiction	The information unit contradicts the source.

The probabilities describe support from the supplied source text, rather than truth independently of that source. For whole-text evaluation, the repository provides the extraction and aggregation steps around this verifier.

Training

The model is fine-tuned from sileod/deberta-v3-large-tasksource-nli. Training combines natural-language entailment data, synthetic atomic-fact entailment data, and DiscInfer discourse-relation entailment data. The training code is available in fdspotter_train, and the accompanying data and model outputs are available in the FDSpotter dataset repository.

Citation

@inproceedings{zhang-etal-2025-structured,
    title = "Structured Discourse Representation for Factual Consistency Verification",
    author = "Zhang, Kun and Balalau, Oana and Manolescu, Ioana",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2025",
    year = "2025",
    month = jul,
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    pages = "820--838",
    url = "https://aclanthology.org/2025.findings-acl.46/",
    doi = "10.18653/v1/2025.findings-acl.46"
}