ai-text-detector (weights)

These are the trained weights of github.com/lucadagati/ai-text-detector. It is a local detector of LLM-generated academic text that works per paragraph and per sentence.

The service is an OR ensemble of two DeBERTa-v3-base classifiers. Each one mean-pools the encoder and outputs a single logit, where higher means more machine-like:

Folder	Training data
deberta-base-v2/	4,960 human paragraphs and abstracts from pre-2020 arXiv papers; 4,718 AI paragraphs from Claude (Sonnet, Opus, Haiku), gpt-oss-20b, Mistral-Small-24B, Qwen2.5-14B, Gemma-3-12B, Llama-3.1-8B and Phi-4. The AI texts are written from scratch, written "to sound human", or are polish, revision and multi-pass rewrites of the human paragraphs.
deberta-base-v3/	The same data, plus the training split of the RAID abstracts domain (11 generators, 11 adversarial attacks).

A paragraph is flagged when either model exceeds its own threshold at 0.5% false positives, so the ensemble stays at 1% or less overall. model.json holds the thresholds, the sentence-window thresholds and the norms of the sentence-structure profile used for explanations.

Results

All results are at 1% false positives, on held-out papers (split by paper).

	Claude	AI-polished human text	RAID clean / attacked
this ensemble	97%	93%	80% / 83%
MAGE (best open detector on RAID)	0.5%	2%	69% / 72%

Claude models left out of training entirely are still detected at 95%. Small open-weight model families left out of training are detected at 59%. Full benchmark, protocol and limits: bench/RESULTS.md in the GitHub repository.

Use

The service in the GitHub repository downloads these weights on first start. To load one model directly:

from huggingface_hub import snapshot_download
from aidet.classifier import load_classifier          # from the GitHub repository
path = snapshot_download("Lukkinen/ai-text-detector", allow_patterns=["deberta-base-v2/*"])
model, tok = load_classifier(f"{path}/deberta-base-v2")

The classifier class (MeanPoolClassifier) is defined in aidet/classifier.py. It is not a stock AutoModelForSequenceClassification head.

Limits

• Trained and validated on English computer-science and engineering academic text.
• A score measures resemblance to LLM text. It is not proof of authorship.
• Detector-guided rewriting (iterating a rewrite until this detector scores it human) evades it.
• Isolated flagged paragraphs occur in about 8% of human papers.