Seq vs Seq: An Open Suite of Paired Encoders and Decoders
Paper • 2507.11412 • Published • 32
Request access to ParaTrace v7
Requests are reviewed by hand.
ParaTrace v7 is released for non-commercial research and evaluation under CC BY-NC-SA 4.0. Its verdicts are evidence, not proof: it flags some human writing and misses some AI text, so it must not be the only basis for a decision about a person.
Log in or Sign Up to review the conditions and access this model content.
ParaTrace v7
The AI-text detector behind paratrace.net. It is a 1B-parameter encoder that scores English text for being AI-generated. For each text it returns a verdict and a confidence level and, on request, which parts of a long text look AI-written.
| Base model | jhu-clsp/ettin-encoder-1b (ModernBERT architecture, 1.0B parameters) |
| Input | English text, scored in 512-token windows |
| Output | P(machine); a verdict at the strict threshold (1% of human texts flagged) or the lenient one (5%); confidence; optionally labelled segments |
| Weights | float32, the served model's file (the service runs it in bfloat16 on GPUs) |
| License | CC BY-NC-SA 4.0, non-commercial (why); access on request |
Performance at a glance
RESULTS.md has every table, the method, and results by generator, attack, domain, source and length.
| Test | Result |
|---|---|
| RAID leaderboard: hidden test set, 11 generators, 11 attacks | TPR 92.81% at 1% FPR, 96.32% at 5%; AUROC 98.65 |
| MELD-eval: 4 recent chat models, 6 attacks, 228k texts | TPR 98.4% at 1% FPR; AUROC 99.85 |
| HC3, MAGE, M4GT, Ghostbuster: AUROC on the texts not in training | 99.87, 99.06, 99.85, 99.86 |
| DetectRL: AUROC | 93.99; 97.68 without its LLM-paraphrased "human" texts |
| Chat-assistant answers, 26,086 from more than 100 models | 99.6% caught |
| Fresh human web text, 66,586 texts | 1.9% flagged |
| UChicago human controls, 7,968 texts | 1.86% flagged |
| AI text in a domain held out of training (poetry) | 80.6% caught at 1% FPR |
| StealthGPT-humanized AI text (UChicago) | 21.4% caught at 1% FPR |
| Texts under 50 words | 12.1% of human texts flagged, 8.7% of AI texts missed |
"Flagged" and "caught" are at the strict threshold, the one the service uses by default.
Known weaknesses
- Humanized text. Text rewritten by humanizer tools is mostly missed: 21% of StealthGPT-humanized texts are caught at 1% FPR.
- Some human writing looks AI to it:
- persuasive essays: 18.4% flagged, 32 of 174;
- recipe-site text: 21.2% flagged;
- human text in chat-transcript formatting: 6.75% flagged, against 0.75% as plain text.
- LLM-polished human text is called AI, by design. A human text an LLM paraphrased or polished counts as AI-assisted.
- Short texts. Below about 50 words, verdicts are unreliable, and the result carries a warning.
- New domains and generators are caught less often: poetry 80.6%. On RAID, text from Cohere's base model is caught 71.5% of the time, and paraphrased text 83.7%.
- English only.
- Benchmark overlap. HC3, MAGE, M4GT and MELD-eval are partly built from corpora that are in the training data. RESULTS.md reports them with and without the overlapping texts.
How to use
Request access on this page.
Log in and download the scoring code:
pip install torch "transformers>=4.57" huggingface_hub hf auth login hf download empirischtech/paratrace-v7 detector.py --local-dir .Score texts:
from detector import Detector detector = Detector.from_hub() # downloads the model (4 GB); or Detector("local/folder") detector.predict(["Text to check ..."]) # strict threshold # [{'probability': 0.9992, 'is_ai': True, 'operating_point': 'strict', 'threshold': 0.4036, # 'n_words': 70, 'n_windows': 1, 'warnings': [], 'confidence': 'high'}] detector.predict(texts, operating_point="lenient") # flags 5% of human texts instead of 1% Detector.from_hub(segments=True).predict(texts) # also: verdict, ai_fraction, segments
The command line works too: python detector.py "Some text.", with --segments and
--operating-point lenient.
How detector.py scores a text, as the service does:
- Prepare. It applies the cleaning used on the training data, then removes invisible characters and folds homoglyphs.
- Score each window. The text is split into 512-token windows that overlap by 128 tokens, and each window is scored.
- Combine. The window scores are averaged into P(machine).
- Verdict. P(machine) is compared with the threshold of the chosen operating point.
With segments=True, each sentence is also labelled. The text's verdict is "ai" when at least
80% of its characters are labelled AI, "human" when at least 90% are labelled human, otherwise
"mixed".
confidence (low / medium / high) says how far the evidence lies from the threshold. On
held-out data, 0.1% of "ai / high" texts and 99.3% of "human / high" texts are human
(RESULTS.md, section 6).
Requirements:
- The model runs in bfloat16 on GPUs and float32 on CPU;
dtype=overrides that. It needs about 2.1 GB of GPU memory. - On transformers older than 4.57, pass
attn_implementation="eager".
Without detector.py. For a single window, without the preparation (scores will differ a
little, more on texts with homoglyphs or invisible characters):
import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer
repo = "empirischtech/paratrace-v7"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForSequenceClassification.from_pretrained(repo).eval()
inputs = tokenizer("Text to check.", truncation=True, max_length=512, return_tensors="pt")
with torch.no_grad():
p_machine = model(**inputs).logits.softmax(-1)[0, 1].item() # labels: 0 human, 1 machine
Training
Ettin encoder 1B was fine-tuned as a human-vs-AI classifier on about 1.9 million English texts:
- Human texts from public corpora:
- reference and academic: Wikipedia, arXiv and peS2o abstracts and papers, PubMed;
- books and fiction: Project Gutenberg, CMU book summaries, WritingPrompts;
- online writing: Reddit (SHP), Stack Exchange, ELI5, blogs, Yelp reviews;
- news: XSum, CNN/DailyMail, CC-News, Global Voices;
- public records: patents, case law, UK Hansard, US government records;
- other: open textbooks, recipes, emails, speeches, and student essays (PERSUADE 2.0, ELLIPSE, ASAP 2.0).
- AI "mirrors" of those human texts. Each mirror is written on the same topic, by more than 20 open-weight and commercial models (GPT, Claude, Gemini, DeepSeek, Qwen, Gemma, Mistral and others), so the model learns authorship rather than topic.
- Other AI text:
- chat-assistant answers (WildChat, LMArena, LMSYS);
- the AI texts of HC3, MAGE, the COLING 2025 MGT shared task and part of RAID's training split;
- LLM paraphrases of human text, labelled AI;
- 13,781 AI texts rewritten by four humanizer prompts.
- Hard negatives: human texts that earlier detectors wrongly flagged, each with an AI mirror.
- Held out for evaluation:
- Anthropic's persuasion essays;
- RAID's poetry domain, GPT-4, Mistral-chat and attacked texts;
- whole generators, and a whole domain (poetry).
Settings: one epoch, learning rate 1e-5, batch 64, 512-token windows, float32. Half of the texts longer than one window were cut at a random position. Development AUROC: 0.9984.
Intended use
- In scope:
- research on AI-generated text and its detection;
- evaluating and comparing detectors;
- one signal among others when a person reviews a text.
- Out of scope:
- using the output as the only basis for decisions about people (academic misconduct, hiring, publishing, moderation);
- non-English text;
- texts under 50 words;
- commercial use (see the license).
License
CC BY-NC-SA 4.0. The training data includes corpora licensed for non-commercial use only: PERSUADE 2.0 and ELLIPSE (CC BY-NC-SA 4.0), and Yelp reviews (Yelp Dataset terms). The base model, Ettin, is released by its authors under the MIT License.
Citation
@misc{paratrace2026v7,
title = {ParaTrace v7},
author = {{ParaTrace}},
year = {2026},
howpublished = {\url{https://huggingface.co/empirischtech/paratrace-v7}}
}
@misc{weller2025seqvsseqopen,
title = {Seq vs Seq: An Open Suite of Paired Encoders and Decoders},
author = {Orion Weller and Kathryn Ricci and Marc Marone and Antoine Chaffin and Dawn Lawrie and Benjamin Van Durme},
year = {2025},
eprint = {2507.11412},
archivePrefix = {arXiv},
primaryClass = {cs.CL}
}
Contact
- Downloads last month
- -
Model tree for empirischtech/paratrace-v7
Base model
jhu-clsp/ettin-encoder-1b