---
license: mit
library_name: onnx
pipeline_tag: text-classification
language:
  - en
tags:
  - ai-text-detection
  - machine-generated-text-detection
  - onnx
  - onnxruntime-web
base_model:
  - jhu-clsp/ettin-encoder-68m
  - jhu-clsp/ettin-encoder-150m
base_model_relation: finetune
datasets:
  - liamdugan/raid
  - yaful/MAGE
  - WUJUNCHAO/DetectRL-X
---

# AI-generated text detectors (ONNX)

Two binary classifiers (human / AI-generated) for English text, exported to ONNX for client-side inference with ONNX
Runtime (Web or Python). In a browser, texts are analyzed on the device and never uploaded. They run in the web app
[frontend-deepfake-detection](https://github.com/Deckard618/frontend-deepfake-detection).

<!-- Add the link to the web demo here once it is published. -->

| Folder | Base model | Parameters | File | Input |
|---|---|---|---|---|
| `ettin-68m` | [Ettin encoder 68M](https://huggingface.co/jhu-clsp/ettin-encoder-68m) (Johns Hopkins) | 68M | 77 MB, int8 | first 512 tokens |
| `ettin-150m` | [Ettin encoder 150M](https://huggingface.co/jhu-clsp/ettin-encoder-150m) (Johns Hopkins) | 149M | 168 MB, int8 | first 512 tokens |

Each folder holds `model.onnx`, the `tokenizer.json` and `tokenizer_config.json` to use with it, a `config.json` for
the web app, and `parity.json` (the int8 export checked against the PyTorch model: same decision on 40 of 40 texts).

- Output: 2 logits, index 0 = `human`, index 1 = `AI-generated`. Apply a softmax to get probabilities.
- Calibrated: the output is shifted so that P(AI) = 50% flags 5% of human texts (2,000 MAGE test texts).
- Tokenizer: use `tokenizer.json` as is. It normalizes the text the way training did (NFKC, zero-width characters
  removed, whitespace collapsed) and adds `[CLS]` / `[SEP]`. Truncate to 512 tokens.

## Results

Held-out test sets, AUROC (1 = perfect, 0.5 = chance), then the share of AI texts caught when 1% of human texts are
flagged:

| Test set | `ettin-68m` | `ettin-150m` |
|---|---|---|
| DetectRL-X rewritten: polished, paraphrased, back-translated, character edits (attacks unseen in training) | 0.980 / 0.77 | 0.964 / 0.75 |
| MAGE GPT-4, unseen domains | 0.946 / 0.39 | 0.933 / 0.10 |
| MAGE GPT-4, paraphrased | 0.822 / 0.13 | 0.803 / 0.02 |
| TextSight 2026, the newest models (197 texts) | 0.898 / 0.50 | 0.937 / 0.54 |
| Beemo, edited chatbot answers | 0.752 / 0.11 | 0.765 / 0.06 |
| DetectRL-X (generators and domains seen in training) | 1.000 / 1.00 | 1.000 / 1.00 |

TextSight 2026 holds only 28 AI texts: differences of a few points there are noise. `ettin-68m` is half the size of
`ettin-150m` and catches more AI text at 1% false positives on most sets.

## Usage (Python)

```python
import numpy as np
import onnxruntime as ort
from tokenizers import Tokenizer

LABELS = ["human", "AI-generated"]

def predict(model_dir, text):
    tokenizer = Tokenizer.from_file(f"{model_dir}/tokenizer.json")
    tokenizer.enable_truncation(max_length=512)
    ids = np.array([tokenizer.encode(text).ids], dtype=np.int64)
    session = ort.InferenceSession(f"{model_dir}/model.onnx")
    logits = session.run(None, {"input_ids": ids, "attention_mask": np.ones_like(ids)})[0][0]
    probs = np.exp(logits - logits.max())
    probs /= probs.sum()
    return dict(zip(LABELS, probs.tolist()))

print(predict("ettin-68m", "Paste a paragraph of English text here."))
```

The web app (onnxruntime-web and tokenizers.js) gives the same token ids and probabilities within 1e-6.

## Intended use and limitations

These models are meant for education, research and personal checks of texts. A score is a probability, not proof.

- English only: other languages get inconsistent scores.
- Edited and assistant-style human text is the weak spot. On Beemo, about half of the human answers are flagged at
  50%.
- Paraphrased AI text is often missed (AUROC about 0.8 on paraphrased GPT-4 texts).
- Short texts are less reliable: every model loses about 0.04 AUROC when it reads only the first 100 words.
- The 5% calibration was measured on MAGE human texts. Human texts from other domains can be flagged more often.
- Do not use a score as the only basis for a decision about a person (academic, legal, disciplinary, professional
  or other).

## Training

All weights of the Ettin encoders were fine-tuned, with a linear head on a learned mix of every layer's mean-pooled
hidden states: 5 epochs of 100,000 texts, with EMA weights. Each training set weighs the same, and so does each class
within it.

| Training set | Texts | Human | AI generators | License |
|---|---|---|---|---|
| [RAID](https://huggingface.co/datasets/liamdugan/raid) (train) | 164,303 | 66,714 | 11 (GPT-2/3/4, ChatGPT, Cohere, Llama-chat, Mistral, MPT), with 11 attacks | MIT |
| [MAGE](https://huggingface.co/datasets/yaful/MAGE) (train) | 69,484 | 28,984 | 27 (OPT, GPT-J, NeoX, LLaMA, FLAN-T5, BLOOM, GLM-130B, text-davinci, GPT-3.5…) | Apache-2.0 |
| [DetectRL-X](https://huggingface.co/datasets/WUJUNCHAO/DetectRL-X) (train, English) | 9,599 | 4,799 | 4 (GPT-4o, Gemini 2.5 Flash, DeepSeek-V3, Qwen-Max) | MIT |

- Formatting is never a shortcut: typography, line breaks, typos, homoglyphs and case changes are applied to both
  classes.
- Half of the training texts are random excerpts, so length and openings are not shortcuts either.
- The models were selected on a validation split, never on the test sets above.

## License

MIT, the license of the Ettin encoders the models are built on (`LICENSE`, Copyright (c) 2025 Johns Hopkins University
Center for Language and Speech Processing). `NOTICE` lists the changes made to the original models. The models are
provided as is, without any warranty.

Sources: Ettin [code](https://github.com/JHU-CLSP/ettin-encoder-vs-decoder) and
[paper](https://arxiv.org/abs/2507.11412).

## Citation

If you use these models, please also cite Ettin:

```bibtex
@misc{weller2025seqvsseq,
  title={Seq vs Seq: An Open Suite of Paired Encoders and Decoders},
  author={Orion Weller and Kathryn Ricci and Marc Marone and Antoine Chaffin and Dawn Lawrie and Benjamin Van Durme},
  year={2025},
  eprint={2507.11412},
  archivePrefix={arXiv},
  primaryClass={cs.CL}
}
```