voi-slop-detect-4B

Scores how much of an English text was written by an LLM. The output is one number from 0 to 1, trained to match the fraction of words a model changed: 0 for untouched human writing, around 0.3 for a human draft a chatbot polished, 1 for text a model wrote from scratch.

This is the large model of the family. The small sibling, voi-slop-detect-380M, is an encoder that runs on a CPU and ships with ONNX graphs.

Model

Base	Qwen/Qwen3.5-4B, sequence-classification head with one output
Fine-tuning	LoRA (r 16, alpha 32) on the attention and MLP projections, merged into the base; the head is trained in full
Weights	bf16 safetensors, 9.1 GB, 4.54B parameters (4.21B language model, 0.33B unused vision tower)
Input	first 512 tokens of the text, fenced code blocks emptied
Output	one logit; sigmoid(logit) is the score
Training	3 epochs, lr 2e-4, batch 64, BCE on the score with band-balanced weights
Language	English
License	Apache 2.0, same as the base model

bands.json holds the operating points:

key	value	meaning
human_max	0.15	below this the text counts as human
generated_min	0.75	at or above this it counts as AI-generated; between the two, AI-edited
flag_min	0.385	at or above this, flag the text as not human; fitted so that 1% of validation human texts are flagged

The two band edges are the labels the model was trained against. flag_min is a separate cut for the yes/no question.

Usage

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

MODEL = "VoightKampffLabs/voi-slop-detect-4B"

tokenizer = AutoTokenizer.from_pretrained(MODEL)
model = AutoModelForSequenceClassification.from_pretrained(MODEL, dtype=torch.bfloat16)
model.to("cuda").eval()

texts = ["Paste the first text here.", "Paste the second text here."]
enc = tokenizer(texts, truncation=True, max_length=512, padding=True, return_tensors="pt")
with torch.no_grad():
    logits = model(**enc.to(model.device)).logits
print(torch.sigmoid(logits.float()).reshape(-1).tolist())  # one score per text, 0 = human, 1 = AI

The model was trained with fenced code blocks emptied to a bare ``` pair; do the same to texts that contain code. Tested with transformers 5.22 on a GPU.

Training data

484k texts built from 98k human documents written before 2023. Every human document has up to three AI versions, and the split (80 / 10 / 10) is by source document.

source	years	human docs	AI rows
Amazon reviews	2020-2021	13,510	57,475
arXiv abstracts	2020-2021	14,983	59,704
Hacker News comments	2020-2023	16,198	55,722
Medium posts	2020-2022	8,725	57,014
Reddit comments and posts	2020-2023	15,699	56,392
StackExchange questions and answers	2020-2022	12,188	41,570
XSum news	2017	16,776	58,198

AI version	rows	how it is made	training target
mirror	128,262	a new document written from a two-line summary and keyword list of the human one; the generator never sees the original	1
rewrite	123,402	the human document rewritten under one of 8 instructions, from "fix grammar" to "rewrite in your own words"	measured fraction of words changed
round trip	134,411	translated to Spanish, German or Russian and back to English	measured fraction of words changed

generator family	models	AI rows
OpenAI	GPT-6 Luna, GPT-6.1 Terra, GPT-6.1 Sol	74,901
Anthropic	Claude Haiku 5.5, Sonnet 5.5, Opus 5.5	59,409
Google Gemini	Gemini 3.5 Flash-Lite, 3.6 Flash, 3.7 Flash, 3.8 Flash	54,179
DeepSeek	DeepSeek Flash	53,850
xAI	Grok 4.3	4,514
Qwen (local)	Qwen3.8-27B	86,078
Gemma (local)	Gemma 4 31B, Gemma 4 26B-A4B	53,144

Rows are dropped when a mirror drifts off topic, an edit changes the meaning (embedding cosine to the source), an edit changes under 2% of the words, or the text has fewer than 30 prose words. Median length is 116 words.

Benchmarks

Both of our models use their shipped thresholds (band edges 0.15 / 0.75 and each model's own flag cut: 0.385 here, 0.412 for the 380M) everywhere except APT-Eval, which follows the paper's protocol of a threshold at 5% false positives on its own human texts. Rows for other detectors are their published numbers or are computed from verdicts stored in the benchmark; none were rerun by us. All results are from a single training seed.

Held-out test split (48,041 texts)

metric	4B	380M
accuracy (3 bands)	0.923	0.891
macro-F1	0.923	0.891
F1 human	0.886	0.840
F1 AI-edited	0.918	0.883
F1 AI-generated	0.965	0.949
AUROC, human vs rest	0.981	0.968
TPR at 1% FPR	0.688	0.589
TPR at 0.1% FPR	0.467	0.372
mean absolute error of the score	0.052	0.070
Spearman, score vs fraction of words changed	0.790	0.699

EditLens

Three-way classification (human, AI-edited, AI-generated) on the EditLens test split. None of its text is in our training data.

Columns: Acc and F1 are three-way accuracy and macro-F1; H, E, G are per-class F1 for human, AI-edited and AI-generated; AI is human vs any AI; Gen is AI-generated vs the rest.

detector	Acc	F1	H	E	G	AI acc	AI F1	Gen acc	Gen F1
EditLens cosine (24B)	0.902	0.904	0.904	0.868	0.941	0.938	0.954	0.964	0.941
EditLens soft n-grams (24B)	0.897	0.899	0.896	0.861	0.941	0.940	0.956	0.943	0.902
voi-slop-detect-4B	0.889	0.886	0.898	0.816	0.944	0.927	0.943	0.962	0.944
voi-slop-detect-380M	0.855	0.851	0.867	0.761	0.925	0.905	0.927	0.949	0.925
GPTZero	0.747	0.727	0.773	0.509	0.898
Pangram (3.x API)	0.730	0.695	0.764	0.432	0.890	0.807	0.837	0.923	0.890
FastDetectGPT	0.606	0.568	0.251	0.610	0.844	0.691	0.805	0.906	0.844
DetectAIve	0.578	0.525	0.799	0.615	0.161
Binoculars	0.485	0.501	0.547	0.374	0.584	0.686	0.814	0.314	0.477

Out-of-distribution splits (macro-F1) and AUROC of the score, human vs rest. Enron is an unseen domain and Llama an unseen generator for EditLens.

detector	Enron F1	Llama F1	AUROC test	AUROC Enron	AUROC Llama
EditLens cosine (24B)	0.866	0.850
voi-slop-detect-4B	0.863	0.829	0.977	0.978	0.982
voi-slop-detect-380M	0.818	0.881	0.966	0.954	0.975

Epoch AI

495 human texts and 594 AI texts by Claude Opus 4.8, GPT-5.5 and Gemini 3.1 Pro, half from a plain prompt and half imitating a given author's style. None of these generators are in our training data. Cells are error counts.

Columns: FP is human texts flagged (of 495); Plain and Imit. are AI texts missed (of 297 each).

detector	FP	Plain	Imit.
voi-slop-detect-4B	1	0	22
voi-slop-detect-380M	19	0	24
Pangram 3.3.2	0	0	30
GPTZero 2026-05-11-base	0	2	32
Originality.ai Turbo 3.0.2	19	1	53

By domain. Humans flagged of 165, AI missed of 99; B, F, S are blog, fiction, scientific.

detector	FP B	FP F	FP S	Plain B	Plain F	Plain S	Imit. B	Imit. F	Imit. S
voi-slop-detect-4B	0	1	0	0	0	0	1	1	20
voi-slop-detect-380M	3	16	0	0	0	0	3	1	20
Pangram 3.3.2	0	0	0	0	0	0	4	1	25
GPTZero 2026-05-11-base	0	0	0	0	1	1	6	2	24
Originality.ai Turbo 3.0.2	2	16	1	0	1	0	19	5	29

By generator. AI missed of 99; Opus is Claude Opus 4.8, Gemini is Gemini 3.1 Pro, GPT is GPT-5.5.

detector	Plain Opus	Plain Gemini	Plain GPT	Imit. Opus	Imit. Gemini	Imit. GPT
voi-slop-detect-4B	0	0	0	2	12	8
voi-slop-detect-380M	0	0	0	3	13	8
Pangram 3.3.2	0	0	0	5	20	5
GPTZero 2026-05-11-base	2	0	0	8	14	10
Originality.ai Turbo 3.0.2	1	0	0	12	14	27

AUROC of the score, plain / imitation: 4B 1.000 / 0.987, 380M 1.000 / 0.974.

Human Detectors

150 human magazine and news articles and 30 AI articles per condition. Cells are error counts.

Columns: FP is humans flagged (of 150); the rest are AI articles missed (of 30). Hum. is humanized o1-pro, Para. is paraphrased GPT-4o.

detector	FP	Claude	GPT-4o	o1-pro	Hum.	Para.
expert majority vote	0	0	0	1	0	0
voi-slop-detect-4B	1	0	0	0	0	4
voi-slop-detect-380M	21	0	0	0	0	0
Pangram humanizers (2025-01)	4	0	0	0	1	0
Pangram (2025-01)	4	0	0	0	3	0
GPTZero 2025-01-09-base	1	1	0	7	16	0
Binoculars (both settings)	2	2	0	8	28	12

AUROC of the score per condition:

model	Claude	GPT-4o	o1-pro	Hum.	Para.
voi-slop-detect-4B	1.000	1.000	1.000	1.000	0.993
voi-slop-detect-380M	0.992	1.000	0.992	0.996	0.962

Learner English (false positives)

Essays by non-native writers; every flag is a false positive. Cells are flagged / total. We did not run ICNALE.

detector	All	ELLIPSE	PELIC	ICNALE	TOEFL
Pangram 4	1 / 24,586	0 / 3,899	1 / 15,005	0 / 5,593	0 / 89
Pangram 3.3.2	2 / 24,586	0 / 3,899	2 / 15,005	0 / 5,593	0 / 89
voi-slop-detect-4B	107 / 19,817	0 / 3,911	107 / 15,815		0 / 91
voi-slop-detect-380M	136 / 19,817	0 / 3,911	134 / 15,815		2 / 91
7 detectors, mean (Liang et al. 2023)					61.3%

APT-Eval

Human texts polished by an LLM to different degrees (APT-Eval), plus 300 pure-AI texts from MixSet as a paired check. As in the paper, each model's threshold is set at 5% false positives on the 300 unpolished human texts.

Columns: Cut is the threshold; AI is pure-AI MixSet texts caught; Pol. is polished texts flagged.

model	Cut	AI	Pol.
voi-slop-detect-4B	0.090	62.7%	69.5%
voi-slop-detect-380M	0.212	51.7%	58.6%

Share of polished texts flagged, by polish degree:

model	Extreme minor	Minor	Slight major	Major
voi-slop-detect-4B	58.6%	82.5%	93.1%	93.6%
voi-slop-detect-380M	42.9%	66.7%	87.2%	91.1%

By polish percent:

model	1%	5%	10%	20%	35%	50%	75%
voi-slop-detect-4B	23.1%	36.8%	55.6%	74.8%	81.2%	83.1%	87.2%
voi-slop-detect-380M	15.7%	24.5%	40.1%	61.1%	69.5%	73.3%	78.7%

By polisher:

model	DeepSeek-V3	GPT-4o	Llama2-7B	Llama3-8B	Llama3.1-70B
voi-slop-detect-4B	75.2%	77.2%	75.0%	55.1%	66.3%
voi-slop-detect-380M	66.8%	64.2%	61.0%	46.2%	55.0%

By domain:

model	Blog	Email	Review	News	Abstract	Speech
voi-slop-detect-4B	65.3%	51.6%	71.9%	73.8%	68.3%	83.2%
voi-slop-detect-380M	56.3%	35.3%	63.0%	68.1%	53.1%	72.0%

Extreme-minor polishes by Llama-2-7B, the one slice with published numbers for commercial detectors (195 texts; the other detectors are at their own operating points):

detector	Flagged
voi-slop-detect-4B	73.3%
GPTZero	64.7%
voi-slop-detect-380M	57.4%
Pangram	42.6%
ZeroGPT	32.3%

Pearson correlation of the score with the benchmark's edit distances (14,650 texts). Lev. is Levenshtein distance, Jac. is Jaccard distance, Sem. is semantic similarity.

detector	Lev.	Jac.	Sem.
EditLens cosine (24B)	0.799	0.781	-0.606
voi-slop-detect-4B	0.698	0.709	-0.482
voi-slop-detect-380M	0.614	0.618	-0.423
Pangram			0.491

Spearman: 4B 0.709 / 0.715 / -0.539, 380M 0.659 / 0.662 / -0.505.

Ablations: other backbones

Every backbone we trained on the same data with the same recipe (3 epochs, 512 tokens, one seed), scored on the held-out test split. Encoders are fine-tuned in full; the Qwen models use LoRA r 16.

Columns: Acc and F1 are three-band accuracy and macro-F1; H, E, G are per-band F1 for human, AI-edited and AI-generated; TPR is at 1% FPR; MAE is the mean absolute error of the score; Spear. is the Spearman correlation of the score with the fraction of words changed.

backbone	Params	Acc	F1	H	E	G	AUROC	TPR	MAE	Spear.
Qwen3.5-4B (this model)	4.2B	0.923	0.923	0.886	0.918	0.965	0.981	0.688	0.052	0.790
Qwen3.5-0.8B	0.8B	0.909	0.908	0.864	0.902	0.959	0.975	0.645	0.059	0.752
UltraBERT-380M (voi-slop-detect-380M)	378M	0.891	0.891	0.840	0.883	0.949	0.968	0.589	0.070	0.699
ModernBERT-large	396M	0.886	0.885	0.828	0.877	0.949	0.963	0.579	0.073	0.687
UltraBERT-150M	151M	0.883	0.882	0.828	0.874	0.946	0.963	0.570	0.074	0.680
BERT-large	335M	0.807	0.786	0.648	0.809	0.902	0.934	0.447	0.113	0.580
BERT-base	109M	0.807	0.785	0.636	0.812	0.907	0.935	0.446	0.114	0.579
ModernBERT-base	150M	0.778	0.774	0.672	0.768	0.882	0.894	0.417	0.128	0.509