BillFisk/ai-text-detector

Clueso AI Text Detector — MAGE LoRA v1

This repository contains the merged FP32 ONNX model used by @bandf/clueso, a local AI-text detector for Node.js.

It is a LoRA fine-tune of wasitaigeneratedcom/ai-text-detector-small. The original four output classes and mask-aware mean-pooling architecture are preserved:

DeBERTa-v3-large
    ↓
mask-aware mean pooling
    ↓
Linear(1024 → 4)
    ├── human
    ├── ai
    ├── ai_edited
    └── humanized

The headline AI score is:

AI score = 1 - P(human)

Input and inference behavior

• English text
• Minimum input: 50 words
• Maximum chunk: 768 tokens including special tokens
• Longer documents: sentence-aware chunks combined using token-weighted class probabilities
• Calibrated AI-score threshold: 0.9974874649196864

The ONNX graph contains the DeBERTa encoder, mask-aware mean pooling, and four-class classifier. No remote code is required.

Fine-tuning

Only the published MAGE training split was used for gradient updates. The accepted training sample contained 4,000 human and 4,000 AI documents, balanced across sources.

Rank-8 LoRA adapters were trained on all 24 layers' query and value projections together with the classifier. This made 790,532 of 434,802,692 parameters trainable. Training ran for one epoch on Apple MPS with an effective batch size of 16 and learning rate 1e-4.

MAGE provides binary labels while this detector has four output classes. The training objective therefore compared the human logit with the combined nonhuman logit:

human_logit = logits[0]
nonhuman_logit = logsumexp(logits[1], logits[2], logits[3])

A KL-divergence preservation term discouraged collapse of the conditional distribution among ai, ai_edited, and humanized.

Training, validation, calibration, development, and sealed evaluation data were separated by published split and source group. Cross-boundary exact normalized-text matches and identical normalized 50-word prefixes were excluded.

See FINE_TUNING.md for the complete methodology and limitations.

Evaluation

On the untouched 1,997-document MAGE validation partition, ROC-AUC improved from 0.7820 to 0.9712.

The package-native threshold was selected on a separate 1,999-document MAGE calibration partition, targeting no more than 0.5% human false positives. At the selected threshold, calibration produced 54.6% AI recall, 0.40% human FPR, and 0.9659 ROC-AUC.

A fresh sealed package-native comparison used 4,000 documents from previously unused HC3 and RAID source groups:

Model	ROC-AUC	AI recall	Human FPR	Balanced accuracy
Original, calibrated	0.9672	66.70%	0.10%	83.30%
MAGE LoRA v1, calibrated	0.9866	83.55%	0.55%	91.50%

Dataset-level results:

Dataset	Original recall	Fine-tuned recall	Original FPR	Fine-tuned FPR
HC3	91.9%	92.6%	0.0%	0.7%
RAID	41.5%	74.5%	0.2%	0.4%

Artifact integrity

File:   onnx/model.onnx
Bytes:  1,737,980,451
SHA-256: 8b0d4d219bb87c645578b9bf590168635d5a5d0d2c7b7cb92a550fef2179d26f

The maximum absolute logit difference between PyTorch and ONNX across three reference inputs, including a 768-token input, was 0.000020682811737060547.

Limitations and responsible use

AI-text detection is probabilistic evidence, not proof of authorship. This model should not be the sole basis for disciplinary, employment, academic-integrity, legal, or other high-impact decisions.

Performance can vary with domain, language, generator, editing level, document length, and author population. The original model's undisclosed training corpus may overlap with public benchmarks. A false-positive rate measured on these samples does not guarantee the same rate in deployment.

License and attribution

Apache License 2.0. See LICENSE and NOTICE.

Base model: wasitaigeneratedcom/ai-text-detector-small.