CoderBak's picture
Add a standalone inference script
3434d4a verified
|
Raw History Blame Contribute Delete
2.28 kB
metadata
base_model: jhu-clsp/ettin-encoder-400m
language:
  - en

ai-detector-dev-11-14

A development checkpoint of a token-level AI-text detector. It labels every token of an English text as human, ai_assisted or ai_generated.

  • Backbone: jhu-clsp/ettin-encoder-400m (395M parameters), fine-tuned.
  • Heads: a 3-way token head, a segment head (how much of a window is AI-written) and a mixed-authorship head.
  • Inference: windows of 512 tokens with stride 256; logits are averaged over overlapping windows. A calibrated linear-chain CRF then gives one label per token.

Files

File Content
backbone/ Fine-tuned encoder weights and config
head.safetensors The three heads
tokenizer/ Tokenizer
model.json Classes, heads, window and stride
crf.json Calibration (W, alpha, delta) and CRF parameters (lambda, gamma, l_min)
inference.py Standalone inference script

Usage

inference.py needs torch, transformers, safetensors, numpy and huggingface_hub.

python inference.py essay.txt            # verdict and labelled segments
python inference.py essay.txt --json     # full result, with per-sentence probabilities
cat essay.txt | python inference.py -
from inference import Detector

detector = Detector()            # or Detector("path/to/this/folder", device="cuda:0")
result = detector.detect(text)
result["verdict"]                # "Human", "Mixed" or "AI"
result["segments"]               # [{"start", "end", "label"}], character positions in result["text"]
result["sentences"]              # the same per sentence, with the probability of each label

The verdict is Human if at least 90% of the text is labelled human, AI if the AI fraction (AI-generated plus half of AI-assisted) is at least 80%, and Mixed otherwise.

Input

The detector was trained on normalised text: straight quotes, -- for em dashes, ... for the ellipsis character, and plain text without Markdown. inference.py applies this normalisation itself, and the positions it returns refer to the normalised text. Feeding raw typographic punctuation without it raises the false-positive rate on human text.