|
Download README.md from CoderBak/ai-detector-dev-11-14: direct link, hf CLI and curl.
- Browser
- Download file 2.28 kB
-
https://huggingface.co/CoderBak/ai-detector-dev-11-14/resolve/main/README.md
- Command line
-
hf download hf://CoderBak/ai-detector-dev-11-14/README.md
-
curl -L -H "Authorization: Bearer $HF_TOKEN" -o README.md https://huggingface.co/CoderBak/ai-detector-dev-11-14/resolve/main/README.md
2.28 kB
| base_model: jhu-clsp/ettin-encoder-400m | |
| language: | |
| - en | |
| # ai-detector-dev-11-14 | |
| A development checkpoint of a token-level AI-text detector. It labels every token of an English text as `human`, `ai_assisted` or `ai_generated`. | |
| - **Backbone:** [jhu-clsp/ettin-encoder-400m](https://huggingface.co/jhu-clsp/ettin-encoder-400m) (395M parameters), fine-tuned. | |
| - **Heads:** a 3-way token head, a segment head (how much of a window is AI-written) and a mixed-authorship head. | |
| - **Inference:** windows of 512 tokens with stride 256; logits are averaged over overlapping windows. A calibrated linear-chain CRF then gives one label per token. | |
| ## Files | |
| | File | Content | | |
| |---|---| | |
| | `backbone/` | Fine-tuned encoder weights and config | | |
| | `head.safetensors` | The three heads | | |
| | `tokenizer/` | Tokenizer | | |
| | `model.json` | Classes, heads, window and stride | | |
| | `crf.json` | Calibration (`W`, `alpha`, `delta`) and CRF parameters (`lambda`, `gamma`, `l_min`) | | |
| | `inference.py` | Standalone inference script | | |
| ## Usage | |
| `inference.py` needs `torch`, `transformers`, `safetensors`, `numpy` and `huggingface_hub`. | |
| ```bash | |
| python inference.py essay.txt # verdict and labelled segments | |
| python inference.py essay.txt --json # full result, with per-sentence probabilities | |
| cat essay.txt | python inference.py - | |
| ``` | |
| ```python | |
| from inference import Detector | |
| detector = Detector() # or Detector("path/to/this/folder", device="cuda:0") | |
| result = detector.detect(text) | |
| result["verdict"] # "Human", "Mixed" or "AI" | |
| result["segments"] # [{"start", "end", "label"}], character positions in result["text"] | |
| result["sentences"] # the same per sentence, with the probability of each label | |
| ``` | |
| The verdict is Human if at least 90% of the text is labelled human, AI if the AI fraction (AI-generated plus half of AI-assisted) is at least 80%, and Mixed otherwise. | |
| ## Input | |
| The detector was trained on normalised text: straight quotes, ` -- ` for em dashes, `...` for the ellipsis character, and plain text without Markdown. `inference.py` applies this normalisation itself, and the positions it returns refer to the normalised text. Feeding raw typographic punctuation without it raises the false-positive rate on human text. | |