---
title: Droid Code Checker
emoji: 🕵️
colorFrom: indigo
colorTo: pink
sdk: gradio
sdk_version: 6.26.0
python_version: '3.12'
app_file: app.py
pinned: false
license: apache-2.0
short_description: Code authorship detection for Code Arena
---

# Code Authorship Checker

Code-authorship detection with **`project-droid/DroidDetect-Large`**
(ModernBERT-large, Apache-2.0) plus a static forensics layer that looks for the
physical traces of pasted text.

Runs unmodified on **either free tier** — CPU basic or ZeroGPU. `detector.py`
switches on the `SPACES_ZERO_GPU` env var HF sets, so changing hardware needs no
code edit. ZeroGPU is much faster per call but is capped by a daily GPU-seconds
quota charged to the *caller*; for batch checking during a live event, CPU basic
is the better choice. See DEPLOY.md for the numbers.

Replaces the CodeT5+ Space. Same endpoint name, same inputs, same four legacy
response fields — Code Arena keeps working with no changes at all.

## What it returns

| | Old CodeT5+ Space | This |
|---|---|---|
| Classes | human / AI | human / AI / **AI-refined** / **AI-adversarial** |
| Verdict levels | 1 (binary) | **3** (binary, ternary, 4-way) |
| Context | 512 tokens (truncated) | 2048 (model ceiling is 8192) |
| Extras | — | forensic signals, style embedding, inconclusive flag |

### The three levels

The model makes one prediction over four classes. It's read back at three
granularities, because they are not equally trustworthy:

| Level | Values | Large's weighted F1 |
|---|---|---|
| `binary` | HUMAN · MACHINE | ~97% |
| **`ternary`** | HUMAN · MACHINE · REFINED | **~88.8%** |
| `detailed` | the raw 4 classes | lowest |

**Show moderators the `ternary` label.** It answers the question a contest
actually cares about — did the participant write this, did a model write it, or
did they start it and let a model finish — and it's the deepest level the model
is reliable at. The 4-way split additionally separates *adversarial* (model
output deliberately styled to look hand-typed) from ordinary model output;
that's the hardest distinction and the one to treat with most suspicion.

`MACHINE_GENERATED_ADVERSARIAL` folds into `MACHINE` at the ternary level,
because model output styled to look hand-written is still model output. It's
also the class where DroidDetect earns its keep: it holds ~0.92 recall there,
where GPTZero drops to ~0.10.

```json
{
  "verdict": "AI-Generated",
  "ai_probability": 0.18,
  "token_count": 412,
  "truncated": false,

  "binary":  { "label": "MACHINE", "confidence": 0.82,
               "probs": { "HUMAN": 0.18, "MACHINE": 0.82 } },
  "ternary": { "label": "REFINED", "confidence": 0.71,
               "probs": { "HUMAN": 0.18, "MACHINE": 0.11, "REFINED": 0.71 },
               "meaning": "Participant started it, a model completed or rewrote it" },
  "detailed": {
    "label": "MACHINE_REFINED",
    "confidence": 0.71,
    "probs": {
      "HUMAN_GENERATED": 0.18,
      "MACHINE_GENERATED": 0.09,
      "MACHINE_REFINED": 0.71,
      "MACHINE_GENERATED_ADVERSARIAL": 0.02
    }
  },
  "inconclusive": false,

  "label": "MACHINE_REFINED",
  "confidence": 0.71,
  "human_probability": 0.18,
  "model_ai_score": 82,
  "embedding": [0.031, -0.118, "…254 more"],

  "features": { "comment_ratio": 0.44, "ai_style_comments": 4, "smart_punct": 1,
                "mixed_indent": false, "crlf": false, "…": "20 fields" },
  "signals": [
    { "code": "unicode_punctuation", "weight": 35,
      "detail": "Curly quotes/dashes — pasted from a rendered page or chat UI, not typed" }
  ],
  "forensic_score": 75,

  "score": 80,
  "model": "project-droid/DroidDetect-Large",
  "pooling": "mean",
  "max_length": 2048
}
```

Field notes:

- **`ai_probability` is a human score.** It's the probability of the
  `HUMAN_GENERATED` class, kept under the old (misleading) name so Code Arena
  keeps behaving identically. `human_probability` is the same number, honestly
  named.
- `model_ai_score` is `100 × P(MACHINE)` from the binary collapse. Three of the
  four classes are machine, so unlike the CodeT5+ Space this is a genuine AI
  probability rather than a distance from a hand-tuned threshold.
- **`inconclusive`** is true when the top class is under 0.45 — the model has
  nothing to say. Show it as "no signal", never as a verdict. A flat
  `0.3 / 0.3 / 0.2 / 0.2` is noise, not evidence.
- `embedding` is a 256-d style vector (128-d on Base). Cosine-compare one
  participant's submissions against each other — the outlier is the answer they
  didn't write the way they write. It separates *human from machine*, not
  *author from author*, so it's a consistency signal, not a plagiarism matcher.
- `score` fuses model and forensics for ranking only. Peer similarity and
  submission timing are deliberately absent: they belong to the platform, which
  holds every submission and every timestamp.

## API

Every request must carry the token. An invalid or missing one returns
`{"error": "Invalid or missing token"}` and **the model never runs**. Empty
input returns `{"error": "No code provided"}`; oversized input is rejected by
character count before tokenization.

```python
from gradio_client import Client

client = Client("your-name/code-authorship-checker")

result = client.predict(
    code="int main(){return 0;}",
    token="YOUR_API_TOKEN",
    api_name="/detect",
)
print(result["ternary"]["label"], result["ternary"]["confidence"])
```

Or over plain HTTP — the two-step Gradio flow Code Arena uses:

```bash
EVENT=$(curl -s -X POST "$SPACE/gradio_api/call/detect" \
  -H 'content-type: application/json' \
  -d '{"data": ["print(1)", "YOUR_API_TOKEN"]}' | jq -r .event_id)

curl -s -N "$SPACE/gradio_api/call/detect/$EVENT"
```

## Configuration

All optional — the Space boots and works with none of them set.

The API token is **not** one of these — it's hardcoded as `API_TOKEN` at the top
of `app.py`. Change it there and push. Because it lives in the source, keep the
Space **private**, and rotate the value if the repo is ever made public.

| Variable | Default | Purpose |
| --- | --- | --- |
| `DROID_REPO` | `project-droid/DroidDetect-Large` | `…/DroidDetect-Base` is ~4× faster for ~2 F1 less |
| `GPU_DURATION` | `30` | ZeroGPU only: seconds the GPU is held per call |
| `POOLING` | `mean` | `mean` or `cls` — run `smoke_test.py` once to confirm |
| `MAX_LENGTH` | `2048` | Tokens per request (model ceiling 8192; higher = slower) |
| `HUMAN_THRESHOLD` | `0.5` | Below this `p_human`, the legacy verdict line says AI-Generated |
| `INCONCLUSIVE_BELOW` | `0.45` | Top-class probability under which `inconclusive` is set |
| `MODEL_WEIGHT` | `0.7` | Model vs forensics weighting in the fused `score` |
| `MAX_CHARS` | `200000` | Reject paste bombs before tokenizing |
| `TORCH_THREADS` | `2` | Matches the free tier's 2 vCPUs |

## Accuracy, honestly

Reported by the Droid paper for Large: **97.0%** weighted F1 binary and
**88.8%** ternary — against 91.6% for GPTSniffer and 76.5% for Fast-DetectGPT.

Those are in-distribution. On unseen languages and domains this class of
detector falls to roughly 0.85 macro-F1, and under combined domain *and*
language shift, benchmarks put it near 0.21. The errors are asymmetric:
detectors reliably spot AI code but flag *unusual human code* as AI — which in
a contest means your strongest participants. Treat every number here as one
input to a human decision, never as a verdict on its own.

## Files

| File | Role |
| --- | --- |
| `app.py` | Gradio UI + token-guarded `detect` endpoint |
| `detector.py` | Checkpoint loader, `TLModel` wrapper, label collapsing |
| `forensics.py` | Static paste-trace analysis — standard library only |
| `labels.py` | Class taxonomy + the binary/ternary collapses |
| `smoke_test.py` | First-run check; resolves the `POOLING` setting |
| `test_forensics.py` | Self-check for the forensics layer, no torch needed |
| `test_collapse.py` | Self-check for the ternary/binary collapse |
| `test_auth.py` | Self-check for the API-token guard |
| `DEPLOY.md` | Deployment walkthrough |