---
title: CLOAK API
emoji: 🛡️
colorFrom: blue
colorTo: indigo
sdk: docker
app_port: 7860
pinned: false
---

# CLOAK model API (Hugging Face Space)

The models and engines the CLOAK server calls, in one Space with one secret. API only: there is
no UI and no interactive API docs (`/docs` and `/openapi.json` are switched off on purpose).

| Path | What it does | Server caller |
|---|---|---|
| `POST /detect` | Personal data in text: nested NER, `Wardline/CLOAK_V3.12_onnx`, ONNX INT8 on CPU | `server/src/detection/pii.ts` |
| `POST /file/classify` · `POST /file/analyze` | What kind of document a text or file is (26 classes) | `server/src/detection/classify.ts` |
| `POST /copilot/parse` · `GET /copilot/examples` | Policy Copilot: a policy described in plain English → drafts the policy forms would save | `server/src/modules/org/routes/copilot.ts` |
| `GET /health` | Status of all three, with no secret needed and none revealed | Settings → backend status |

Every path except `/` and `/health` needs `Authorization: Bearer <API_SECRET>`.

## Deploy

1. Push this folder to the Space repo. It is a **Docker Space** (`sdk: docker` above): HF builds
   `Dockerfile`, which pins Python 3.11.9 and runs as uid 1000. If the Space was created as a
   Gradio Space, the front-matter above switches it; HF rebuilds on push.
2. Space secrets:
   - `API_SECRET`: a long random value (32+ characters). CLOAK sends it as the Bearer token.
     If unset it falls back to `HF_TOKEN`; that works, but puts the HF account token in CLOAK's
     settings, so don't.
   - `HF_TOKEN`: read access to the model repos.
   - Optional: `MODEL_REPO`, `MODEL_FILE`, `DOCCLASS_*` (below), `LOG_LEVEL`.
3. In the CLOAK dashboard → Settings: **Backend URL** = the Space URL
   (`https://<space>.hf.space`), **API key** = `API_SECRET`. The Policy Copilot uses the same
   two settings; nothing else to configure.

The same image runs on a customer's own host, for installs that must not send text or files to
Hugging Face:

```bash
docker build -t cloak-model-api .
docker run -p 7860:7860 -e API_SECRET=... -e HF_TOKEN=... cloak-model-api
```

Point CLOAK's Backend URL at it. (`POLICY_COPILOT_URL` on the server can point the copilot at a
different host, if it ever has to run apart from the models.)

## Security

- One Bearer secret for every data path, compared in constant time.
- Bodies are bounded before they are read: `/detect` by `MAX_INPUT_CHARS`, `/copilot/*` at 2 MB
  (text at most 2,000 characters); `/file/*` by docclass's own limits.
- Failures are real HTTP errors (401, 411, 413, 422, 503) with generic messages; the reason goes
  to the Space log only.
- Responses carry `Cache-Control: no-store`, `nosniff`, `DENY` framing and no referrer.
- Nothing is stored. The admin's copilot text is never logged: only request id, outcome and timing.
- The HF token is passed to each download and never written to disk.
- The service user cannot modify the code in the image; only its model cache is writable.

## `/detect` contract (must stay in sync with `server/src/detection/pii.ts`)

- `POST /detect` · `{"text": "..."}`
  → `{entities: [{type, text, start, end, confidence}], nested, redacted_text, model, processing_time_ms}`
- Failures are never 200-with-empty, so DSPM records scan **errors**, not false "clean".
- `MAX_INPUT_CHARS = 80_000` here must be ≥ `DETECT_MAX_CHARS` in `pii.ts`.
- `GET /health` → `{status, model, engine, threads, classes, max_input_chars, auth_configured, docclass, copilot}`.

## Document classification (`docclass.py`)

`/detect` finds personal details in text. The `/file` paths say what KIND of document a file is
(26 classes, including `credentials`) and suggest an action. CLOAK stores both.

| Path | Body | Runs |
|---|---|---|
| `POST /file/classify` | `{"text": "...", "column_headers": [...]}` | Layer 3 on text CLOAK already extracted (~5 ms) |
| `POST /file/analyze` | raw bytes, `Content-Type: application/octet-stream`, `X-Filename: <name>` | Layers 1+2+3: real type from bytes, archive contents, OCR for images and scanned PDFs |

- **Python 3.11.9 only** (the Dockerfile pins it): the model is a scikit-learn pickle built on
  3.11.9, and `requirements.txt` pins its exact libraries. Other versions can load it and
  silently behave differently.
- The model is fetched from `DOCCLASS_REPO` (default: `MODEL_REPO`) / `DOCCLASS_FILE` (default
  `layer3_25class.joblib`) and **refused unless its SHA-256 matches** `DOCCLASS_SHA256` (pinned in
  `docclass.py`). Upload a new model → update the hash in the same change.
- If it can't load, `/file/*` returns 503 and the other paths are unaffected.
- Limits: 25 MB per file, archives 3 levels / 500 entries / 150 MB unpacked, at most 2 OCR jobs at
  once (`DOCCLASS_OCR_CONCURRENCY`), first 3 pages of a scanned PDF (`DOCCLASS_PDF_OCR_PAGES`).
- Always load the model through `Layer3DocType`, never the raw pickle.

## Policy Copilot (`copilot/`, `copilot_api.py`)

Turns "block Aadhaar and PAN in ChatGPT for interns" into the exact body the policy form would
save, with the questions and warnings an admin needs before saving. Rules and vocabulary only:
no model, no network, a few milliseconds per request.

- **Stateless and write-free.** The CLOAK server sends the tenant's context with each request
  (groups, rules, detectors, block lists, connectors, policies) and saves nothing until the admin
  confirms in the dashboard, through the normal policy routes, with the admin's own permissions.
  Drafts are always saved switched off.
- **Adds to existing policies instead of duplicating them.** When a request overlaps a policy that
  already exists (same group, channel or mail service), the admin chooses to add it there or create
  a separate one; "add printing to the USB Lock policy" picks it directly. Only a list of changes is
  sent (switch on, add to a list, make the action stricter), applied to the policy as it is at save
  time. A merge never removes anything, and never changes the name, group or on/off state.
- **Validated twice.** Every draft passes `copilot/check.py` against the form's schema before it
  leaves the Space; the server then accepts only save steps to the three policy-creation routes.
- **Platform facts are synced, not hand-copied.** Built-in rules, the 42 built-in block lists, AI
  sites and every form's defaults come from the cloakv2 source:

  ```bash
  node tools/sync_platform.mjs ..        # from hf-space/, with cloakv2 as the parent
  ```

  Run it whenever a policy form, preset or block list changes, review `git diff copilot/platform`,
  and redeploy the Space. Phrases live in `copilot/lexicon/vocab.yaml`; the loader refuses a phrase
  that points at a rule, list or setting the snapshot doesn't have.
- If the copilot can't load, `/copilot/*` returns 503, the dashboard tells the admin to use the
  form, and the other paths are unaffected.

Try it in a terminal: `python -m copilot.cli "block usb for interns"`.

## Files

```
app.py              PII model, /detect, /health, shared auth and limits; mounts the other two
docclass.py         document pipeline (type from bytes, archives, extraction, OCR, Layer 3)
copilot_api.py      /copilot routes over copilot/
copilot/            the Policy Copilot engine; platform/ holds the synced snapshot
tools/              sync_platform.mjs (not shipped in the image)
Dockerfile          the Space image; .dockerignore keeps tools and docs out
```