README.md

metadata

title: CLOAK API
emoji: 🛡️
colorFrom: blue
colorTo: indigo
sdk: docker
app_port: 7860
pinned: false

CLOAK model API (Hugging Face Space)

The models and engines the CLOAK server calls, in one Space with one secret. API only: there is no UI and no interactive API docs (/docs and /openapi.json are switched off on purpose).

Path	What it does	Server caller
POST /detect	Personal data in text: nested NER, Wardline/CLOAK_V3.12_onnx, ONNX INT8 on CPU	server/src/detection/pii.ts
POST /file/classify · POST /file/analyze	What kind of document a text or file is (26 classes)	server/src/detection/classify.ts
POST /copilot/parse · GET /copilot/examples	Policy Copilot: a policy described in plain English → drafts the policy forms would save	server/src/modules/org/routes/copilot.ts
GET /health	Status of all three, with no secret needed and none revealed	Settings → backend status

Every path except / and /health needs Authorization: Bearer <API_SECRET>.

Deploy

1. Push this folder to the Space repo. It is a Docker Space (sdk: docker above): HF builds Dockerfile, which pins Python 3.11.9 and runs as uid 1000. If the Space was created as a Gradio Space, the front-matter above switches it; HF rebuilds on push.
2. Space secrets:
   • API_SECRET: a long random value (32+ characters). CLOAK sends it as the Bearer token. If unset it falls back to HF_TOKEN; that works, but puts the HF account token in CLOAK's settings, so don't.
   • HF_TOKEN: read access to the model repos.
   • Optional: MODEL_REPO, MODEL_FILE, DOCCLASS_* (below), LOG_LEVEL.
3. In the CLOAK dashboard → Settings: Backend URL = the Space URL (https://<space>.hf.space), API key = API_SECRET. The Policy Copilot uses the same two settings; nothing else to configure.

The same image runs on a customer's own host, for installs that must not send text or files to Hugging Face:

docker build -t cloak-model-api .
docker run -p 7860:7860 -e API_SECRET=... -e HF_TOKEN=... cloak-model-api

Point CLOAK's Backend URL at it. (POLICY_COPILOT_URL on the server can point the copilot at a different host, if it ever has to run apart from the models.)

Security

• One Bearer secret for every data path, compared in constant time.
• Bodies are bounded before they are read: /detect by MAX_INPUT_CHARS, /copilot/* at 2 MB (text at most 2,000 characters); /file/* by docclass's own limits.
• Failures are real HTTP errors (401, 411, 413, 422, 503) with generic messages; the reason goes to the Space log only.
• Responses carry Cache-Control: no-store, nosniff, DENY framing and no referrer.
• Nothing is stored. The admin's copilot text is never logged: only request id, outcome and timing.
• The HF token is passed to each download and never written to disk.
• The service user cannot modify the code in the image; only its model cache is writable.

/detect contract (must stay in sync with server/src/detection/pii.ts)

• POST /detect · {"text": "..."} → {entities: [{type, text, start, end, confidence}], nested, redacted_text, model, processing_time_ms}
• Failures are never 200-with-empty, so DSPM records scan errors, not false "clean".
• MAX_INPUT_CHARS = 80_000 here must be ≥ DETECT_MAX_CHARS in pii.ts.
• GET /health → {status, model, engine, threads, classes, max_input_chars, auth_configured, docclass, copilot}.

Document classification (docclass.py)

/detect finds personal details in text. The /file paths say what KIND of document a file is (26 classes, including credentials) and suggest an action. CLOAK stores both.

Path	Body	Runs
POST /file/classify	{"text": "...", "column_headers": [...]}	Layer 3 on text CLOAK already extracted (~5 ms)
POST /file/analyze	raw bytes, Content-Type: application/octet-stream, X-Filename: <name>	Layers 1+2+3: real type from bytes, archive contents, OCR for images and scanned PDFs

• Python 3.11.9 only (the Dockerfile pins it): the model is a scikit-learn pickle built on 3.11.9, and requirements.txt pins its exact libraries. Other versions can load it and silently behave differently.
• The model is fetched from DOCCLASS_REPO (default: MODEL_REPO) / DOCCLASS_FILE (default layer3_25class.joblib) and refused unless its SHA-256 matches DOCCLASS_SHA256 (pinned in docclass.py). Upload a new model → update the hash in the same change.
• If it can't load, /file/* returns 503 and the other paths are unaffected.
• Limits: 25 MB per file, archives 3 levels / 500 entries / 150 MB unpacked, at most 2 OCR jobs at once (DOCCLASS_OCR_CONCURRENCY), first 3 pages of a scanned PDF (DOCCLASS_PDF_OCR_PAGES).
• Always load the model through Layer3DocType, never the raw pickle.

Policy Copilot (copilot/, copilot_api.py)

Turns "block Aadhaar and PAN in ChatGPT for interns" into the exact body the policy form would save, with the questions and warnings an admin needs before saving. Rules and vocabulary only: no model, no network, a few milliseconds per request.

• Stateless and write-free. The CLOAK server sends the tenant's context with each request (groups, rules, detectors, block lists, connectors, policies) and saves nothing until the admin confirms in the dashboard, through the normal policy routes, with the admin's own permissions. Drafts are always saved switched off.

• Adds to existing policies instead of duplicating them. When a request overlaps a policy that already exists (same group, channel or mail service), the admin chooses to add it there or create a separate one; "add printing to the USB Lock policy" picks it directly. Only a list of changes is sent (switch on, add to a list, make the action stricter), applied to the policy as it is at save time. A merge never removes anything, and never changes the name, group or on/off state.

• Validated twice. Every draft passes copilot/check.py against the form's schema before it leaves the Space; the server then accepts only save steps to the three policy-creation routes.

• Platform facts are synced, not hand-copied. Built-in rules, the 42 built-in block lists, AI sites and every form's defaults come from the cloakv2 source:

  node tools/sync_platform.mjs ..        # from hf-space/, with cloakv2 as the parent
  

  Run it whenever a policy form, preset or block list changes, review git diff copilot/platform, and redeploy the Space. Phrases live in copilot/lexicon/vocab.yaml; the loader refuses a phrase that points at a rule, list or setting the snapshot doesn't have.

• If the copilot can't load, /copilot/* returns 503, the dashboard tells the admin to use the form, and the other paths are unaffected.

Try it in a terminal: python -m copilot.cli "block usb for interns".

Files

app.py              PII model, /detect, /health, shared auth and limits; mounts the other two
docclass.py         document pipeline (type from bytes, archives, extraction, OCR, Layer 3)
copilot_api.py      /copilot routes over copilot/
copilot/            the Policy Copilot engine; platform/ holds the synced snapshot
tools/              sync_platform.mjs (not shipped in the image)
Dockerfile          the Space image; .dockerignore keeps tools and docs out