n1th1sh commited on
Commit
aa3f8ec
·
verified ·
1 Parent(s): 119ad66

Delete README.md

Browse files
Files changed (1) hide show
  1. README.md +0 -87
README.md DELETED
@@ -1,87 +0,0 @@
1
- # CLOAK PII API — HF Space
2
-
3
- ONNX INT8 CPU inference for `Wardline/CLOAK_V3.10_onnx` (model file
4
- `model.fused.int8.onnx`) + Gradio demo UI + the REST API the CLOAK server calls.
5
-
6
- Override the repo/file without editing code via Space secrets `MODEL_REPO`
7
- and `MODEL_FILE`.
8
-
9
- ## Deploy
10
- 1. Copy `app.py` + `requirements.txt` to the Space repo and push.
11
- 2. Space secrets: `HF_TOKEN` (model repo read access) and `API_SECRET`
12
- (Bearer token for `/detect`). If `API_SECRET` is unset it falls back to
13
- `HF_TOKEN` — works, but don't: that puts your HF account token in CLOAK's
14
- settings. Use a dedicated random secret.
15
- 3. Gotchas (learned the hard way):
16
- - On a Gradio-SDK Space the Gradio version comes from `sdk_version` in the
17
- Space README front-matter — the `requirements.txt` pin is ignored.
18
- - HF's image sets `GRADIO_SSR_MODE=true`, which spawns a second server on
19
- port 7861 and crashes the mount_gradio_app + uvicorn pattern (exit 3,
20
- "address already in use"). `app.py` forces it off before importing gradio.
21
-
22
- ## Wire up CLOAK
23
- In the CLOAK dashboard → Settings:
24
- - **Backend URL** → the Space URL (e.g. `https://<space>.hf.space`)
25
- - **API key** → the `API_SECRET` value
26
-
27
- The server auto-detects the protocol: it tries `POST /detect` first and falls
28
- back to the legacy Gradio API if the URL still points at an old Space.
29
-
30
- ## Contract (must stay in sync with `server/src/detection/pii.ts`)
31
- - `POST /detect` · `Authorization: Bearer <API_SECRET>` · `{"text": "..."}`
32
- → `{entities: [{type, text, start, end, confidence}], nested, redacted_text, model, processing_time_ms}`
33
- - Failures are real HTTP errors (401 bad key / 422 oversize / 503 inference) —
34
- never 200-with-empty, so DSPM records scan **errors**, not false "clean".
35
- - `MAX_TEXT_CHARS = 30_000` here ↔ `DETECT_MAX_CHARS = 30_000` in `pii.ts`.
36
- - `GET /health` → `{status, model, engine, threads, classes, max_chars}` —
37
- used by the Settings backend-status probe.
38
-
39
- ## Second path: document classification (`docclass.py`)
40
-
41
- `/detect` finds personal details in text. The `/file` paths say what KIND of
42
- document a file is (26 classes, including `credentials`) and suggest an action.
43
- CLOAK stores both; neither replaces the other.
44
-
45
- | Path | Body | Runs |
46
- |---|---|---|
47
- | `POST /file/classify` | `{"text": "...", "column_headers": [...]}` | Layer 3 on text CLOAK already extracted (~5 ms) |
48
- | `POST /file/analyze` | raw bytes, `Content-Type: application/octet-stream`, `X-Filename: <name>` | Layers 1+2+3: real type from bytes, archive contents, OCR for images and scanned PDFs |
49
-
50
- Same `Authorization: Bearer <API_SECRET>` as `/detect`. `/health` reports it
51
- under `docclass` (loaded, error, threshold, ocr).
52
-
53
- Code tree (the handover's layout, unchanged):
54
-
55
- ```
56
- docclass.py router, model loading, OCR concurrency, limits
57
- src/analyze.py layers 1 + 2 entry point
58
- src/layer1_format/ magic-byte identification
59
- src/layer2_containers/ zip / gzip / 7z unpacking
60
- src/content_extraction/ pdf · docx · xlsx · csv · text · ocr.py
61
- src/layer3_doctype/ Layer3DocType — ALWAYS load through this, never the raw pickle
62
- reference/policy_map.py class -> action table (configuration)
63
- tests/test_docclass.py contract + HTTP tests
64
- ```
65
-
66
- Deploy notes:
67
- - **The Space must run Python 3.11** — set `python_version: "3.11"` in the Space
68
- README front-matter. The model is a scikit-learn pickle built on 3.11.9 and
69
- `requirements.txt` pins its exact libraries; other versions can load it and
70
- silently behave differently.
71
- - The model is fetched from `DOCCLASS_REPO` (default: `MODEL_REPO`) /
72
- `DOCCLASS_FILE` (default `layer3_25class.joblib`) and **refused unless its
73
- SHA-256 matches** `DOCCLASS_SHA256` (pinned in `docclass.py`). Upload a new
74
- model → update the hash in the same change.
75
- - If the model can't load, `/file/*` returns 503 and `/detect` is unaffected.
76
- - Limits: 25 MB per file, archives 3 levels / 500 entries / 150 MB unpacked,
77
- at most 2 OCR jobs at once (`DOCCLASS_OCR_CONCURRENCY`), first 3 pages of a
78
- scanned PDF (`DOCCLASS_PDF_OCR_PAGES`).
79
- - `/file/analyze` receives whole files (including photos of ID cards). For
80
- client installs that must not send files to Hugging Face, run this same
81
- Space container on their own host and point CLOAK at it.
82
-
83
- Test locally:
84
-
85
- ```bash
86
- DOCCLASS_MODEL_PATH=path/to/layer3_25class.joblib python -m pytest tests -q
87
- ```