n1th1sh commited on
Commit
c4a3124
Β·
verified Β·
1 Parent(s): e7cd337

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +61 -45
README.md CHANGED
@@ -1,40 +1,52 @@
1
- # CLOAK PII API β€” HF Space
2
-
3
- ONNX INT8 CPU inference for `Wardline/CLOAK_V3.10_onnx` (model file
4
- `model.fused.int8.onnx`) + Gradio demo UI + the REST API the CLOAK server calls.
5
-
6
- Override the repo/file without editing code via Space secrets `MODEL_REPO`
7
- and `MODEL_FILE`.
8
-
9
- ## Deploy
10
- 1. Copy `app.py` + `requirements.txt` to the Space repo and push.
11
- 2. Space secrets: `HF_TOKEN` (model repo read access) and `API_SECRET`
12
- (Bearer token for `/detect`). If `API_SECRET` is unset it falls back to
13
- `HF_TOKEN` β€” works, but don't: that puts your HF account token in CLOAK's
14
- settings. Use a dedicated random secret.
15
- 3. Gotchas (learned the hard way):
16
- - On a Gradio-SDK Space the Gradio version comes from `sdk_version` in the
17
- Space README front-matter β€” the `requirements.txt` pin is ignored.
18
- - HF's image sets `GRADIO_SSR_MODE=true`, which spawns a second server on
19
- port 7861 and crashes the mount_gradio_app + uvicorn pattern (exit 3,
20
- "address already in use"). `app.py` forces it off before importing gradio.
21
-
22
- ## Wire up CLOAK
23
- In the CLOAK dashboard β†’ Settings:
24
- - **Backend URL** β†’ the Space URL (e.g. `https://<space>.hf.space`)
25
- - **API key** β†’ the `API_SECRET` value
26
-
27
- The server auto-detects the protocol: it tries `POST /detect` first and falls
28
- back to the legacy Gradio API if the URL still points at an old Space.
29
-
30
- ## Contract (must stay in sync with `server/src/detection/pii.ts`)
31
- - `POST /detect` Β· `Authorization: Bearer <API_SECRET>` Β· `{"text": "..."}`
32
- β†’ `{entities: [{type, text, start, end, confidence}], nested, redacted_text, model, processing_time_ms}`
33
- - Failures are real HTTP errors (401 bad key / 422 oversize / 503 inference) β€”
34
- never 200-with-empty, so DSPM records scan **errors**, not false "clean".
35
- - `MAX_TEXT_CHARS = 30_000` here ↔ `DETECT_MAX_CHARS = 30_000` in `pii.ts`.
36
- - `GET /health` β†’ `{status, model, engine, threads, classes, max_chars}` β€”
37
- used by the Settings backend-status probe.
 
 
 
 
 
 
 
 
 
 
 
 
38
 
39
  ## Second path: document classification (`docclass.py`)
40
 
@@ -50,19 +62,23 @@ CLOAK stores both; neither replaces the other.
50
  Same `Authorization: Bearer <API_SECRET>` as `/detect`. `/health` reports it
51
  under `docclass` (loaded, error, threshold, ocr).
52
 
53
- Code tree (the handover's layout, unchanged):
54
 
55
  ```
56
- docclass.py router, model loading, OCR concurrency, limits
57
- src/analyze.py layers 1 + 2 entry point
58
- src/layer1_format/ magic-byte identification
59
- src/layer2_containers/ zip / gzip / 7z unpacking
60
- src/content_extraction/ pdf Β· docx Β· xlsx Β· csv Β· text Β· ocr.py
61
- src/layer3_doctype/ Layer3DocType β€” ALWAYS load through this, never the raw pickle
62
- reference/policy_map.py class -> action table (configuration)
 
 
63
  tests/test_docclass.py contract + HTTP tests
64
  ```
65
 
 
 
66
  Deploy notes:
67
  - **The Space must run Python 3.11** β€” set `python_version: "3.11"` in the Space
68
  README front-matter. The model is a scikit-learn pickle built on 3.11.9 and
 
1
+ ---
2
+ title: CLOAK API
3
+ emoji: πŸ›‘οΈ
4
+ colorFrom: blue
5
+ colorTo: indigo
6
+ sdk: gradio
7
+ sdk_version: "5.50.0"
8
+ python_version: "3.11"
9
+ app_file: app.py
10
+ pinned: false
11
+ ---
12
+
13
+ # CLOAK PII API β€” HF Space
14
+
15
+ ONNX INT8 CPU inference for `Wardline/CLOAK_V3.12_onnx` (model file
16
+ `model.fused.int8.onnx`) + Gradio demo UI + the REST API the CLOAK server calls.
17
+
18
+ Override the repo/file without editing code via Space secrets `MODEL_REPO`
19
+ and `MODEL_FILE`.
20
+
21
+ ## Deploy
22
+ 1. Copy `app.py` + `requirements.txt` to the Space repo and push.
23
+ 2. Space secrets: `HF_TOKEN` (model repo read access) and `API_SECRET`
24
+ (Bearer token for `/detect`). If `API_SECRET` is unset it falls back to
25
+ `HF_TOKEN` β€” works, but don't: that puts your HF account token in CLOAK's
26
+ settings. Use a dedicated random secret.
27
+ 3. Gotchas (learned the hard way):
28
+ - On a Gradio-SDK Space the Gradio version comes from `sdk_version` in the
29
+ Space README front-matter β€” the `requirements.txt` pin is ignored.
30
+ - HF's image sets `GRADIO_SSR_MODE=true`, which spawns a second server on
31
+ port 7861 and crashes the mount_gradio_app + uvicorn pattern (exit 3,
32
+ "address already in use"). `app.py` forces it off before importing gradio.
33
+
34
+ ## Wire up CLOAK
35
+ In the CLOAK dashboard β†’ Settings:
36
+ - **Backend URL** β†’ the Space URL (e.g. `https://<space>.hf.space`)
37
+ - **API key** β†’ the `API_SECRET` value
38
+
39
+ The server auto-detects the protocol: it tries `POST /detect` first and falls
40
+ back to the legacy Gradio API if the URL still points at an old Space.
41
+
42
+ ## Contract (must stay in sync with `server/src/detection/pii.ts`)
43
+ - `POST /detect` Β· `Authorization: Bearer <API_SECRET>` Β· `{"text": "..."}`
44
+ β†’ `{entities: [{type, text, start, end, confidence}], nested, redacted_text, model, processing_time_ms}`
45
+ - Failures are real HTTP errors (401 bad key / 422 oversize / 503 inference) β€”
46
+ never 200-with-empty, so DSPM records scan **errors**, not false "clean".
47
+ - `MAX_TEXT_CHARS = 30_000` here ↔ `DETECT_MAX_CHARS = 30_000` in `pii.ts`.
48
+ - `GET /health` β†’ `{status, model, engine, threads, classes, max_chars}` β€”
49
+ used by the Settings backend-status probe.
50
 
51
  ## Second path: document classification (`docclass.py`)
52
 
 
62
  Same `Authorization: Bearer <API_SECRET>` as `/detect`. `/health` reports it
63
  under `docclass` (loaded, error, threshold, ocr).
64
 
65
+ Files:
66
 
67
  ```
68
+ app.py PII model + /detect, and mounts the document routes
69
+ docclass.py the whole document pipeline, one file, in sections:
70
+ Layer 1 real file type from the bytes
71
+ Layer 2 zip / gzip / 7z unpacking, bounded
72
+ Extraction pdf Β· docx Β· xlsx Β· csv Β· text Β· OCR
73
+ Layer 3 Layer3DocType (abstain guard + credentials rule)
74
+ Policy table class -> suggested action
75
+ API model loading, limits, /file routes
76
+ requirements.txt
77
  tests/test_docclass.py contract + HTTP tests
78
  ```
79
 
80
+ Always load the model through `Layer3DocType`, never the raw pickle.
81
+
82
  Deploy notes:
83
  - **The Space must run Python 3.11** β€” set `python_version: "3.11"` in the Space
84
  README front-matter. The model is a scikit-learn pickle built on 3.11.9 and