Spaces:
Running
Running
deploy with engine 2cfbe0d5a64aec03c49078eb558d71fa0c6ae05d
Browse files- README.md +277 -2
- app/guard.py +19 -3
- smoke.sh +105 -0
- tests/test_guard.py +25 -1
README.md
CHANGED
|
@@ -12,5 +12,280 @@ pinned: false
|
|
| 12 |
# postkitty API
|
| 13 |
|
| 14 |
The HTTP side of [postkitty](https://github.com/mertkayacs/postkitty). Send one post (text, photos or a
|
| 15 |
-
landscape video) and get back a file for each social platform,
|
| 16 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 12 |
# postkitty API
|
| 13 |
|
| 14 |
The HTTP side of [postkitty](https://github.com/mertkayacs/postkitty). Send one post (text, photos or a
|
| 15 |
+
landscape video) and get back a file for each social platform: Instagram, TikTok, YouTube Shorts, X,
|
| 16 |
+
LinkedIn, Threads, Bluesky and Facebook. Every file is checked against that platform's published limits
|
| 17 |
+
and repaired when it fails.
|
| 18 |
+
|
| 19 |
+
It runs the same engine as the postkitty plugin for Hermes Agent. There is no language model behind it, so
|
| 20 |
+
for text it counts, checks and splits into threads, and it never rewrites. It takes uploads only, never
|
| 21 |
+
fetches a URL, never posts anywhere, and deletes each job 60 minutes after it finishes.
|
| 22 |
+
|
| 23 |
+
- Guide: https://postkitty.mertkayacs.com/docs/
|
| 24 |
+
- Try it in the browser: https://postkitty.mertkayacs.com/try/
|
| 25 |
+
- Interactive reference: [`/docs`](https://mertkayacs-postkitty-api.hf.space/docs), OpenAPI document:
|
| 26 |
+
[`openapi.json`](openapi.json)
|
| 27 |
+
|
| 28 |
+
## Endpoints
|
| 29 |
+
|
| 30 |
+
Base URL: `https://mertkayacs-postkitty-api.hf.space`
|
| 31 |
+
|
| 32 |
+
| Method and path | What it does |
|
| 33 |
+
|---|---|
|
| 34 |
+
| `GET /v1/health` | Service, engine and ffmpeg versions |
|
| 35 |
+
| `GET /v1/limits` | Every limit below, as numbers |
|
| 36 |
+
| `GET /v1/platforms` | Platforms, targets, sizes and limits with their sources |
|
| 37 |
+
| `POST /v1/text` | Count a text with each platform's own rule |
|
| 38 |
+
| `POST /v1/check` | Check a file you already have against one target |
|
| 39 |
+
| `POST /v1/jobs` | Turn one post into files for every platform |
|
| 40 |
+
| `GET /v1/jobs/{id}` | Status, progress and results of a job |
|
| 41 |
+
| `GET /v1/jobs/{id}/files/{path}` | One result file |
|
| 42 |
+
| `GET /v1/jobs/{id}/download` | Every result as one zip |
|
| 43 |
+
| `DELETE /v1/jobs/{id}` | Delete a job and its files now |
|
| 44 |
+
|
| 45 |
+
The responses below are copied from the running service.
|
| 46 |
+
|
| 47 |
+
### Count a text
|
| 48 |
+
|
| 49 |
+
```bash
|
| 50 |
+
curl -X POST "$API/v1/text" -H 'content-type: application/json' -d '{
|
| 51 |
+
"text": "We shipped postkitty 1.0 today. One post in, every platform out. https://postkitty.mertkayacs.com #launch",
|
| 52 |
+
"platforms": ["x", "instagram"]}'
|
| 53 |
+
```
|
| 54 |
+
|
| 55 |
+
```json
|
| 56 |
+
{
|
| 57 |
+
"all_ok": true,
|
| 58 |
+
"platforms": {
|
| 59 |
+
"x": {
|
| 60 |
+
"label": "X", "ok": true, "enforcement": "hard", "unit": "x_weighted", "limit": 280, "count": 96,
|
| 61 |
+
"overflow": 0, "split": false,
|
| 62 |
+
"parts": [{"text": "We shipped postkitty 1.0 today. One post in, every platform out. https://postkitty.mertkayacs.com #launch", "count": 96}],
|
| 63 |
+
"hashtags": 1, "mentions": 0, "urls": 1, "fold": null, "notes": []
|
| 64 |
+
},
|
| 65 |
+
"instagram": {
|
| 66 |
+
"label": "Instagram", "ok": true, "enforcement": "hard", "unit": "codepoints", "limit": 2200, "count": 105,
|
| 67 |
+
"overflow": 0, "split": false,
|
| 68 |
+
"parts": [{"text": "We shipped postkitty 1.0 today. One post in, every platform out. https://postkitty.mertkayacs.com #launch", "count": 105}],
|
| 69 |
+
"hashtags": 1, "mentions": 0, "urls": 1,
|
| 70 |
+
"fold": {"limit": 125, "shown": 105, "text": "We shipped postkitty 1.0 today. One post in, every platform out. https://postkitty.mertkayacs.com #launch"},
|
| 71 |
+
"notes": []
|
| 72 |
+
}
|
| 73 |
+
}
|
| 74 |
+
}
|
| 75 |
+
```
|
| 76 |
+
|
| 77 |
+
The same text is 96 on X and 105 on Instagram because X counts every link as 23. `platforms` takes platform
|
| 78 |
+
names or target ids; leave it out for all eight. Text over the limit on X, Threads or Bluesky comes back
|
| 79 |
+
split into numbered `parts` (`split: true`); send `"split_threads": false` to switch that off. `fold` is the
|
| 80 |
+
part of the text a platform shows before it cuts the post with "more".
|
| 81 |
+
|
| 82 |
+
### Check a file you already have
|
| 83 |
+
|
| 84 |
+
```bash
|
| 85 |
+
curl -F file=@team.jpg -F target=x_image "$API/v1/check"
|
| 86 |
+
```
|
| 87 |
+
|
| 88 |
+
```json
|
| 89 |
+
{
|
| 90 |
+
"file": "team.jpg", "kind": "image", "target": "x_image", "pass": false,
|
| 91 |
+
"checks": [
|
| 92 |
+
{"id": "image.width", "status": "fail", "hard": true, "measured": 2752, "expected": 1600,
|
| 93 |
+
"message": "The image width is not exact.", "source_url": null, "checked_on": null},
|
| 94 |
+
{"id": "image.height", "status": "fail", "hard": true, "measured": 1536, "expected": 900,
|
| 95 |
+
"message": "The image height is not exact.", "source_url": null, "checked_on": null},
|
| 96 |
+
{"id": "image.aspect", "status": "fail", "hard": true, "measured": 1.7916666666666667, "expected": 1.7777777777777777,
|
| 97 |
+
"message": "The image aspect ratio differs by more than 0.5%.", "source_url": null, "checked_on": null}
|
| 98 |
+
]
|
| 99 |
+
}
|
| 100 |
+
```
|
| 101 |
+
|
| 102 |
+
A file that fits answers `{"file": "image.jpg", "kind": "image", "target": "x_image", "pass": true, "checks": []}`.
|
| 103 |
+
`target` is a target id from `GET /v1/platforms`, for example `instagram_feed` or `tiktok`.
|
| 104 |
+
|
| 105 |
+
### Make the files for a post
|
| 106 |
+
|
| 107 |
+
```bash
|
| 108 |
+
curl -F files=@team.jpg -F "text=Meet the team behind postkitty. #launch" "$API/v1/jobs"
|
| 109 |
+
```
|
| 110 |
+
|
| 111 |
+
```json
|
| 112 |
+
{"id": "5q4UbSK9M9xX6Y-XzozB_UoE6jjHIk1-", "status": "queued",
|
| 113 |
+
"links": {"status": "/v1/jobs/5q4UbSK9M9xX6Y-XzozB_UoE6jjHIk1-",
|
| 114 |
+
"download": "/v1/jobs/5q4UbSK9M9xX6Y-XzozB_UoE6jjHIk1-/download"}}
|
| 115 |
+
```
|
| 116 |
+
|
| 117 |
+
Form fields, all optional, with at least one of `files` and `text`:
|
| 118 |
+
|
| 119 |
+
| Field | Value |
|
| 120 |
+
|---|---|
|
| 121 |
+
| `files` | repeat the field: up to 6 photos, or 1 video |
|
| 122 |
+
| `text` | the post text |
|
| 123 |
+
| `platforms` | platform names or target ids, repeated or comma-separated. Leave out for every target that fits the files |
|
| 124 |
+
| `layout` | `auto`, `crop`, `split`, `speaker`, `seam` or `pad`, applied to every media target (`speaker` is for video, `seam` for photos) |
|
| 125 |
+
| `focus` | `auto`, `left`, `center`, `right`, `person:N` (1 is the leftmost person) or `x:0.0` to `x:1.0` |
|
| 126 |
+
|
| 127 |
+
Poll `links.status` every 2 seconds. `status` goes from `queued` to `running` to `done` or `failed`, and
|
| 128 |
+
`progress` says where the job is (`{"stage": "render", "note": "Rendering and checking 7 files.", "position": null}`).
|
| 129 |
+
The job above, finished (two of its seven targets shown):
|
| 130 |
+
|
| 131 |
+
```json
|
| 132 |
+
{
|
| 133 |
+
"id": "5q4UbSK9M9xX6Y-XzozB_UoE6jjHIk1-",
|
| 134 |
+
"status": "done",
|
| 135 |
+
"progress": {"stage": "done", "note": "Finished.", "position": null},
|
| 136 |
+
"created_at": "2026-11-11T16:17:55Z",
|
| 137 |
+
"started_at": "2026-11-11T16:17:55Z",
|
| 138 |
+
"finished_at": "2026-11-11T16:17:59Z",
|
| 139 |
+
"expires_at": "2026-11-11T17:17:59Z",
|
| 140 |
+
"inputs": [{"name": "team.jpg", "kind": "image", "width": 2752, "height": 1536, "duration": null, "people": 5}],
|
| 141 |
+
"targets": [
|
| 142 |
+
{"id": "instagram_feed", "platform": "instagram", "kind": "image", "status": "ready",
|
| 143 |
+
"file": "instagram/feed.jpg",
|
| 144 |
+
"url": "/v1/jobs/5q4UbSK9M9xX6Y-XzozB_UoE6jjHIk1-/files/instagram/feed.jpg",
|
| 145 |
+
"cover": null, "cover_url": null, "bytes": 291225, "width": 1080, "height": 1350, "duration": null,
|
| 146 |
+
"layout": "pad", "note": "crop would cut a subject",
|
| 147 |
+
"problems": [], "advisories": [], "repairs": [], "reason": null},
|
| 148 |
+
{"id": "x_image", "platform": "x", "kind": "image", "status": "ready",
|
| 149 |
+
"file": "x/image.jpg",
|
| 150 |
+
"url": "/v1/jobs/5q4UbSK9M9xX6Y-XzozB_UoE6jjHIk1-/files/x/image.jpg",
|
| 151 |
+
"cover": null, "cover_url": null, "bytes": 413696, "width": 1600, "height": 900, "duration": null,
|
| 152 |
+
"layout": "crop", "note": "all subjects fit in one crop",
|
| 153 |
+
"problems": [], "advisories": [], "repairs": [], "reason": null}
|
| 154 |
+
],
|
| 155 |
+
"text": {
|
| 156 |
+
"x": {"label": "X", "ok": true, "enforcement": "hard", "unit": "x_weighted", "limit": 280, "count": 39,
|
| 157 |
+
"overflow": 0, "split": false,
|
| 158 |
+
"parts": [{"text": "Meet the team behind postkitty. #launch", "count": 39}],
|
| 159 |
+
"hashtags": 1, "mentions": 0, "urls": 0, "fold": null, "notes": [], "files": ["x/post.txt"]}
|
| 160 |
+
},
|
| 161 |
+
"files": [
|
| 162 |
+
{"path": "instagram/feed.jpg", "url": "/v1/jobs/5q4UbSK9M9xX6Y-XzozB_UoE6jjHIk1-/files/instagram/feed.jpg",
|
| 163 |
+
"platform": "instagram", "kind": "image", "bytes": 291225},
|
| 164 |
+
{"path": "x/post.txt", "url": "/v1/jobs/5q4UbSK9M9xX6Y-XzozB_UoE6jjHIk1-/files/x/post.txt",
|
| 165 |
+
"platform": "x", "kind": "text", "bytes": 40}
|
| 166 |
+
],
|
| 167 |
+
"notes": [],
|
| 168 |
+
"error": null,
|
| 169 |
+
"links": {"status": "/v1/jobs/5q4UbSK9M9xX6Y-XzozB_UoE6jjHIk1-",
|
| 170 |
+
"download": "/v1/jobs/5q4UbSK9M9xX6Y-XzozB_UoE6jjHIk1-/download"}
|
| 171 |
+
}
|
| 172 |
+
```
|
| 173 |
+
|
| 174 |
+
A target's `status` is `ready`, `repaired` (a check failed and the file was fixed), `failed` (see
|
| 175 |
+
`problems`) or `skipped` (see `reason`). Only `ready` and `repaired` targets have a file. A video target
|
| 176 |
+
also has a `cover` frame and a `duration`:
|
| 177 |
+
|
| 178 |
+
```json
|
| 179 |
+
{"id": "tiktok", "platform": "tiktok", "kind": "video", "status": "ready",
|
| 180 |
+
"file": "tiktok/tiktok.mp4", "url": "/v1/jobs/EvmyTgH4E8Gg602QxsKYTWtTjTG3PliN/files/tiktok/tiktok.mp4",
|
| 181 |
+
"cover": "tiktok/tiktok_cover.jpg",
|
| 182 |
+
"cover_url": "/v1/jobs/EvmyTgH4E8Gg602QxsKYTWtTjTG3PliN/files/tiktok/tiktok_cover.jpg",
|
| 183 |
+
"bytes": 11142368, "width": 1080, "height": 1920, "duration": 15.1,
|
| 184 |
+
"layout": "split", "note": "two subjects cannot share one vertical window",
|
| 185 |
+
"problems": [], "advisories": [], "repairs": [], "reason": null}
|
| 186 |
+
```
|
| 187 |
+
|
| 188 |
+
Every `url` and every link starts with `/v1/`; put the base URL in front. Files download as attachments.
|
| 189 |
+
The zip from `links.download` holds `ready/<platform>/...` plus `board.html`, a review page for the whole
|
| 190 |
+
job. The job id is the only key to a job, so keep it.
|
| 191 |
+
|
| 192 |
+
### Errors
|
| 193 |
+
|
| 194 |
+
Every error has the same shape: a stable `code`, one plain sentence, and what to do about it.
|
| 195 |
+
|
| 196 |
+
```json
|
| 197 |
+
{"error": {"code": "unsupported_type",
|
| 198 |
+
"message": "a.jpg is not a photo or video this service accepts.",
|
| 199 |
+
"hint": "Upload a JPEG, PNG, WebP or GIF photo, or an MP4, MOV, MKV or WebM video."}}
|
| 200 |
+
```
|
| 201 |
+
|
| 202 |
+
| HTTP | Codes |
|
| 203 |
+
|---|---|
|
| 204 |
+
| 403 | `origin_not_allowed` |
|
| 205 |
+
| 404 | `not_found`, `job_not_found`, `file_not_found` |
|
| 206 |
+
| 405 | `method_not_allowed` |
|
| 207 |
+
| 409 | `job_not_ready` |
|
| 208 |
+
| 413 | `upload_too_large`, `file_too_large` |
|
| 209 |
+
| 415 | `unsupported_type` |
|
| 210 |
+
| 422 | `invalid_request`, `empty_text`, `empty_job`, `text_too_long`, `too_many_files`, `unknown_platform`, `unknown_target`, `wrong_media_kind`, `no_matching_target`, `invalid_layout`, `invalid_focus`, `unreadable_file`, `image_too_large`, `animated_image`, `video_too_long`, `video_too_large`, `check_failed` |
|
| 211 |
+
| 429 | `rate_limited` (with `Retry-After`) |
|
| 212 |
+
| 503 | `queue_full` (with `Retry-After`) |
|
| 213 |
+
| 504 | `check_timeout` |
|
| 214 |
+
| 500 | `internal_error` |
|
| 215 |
+
|
| 216 |
+
A job that fails after it was accepted reports `job_failed`, `job_timeout` or `job_crashed` in the `error`
|
| 217 |
+
field of its status.
|
| 218 |
+
|
| 219 |
+
## Limits
|
| 220 |
+
|
| 221 |
+
`GET /v1/limits` returns these as numbers.
|
| 222 |
+
|
| 223 |
+
| What | Limit |
|
| 224 |
+
|---|---|
|
| 225 |
+
| Files per job | 6 photos, or 1 video |
|
| 226 |
+
| Photo | JPEG, PNG, WebP or GIF (not animated), 20 MB, 50 megapixels |
|
| 227 |
+
| Video | MP4, MOV, MKV or WebM, 80 MB, 120 seconds, 4096 pixels on the longer side |
|
| 228 |
+
| Text | 20,000 characters |
|
| 229 |
+
| Queue | 8 jobs waiting, one running |
|
| 230 |
+
| Time per job | 15 minutes, then the job is stopped and reported as failed |
|
| 231 |
+
| Requests per hour and address | 12 jobs, 120 text or check calls, 3,600 reads |
|
| 232 |
+
| Kept for | 60 minutes after a job finishes, or until `DELETE` |
|
| 233 |
+
|
| 234 |
+
## What happens to an upload
|
| 235 |
+
|
| 236 |
+
A file is accepted only when its first bytes are a JPEG, PNG, WebP, GIF, MP4, MOV, MKV or WebM signature,
|
| 237 |
+
and then only when Pillow or ffprobe reads it as that type. Playlists, concat scripts, SVG and every other
|
| 238 |
+
text format are refused, because those are the files that make ffmpeg or a browser read local files or
|
| 239 |
+
fetch URLs. Pixel count, video size and video length are checked before anything is decoded.
|
| 240 |
+
|
| 241 |
+
Uploads are stored under random names; the file name you send is only shown back as a label. The upload
|
| 242 |
+
and all working files are deleted when the job finishes. The results stay for 60 minutes and are then
|
| 243 |
+
deleted too. Job ids are random and long, and a result path is served only if the job itself listed it.
|
| 244 |
+
The review board is HTML with your text in it, so it is only ever inside the zip. No file contents and no
|
| 245 |
+
addresses are written to the log.
|
| 246 |
+
|
| 247 |
+
Each job runs in its own process with a time limit, so a render that hangs is stopped and one that crashes
|
| 248 |
+
does not take the service down.
|
| 249 |
+
|
| 250 |
+
## Run the tests
|
| 251 |
+
|
| 252 |
+
The tests need a checkout of the engine and ffmpeg. They never load the face model: the engine replays
|
| 253 |
+
recorded detections when `POSTKITTY_DETECTIONS` is set, and the test setup sets it.
|
| 254 |
+
|
| 255 |
+
```bash
|
| 256 |
+
git clone https://github.com/mertkayacs/postkitty /somewhere/postkitty
|
| 257 |
+
uv venv .venv && uv pip install -p .venv/bin/python -r requirements.txt pytest httpx
|
| 258 |
+
POSTKITTY_SRC=/somewhere/postkitty .venv/bin/python -m pytest
|
| 259 |
+
```
|
| 260 |
+
|
| 261 |
+
## Deploy
|
| 262 |
+
|
| 263 |
+
The Space builds the `Dockerfile`: Ubuntu 24.04 with Python 3.12 and ffmpeg 6.1, the engine at the commit
|
| 264 |
+
named by `POSTKITTY_REF`, and the face model fetched at build time.
|
| 265 |
+
|
| 266 |
+
```bash
|
| 267 |
+
./deploy.sh # push this folder as it is
|
| 268 |
+
./deploy.sh <commit> # pin the engine to that commit of mertkayacs/postkitty, then push
|
| 269 |
+
```
|
| 270 |
+
|
| 271 |
+
Then run the smoke test against the live service with a real photo and a short clip:
|
| 272 |
+
|
| 273 |
+
```bash
|
| 274 |
+
./smoke.sh https://mertkayacs-postkitty-api.hf.space team.jpg clip.mp4 /tmp/postkitty-smoke
|
| 275 |
+
```
|
| 276 |
+
|
| 277 |
+
## Layout
|
| 278 |
+
|
| 279 |
+
```
|
| 280 |
+
app/main.py routes, error handlers, the one page at /
|
| 281 |
+
app/engine.py the only module that imports and calls the postkitty engine
|
| 282 |
+
app/runner.py the child process that runs one job or one check
|
| 283 |
+
app/jobs.py jobs in memory, the worker, the queue, the time limit, deletion
|
| 284 |
+
app/uploads.py what an upload is, from its bytes
|
| 285 |
+
app/guard.py origin check, rate limits and the upload size cap, before a body is read
|
| 286 |
+
app/schemas.py request and response shapes
|
| 287 |
+
app/config.py the limits
|
| 288 |
+
tests/ pytest suite
|
| 289 |
+
```
|
| 290 |
+
|
| 291 |
+
postkitty by Mert Kaya. MIT license.
|
app/guard.py
CHANGED
|
@@ -1,4 +1,4 @@
|
|
| 1 |
-
"""What happens before any request body is read:
|
| 2 |
|
| 3 |
from __future__ import annotations
|
| 4 |
|
|
@@ -6,7 +6,7 @@ import math
|
|
| 6 |
import time
|
| 7 |
from collections.abc import Callable
|
| 8 |
|
| 9 |
-
from .config import MB
|
| 10 |
from .errors import ApiError, error_response
|
| 11 |
|
| 12 |
WINDOW = 3600.0
|
|
@@ -62,8 +62,19 @@ def request_class(method: str, path: str) -> str | None:
|
|
| 62 |
return "read"
|
| 63 |
|
| 64 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 65 |
class Guard:
|
| 66 |
-
"""ASGI middleware: rate limit, cap the body size
|
| 67 |
|
| 68 |
def __init__(self, app, limiter: RateLimiter, body_caps: dict[str, int]):
|
| 69 |
self.app = app
|
|
@@ -86,6 +97,11 @@ class Guard:
|
|
| 86 |
await send(message)
|
| 87 |
|
| 88 |
kind = request_class(scope["method"], path)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 89 |
if kind:
|
| 90 |
wait = self.limiter.take(client_ip(headers, (scope.get("client") or ("",))[0]), kind)
|
| 91 |
if wait:
|
|
|
|
| 1 |
+
"""What happens before any request body is read: the origin check, rate limits and the upload size cap."""
|
| 2 |
|
| 3 |
from __future__ import annotations
|
| 4 |
|
|
|
|
| 6 |
import time
|
| 7 |
from collections.abc import Callable
|
| 8 |
|
| 9 |
+
from .config import ALLOWED_ORIGINS, MB
|
| 10 |
from .errors import ApiError, error_response
|
| 11 |
|
| 12 |
WINDOW = 3600.0
|
|
|
|
| 62 |
return "read"
|
| 63 |
|
| 64 |
|
| 65 |
+
def foreign_origin(headers: dict[bytes, bytes]) -> bool:
|
| 66 |
+
"""True when a browser sends a cross-site request from a page that is not on the allowlist.
|
| 67 |
+
|
| 68 |
+
CORS response headers alone do not keep other websites out, because a proxy in front of the service can
|
| 69 |
+
add its own. Browsers set Sec-Fetch-Site themselves and a page cannot change it, so the allowlist is
|
| 70 |
+
enforced on the request. A site that proxies this API under its own address is same-origin and passes.
|
| 71 |
+
"""
|
| 72 |
+
origin = headers.get(b"origin", b"").decode("latin-1")
|
| 73 |
+
return bool(origin) and headers.get(b"sec-fetch-site") == b"cross-site" and origin not in ALLOWED_ORIGINS
|
| 74 |
+
|
| 75 |
+
|
| 76 |
class Guard:
|
| 77 |
+
"""ASGI middleware: refuse other websites, rate limit, cap the body size, and harden every response."""
|
| 78 |
|
| 79 |
def __init__(self, app, limiter: RateLimiter, body_caps: dict[str, int]):
|
| 80 |
self.app = app
|
|
|
|
| 97 |
await send(message)
|
| 98 |
|
| 99 |
kind = request_class(scope["method"], path)
|
| 100 |
+
if kind and foreign_origin(headers):
|
| 101 |
+
refusal = error_response(403, "origin_not_allowed", "This API does not take requests from other websites.",
|
| 102 |
+
f"Use it on {ALLOWED_ORIGINS[0]}, or call it from your own server.")
|
| 103 |
+
await refusal(scope, receive, send_hardened)
|
| 104 |
+
return
|
| 105 |
if kind:
|
| 106 |
wait = self.limiter.take(client_ip(headers, (scope.get("client") or ("",))[0]), kind)
|
| 107 |
if wait:
|
smoke.sh
ADDED
|
@@ -0,0 +1,105 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env bash
|
| 2 |
+
# Smoke test against a running postkitty API with real media. Prints one line per step with its timing
|
| 3 |
+
# and exits non-zero on the first failed expectation.
|
| 4 |
+
#
|
| 5 |
+
# ./smoke.sh <base-url> <photo-with-people> <short-video> <output-folder>
|
| 6 |
+
#
|
| 7 |
+
# Needs curl, python3 and ffprobe. Results are downloaded into the output folder so they can be looked at.
|
| 8 |
+
set -euo pipefail
|
| 9 |
+
|
| 10 |
+
BASE="${1:?base url, for example https://mertkayacs-postkitty-api.hf.space}"
|
| 11 |
+
PHOTO="${2:?path of a photo with people in it}"
|
| 12 |
+
VIDEO="${3:?path of a short landscape video}"
|
| 13 |
+
OUT="${4:?folder for the downloaded results}"
|
| 14 |
+
mkdir -p "$OUT"
|
| 15 |
+
|
| 16 |
+
json() { python3 -c "import json,sys; d=json.load(sys.stdin); print($1)"; }
|
| 17 |
+
fail() { echo "FAIL: $*" >&2; exit 1; }
|
| 18 |
+
status_of() { curl -sS -o /dev/null -w '%{http_code}' "$@"; }
|
| 19 |
+
|
| 20 |
+
step() { # step <label> <curl args...>: runs curl, prints "<label> <http> <seconds>s", leaves the body in $BODY
|
| 21 |
+
local label="$1"; shift
|
| 22 |
+
local meta
|
| 23 |
+
meta=$(curl -sS -o "$OUT/.body" -w '%{http_code} %{time_total}' "$@")
|
| 24 |
+
BODY=$(cat "$OUT/.body")
|
| 25 |
+
echo "$label: HTTP ${meta% *} in ${meta#* }s"
|
| 26 |
+
CODE="${meta% *}"
|
| 27 |
+
}
|
| 28 |
+
|
| 29 |
+
wait_for_job() { # wait_for_job <id>: polls until done or failed, prints stage changes with elapsed seconds
|
| 30 |
+
local id="$1" start last="" stage state
|
| 31 |
+
start=$(date +%s)
|
| 32 |
+
while true; do
|
| 33 |
+
BODY=$(curl -sS "$BASE/v1/jobs/$id")
|
| 34 |
+
state=$(echo "$BODY" | json 'd["status"]')
|
| 35 |
+
stage=$(echo "$BODY" | json 'd["progress"]["stage"] + ": " + d["progress"]["note"]')
|
| 36 |
+
if [ "$stage" != "$last" ]; then echo " +$(( $(date +%s) - start ))s $stage"; last="$stage"; fi
|
| 37 |
+
case "$state" in done|failed) break ;; esac
|
| 38 |
+
sleep 2
|
| 39 |
+
done
|
| 40 |
+
echo " job $state after $(( $(date +%s) - start ))s"
|
| 41 |
+
[ "$state" = done ] || fail "job $id: $(echo "$BODY" | json 'd["error"]')"
|
| 42 |
+
}
|
| 43 |
+
|
| 44 |
+
step "health" "$BASE/v1/health"
|
| 45 |
+
[ "$CODE" = 200 ] || fail "health"
|
| 46 |
+
echo " $BODY"
|
| 47 |
+
|
| 48 |
+
step "text" -X POST "$BASE/v1/text" -H 'content-type: application/json' \
|
| 49 |
+
-d '{"text": "We shipped postkitty 1.0 today. One post in, every platform out. https://postkitty.mertkayacs.com #launch"}'
|
| 50 |
+
[ "$(echo "$BODY" | json 'd["all_ok"], len(d["platforms"]), d["platforms"]["x"]["count"]')" = "True 8 96" ] || fail "text: $BODY"
|
| 51 |
+
|
| 52 |
+
step "check photo against instagram_feed" -F "file=@$PHOTO" -F target=instagram_feed "$BASE/v1/check"
|
| 53 |
+
[ "$CODE" = 200 ] || fail "check: $BODY"
|
| 54 |
+
echo " pass=$(echo "$BODY" | json 'd["pass"]') checks=$(echo "$BODY" | json '[c["id"] for c in d["checks"]]')"
|
| 55 |
+
|
| 56 |
+
step "photo job accepted" -F "files=@$PHOTO" -F "text=Meet the team behind postkitty. #launch" "$BASE/v1/jobs"
|
| 57 |
+
[ "$CODE" = 202 ] || fail "photo job: $BODY"
|
| 58 |
+
PHOTO_JOB=$(echo "$BODY" | json 'd["id"]')
|
| 59 |
+
wait_for_job "$PHOTO_JOB"
|
| 60 |
+
echo "$BODY" > "$OUT/photo-job.json"
|
| 61 |
+
echo " people found: $(echo "$BODY" | json 'd["inputs"][0]["people"]')"
|
| 62 |
+
echo "$BODY" | json '"\n".join(" %-16s %-9s %sx%s %s bytes %s" % (t["id"], t["status"], t["width"], t["height"], t["bytes"], t["layout"]) for t in d["targets"])'
|
| 63 |
+
[ "$(echo "$BODY" | json 'all(t["status"] in ("ready", "repaired") for t in d["targets"]) and len(d["targets"]) > 0')" = True ] \
|
| 64 |
+
|| fail "a photo target is not ready"
|
| 65 |
+
for path in $(echo "$BODY" | json '" ".join(f["path"] for f in d["files"] if f["kind"] == "image")'); do
|
| 66 |
+
mkdir -p "$OUT/photo/$(dirname "$path")"
|
| 67 |
+
curl -sS -o "$OUT/photo/$path" "$BASE/v1/jobs/$PHOTO_JOB/files/$path"
|
| 68 |
+
done
|
| 69 |
+
headers=$(curl -sS -D - -o /dev/null "$BASE/v1/jobs/$PHOTO_JOB/files/instagram/feed.jpg" | tr -d '\r' | tr 'A-Z' 'a-z')
|
| 70 |
+
echo "$headers" | grep -q '^content-disposition: attachment; filename="instagram-feed.jpg"' || fail "no attachment header"
|
| 71 |
+
echo "$headers" | grep -q '^x-content-type-options: nosniff' || fail "no nosniff header"
|
| 72 |
+
echo "$headers" | grep -q '^content-type: image/jpeg' || fail "wrong content type"
|
| 73 |
+
meta=$(curl -sS -o "$OUT/photo.zip" -w '%{http_code} %{time_total}' "$BASE/v1/jobs/$PHOTO_JOB/download")
|
| 74 |
+
echo "photo zip: HTTP ${meta% *} in ${meta#* }s"
|
| 75 |
+
python3 -c "import sys,zipfile; n=zipfile.ZipFile(sys.argv[1]).namelist(); assert 'board.html' in n and any(x.startswith('ready/') for x in n), n; print(' zip holds', len(n), 'files')" "$OUT/photo.zip"
|
| 76 |
+
|
| 77 |
+
step "video job accepted" -F "files=@$VIDEO" -F "text=Two minutes on why we built postkitty." "$BASE/v1/jobs"
|
| 78 |
+
[ "$CODE" = 202 ] || fail "video job: $BODY"
|
| 79 |
+
VIDEO_JOB=$(echo "$BODY" | json 'd["id"]')
|
| 80 |
+
wait_for_job "$VIDEO_JOB"
|
| 81 |
+
echo "$BODY" > "$OUT/video-job.json"
|
| 82 |
+
echo " people found: $(echo "$BODY" | json 'd["inputs"][0]["people"]')"
|
| 83 |
+
echo "$BODY" | json '"\n".join(" %-16s %-9s %sx%s %ss %s bytes %s" % (t["id"], t["status"], t["width"], t["height"], t["duration"], t["bytes"], t["layout"]) for t in d["targets"])'
|
| 84 |
+
READY=$(echo "$BODY" | json '" ".join(t["file"] for t in d["targets"] if t["file"])')
|
| 85 |
+
[ -n "$READY" ] || fail "no video target is ready"
|
| 86 |
+
for path in $READY $(echo "$BODY" | json '" ".join(t["cover"] for t in d["targets"] if t["cover"])'); do
|
| 87 |
+
mkdir -p "$OUT/video/$(dirname "$path")"
|
| 88 |
+
curl -sS -o "$OUT/video/$path" "$BASE/v1/jobs/$VIDEO_JOB/files/$path"
|
| 89 |
+
done
|
| 90 |
+
for path in $READY; do
|
| 91 |
+
echo " $path: $(ffprobe -v error -show_entries stream=codec_name,width,height:format=duration -of csv=p=0 "$OUT/video/$path" | tr '\n' ' ')"
|
| 92 |
+
done
|
| 93 |
+
|
| 94 |
+
[ "$(status_of -F "files=@$0;filename=clip.mp4" "$BASE/v1/jobs")" = 415 ] || fail "a shell script named clip.mp4 was accepted"
|
| 95 |
+
[ "$(status_of -H 'Origin: https://other.example' -H 'Sec-Fetch-Site: cross-site' "$BASE/v1/health")" = 403 ] \
|
| 96 |
+
|| fail "a cross-site request from another website was let in"
|
| 97 |
+
[ "$(python3 -c "print('{\"text\": \"' + 'x' * 2100000 + '\"}')" | status_of -X POST -H 'content-type: application/json' --data-binary @- "$BASE/v1/text")" = 413 ] \
|
| 98 |
+
|| fail "a 2 MB text body was read"
|
| 99 |
+
[ "$(status_of "$BASE/v1/jobs/$PHOTO_JOB/files/..%2F..%2Frequest.json")" = 404 ] || fail "path traversal"
|
| 100 |
+
[ "$(status_of "$BASE/v1/jobs/not-a-job")" = 404 ] || fail "unknown job"
|
| 101 |
+
step "delete photo job" -X DELETE "$BASE/v1/jobs/$PHOTO_JOB"
|
| 102 |
+
[ "$(status_of "$BASE/v1/jobs/$PHOTO_JOB")" = 404 ] || fail "job still there after delete"
|
| 103 |
+
curl -sS -o /dev/null -X DELETE "$BASE/v1/jobs/$VIDEO_JOB"
|
| 104 |
+
rm -f "$OUT/.body"
|
| 105 |
+
echo "OK: results are in $OUT"
|
tests/test_guard.py
CHANGED
|
@@ -4,7 +4,7 @@ from __future__ import annotations
|
|
| 4 |
|
| 5 |
from conftest import error_of
|
| 6 |
|
| 7 |
-
from app.guard import RateLimiter, client_ip, request_class
|
| 8 |
|
| 9 |
|
| 10 |
class Clock:
|
|
@@ -110,3 +110,27 @@ def test_upload_without_a_length_is_cut_off_at_the_cap(make_client):
|
|
| 110 |
|
| 111 |
def test_text_body_has_a_small_cap_of_its_own(client):
|
| 112 |
error_of(client.post("/v1/text", json={"text": "x" * (2 * 1024 * 1024)}), 413, "upload_too_large")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4 |
|
| 5 |
from conftest import error_of
|
| 6 |
|
| 7 |
+
from app.guard import RateLimiter, client_ip, foreign_origin, request_class
|
| 8 |
|
| 9 |
|
| 10 |
class Clock:
|
|
|
|
| 110 |
|
| 111 |
def test_text_body_has_a_small_cap_of_its_own(client):
|
| 112 |
error_of(client.post("/v1/text", json={"text": "x" * (2 * 1024 * 1024)}), 413, "upload_too_large")
|
| 113 |
+
|
| 114 |
+
|
| 115 |
+
def test_other_websites_are_refused_whatever_a_proxy_adds_to_the_response(client):
|
| 116 |
+
def from_page(origin: str) -> dict:
|
| 117 |
+
return {"Origin": origin, "Sec-Fetch-Site": "cross-site"}
|
| 118 |
+
|
| 119 |
+
error = error_of(client.post("/v1/text", json={"text": "hello"}, headers=from_page("https://evil.example")),
|
| 120 |
+
403, "origin_not_allowed")
|
| 121 |
+
assert "https://postkitty.mertkayacs.com" in error["hint"]
|
| 122 |
+
error_of(client.get("/v1/platforms", headers=from_page("null")), 403, "origin_not_allowed")
|
| 123 |
+
error_of(client.post("/v1/jobs", data={"text": "hello"}, headers=from_page("http://localhost:4173")),
|
| 124 |
+
403, "origin_not_allowed")
|
| 125 |
+
assert list(client.app.state.store._jobs_dir.iterdir()) == []
|
| 126 |
+
|
| 127 |
+
|
| 128 |
+
def test_the_site_local_dev_same_origin_proxies_and_non_browsers_are_let_in(client):
|
| 129 |
+
for origin in ("https://postkitty.mertkayacs.com", "http://localhost:5173"):
|
| 130 |
+
headers = {"Origin": origin, "Sec-Fetch-Site": "cross-site"}
|
| 131 |
+
assert client.post("/v1/text", json={"text": "hello"}, headers=headers).status_code == 200, origin
|
| 132 |
+
proxied = {"Origin": "http://127.0.0.1:18088", "Sec-Fetch-Site": "same-origin"}
|
| 133 |
+
assert client.post("/v1/text", json={"text": "hello"}, headers=proxied).status_code == 200
|
| 134 |
+
assert client.post("/v1/text", json={"text": "hello"}).status_code == 200
|
| 135 |
+
assert client.get("/v1/health", headers={"Sec-Fetch-Site": "cross-site"}).status_code == 200
|
| 136 |
+
assert foreign_origin({b"origin": b"https://evil.example"}) is False
|