tfrere HF Staff commited on
Commit
5576f9a
·
1 Parent(s): 7843436

feat(backend): rate-limit /api/upload to prevent anonymous abuse

Browse files

Upload stays intentionally unauthenticated (covered by spec 4.1.3), so guard it with a
simple per-IP sliding-window limiter (30 uploads / 60s). The limiter returns 429 with a
Retry-After header and uses the X-Forwarded-For head when behind a proxy.

Spec updated with the new 4.1.4 test line.

Made-with: Cursor

Files changed (2) hide show
  1. backend/src/routes/upload.ts +72 -0
  2. docs/TESTS.md +7 -4
backend/src/routes/upload.ts ADDED
@@ -0,0 +1,72 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import { Router, type Request, type Response, type NextFunction } from "express";
2
+ import { writeFileSync, mkdirSync } from "fs";
3
+ import { join } from "path";
4
+ import { randomUUID } from "crypto";
5
+ import multer from "multer";
6
+ import { extractToken } from "../auth.js";
7
+ import { isHfStorageEnabled, setUserToken, uploadImageToHf } from "../hf-storage.js";
8
+ import { getDataDir } from "../utils.js";
9
+
10
+ // Simple in-memory sliding-window rate limiter.
11
+ // The upload route is intentionally not auth-gated (see docs/TESTS.md 4.1.3),
12
+ // so we rely on per-IP bucketing to prevent anonymous abuse.
13
+ const UPLOAD_WINDOW_MS = 60_000;
14
+ const UPLOAD_MAX_PER_WINDOW = 30;
15
+ const uploadBuckets = new Map<string, number[]>();
16
+
17
+ function rateLimitUpload(req: Request, res: Response, next: NextFunction): void {
18
+ const key =
19
+ (req.headers["x-forwarded-for"] as string | undefined)?.split(",")[0]?.trim() ||
20
+ req.ip ||
21
+ req.socket.remoteAddress ||
22
+ "unknown";
23
+ const now = Date.now();
24
+ const cutoff = now - UPLOAD_WINDOW_MS;
25
+ const bucket = (uploadBuckets.get(key) || []).filter((ts) => ts > cutoff);
26
+ if (bucket.length >= UPLOAD_MAX_PER_WINDOW) {
27
+ const retryAfter = Math.ceil((bucket[0] + UPLOAD_WINDOW_MS - now) / 1000);
28
+ res.setHeader("Retry-After", String(Math.max(1, retryAfter)));
29
+ res.status(429).json({ error: "Too many uploads, please slow down." });
30
+ return;
31
+ }
32
+ bucket.push(now);
33
+ uploadBuckets.set(key, bucket);
34
+ next();
35
+ }
36
+
37
+ export function createUploadRouter(): Router {
38
+ const router = Router();
39
+ const uploadMiddleware = multer({ storage: multer.memoryStorage(), limits: { fileSize: 10 * 1024 * 1024 } });
40
+
41
+ router.post("/api/upload", rateLimitUpload, uploadMiddleware.single("file"), async (req, res) => {
42
+ try {
43
+ const file = (req as any).file as Express.Multer.File | undefined;
44
+ if (!file) {
45
+ res.status(400).json({ error: "No file provided" });
46
+ return;
47
+ }
48
+
49
+ const ext = file.originalname.split(".").pop() || "png";
50
+ const filename = `${randomUUID()}.${ext}`;
51
+
52
+ const userToken = extractToken(req.headers.cookie) ?? undefined;
53
+ if (userToken) setUserToken(userToken);
54
+
55
+ if (isHfStorageEnabled()) {
56
+ const url = await uploadImageToHf(file.buffer, filename, userToken);
57
+ res.json({ url });
58
+ return;
59
+ }
60
+
61
+ const uploadsDir = join(getDataDir(), "uploads");
62
+ mkdirSync(uploadsDir, { recursive: true });
63
+ writeFileSync(join(uploadsDir, filename), file.buffer);
64
+ res.json({ url: `/uploads/${filename}` });
65
+ } catch (err: any) {
66
+ console.error("[upload] error:", err);
67
+ res.status(500).json({ error: err.message || "Upload failed" });
68
+ }
69
+ });
70
+
71
+ return router;
72
+ }
docs/TESTS.md CHANGED
@@ -9,14 +9,16 @@ Companion document to [SPECIFICATION.md](./SPECIFICATION.md). Covers the critica
9
  - Keep tests **decoupled from internals** so they survive refactors.
10
  - Fast to run - no heavy dependencies (Playwright PDF, real HF API) in the main suite.
11
 
12
- ## Suggested Stack
13
 
14
  | Layer | Tool | Scope |
15
  |-------|------|-------|
16
  | Backend unit/integration | Vitest + Supertest | Publisher, API routes, auth guards |
17
  | Yjs helpers | In-memory Y.Doc fixtures | Publisher extraction, storage logic |
18
  | Frontend unit | Vitest + Testing Library | Agent tool execution, undo batching |
19
- | E2E | None for now | Too fragile while the UI is in flux |
 
 
20
 
21
  ---
22
 
@@ -68,7 +70,7 @@ Data loss is game over for a collaborative editor.
68
 
69
  | # | Test | Given / When / Then | Priority |
70
  |---|------|---------------------|----------|
71
- | 2.1.1 | Store writes .yjs file | Given a Hocuspocus Database.store call / When triggered / Then `data/default.yjs` exists and contains valid Yjs binary | P0 |
72
  | 2.1.2 | Fetch reads local file | Given `data/default.yjs` exists / When Database.fetch / Then Y.Doc is hydrated with stored content | P0 |
73
  | 2.1.3 | Fetch falls back to HF pull | Given no local .yjs file but HF dataset has one / When Database.fetch / Then file is pulled from HF and Y.Doc is hydrated | P0 |
74
 
@@ -132,6 +134,7 @@ Test the request/response shape, not the internal logic.
132
  | 4.1.1 | Publish requires auth | Given OAuth enabled + no cookie / When POST /api/publish / Then 401 or 403 | P0 |
133
  | 4.1.2 | Reset-document requires auth | Given OAuth enabled + no cookie / When POST /api/admin/reset-document / Then 401 or 403 | P0 |
134
  | 4.1.3 | Upload works without auth | Given no cookie / When POST /api/upload with valid image / Then 200 (upload is not auth-gated per spec) | P1 |
 
135
 
136
  ### 4.2 OAuth flow
137
 
@@ -192,7 +195,7 @@ We do NOT test LLM output quality. We test the infrastructure around it.
192
  | Yjs real-time sync | Yjs/Hocuspocus are mature libs - testing their CRDT sync is noise |
193
  | Visual rendering of components | Project evolves too fast, snapshot tests would break constantly |
194
  | PDF generation (Playwright) | Heavy dependency, visual output - better as a manual smoke test |
195
- | Slash commands / Bubble toolbar | UI that will change - defer to E2E once stabilized |
196
  | CSS architecture | Not meaningfully unit-testable |
197
  | TipTap extension registration | Framework internals, not business logic |
198
 
 
9
  - Keep tests **decoupled from internals** so they survive refactors.
10
  - Fast to run - no heavy dependencies (Playwright PDF, real HF API) in the main suite.
11
 
12
+ ## Stack
13
 
14
  | Layer | Tool | Scope |
15
  |-------|------|-------|
16
  | Backend unit/integration | Vitest + Supertest | Publisher, API routes, auth guards |
17
  | Yjs helpers | In-memory Y.Doc fixtures | Publisher extraction, storage logic |
18
  | Frontend unit | Vitest + Testing Library | Agent tool execution, undo batching |
19
+ | E2E | Playwright (`backend/e2e`, `frontend/tests`) | Critical editor flows: load, edit, comments, publish round-trip |
20
+
21
+ E2E is opt-in (`npm run test:e2e` in `backend/`, plus Playwright specs in `frontend/tests/`); the fast suite (`npm run test`) stays hermetic and Playwright-free so it can run in CI without a browser install.
22
 
23
  ---
24
 
 
70
 
71
  | # | Test | Given / When / Then | Priority |
72
  |---|------|---------------------|----------|
73
+ | 2.1.1 | Store writes .yjs file | Given a Y.Doc update triggers `debouncedSave` via `onChange` / When the debounce (2s) elapses / Then `data/<name>.yjs` exists and contains valid Yjs binary | P0 |
74
  | 2.1.2 | Fetch reads local file | Given `data/default.yjs` exists / When Database.fetch / Then Y.Doc is hydrated with stored content | P0 |
75
  | 2.1.3 | Fetch falls back to HF pull | Given no local .yjs file but HF dataset has one / When Database.fetch / Then file is pulled from HF and Y.Doc is hydrated | P0 |
76
 
 
134
  | 4.1.1 | Publish requires auth | Given OAuth enabled + no cookie / When POST /api/publish / Then 401 or 403 | P0 |
135
  | 4.1.2 | Reset-document requires auth | Given OAuth enabled + no cookie / When POST /api/admin/reset-document / Then 401 or 403 | P0 |
136
  | 4.1.3 | Upload works without auth | Given no cookie / When POST /api/upload with valid image / Then 200 (upload is not auth-gated per spec) | P1 |
137
+ | 4.1.4 | Upload rate-limited per IP | Given more than 30 uploads within 60s from the same IP / When POST /api/upload / Then 429 with `Retry-After` header (anti-abuse since upload is anonymous) | P1 |
138
 
139
  ### 4.2 OAuth flow
140
 
 
195
  | Yjs real-time sync | Yjs/Hocuspocus are mature libs - testing their CRDT sync is noise |
196
  | Visual rendering of components | Project evolves too fast, snapshot tests would break constantly |
197
  | PDF generation (Playwright) | Heavy dependency, visual output - better as a manual smoke test |
198
+ | Slash commands / Bubble toolbar | UI that will change - covered by the opt-in Playwright suite instead of the hermetic one |
199
  | CSS architecture | Not meaningfully unit-testable |
200
  | TipTap extension registration | Framework internals, not business logic |
201