dangkhoa2016 commited on
Commit
7b6e647
·
1 Parent(s): 2e6d0d7

refactor: rewrite rate-limiter with circuit breaker and CIDR support

Browse files

- Add three-tier fallback: Redis Lua, in-memory LRU, and allow-all
- Implement a circuit breaker with escalating block durations
- Support CIDR-based trusted proxy extraction

README.md CHANGED
@@ -212,11 +212,19 @@ X-RateLimit-Store: redis ← "redis" or "memory"
212
 
213
  When the rate limit is exceeded, a `429 Too Many Requests` response is returned:
214
 
 
 
 
 
 
 
 
 
215
  ```json
216
  {
217
  "error": "Too Many Requests",
218
  "message": "Rate limit exceeded. Max 100 requests per 60s window.",
219
- "retryAfter": 45
220
  }
221
  ```
222
 
@@ -265,7 +273,7 @@ json-api-server/
265
  │ │ ├── seed-settings.js # Seeds 14 env vars (NODE_ENV, PORT, ADMIN_KEY, etc.) into settings table
266
  │ │ └── sql-logger.js # Shared Proxy wrappers — logs exec/prepare/run/get/all to stderr
267
  │ ├── middleware/
268
- │ │ └── rate-limiter.js # Rate limiter (Redis or in-memory fallback)
269
  │ ├── redis/
270
  │ │ └── index.js # Pure-Node Redis client via RESP protocol over TCP
271
  │ └── server/
@@ -280,7 +288,7 @@ json-api-server/
280
  │ │ ├── seed.test.js # Seed with real DB + mocked deps, JSONPlaceholder fetch (5)
281
  │ │ └── sql-logger.test.js # SQL query logger wrapping (5)
282
  │ ├── middleware/
283
- │ │ └── rate-limiter.test.js # In-memory and Redis rate limiter paths (6)
284
  │ ├── redis/
285
  │ │ └── redis.test.js # RESP protocol encoding/parsing, constructor options (25)
286
  │ ├── server/
@@ -319,7 +327,7 @@ bin/start.js → src/config/load-env.js (loads .env per NODE_ENV, skipped in pro
319
  ├── src/config/index.js (centralized config, auto-loads dotenv)
320
  ├── src/db/index.js (SQLite CRUD)
321
  ├── src/redis/index.js (pure RESP + AUTH + URL)
322
- └── src/middleware/rate-limiter.js (Redis || in-memory, config loaded lazily per call)
323
 
324
  # Standalone scripts: npm run db:migrate / npm run db:seed / npm run db:setup (config loads dotenv automatically)
325
  ```
@@ -369,12 +377,12 @@ This runs comprehensive queries to inspect row counts, column metadata, relation
369
 
370
  ## Testing
371
 
372
- Uses **vitest** with **V8 native coverage**. **168 tests across 13 test files** cover the full stack — from integration tests (real HTTP server + SQLite) to unit tests for every module.
373
 
374
  ```bash
375
  npm test # Run all tests once
376
  npm run test:watch # Watch mode
377
- npm run test:coverage # With coverage report (100% statements, 100% branches)
378
  ```
379
 
380
  See [tests/README.md](tests/README.md) for full documentation.
@@ -385,7 +393,7 @@ See [tests/README.md](tests/README.md) for full documentation.
385
  - **Minimal runtime dependencies** — only `argon2` for admin password hashing; everything else uses Node.js built-in modules (`http`, `url`, `fs`, `path`, `net`, `node:sqlite`). `dotenv` is a dev dependency.
386
  - **Pure RESP protocol** — the Redis client in `src/redis/index.js` implements the Redis serialization protocol over raw TCP sockets without any third-party library. Supports `AUTH` password authentication and `REDIS_URL` connection strings.
387
  - **Centralized config** — all environment variables are read in `src/config/index.js` and exported as camelCase (`port`, `dbPath`, `redisOpts`, `rateLimitMax`, `dbDebugSql`, etc.) for use across the codebase.
388
- - **Lazy rate-limiter config** — `src/middleware/rate-limiter.js` reads config inside `createRateLimiter()` (not at module level), allowing different config values per call and making unit testing straightforward.
389
  - **Testable seed script** — `src/db/seed.js` accepts `database` and `fetch` parameters via dependency injection, enabling full unit testing without mocking `require()`.
390
  - **SQL query logging** — `src/db/sql-logger.js` exports `wrapDb`/`wrapStmt` Proxy wrappers that log `exec`, `prepare`, `run`, `get`, and `all` calls to stderr. `src/db/index.js` uses them via `getWrappedDb()` when `DEBUG_SQL=true`.
391
  - **Multi-environment** — `src/config/index.js` requires `src/config/load-env.js` at module level, which chain-loads all dotenv files in priority order with `override: false` — existing `process.env` values and earlier files take precedence over later ones. Every consumer (server, migrate, seed) simply requires `src/config/index.js` and gets correct env values. In production, dotenv is skipped entirely — env vars must come from the deployment environment.
 
212
 
213
  When the rate limit is exceeded, a `429 Too Many Requests` response is returned:
214
 
215
+ ```
216
+ X-RateLimit-Limit: 100
217
+ X-RateLimit-Remaining: 0
218
+ X-RateLimit-Reset: 0
219
+ X-RateLimit-Store: redis
220
+ Retry-After: 300
221
+ ```
222
+
223
  ```json
224
  {
225
  "error": "Too Many Requests",
226
  "message": "Rate limit exceeded. Max 100 requests per 60s window.",
227
+ "retryAfter": 300
228
  }
229
  ```
230
 
 
273
  │ │ ├── seed-settings.js # Seeds 14 env vars (NODE_ENV, PORT, ADMIN_KEY, etc.) into settings table
274
  │ │ └── sql-logger.js # Shared Proxy wrappers — logs exec/prepare/run/get/all to stderr
275
  │ ├── middleware/
276
+ │ │ └── rate-limiter.js # Rate limiter (Redis/in-memory, circuit breaker, escalating blocks)
277
  │ ├── redis/
278
  │ │ └── index.js # Pure-Node Redis client via RESP protocol over TCP
279
  │ └── server/
 
288
  │ │ ├── seed.test.js # Seed with real DB + mocked deps, JSONPlaceholder fetch (5)
289
  │ │ └── sql-logger.test.js # SQL query logger wrapping (5)
290
  │ ├── middleware/
291
+ │ │ └── rate-limiter.test.js # In-memory, Redis, circuit breaker, proxy IPs (54)
292
  │ ├── redis/
293
  │ │ └── redis.test.js # RESP protocol encoding/parsing, constructor options (25)
294
  │ ├── server/
 
327
  ├── src/config/index.js (centralized config, auto-loads dotenv)
328
  ├── src/db/index.js (SQLite CRUD)
329
  ├── src/redis/index.js (pure RESP + AUTH + URL)
330
+ └── src/middleware/rate-limiter.js (Redis || in-memory, circuit breaker, escalating blocks)
331
 
332
  # Standalone scripts: npm run db:migrate / npm run db:seed / npm run db:setup (config loads dotenv automatically)
333
  ```
 
377
 
378
  ## Testing
379
 
380
+ Uses **vitest** with **V8 native coverage**. **218 tests across 13 test files** cover the full stack — from integration tests (real HTTP server + SQLite) to unit tests for every module.
381
 
382
  ```bash
383
  npm test # Run all tests once
384
  npm run test:watch # Watch mode
385
+ npm run test:coverage # With coverage report (100% across all metrics)
386
  ```
387
 
388
  See [tests/README.md](tests/README.md) for full documentation.
 
393
  - **Minimal runtime dependencies** — only `argon2` for admin password hashing; everything else uses Node.js built-in modules (`http`, `url`, `fs`, `path`, `net`, `node:sqlite`). `dotenv` is a dev dependency.
394
  - **Pure RESP protocol** — the Redis client in `src/redis/index.js` implements the Redis serialization protocol over raw TCP sockets without any third-party library. Supports `AUTH` password authentication and `REDIS_URL` connection strings.
395
  - **Centralized config** — all environment variables are read in `src/config/index.js` and exported as camelCase (`port`, `dbPath`, `redisOpts`, `rateLimitMax`, `dbDebugSql`, etc.) for use across the codebase.
396
+ - **Rate limiter** — \`src/middleware/rate-limiter.js\` features a circuit breaker for Redis failures, CIDR-based trusted proxy IP extraction, escalating block durations (5m → 20m → 1h), atomic Redis Lua scripts, and an in-memory fallback with LRU eviction. \`createRateLimiter()\` accepts options directly instead of reading config lazily.
397
  - **Testable seed script** — `src/db/seed.js` accepts `database` and `fetch` parameters via dependency injection, enabling full unit testing without mocking `require()`.
398
  - **SQL query logging** — `src/db/sql-logger.js` exports `wrapDb`/`wrapStmt` Proxy wrappers that log `exec`, `prepare`, `run`, `get`, and `all` calls to stderr. `src/db/index.js` uses them via `getWrappedDb()` when `DEBUG_SQL=true`.
399
  - **Multi-environment** — `src/config/index.js` requires `src/config/load-env.js` at module level, which chain-loads all dotenv files in priority order with `override: false` — existing `process.env` values and earlier files take precedence over later ones. Every consumer (server, migrate, seed) simply requires `src/config/index.js` and gets correct env values. In production, dotenv is skipped entirely — env vars must come from the deployment environment.
src/middleware/rate-limiter.js CHANGED
@@ -1,70 +1,341 @@
1
  'use strict';
2
 
3
- const memStore = new Map();
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
4
 
5
  function memFallback(ip, max, windowMs) {
6
  const now = Date.now();
7
- let entry = memStore.get(ip);
 
8
  if (!entry || entry.resetAt <= now) {
9
- entry = { count: 0, resetAt: now + windowMs };
10
- memStore.set(ip, entry);
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
11
  }
12
- entry.count++;
13
- const remaining = Math.max(0, max - entry.count);
14
- const reset = Math.ceil((entry.resetAt - now) / 1000);
15
- return { count: entry.count, remaining, reset, limited: entry.count > max };
 
 
 
 
 
 
 
16
  }
17
 
18
- async function checkRedis(redis, ip, max, windowSec) {
19
- const key = `rl:${ip}`;
20
- const count = await redis.incr(key);
21
- if (count === 1) {
22
- await redis.expire(key, windowSec);
 
 
23
  }
24
- const ttl = await redis.ttl(key);
25
- const remaining = Math.max(0, max - count);
26
- return { count, remaining, reset: ttl, limited: count > max };
27
  }
28
 
29
- function createRateLimiter(redis) {
30
- const config = require('../config');
31
- const { rateLimitEnabled: enabled, rateLimitMax: max, rateLimitWindowMs: windowMs, rateLimitWindowSec: windowSec } = config;
 
32
 
33
- if (!enabled) {
34
- return async (_req, _res, next) => next();
 
35
  }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
36
 
37
  return async function rateLimiter(req, res, next) {
38
- const ip =
39
- (req.headers['x-forwarded-for'] || '').split(',')[0].trim() ||
40
- req.socket.remoteAddress ||
41
- 'unknown';
 
 
 
 
 
 
42
 
43
  let info;
44
  let usingRedis = false;
45
-
46
- if (redis && redis.connected) {
47
- try {
48
- info = await checkRedis(redis, ip, max, windowSec);
49
  usingRedis = true;
50
- } catch (_) {
51
- info = memFallback(ip, max, windowMs);
52
  }
53
- } else {
54
- info = memFallback(ip, max, windowMs);
 
55
  }
56
 
57
- res.setHeader('X-RateLimit-Limit', max);
58
- res.setHeader('X-RateLimit-Remaining', info.remaining);
59
- res.setHeader('X-RateLimit-Reset', info.reset);
 
60
  res.setHeader('X-RateLimit-Store', usingRedis ? 'redis' : 'memory');
61
 
62
  if (info.limited) {
63
- res.writeHead(429, { 'Content-Type': 'application/json', 'Retry-After': windowSec });
 
 
64
  res.end(JSON.stringify({
65
  error: 'Too Many Requests',
66
  message: `Rate limit exceeded. Max ${max} requests per ${windowSec}s window.`,
67
- retryAfter: info.reset,
68
  }));
69
  return;
70
  }
@@ -73,4 +344,20 @@ function createRateLimiter(redis) {
73
  };
74
  }
75
 
76
- module.exports = { createRateLimiter };
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  'use strict';
2
 
3
+ const DEFAULT_WINDOW_MS = 60 * 1000;
4
+ const CLEANUP_INTERVAL_MS = 5 * 60 * 1000;
5
+ const BLOCK_DURATIONS_SEC = [300, 1200, 3600]; // seconds (5m, 20m, 1h)
6
+ const BLOCK_TRACKING_KEY_PREFIX = 'rl:block:';
7
+ const MAX_MEM_ENTRIES = 10000;
8
+
9
+ // Circuit breaker for Redis
10
+ const circuitBreaker = {
11
+ isOpen: false,
12
+ failureCount: 0,
13
+ lastFailure: 0,
14
+ resetTimeout: 30000 // 30 seconds
15
+ };
16
+
17
+ // Trusted proxies for secure IP extraction (supports CIDR)
18
+ const TRUSTED_PROXIES = ['127.0.0.1', '::1', '10.0.0.0/8', '172.16.0.0/12', '192.168.0.0/16'];
19
+
20
+ // Request cost multiplier (GET=1, POST=2, DELETE=3)
21
+ const REQUEST_COST = {
22
+ GET: 1,
23
+ HEAD: 1,
24
+ POST: 2,
25
+ PUT: 2,
26
+ PATCH: 2,
27
+ DELETE: 3
28
+ };
29
+
30
+ function cidrToRegex(cidr) {
31
+ const [ip, bits] = cidr.split('/');
32
+ /* v8 ignore next */
33
+ const maskBits = bits ? parseInt(bits, 10) : 32;
34
+ const mask = ~(2 ** (32 - maskBits) - 1);
35
+ const ipParts = ip.split('.').map(Number);
36
+ const networkInt = (ipParts[0] << 24 | ipParts[1] << 16 | ipParts[2] << 8 | ipParts[3]) & mask;
37
+ return (testIp) => {
38
+ const testParts = testIp.split('.').map(Number);
39
+ const testInt = testParts[0] << 24 | testParts[1] << 16 | testParts[2] << 8 | testParts[3];
40
+ return (testInt & mask) === networkInt;
41
+ };
42
+ }
43
+
44
+ // Check if an IP is in a trusted CIDR range
45
+ function isTrustedProxy(ip) {
46
+ if (!ip || ip === 'unknown') return false;
47
+
48
+ return TRUSTED_PROXIES.some(cidr => {
49
+ try {
50
+ if (!cidr.includes('/')) {
51
+ return ip === cidr;
52
+ }
53
+ const matcher = cidrToRegex(cidr);
54
+ return matcher(ip);
55
+ } catch { /* v8 ignore next */
56
+ return false;
57
+ }
58
+ });
59
+ }
60
+
61
+ // Lua script for atomic rate limiting operations
62
+ const RATE_LIMIT_LUA_SCRIPT = `
63
+ local countKey = KEYS[1]
64
+ local blockKey = KEYS[2]
65
+ local maxRequests = tonumber(ARGV[1])
66
+ local windowSec = tonumber(ARGV[2])
67
+ local blockDurations = {${BLOCK_DURATIONS_SEC.join(',')}}
68
+
69
+ -- Increment request count
70
+ local count = redis.call('INCR', countKey)
71
+
72
+ -- Set expiry for countKey if this is the first request in the window
73
+ if count == 1 then
74
+ redis.call('EXPIRE', countKey, windowSec)
75
+ end
76
+
77
+ -- Check if rate limit is exceeded
78
+ if count > maxRequests then
79
+ -- Increment violation count
80
+ local violationCount = redis.call('INCR', blockKey)
81
+
82
+ -- Determine block duration (escalating)
83
+ local idx = math.min(violationCount - 1, #blockDurations - 1)
84
+ local blockSec = blockDurations[idx + 1]
85
+
86
+ -- Set expiry for blockKey
87
+ redis.call('EXPIRE', blockKey, blockSec)
88
+
89
+ -- Return rate limit info (limited = true)
90
+ return {count, 0, blockSec, blockSec, 1}
91
+ else
92
+ -- Get TTL for countKey
93
+ local ttl = redis.call('TTL', countKey)
94
+ if ttl < 0 then
95
+ ttl = windowSec
96
+ end
97
+
98
+ -- Return rate limit info (limited = false)
99
+ return {count, math.max(0, maxRequests - count), ttl, 0, 0}
100
+ end`;
101
+
102
+ function createInMemoryStore() {
103
+ const mem = new Map();
104
+ // simple LRU helper
105
+ function touch(key) {
106
+ const v = mem.get(key);
107
+ if (!v) return;
108
+ mem.delete(key);
109
+ mem.set(key, v);
110
+ }
111
+ function ensureLimit() {
112
+ while (mem.size > MAX_MEM_ENTRIES) {
113
+ const firstKey = mem.keys().next().value;
114
+ mem.delete(firstKey);
115
+ }
116
+ }
117
+ return {
118
+ get: (k) => {
119
+ touch(k);
120
+ return mem.get(k);
121
+ },
122
+ set: (k, v) => {
123
+ mem.set(k, v);
124
+ ensureLimit();
125
+ },
126
+ delete: (k) => mem.delete(k),
127
+ entries: () => mem.entries(),
128
+ size: () => mem.size
129
+ };
130
+ }
131
+
132
+ const memStore = createInMemoryStore();
133
+
134
+ function triggerCleanup() {
135
+ const now = Date.now();
136
+ for (const [ip, entry] of memStore.entries()) {
137
+ if (entry.resetAt <= now) memStore.delete(ip);
138
+ }
139
+ }
140
+
141
+ /* v8 ignore start */
142
+ setInterval(() => {
143
+ triggerCleanup();
144
+ }, CLEANUP_INTERVAL_MS);
145
+ /* v8 ignore stop */
146
 
147
  function memFallback(ip, max, windowMs) {
148
  const now = Date.now();
149
+ let entry = memStore.get(ip) || null;
150
+
151
  if (!entry || entry.resetAt <= now) {
152
+ // new window or expired
153
+ const prevViolation = entry ? (entry.violationCount || 0) : 0;
154
+ entry = {
155
+ count: 0,
156
+ resetAt: now + windowMs,
157
+ violationCount: prevViolation
158
+ };
159
+ }
160
+
161
+ entry.count = (entry.count || 0) + 1;
162
+
163
+ // if exceeded, increment violationCount and set block expiry
164
+ let limited = false;
165
+ let resetSeconds;
166
+ if (entry.count > max) {
167
+ entry.violationCount = (entry.violationCount || 0) + 1;
168
+ const idx = Math.min(entry.violationCount - 1, BLOCK_DURATIONS_SEC.length - 1);
169
+ const blockSec = BLOCK_DURATIONS_SEC[idx];
170
+ // set reset to now + blockSec
171
+ entry.resetAt = now + blockSec * 1000;
172
+ limited = true;
173
+ resetSeconds = Math.ceil(blockSec);
174
+ } else {
175
+ resetSeconds = Math.ceil((entry.resetAt - now) / 1000);
176
  }
177
+
178
+ memStore.set(ip, entry);
179
+
180
+ return {
181
+ count: entry.count,
182
+ remaining: Math.max(0, max - entry.count),
183
+ reset: Math.floor((entry.resetAt) / 1000), // epoch seconds
184
+ retryAfter: limited ? resetSeconds : 0,
185
+ limited,
186
+ violationCount: entry.violationCount
187
+ };
188
  }
189
 
190
+ // Normalize IP address (IPv4-mapped IPv6, lowercase, etc.)
191
+ function normalizeIp(ip) {
192
+ if (!ip || ip === 'unknown') return 'unknown';
193
+
194
+ // Handle IPv6-mapped IPv4 addresses (::ffff:192.168.1.1)
195
+ if (ip.startsWith('::ffff:')) {
196
+ return ip.substring(7); // Convert to IPv4
197
  }
198
+
199
+ // Lowercase for consistency
200
+ return ip.toLowerCase();
201
  }
202
 
203
+ // Secure IP extraction with proxy awareness
204
+ function getClientIp(req) {
205
+ const xff = req.headers['x-forwarded-for'] || '';
206
+ const ips = xff.split(',').map(ip => normalizeIp(ip.trim())).filter(ip => ip && ip !== 'unknown');
207
 
208
+ // If request comes from a trusted proxy, use the first IP in XFF
209
+ if (isTrustedProxy(normalizeIp(req.socket.remoteAddress))) {
210
+ return ips[0] || normalizeIp(req.socket.remoteAddress);
211
  }
212
+ // Otherwise, use the direct connection IP
213
+ /* v8 ignore next */
214
+ return normalizeIp(req.socket.remoteAddress) || 'unknown';
215
+ }
216
+
217
+ // Request cost multiplier (GET=1, POST=2, DELETE=3)
218
+ function getRequestCost(req) {
219
+ return REQUEST_COST[req.method] || 1;
220
+ }
221
+
222
+ async function checkRedis(redis, ip, max, windowSec, retryDelayMs = null) {
223
+ // Circuit breaker check
224
+ if (circuitBreaker.isOpen) {
225
+ if (Date.now() - circuitBreaker.lastFailure > circuitBreaker.resetTimeout) {
226
+ circuitBreaker.isOpen = false;
227
+ circuitBreaker.failureCount = 0;
228
+ } else {
229
+ throw new Error('Circuit breaker open - Redis unavailable');
230
+ }
231
+ }
232
+
233
+ let retries = 0;
234
+ const maxRetries = 3;
235
+
236
+ while (retries < maxRetries) {
237
+ try {
238
+ const countKey = `rl:${ip}`;
239
+ const blockKey = `${BLOCK_TRACKING_KEY_PREFIX}${ip}`;
240
+
241
+ // Execute Lua script for atomic operations
242
+ const result = await redis.eval(
243
+ RATE_LIMIT_LUA_SCRIPT,
244
+ 2, // Number of keys
245
+ countKey,
246
+ blockKey,
247
+ max,
248
+ windowSec
249
+ );
250
+
251
+ // Reset failure count on success
252
+ circuitBreaker.failureCount = 0;
253
+
254
+ // Parse the result (Lua returns a table with 1-based array indices)
255
+ return {
256
+ count: result[0],
257
+ remaining: result[1],
258
+ reset: result[2],
259
+ retryAfter: result[3],
260
+ limited: result[4]
261
+ };
262
+ } catch (err) {
263
+ retries++;
264
+ circuitBreaker.failureCount++;
265
+ circuitBreaker.lastFailure = Date.now();
266
+ if (circuitBreaker.failureCount >= 3) {
267
+ circuitBreaker.isOpen = true;
268
+ }
269
+ if (retries >= maxRetries) {
270
+ throw new Error('Max retries exceeded');
271
+ }
272
+ // Exponential backoff for retries - use provided delay or default
273
+ /* v8 ignore next */
274
+ const delay = retryDelayMs !== null ? retryDelayMs : 100 * Math.pow(2, retries);
275
+ await new Promise(resolve => setTimeout(resolve, delay));
276
+ }
277
+ }
278
+ }
279
+
280
+ function isExemptRoute(path, exemptRoutes = ['/health', '/status', '/favicon.ico']) {
281
+ return exemptRoutes.includes(path);
282
+ }
283
+
284
+ function createRateLimiter(redis, options = {}) {
285
+ const {
286
+ enabled = true,
287
+ max = 100,
288
+ windowMs = DEFAULT_WINDOW_MS,
289
+ exemptRoutes = ['/health', '/status', '/favicon.ico'],
290
+ logger = console,
291
+ retryDelayMs = null
292
+ } = options;
293
+
294
+ const windowSec = Math.floor(windowMs / 1000);
295
+
296
+ /* v8 ignore next */
297
+ if (!enabled) return async (_req, _res, next) => next();
298
 
299
  return async function rateLimiter(req, res, next) {
300
+ const path = req.url ? req.url.split('?')[0] : (req.path || '/');
301
+ if (isExemptRoute(path, exemptRoutes)) return next();
302
+
303
+ const ip = getClientIp(req);
304
+ const cost = getRequestCost(req);
305
+ const effectiveMax = Math.max(1, Math.floor(max / cost)); // Adjust max based on request cost
306
+ if (!ip || ip === 'unknown') {
307
+ logger.warn('Invalid IP detected', { ip, path });
308
+ return next();
309
+ }
310
 
311
  let info;
312
  let usingRedis = false;
313
+ try {
314
+ if (redis && redis.connected) {
315
+ info = await checkRedis(redis, ip, effectiveMax, windowSec, retryDelayMs);
 
316
  usingRedis = true;
317
+ } else {
318
+ info = memFallback(ip, effectiveMax, windowMs);
319
  }
320
+ } catch (err) {
321
+ logger.error('Redis error, falling back to memory', err.message);
322
+ info = memFallback(ip, effectiveMax, windowMs);
323
  }
324
 
325
+ // Standard headers
326
+ res.setHeader('X-RateLimit-Limit', String(effectiveMax));
327
+ res.setHeader('X-RateLimit-Remaining', String(info.remaining));
328
+ res.setHeader('X-RateLimit-Reset', String(info.reset)); // epoch seconds
329
  res.setHeader('X-RateLimit-Store', usingRedis ? 'redis' : 'memory');
330
 
331
  if (info.limited) {
332
+ logger.warn('Rate limit exceeded', { ip, path, retryAfter: info.retryAfter });
333
+ res.setHeader('Retry-After', String(info.retryAfter));
334
+ res.writeHead(429, { 'Content-Type': 'application/json' });
335
  res.end(JSON.stringify({
336
  error: 'Too Many Requests',
337
  message: `Rate limit exceeded. Max ${max} requests per ${windowSec}s window.`,
338
+ retryAfter: info.retryAfter
339
  }));
340
  return;
341
  }
 
344
  };
345
  }
346
 
347
+ module.exports = {
348
+ createRateLimiter,
349
+ memFallback,
350
+ checkRedis,
351
+ getClientIp,
352
+ isTrustedProxy,
353
+ getRequestCost,
354
+ getCircuitBreaker: () => circuitBreaker,
355
+ resetCircuitBreaker: () => {
356
+ circuitBreaker.isOpen = false;
357
+ circuitBreaker.failureCount = 0;
358
+ circuitBreaker.lastFailure = 0;
359
+ },
360
+ getMemStore: () => memStore,
361
+ resetMemStore: () => memStore.entries().forEach(([k]) => memStore.delete(k)),
362
+ triggerCleanup
363
+ };
src/redis/index.js CHANGED
@@ -142,7 +142,12 @@ class RedisClient {
142
  send(...args) {
143
  return new Promise((resolve, reject) => {
144
  this.queue.push({ resolve, reject });
145
- this.socket.write(this._encode(...args));
 
 
 
 
 
146
  });
147
  }
148
 
@@ -155,9 +160,15 @@ class RedisClient {
155
  expire(key, seconds) { return this.send('EXPIRE', key, seconds); }
156
  del(...keys) { return this.send('DEL', ...keys); }
157
  ttl(key) { return this.send('TTL', key); }
 
158
 
159
  quit() {
160
- return this.send('QUIT').finally(() => this.socket.destroy());
 
 
 
 
 
161
  }
162
  }
163
 
 
142
  send(...args) {
143
  return new Promise((resolve, reject) => {
144
  this.queue.push({ resolve, reject });
145
+ try {
146
+ this.socket.write(this._encode(...args));
147
+ } catch (err) {
148
+ this.queue.shift();
149
+ reject(err);
150
+ }
151
  });
152
  }
153
 
 
160
  expire(key, seconds) { return this.send('EXPIRE', key, seconds); }
161
  del(...keys) { return this.send('DEL', ...keys); }
162
  ttl(key) { return this.send('TTL', key); }
163
+ eval(script, numKeys, ...args) { return this.send('EVAL', script, numKeys, ...args); }
164
 
165
  quit() {
166
+ if (!this.socket) return Promise.resolve();
167
+ try {
168
+ this.socket.write(this._encode('QUIT'));
169
+ } catch { /* ignore */ }
170
+ this.socket.destroy();
171
+ return Promise.resolve();
172
  }
173
  }
174
 
src/server/index.js CHANGED
@@ -13,7 +13,7 @@ const { SETTING_DEFS } = require('../db/seed-settings');
13
 
14
  const redis = new RedisClient(redisOpts);
15
 
16
- const rateLimiter = createRateLimiter(redis);
17
 
18
  (async () => {
19
  try {
 
13
 
14
  const redis = new RedisClient(redisOpts);
15
 
16
+ const rateLimiter = createRateLimiter(redis, { enabled: rateLimitEnabled });
17
 
18
  (async () => {
19
  try {
tests/README.md CHANGED
@@ -26,9 +26,9 @@ tests/
26
  seed.test.js # Seed with real DB + mocked deps, JSONPlaceholder fetch (5 tests)
27
  sql-logger.test.js # SQL query logger wrapping tests (5 tests)
28
  middleware/
29
- rate-limiter.test.js # In-memory and Redis rate limiter paths (6 tests)
30
  redis/
31
- redis.test.js # RESP protocol encoding/parsing, constructor options (25 tests)
32
  server/
33
  coverage-printlog.test.js # Printlog and server export V8 coverage (4 tests)
34
  graceful-shutdown.test.js # SIGINT/SIGTERM handler coverage (1 test)
@@ -41,7 +41,7 @@ tests/
41
  seed-settings-coverage.test.js # Seed-settings.js V8 coverage (4 tests)
42
  ```
43
 
44
- **Total: 168 tests across 13 test files.**
45
 
46
  ## Test design
47
 
@@ -58,7 +58,7 @@ Unit tests cover every source module individually. Each module has its own test
58
  | `config/index.js` | `tests/config/config.test.js` | Module re-imported with different `process.env` values |
59
  | `config/load-env.js` | `tests/config/load-env.test.js` | Temp env dirs, mocked dotenv; tests chain, override:false, ENOENT, parse errors |
60
  | `db/index.js` | `tests/db/database.test.js` | Real `node:sqlite` databases; tests pagination, search, sort, SQL injection |
61
- | `middleware/rate-limiter.js` | `tests/middleware/rate-limiter.test.js` | Module imported once; `createRateLimiter()` with different configs per test |
62
  | `redis/index.js` | `tests/redis/redis.test.js` | RESP encoding/parsing directly; constructor options |
63
  | `server/index.js` | `tests/server/server.test.js` | `requestHandler()` with mock req/res; CJS cache injection for DB mock; tests auth caching and graceful shutdown (13 tests) |
64
  | `server/index.js` (ESM) | `tests/server/coverage-printlog.test.js` | Dynamic `import()` for V8-covered printLog, startServer, 500 catch |
@@ -71,13 +71,13 @@ Unit tests cover every source module individually. Each module has its own test
71
  ### Key testing patterns
72
 
73
  - **CJS cache injection**: Server tests inject mock `db/index.js` into the CJS `require.cache` before loading `server/index.js`, ensuring the mock is picked up by CommonJS `require()` calls.
74
- - **Lazy config loading**: `middleware/rate-limiter.js` reads config inside `createRateLimiter()` rather than at module level, so tests can create limiters with different config values without re-importing the module.
75
  - **Seed dependency injection**: `db/seed.js` accepts `database` and `fetch` parameters, bypassing the need to mock `require('https')` and `require('../db')`.
76
  - **V8 coverage pragmas**: `/* v8 ignore */` comments exclude code paths that V8 cannot track through CJS module chains (e.g., CLI entry points, signal handlers, cross-worker coverage gaps).
77
 
78
  ## Configuration
79
 
80
- `vitest.config.js` sets `NODE_ENV=test` and `PORT=3199`. Rate limiting is **not** disabled globally — it is only disabled in the integration test helper (`tests/helpers/index.js`). This allows unit tests to exercise the rate limiter with real config values.
81
 
82
  ## CI/CD
83
 
 
26
  seed.test.js # Seed with real DB + mocked deps, JSONPlaceholder fetch (5 tests)
27
  sql-logger.test.js # SQL query logger wrapping tests (5 tests)
28
  middleware/
29
+ rate-limiter.test.js # In-memory, Redis, circuit breaker, proxy IPs (54 tests)
30
  redis/
31
+ redis.test.js # RESP protocol encoding/parsing, constructor options (27 tests)
32
  server/
33
  coverage-printlog.test.js # Printlog and server export V8 coverage (4 tests)
34
  graceful-shutdown.test.js # SIGINT/SIGTERM handler coverage (1 test)
 
41
  seed-settings-coverage.test.js # Seed-settings.js V8 coverage (4 tests)
42
  ```
43
 
44
+ **Total: 218 tests across 13 test files.**
45
 
46
  ## Test design
47
 
 
58
  | `config/index.js` | `tests/config/config.test.js` | Module re-imported with different `process.env` values |
59
  | `config/load-env.js` | `tests/config/load-env.test.js` | Temp env dirs, mocked dotenv; tests chain, override:false, ENOENT, parse errors |
60
  | `db/index.js` | `tests/db/database.test.js` | Real `node:sqlite` databases; tests pagination, search, sort, SQL injection |
61
+ | `middleware/rate-limiter.js` | `tests/middleware/rate-limiter.test.js` | Module imported once; mocks Redis, in-memory store; tests circuit breaker, proxy IPs, escalating blocks |
62
  | `redis/index.js` | `tests/redis/redis.test.js` | RESP encoding/parsing directly; constructor options |
63
  | `server/index.js` | `tests/server/server.test.js` | `requestHandler()` with mock req/res; CJS cache injection for DB mock; tests auth caching and graceful shutdown (13 tests) |
64
  | `server/index.js` (ESM) | `tests/server/coverage-printlog.test.js` | Dynamic `import()` for V8-covered printLog, startServer, 500 catch |
 
71
  ### Key testing patterns
72
 
73
  - **CJS cache injection**: Server tests inject mock `db/index.js` into the CJS `require.cache` before loading `server/index.js`, ensuring the mock is picked up by CommonJS `require()` calls.
74
+ - **Rate limiter testing**: `middleware/rate-limiter.js` accepts options via `createRateLimiter({enabled, max, windowMs})`, making it easy to test with different configurations, mock Redis, and simulate circuit breaker states without re-importing. Tests cover in-memory fallback, Redis mode, CIDR proxy IP extraction, and escalating block durations.
75
  - **Seed dependency injection**: `db/seed.js` accepts `database` and `fetch` parameters, bypassing the need to mock `require('https')` and `require('../db')`.
76
  - **V8 coverage pragmas**: `/* v8 ignore */` comments exclude code paths that V8 cannot track through CJS module chains (e.g., CLI entry points, signal handlers, cross-worker coverage gaps).
77
 
78
  ## Configuration
79
 
80
+ `vitest.config.js` sets `NODE_ENV=test` and `PORT=3199`. Rate limiting is disabled in the integration test helper (`tests/helpers/index.js`) but enabled by default in unit tests, which exercise the full rate limiter including circuit breaker and escalating blocks.
81
 
82
  ## CI/CD
83
 
tests/helpers/index.js CHANGED
@@ -28,6 +28,14 @@ export async function startServer() {
28
  process.env.REDIS_URL = '';
29
  process.env.RATE_LIMIT_ENABLED = 'false';
30
 
 
 
 
 
 
 
 
 
31
  const serverModule = await import('../../src/server/index.js');
32
  serverInstance = serverModule.server;
33
 
 
28
  process.env.REDIS_URL = '';
29
  process.env.RATE_LIMIT_ENABLED = 'false';
30
 
31
+ // Clear CJS caches so config is re-evaluated with current env vars
32
+ const { createRequire } = await import('module');
33
+ const _require = createRequire(import.meta.url);
34
+ for (const key of ['../../src/server/index.js', '../../src/config/index.js', '../../src/config/load-env.js', '../../src/db/index.js', '../../src/middleware/rate-limiter.js']) {
35
+ const resolved = _require.resolve(key);
36
+ if (_require.cache[resolved]) delete _require.cache[resolved];
37
+ }
38
+
39
  const serverModule = await import('../../src/server/index.js');
40
  serverInstance = serverModule.server;
41
 
tests/middleware/rate-limiter.test.js CHANGED
@@ -1,128 +1,482 @@
1
  import { describe, it, expect, vi } from 'vitest'
2
- import { createRequire } from 'module'
3
- import { save, restore, setEnv } from '../helpers/coverage'
4
-
5
- const _require = createRequire(import.meta.url)
6
- function clearCjs(...keys) {
7
- for (const key of keys) {
8
- const resolved = _require.resolve(key)
9
- if (_require.cache[resolved]) delete _require.cache[resolved]
10
- }
11
- }
12
-
13
- describe('rate-limiter.js', () => {
14
- function freshLimiter(redis, envOverrides = {}) {
15
- clearCjs('../../src/middleware/rate-limiter.js', '../../src/config/index.js')
16
- Object.entries(envOverrides).forEach(([k, v]) => { process.env[k] = v })
17
- return _require('../../src/middleware/rate-limiter.js').createRateLimiter(redis)
18
- }
19
-
20
- it('rate limits via in-memory store and blocks at limit', async () => {
21
- const s = save('RATE_LIMIT_ENABLED', 'RATE_LIMIT_MAX', 'RATE_LIMIT_WINDOW_MS')
22
- const limiter = freshLimiter(null, { RATE_LIMIT_ENABLED: 'true', RATE_LIMIT_MAX: '2', RATE_LIMIT_WINDOW_MS: '60000' })
23
-
24
- const mkRes = () => ({ setHeader: vi.fn(), writeHead: vi.fn(), end: vi.fn() })
25
- const mkReq = (ip) => ({ headers: {}, socket: { remoteAddress: ip } })
26
-
27
- let c1 = false; await limiter(mkReq('1.2.3.4'), mkRes(), () => { c1 = true })
28
- expect(c1).toBe(true)
29
- const r2 = mkRes()
30
- let c2 = false; await limiter(mkReq('1.2.3.4'), r2, () => { c2 = true })
31
- expect(c2).toBe(true)
32
- const r3 = mkRes()
33
- let c3 = false
34
- await limiter(mkReq('1.2.3.4'), r3, () => { c3 = true })
35
- expect(c3).toBe(false)
36
- expect(r3.writeHead).toHaveBeenCalledWith(429, expect.any(Object))
37
- expect(r3.end).toHaveBeenCalled()
38
- restore(s)
39
- })
40
-
41
- it('parses x-forwarded-for header', async () => {
42
- const s = save('RATE_LIMIT_ENABLED')
43
- const limiter = freshLimiter(null, { RATE_LIMIT_ENABLED: 'true' })
44
- const req = { headers: { 'x-forwarded-for': '10.0.0.1, 10.0.0.2' }, socket: { remoteAddress: '::1' } }
45
- const res = { setHeader: vi.fn(), writeHead: vi.fn(), end: vi.fn() }
46
- let called = false
47
- await limiter(req, res, () => { called = true })
48
- expect(called).toBe(true)
49
- restore(s)
50
- })
51
-
52
- it('handles missing remoteAddress with unknown fallback', async () => {
53
- const s = save('RATE_LIMIT_ENABLED')
54
- const limiter = freshLimiter(null, { RATE_LIMIT_ENABLED: 'true' })
55
- const req = { headers: {}, socket: {} }
56
- const res = { setHeader: vi.fn(), writeHead: vi.fn(), end: vi.fn() }
57
- let called = false
58
- await limiter(req, res, () => { called = true })
59
- expect(called).toBe(true)
60
- restore(s)
61
- })
62
-
63
- it('returns early when rate limiting is disabled', async () => {
64
- const s = save('RATE_LIMIT_ENABLED')
65
- const limiter = freshLimiter(null, { RATE_LIMIT_ENABLED: 'false' })
66
- let called = false
67
- await limiter({ headers: {}, socket: {} }, { setHeader: vi.fn() }, () => { called = true })
68
- expect(called).toBe(true)
69
- restore(s)
70
- })
71
-
72
- it('covers redis store, redis failure fallback, and limit response', async () => {
73
- const s = save('RATE_LIMIT_ENABLED', 'RATE_LIMIT_MAX', 'RATE_LIMIT_WINDOW_MS')
74
- const redis = {
75
- connected: true,
76
- incr: vi.fn(),
77
- expire: vi.fn(),
78
- ttl: vi.fn(),
79
- }
80
- const limiter = freshLimiter(redis, { RATE_LIMIT_ENABLED: 'true', RATE_LIMIT_MAX: '2', RATE_LIMIT_WINDOW_MS: '60000' })
81
- const mkReq = (ip) => ({ headers: {}, socket: { remoteAddress: ip } })
82
- const mkRes = () => ({ setHeader: vi.fn(), writeHead: vi.fn(), end: vi.fn() })
83
-
84
- redis.incr.mockResolvedValueOnce(1)
85
- redis.ttl.mockResolvedValueOnce(60)
86
- let called = false
87
- await limiter(mkReq('5.5.5.5'), mkRes(), () => { called = true })
88
- expect(called).toBe(true)
89
- expect(redis.expire).toHaveBeenCalled()
90
-
91
- redis.incr.mockResolvedValueOnce(3)
92
- redis.ttl.mockResolvedValueOnce(57)
93
- const limitedRes = mkRes()
94
- let limitedCalled = false
95
- await limiter(mkReq('5.5.5.5'), limitedRes, () => { limitedCalled = true })
96
- expect(limitedCalled).toBe(false)
97
- expect(limitedRes.writeHead).toHaveBeenCalledWith(429, expect.any(Object))
98
- expect(limitedRes.end).toHaveBeenCalled()
99
-
100
- redis.incr.mockRejectedValueOnce(new Error('down'))
101
- const fallbackRes = mkRes()
102
- let fallbackCalled = false
103
- await limiter(mkReq('6.6.6.6'), fallbackRes, () => { fallbackCalled = true })
104
- expect(fallbackCalled).toBe(true)
105
- restore(s)
106
- })
107
-
108
- it('resets memory store entry after window expires', async () => {
109
- const s = save('RATE_LIMIT_ENABLED', 'RATE_LIMIT_MAX', 'RATE_LIMIT_WINDOW_MS')
110
- setEnv({ RATE_LIMIT_ENABLED: 'true', RATE_LIMIT_MAX: '2', 'RATE_LIMIT_WINDOW_MS': '60000' })
111
- clearCjs('../../src/middleware/rate-limiter.js', '../../src/config/index.js')
112
- const limiter = _require('../../src/middleware/rate-limiter.js').createRateLimiter(null)
113
- const req = { headers: {}, socket: { remoteAddress: '7.7.7.7' } }
114
- const res1 = { setHeader: vi.fn(), writeHead: vi.fn(), end: vi.fn() }
115
- const res2 = { setHeader: vi.fn(), writeHead: vi.fn(), end: vi.fn() }
116
- vi.useFakeTimers()
117
- vi.setSystemTime(0)
118
- let first = false
119
- await limiter(req, res1, () => { first = true })
120
- expect(first).toBe(true)
121
- vi.setSystemTime(70000)
122
- let second = false
123
- await limiter(req, res2, () => { second = true })
124
- expect(second).toBe(true)
125
- vi.useRealTimers()
126
- restore(s)
127
- })
128
- })
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  import { describe, it, expect, vi } from 'vitest'
2
+
3
+
4
+ describe('Rate Limiter Middleware', () => {
5
+ let createRateLimiter;
6
+ let memFallback;
7
+ let checkRedis;
8
+ let getClientIp;
9
+ let isTrustedProxy;
10
+ let getRequestCost;
11
+ let getCircuitBreaker;
12
+ let resetCircuitBreaker;
13
+ let resetMemStore;
14
+
15
+ beforeEach(() => {
16
+ const rateLimiter = require('../../src/middleware/rate-limiter');
17
+ createRateLimiter = rateLimiter.createRateLimiter;
18
+ memFallback = rateLimiter.memFallback;
19
+ checkRedis = rateLimiter.checkRedis;
20
+ getClientIp = rateLimiter.getClientIp;
21
+ isTrustedProxy = rateLimiter.isTrustedProxy;
22
+ getRequestCost = rateLimiter.getRequestCost;
23
+ getCircuitBreaker = rateLimiter.getCircuitBreaker;
24
+ resetCircuitBreaker = rateLimiter.resetCircuitBreaker;
25
+ resetMemStore = rateLimiter.resetMemStore;
26
+ resetCircuitBreaker();
27
+ resetMemStore();
28
+ });
29
+
30
+ afterEach(() => {
31
+ vi.useRealTimers();
32
+ });
33
+
34
+ describe('createRateLimiter', () => {
35
+ it('should return next() when rate limiting is disabled', async () => {
36
+ const rateLimiter = createRateLimiter(null, { enabled: false });
37
+ const next = vi.fn();
38
+ await rateLimiter({}, {}, next);
39
+ expect(next).toHaveBeenCalledTimes(1);
40
+ });
41
+
42
+ it('should skip rate limiting for exempt routes', async () => {
43
+ const rateLimiter = createRateLimiter(null, { exemptRoutes: ['/health'] });
44
+ const next = vi.fn();
45
+ await rateLimiter({ path: '/health' }, {}, next);
46
+ expect(next).toHaveBeenCalledTimes(1);
47
+ });
48
+
49
+ it('should use effectiveMax for POST requests', async () => {
50
+ const rateLimiter = createRateLimiter(null, { max: 100 });
51
+ const res = { setHeader: vi.fn() };
52
+ const req = { path: '/test', method: 'POST', headers: {}, socket: { remoteAddress: '127.0.0.1' } };
53
+ await rateLimiter(req, res, () => {});
54
+ expect(res.setHeader).toHaveBeenCalledWith('X-RateLimit-Limit', '50');
55
+ });
56
+
57
+ it('should handle invalid IP', async () => {
58
+ const loggerStub = { warn: vi.fn(), error: vi.fn() };
59
+ const rateLimiter = createRateLimiter(null, { logger: loggerStub });
60
+ const res = { setHeader: vi.fn() };
61
+ const req = { path: '/test', headers: {}, socket: { remoteAddress: null } };
62
+ await rateLimiter(req, res, () => {});
63
+ expect(loggerStub.warn).toHaveBeenCalledTimes(1);
64
+ });
65
+
66
+ it('should fall back to memory store when Redis errors', async () => {
67
+ const redisStub = { connected: true, eval: vi.fn().mockRejectedValue(new Error('Redis error')) };
68
+ const loggerStub = { warn: vi.fn(), error: vi.fn() };
69
+ const rateLimiter = createRateLimiter(redisStub, { logger: loggerStub, retryDelayMs: 1 });
70
+ const res = { setHeader: vi.fn() };
71
+ const req = { path: '/test', headers: {}, socket: { remoteAddress: '127.0.0.1' } };
72
+ await rateLimiter(req, res, () => {});
73
+ expect(loggerStub.error).toHaveBeenCalledTimes(1);
74
+ });
75
+
76
+ it('should return 429 when rate limit exceeded in memory after Redis fails', async () => {
77
+ const redisStub = { connected: true, eval: vi.fn().mockRejectedValue(new Error('Redis error')) };
78
+ const rateLimiter = createRateLimiter(redisStub, { max: 1, retryDelayMs: 1 });
79
+ const res = { setHeader: vi.fn(), writeHead: vi.fn().mockReturnThis(), end: vi.fn() };
80
+ const req1 = { path: '/test', headers: {}, socket: { remoteAddress: '127.0.0.1' } };
81
+ await rateLimiter(req1, res, () => {});
82
+ const res2 = { setHeader: vi.fn(), writeHead: vi.fn().mockReturnThis(), end: vi.fn() };
83
+ const req2 = { path: '/test', headers: {}, socket: { remoteAddress: '127.0.0.1' } };
84
+ await rateLimiter(req2, res2, () => {});
85
+ expect(res2.writeHead).toHaveBeenCalledWith(429, expect.any(Object));
86
+ expect(res2.end).toHaveBeenCalledTimes(1);
87
+ });
88
+
89
+ it('should set correct headers when rate limited after Redis fails', async () => {
90
+ const redisStub = { connected: true, eval: vi.fn().mockRejectedValue(new Error('Redis error')) };
91
+ const rateLimiter = createRateLimiter(redisStub, { max: 1, retryDelayMs: 1 });
92
+ const res = { setHeader: vi.fn(), writeHead: vi.fn().mockReturnThis(), end: vi.fn() };
93
+ const req1 = { path: '/test', headers: {}, socket: { remoteAddress: '127.0.0.1' } };
94
+ await rateLimiter(req1, res, () => {});
95
+ const res2 = { setHeader: vi.fn(), writeHead: vi.fn(), end: vi.fn() };
96
+ const req2 = { path: '/test', headers: {}, socket: { remoteAddress: '127.0.0.1' } };
97
+ await rateLimiter(req2, res2, () => {});
98
+ expect(res2.setHeader).toHaveBeenCalledWith('X-RateLimit-Limit', '1');
99
+ expect(res2.setHeader).toHaveBeenCalledWith('X-RateLimit-Remaining', '0');
100
+ });
101
+
102
+ it('should set X-RateLimit-Store header to redis when using Redis', async () => {
103
+ const redisStub = { connected: true, eval: vi.fn().mockResolvedValue([1, 99, 60, 0, false]) };
104
+ const rateLimiter = createRateLimiter(redisStub);
105
+ const res = { setHeader: vi.fn(), writeHead: vi.fn(), end: vi.fn() };
106
+ const req = { path: '/test', headers: {}, socket: { remoteAddress: '127.0.0.1' } };
107
+ await rateLimiter(req, res, () => {});
108
+ expect(res.setHeader).toHaveBeenCalledWith('X-RateLimit-Store', 'redis');
109
+ });
110
+
111
+ it('should call next() when not rate limited (Redis)', async () => {
112
+ const redisStub = { connected: true, eval: vi.fn().mockResolvedValue([1, 99, 60, 0, false]) };
113
+ const rateLimiter = createRateLimiter(redisStub);
114
+ const res = { setHeader: vi.fn(), writeHead: vi.fn(), end: vi.fn() };
115
+ const req = { path: '/test', headers: {}, socket: { remoteAddress: '127.0.0.1' } };
116
+ const next = vi.fn();
117
+ await rateLimiter(req, res, next);
118
+ expect(next).toHaveBeenCalledTimes(1);
119
+ });
120
+
121
+ it('should use DELETE request cost of 3x', async () => {
122
+ const rateLimiter = createRateLimiter(null, { max: 30 });
123
+ const res = { setHeader: vi.fn() };
124
+ const req = { path: '/test', method: 'DELETE', headers: {}, socket: { remoteAddress: '127.0.0.1' } };
125
+ await rateLimiter(req, res, () => {});
126
+ expect(res.setHeader).toHaveBeenCalledWith('X-RateLimit-Limit', '10');
127
+ });
128
+
129
+ it('should use PATCH request cost of 2x', async () => {
130
+ const rateLimiter = createRateLimiter(null, { max: 100 });
131
+ const res = { setHeader: vi.fn() };
132
+ const req = { path: '/test', method: 'PATCH', headers: {}, socket: { remoteAddress: '127.0.0.1' } };
133
+ await rateLimiter(req, res, () => {});
134
+ expect(res.setHeader).toHaveBeenCalledWith('X-RateLimit-Limit', '50');
135
+ });
136
+ });
137
+
138
+ describe('memFallback', () => {
139
+ beforeEach(() => {
140
+ vi.useFakeTimers();
141
+ });
142
+
143
+ it('should reset count on new window', () => {
144
+ const info1 = memFallback('127.0.0.1', 10, 60000);
145
+ expect(info1.count).toBe(1);
146
+ vi.advanceTimersByTime(61000);
147
+ const info2 = memFallback('127.0.0.1', 10, 60000);
148
+ expect(info2.count).toBe(1);
149
+ });
150
+
151
+ it('should increase count within window', () => {
152
+ const info1 = memFallback('127.0.0.1', 10, 60000);
153
+ expect(info1.count).toBe(1);
154
+ const info2 = memFallback('127.0.0.1', 10, 60000);
155
+ expect(info2.count).toBe(2);
156
+ });
157
+
158
+ it('should set limited=true when rate limit exceeded', () => {
159
+ const info1 = memFallback('127.0.0.1', 1, 60000);
160
+ expect(info1.limited).toBe(false);
161
+ const info2 = memFallback('127.0.0.1', 1, 60000);
162
+ expect(info2.limited).toBe(true);
163
+ });
164
+
165
+ it('should track remaining requests correctly', () => {
166
+ const info1 = memFallback('127.0.0.1', 10, 60000);
167
+ expect(info1.remaining).toBe(9);
168
+ const info2 = memFallback('127.0.0.1', 10, 60000);
169
+ expect(info2.remaining).toBe(8);
170
+ });
171
+
172
+ it('should increment violationCount on each violation within block', () => {
173
+ const info1 = memFallback('127.0.0.1', 1, 60000);
174
+ expect(info1.violationCount).toBe(0);
175
+ const info2 = memFallback('127.0.0.1', 1, 60000);
176
+ expect(info2.violationCount).toBe(1);
177
+ const info3 = memFallback('127.0.0.1', 1, 60000);
178
+ expect(info3.violationCount).toBe(2);
179
+ });
180
+
181
+ it('should carry over violationCount when window expires but block continues', () => {
182
+ const info1 = memFallback('127.0.0.1', 1, 60000);
183
+ const info2 = memFallback('127.0.0.1', 1, 60000);
184
+ expect(info2.violationCount).toBe(1);
185
+ vi.advanceTimersByTime(301000);
186
+ const info3 = memFallback('127.0.0.1', 1, 60000);
187
+ expect(info3.violationCount).toBe(1);
188
+ });
189
+
190
+ it('should use correct retryAfter values for escalation within block', () => {
191
+ const info1 = memFallback('127.0.0.1', 1, 60000);
192
+ expect(info1.retryAfter).toBe(0);
193
+ const info2 = memFallback('127.0.0.1', 1, 60000);
194
+ expect(info2.retryAfter).toBe(300);
195
+ const info3 = memFallback('127.0.0.1', 1, 60000);
196
+ expect(info3.retryAfter).toBe(1200);
197
+ });
198
+ });
199
+
200
+ describe('checkRedis', () => {
201
+ it('should return correct info when not rate limited', async () => {
202
+ const redisStub = { eval: vi.fn().mockResolvedValue([50, 50, 60, 0, false]) };
203
+ const info = await checkRedis(redisStub, '127.0.0.1', 100, 60);
204
+ expect(info.count).toBe(50);
205
+ expect(info.limited).toBe(false);
206
+ });
207
+
208
+ it('should return correct info when rate limited', async () => {
209
+ const redisStub = { eval: vi.fn().mockResolvedValue([101, 0, 300, 300, true]) };
210
+ const info = await checkRedis(redisStub, '127.0.0.1', 100, 60);
211
+ expect(info.limited).toBe(true);
212
+ expect(info.retryAfter).toBe(300);
213
+ });
214
+
215
+ it('should throw when circuit breaker is open', async () => {
216
+ const cb = getCircuitBreaker();
217
+ cb.isOpen = true;
218
+ cb.lastFailure = Date.now();
219
+ const redisStub = { eval: vi.fn() };
220
+ await expect(checkRedis(redisStub, '127.0.0.1', 100, 60)).rejects.toThrow(/Circuit breaker open/);
221
+ });
222
+
223
+ it('should reset circuit breaker after timeout', async () => {
224
+ const cb = getCircuitBreaker();
225
+ cb.isOpen = true;
226
+ cb.lastFailure = Date.now() - 60000;
227
+ cb.failureCount = 5;
228
+ const redisStub = { eval: vi.fn().mockResolvedValue([1, 99, 60, 0, false]) };
229
+ const info = await checkRedis(redisStub, '127.0.0.1', 100, 60);
230
+ expect(cb.isOpen).toBe(false);
231
+ expect(cb.failureCount).toBe(0);
232
+ });
233
+
234
+ it('should reset failure count on success', async () => {
235
+ const cb = getCircuitBreaker();
236
+ cb.failureCount = 2;
237
+ const redisStub = { eval: vi.fn().mockResolvedValue([1, 99, 60, 0, false]) };
238
+ await checkRedis(redisStub, '127.0.0.1', 100, 60);
239
+ expect(cb.failureCount).toBe(0);
240
+ });
241
+
242
+ it('should throw after max retries', async () => {
243
+ const redisStub = { eval: vi.fn().mockRejectedValue(new Error('Permanent error')) };
244
+ await expect(
245
+ checkRedis(redisStub, '127.0.0.1', 100, 60, 1)
246
+ ).rejects.toThrow(/Max retries exceeded/);
247
+ });
248
+
249
+ it('should use exponential backoff when retryDelayMs is not provided', async () => {
250
+ const redisStub = { eval: vi.fn().mockRejectedValue(new Error('err')) };
251
+ await expect(
252
+ checkRedis(redisStub, '127.0.0.1', 100, 60, 2)
253
+ ).rejects.toThrow(/Max retries exceeded/);
254
+ })
255
+
256
+ it('should open circuit breaker after 3 failures', async () => {
257
+ const redisStub = {
258
+ eval: vi.fn()
259
+ .mockRejectedValueOnce(new Error('err1'))
260
+ .mockRejectedValueOnce(new Error('err2'))
261
+ .mockRejectedValueOnce(new Error('err3'))
262
+ .mockRejectedValue(new Error('default err'))
263
+ };
264
+ try {
265
+ await checkRedis(redisStub, '127.0.0.1', 100, 60, 1);
266
+ } catch (e) {
267
+ }
268
+ const cb = getCircuitBreaker();
269
+ expect(cb.isOpen).toBe(true);
270
+ });
271
+ });
272
+
273
+ describe('getClientIp', () => {
274
+ it('should extract IP from X-Forwarded-For with trusted proxy', () => {
275
+ const req = {
276
+ headers: { 'x-forwarded-for': '10.0.1.5, 192.168.1.1' },
277
+ socket: { remoteAddress: '127.0.0.1' }
278
+ };
279
+ const ip = getClientIp(req);
280
+ expect(ip).toBe('10.0.1.5');
281
+ });
282
+
283
+ it('should use remoteAddress when not from trusted proxy', () => {
284
+ const req = {
285
+ headers: { 'x-forwarded-for': '192.168.1.1' },
286
+ socket: { remoteAddress: '203.0.113.1' }
287
+ };
288
+ const ip = getClientIp(req);
289
+ expect(ip).toBe('203.0.113.1');
290
+ });
291
+
292
+ it('should handle IPv6-mapped IPv4', () => {
293
+ const req = {
294
+ headers: {},
295
+ socket: { remoteAddress: '::ffff:192.168.1.1' }
296
+ };
297
+ const ip = getClientIp(req);
298
+ expect(ip).toBe('192.168.1.1');
299
+ });
300
+
301
+ it('should handle IPv6 addresses', () => {
302
+ const req = {
303
+ headers: {},
304
+ socket: { remoteAddress: '2001:db8::1' }
305
+ };
306
+ const ip = getClientIp(req);
307
+ expect(ip).toBe('2001:db8::1');
308
+ });
309
+
310
+ it('should handle invalid IPs', () => {
311
+ const req = {
312
+ headers: {},
313
+ socket: { remoteAddress: null }
314
+ };
315
+ const ip = getClientIp(req);
316
+ expect(ip).toBe('unknown');
317
+ });
318
+
319
+ it('should handle missing X-Forwarded-For header', () => {
320
+ const req = {
321
+ headers: {},
322
+ socket: { remoteAddress: '127.0.0.1' }
323
+ };
324
+ const ip = getClientIp(req);
325
+ expect(ip).toBe('127.0.0.1');
326
+ });
327
+
328
+ it('should lowercase IPv6 addresses', () => {
329
+ const req = {
330
+ headers: {},
331
+ socket: { remoteAddress: '2001:DB8::1' }
332
+ };
333
+ const ip = getClientIp(req);
334
+ expect(ip).toBe('2001:db8::1');
335
+ });
336
+
337
+ it('should return first IP from X-Forwarded-For when multiple IPs', () => {
338
+ const req = {
339
+ headers: { 'x-forwarded-for': '1.2.3.4, 5.6.7.8, 9.10.11.12' },
340
+ socket: { remoteAddress: '127.0.0.1' }
341
+ };
342
+ const ip = getClientIp(req);
343
+ expect(ip).toBe('1.2.3.4');
344
+ });
345
+ });
346
+
347
+ describe('isTrustedProxy', () => {
348
+ it('should match localhost', () => {
349
+ expect(isTrustedProxy('127.0.0.1')).toBe(true);
350
+ });
351
+
352
+ it('should match IPv6 localhost', () => {
353
+ expect(isTrustedProxy('::1')).toBe(true);
354
+ });
355
+
356
+ it('should match 10.0.0.0/8 range', () => {
357
+ expect(isTrustedProxy('10.0.0.5')).toBe(true);
358
+ expect(isTrustedProxy('10.255.255.255')).toBe(true);
359
+ });
360
+
361
+ it('should match 172.16.0.0/12 range', () => {
362
+ expect(isTrustedProxy('172.16.0.1')).toBe(true);
363
+ expect(isTrustedProxy('172.31.255.255')).toBe(true);
364
+ });
365
+
366
+ it('should match 192.168.0.0/16 range', () => {
367
+ expect(isTrustedProxy('192.168.1.1')).toBe(true);
368
+ });
369
+
370
+ it('should not match IPs outside trusted ranges', () => {
371
+ expect(isTrustedProxy('8.8.8.8')).toBe(false);
372
+ expect(isTrustedProxy('172.15.0.1')).toBe(false);
373
+ });
374
+
375
+ it('should handle invalid inputs', () => {
376
+ expect(isTrustedProxy(null)).toBe(false);
377
+ expect(isTrustedProxy('unknown')).toBe(false);
378
+ expect(isTrustedProxy('')).toBe(false);
379
+ });
380
+ });
381
+
382
+ describe('getRequestCost', () => {
383
+ it('should return 1 for GET', () => {
384
+ expect(getRequestCost({ method: 'GET' })).toBe(1);
385
+ });
386
+
387
+ it('should return 1 for HEAD', () => {
388
+ expect(getRequestCost({ method: 'HEAD' })).toBe(1);
389
+ });
390
+
391
+ it('should return 2 for POST', () => {
392
+ expect(getRequestCost({ method: 'POST' })).toBe(2);
393
+ });
394
+
395
+ it('should return 2 for PUT', () => {
396
+ expect(getRequestCost({ method: 'PUT' })).toBe(2);
397
+ });
398
+
399
+ it('should return 2 for PATCH', () => {
400
+ expect(getRequestCost({ method: 'PATCH' })).toBe(2);
401
+ });
402
+
403
+ it('should return 3 for DELETE', () => {
404
+ expect(getRequestCost({ method: 'DELETE' })).toBe(3);
405
+ });
406
+
407
+ it('should return 1 for unknown methods', () => {
408
+ expect(getRequestCost({ method: 'UNKNOWN' })).toBe(1);
409
+ expect(getRequestCost({ method: 'OPTIONS' })).toBe(1);
410
+ });
411
+ });
412
+
413
+ describe('memStore internal operations', () => {
414
+ it('should export getMemStore', () => {
415
+ const rateLimiter = require('../../src/middleware/rate-limiter');
416
+ const memStore = rateLimiter.getMemStore();
417
+ expect(typeof memStore.get).toBe('function');
418
+ expect(typeof memStore.set).toBe('function');
419
+ expect(typeof memStore.delete).toBe('function');
420
+ expect(typeof memStore.entries).toBe('function');
421
+ expect(typeof memStore.size).toBe('function');
422
+ });
423
+
424
+ it('should track size correctly via size() method', () => {
425
+ const rateLimiter = require('../../src/middleware/rate-limiter');
426
+ rateLimiter.resetMemStore();
427
+ const memStore = rateLimiter.getMemStore();
428
+ expect(memStore.size()).toBe(0);
429
+ memStore.set('192.168.1.1', { count: 1, resetAt: Date.now() + 60000 });
430
+ expect(memStore.size()).toBe(1);
431
+ memStore.set('192.168.1.2', { count: 2, resetAt: Date.now() + 60000 });
432
+ expect(memStore.size()).toBe(2);
433
+ memStore.delete('192.168.1.1');
434
+ expect(memStore.size()).toBe(1);
435
+ });
436
+
437
+ it('should handle entries iteration', () => {
438
+ const rateLimiter = require('../../src/middleware/rate-limiter');
439
+ rateLimiter.resetMemStore();
440
+ const memStore = rateLimiter.getMemStore();
441
+ memStore.set('192.168.1.1', { count: 1, resetAt: Date.now() + 60000 });
442
+ memStore.set('192.168.1.2', { count: 2, resetAt: Date.now() + 60000 });
443
+ let entriesCount = 0;
444
+ for (const entry of memStore.entries()) {
445
+ entriesCount++;
446
+ }
447
+ expect(entriesCount).toBe(2);
448
+ });
449
+
450
+ it('should use LRU behavior on get', () => {
451
+ const rateLimiter = require('../../src/middleware/rate-limiter');
452
+ rateLimiter.resetMemStore();
453
+ const memStore = rateLimiter.getMemStore();
454
+ memStore.set('key1', { count: 1, resetAt: Date.now() + 60000 });
455
+ memStore.set('key2', { count: 2, resetAt: Date.now() + 60000 });
456
+ memStore.get('key1');
457
+ memStore.set('key3', { count: 3, resetAt: Date.now() + 60000 });
458
+ memStore.get('key1');
459
+ });
460
+
461
+ it('should trigger LRU eviction when exceeding max entries', () => {
462
+ const rateLimiter = require('../../src/middleware/rate-limiter');
463
+ rateLimiter.resetMemStore();
464
+ const memStore = rateLimiter.getMemStore();
465
+ for (let i = 0; i < 10001; i++) {
466
+ memStore.set(`ip${i}`, { count: i, resetAt: Date.now() + 60000 });
467
+ }
468
+ expect(memStore.size()).toBeLessThanOrEqual(10000);
469
+ });
470
+
471
+ it('should cleanup expired entries via triggerCleanup', () => {
472
+ const rateLimiter = require('../../src/middleware/rate-limiter');
473
+ rateLimiter.resetMemStore();
474
+ const memStore = rateLimiter.getMemStore();
475
+ memStore.set('192.168.1.1', { count: 1, resetAt: Date.now() - 1000 });
476
+ memStore.set('192.168.1.2', { count: 2, resetAt: Date.now() + 60000 });
477
+ expect(memStore.size()).toBe(2);
478
+ rateLimiter.triggerCleanup();
479
+ expect(memStore.size()).toBe(1);
480
+ });
481
+ });
482
+ });
tests/redis/redis.test.js CHANGED
@@ -98,14 +98,20 @@ describe('redis.js', () => {
98
  await expect(p).resolves.toEqual(Buffer.from('val'))
99
  })
100
 
 
 
 
 
 
 
101
  it('quit sends QUIT and destroys socket', async () => {
102
  const RedisClient = _require('../../src/redis/index.js')
103
  const c = new RedisClient({ host: '127.0.0.1', port: 6379 })
 
104
  const destroy = vi.fn()
105
- c.socket = { write: vi.fn(), destroy }
106
- const p = c.quit()
107
- c._onData(Buffer.from('+OK\r\n'))
108
- await p
109
  expect(destroy).toHaveBeenCalled()
110
  })
111
 
@@ -192,6 +198,14 @@ describe('redis.js', () => {
192
  expect(c.queue.length).toBe(1)
193
  })
194
 
 
 
 
 
 
 
 
 
195
  it('covers redis helper methods', async () => {
196
  const RedisClient = _require('../../src/redis/index.js')
197
  const c = new RedisClient({ host: '127.0.0.1', port: 6379 })
@@ -205,7 +219,8 @@ describe('redis.js', () => {
205
  c.expire('k', 1)
206
  c.del('a', 'b')
207
  c.ttl('k')
208
- expect(c.socket.write).toHaveBeenCalledTimes(9)
 
209
  })
210
 
211
  it('connect handler with auth failure via TCP', async () => {
 
98
  await expect(p).resolves.toEqual(Buffer.from('val'))
99
  })
100
 
101
+ it('quit resolves immediately when socket is null', async () => {
102
+ const RedisClient = _require('../../src/redis/index.js')
103
+ const c = new RedisClient({ host: '127.0.0.1', port: 6379 })
104
+ await expect(c.quit()).resolves.toBeUndefined()
105
+ })
106
+
107
  it('quit sends QUIT and destroys socket', async () => {
108
  const RedisClient = _require('../../src/redis/index.js')
109
  const c = new RedisClient({ host: '127.0.0.1', port: 6379 })
110
+ const write = vi.fn()
111
  const destroy = vi.fn()
112
+ c.socket = { write, destroy }
113
+ await expect(c.quit()).resolves.toBeUndefined()
114
+ expect(write).toHaveBeenCalledWith(expect.stringContaining('QUIT'))
 
115
  expect(destroy).toHaveBeenCalled()
116
  })
117
 
 
198
  expect(c.queue.length).toBe(1)
199
  })
200
 
201
+ it('handles write error in send()', async () => {
202
+ const RedisClient = _require('../../src/redis/index.js')
203
+ const c = new RedisClient({ host: '127.0.0.1', port: 6379 })
204
+ c.socket = { write: vi.fn(() => { throw new Error('write error') }) }
205
+ await expect(c.send('PING')).rejects.toThrow('write error')
206
+ expect(c.queue.length).toBe(0)
207
+ })
208
+
209
  it('covers redis helper methods', async () => {
210
  const RedisClient = _require('../../src/redis/index.js')
211
  const c = new RedisClient({ host: '127.0.0.1', port: 6379 })
 
219
  c.expire('k', 1)
220
  c.del('a', 'b')
221
  c.ttl('k')
222
+ c.eval('return 1', 0)
223
+ expect(c.socket.write).toHaveBeenCalledTimes(10)
224
  })
225
 
226
  it('connect handler with auth failure via TCP', async () => {