Nodejs-JSON-API-Server / docs /TECHNICAL.md
dangkhoa2016's picture
docs: extract technical content into bilingual docs
42e30c2
|
Raw History Blame
9.96 kB
# Technical Documentation
> 🌐 Language / NgΓ΄n ngα»―: **English** | [TiαΊΏng Việt](TECHNICAL.vi.md)
> Architecture, implementation notes, and internal design of json-api-server.
## Architecture
```
json-api-server/
β”œβ”€β”€ bin/
β”‚ └── start.js # Entry point β€” loads .env via src/config/load-env.js, starts server
β”œβ”€β”€ src/
β”‚ β”œβ”€β”€ config/
β”‚ β”‚ β”œβ”€β”€ index.js # Centralized config β€” auto-loads dotenv via load-env.js, exports camelCase
β”‚ β”‚ β”œβ”€β”€ load-env.js # Shared dotenv loader β€” auto-run on require, skips in production
β”‚ β”‚ β”œβ”€β”€ runtime-config.js # Thread-safe in-memory config overrides for runtime updates (rate-limit, Redis)
β”‚ β”‚ └── setting-defs.js # Setting definitions for 14 env vars (NODE_ENV, PORT, ADMIN_KEY, etc.)
β”‚ β”œβ”€β”€ db/
β”‚ β”‚ β”œβ”€β”€ index.js # SQLite layer (node:sqlite) β€” CRUD operations (reads config)
β”‚ β”‚ β”œβ”€β”€ migrate.js # Table creation (standalone via npm run db:migrate)
β”‚ β”‚ β”œβ”€β”€ seed.js # Fetches seed data from JSONPlaceholder API, auto-runs migrate
β”‚ β”‚ β”œβ”€β”€ seed-settings.js # Seeds 14 env vars (NODE_ENV, PORT, ADMIN_KEY, etc.) into settings table
β”‚ β”‚ └── sql-logger.js # Shared Proxy wrappers β€” logs exec/prepare/run/get/all to stderr
β”‚ β”œβ”€β”€ middleware/
β”‚ β”‚ └── rate-limiter.js # Rate limiter (Redis/in-memory, circuit breaker, escalating blocks)
β”‚ β”œβ”€β”€ redis/
β”‚ β”‚ └── index.js # Pure-Node Redis client via RESP protocol over TCP
β”‚ β”œβ”€β”€ public/
β”‚ β”‚ β”œβ”€β”€ favicon.ico # Favicon (ICO format)
β”‚ β”‚ β”œβ”€β”€ favicon.png # Favicon (PNG format)
β”‚ β”‚ └── license.md # Flaticon license file
β”‚ └── server/
β”‚ β”œβ”€β”€ index.js # HTTP server, graceful shutdown, startup orchestration
β”‚ └── route.js # Route parser, request handlers, admin auth, favicon, runtime config updates
β”œβ”€β”€ tests/
β”‚ β”œβ”€β”€ config/
β”‚ β”‚ β”œβ”€β”€ config.test.js # Config defaults and env var branches (9)
β”‚ β”‚ └── load-env.test.js # Load-env file loading chain and error paths (7)
β”‚ β”œβ”€β”€ db/
β”‚ β”‚ β”œβ”€β”€ database.test.js # Database CRUD, pagination, search, sort, SQL injection, cascade delete (11)
β”‚ β”‚ β”œβ”€β”€ migrate.test.js # Migration success and failure paths (2)
β”‚ β”‚ β”œβ”€β”€ seed.test.js # Seed with real DB + mocked deps, JSONPlaceholder fetch (5)
β”‚ β”‚ └── sql-logger.test.js # SQL query logger wrapping (5)
β”‚ β”œβ”€β”€ middleware/
β”‚ β”‚ └── rate-limiter.test.js # In-memory, Redis, circuit breaker, proxy IPs, updateConfig (57)
β”‚ β”œβ”€β”€ redis/
β”‚ β”‚ └── redis.test.js # RESP protocol encoding/parsing, constructor options, eval, reconnect (33)
β”‚ β”œβ”€β”€ server/
β”‚ β”‚ β”œβ”€β”€ coverage-printlog.test.js # Printlog and server export V8 coverage (4)
β”‚ β”‚ β”œβ”€β”€ graceful-shutdown.test.js # SIGINT/SIGTERM handler coverage (1)
β”‚ β”‚ β”œβ”€β”€ index.test.js # Server request handler, admin auth, graceful shutdown (13)
β”‚ β”‚ β”œβ”€β”€ integration.test.js # API integration tests β€” real HTTP + SQLite, runtime PATCH (81)
β”‚ β”‚ └── route.test.js # Route parsing, favicon, health, admin auth cache, runtime config (18)
β”‚ β”œβ”€β”€ README.md # Testing documentation (English)
β”‚ β”œβ”€β”€ README.vi.md # Testing documentation (Vietnamese)
β”‚ └── helpers/
β”‚ β”œβ”€β”€ coverage.js # Test-coverage utilities (save/restore/setEnv/clearCjs/configMockFactory)
β”‚ β”œβ”€β”€ index.js # startServer / stopServer / request utilities
β”‚ β”œβ”€β”€ mock-factory.js # Mock factory helpers (mkDb/mkReq/mkRes/mkRedis/mkSettingsTable)
β”‚ └── seed.js # Standalone script to create & seed temp DB
β”‚ └── seed-settings-coverage.test.js # Seed-settings.js V8 coverage (4)
β”œβ”€β”€ manual/
β”‚ β”œβ”€β”€ admin.sh # Admin panel curl commands
β”‚ β”œβ”€β”€ albums.sh # Albums endpoints
β”‚ β”œβ”€β”€ comments.sh # Comments endpoints
β”‚ β”œβ”€β”€ curl.sh # Quick curl commands
β”‚ β”œβ”€β”€ health.sh # Health endpoint
β”‚ β”œβ”€β”€ inspect-queries.sql # SQL queries for database inspection
β”‚ β”œβ”€β”€ inspect.sh # Database inspection script
β”‚ β”œβ”€β”€ inspect-docker-data.sh # Docker data inspection script
β”‚ β”œβ”€β”€ photos.sh # Photos endpoints
β”‚ β”œβ”€β”€ posts.sh # Posts endpoints
β”‚ β”œβ”€β”€ todos.sh # Todos endpoints
β”‚ └── users.sh # Users endpoints
β”œβ”€β”€ manual-test-coverage/
β”‚ β”œβ”€β”€ README.md # Coverage verification documentation
β”‚ └── verify-commit-coverage.sh # Coverage verification script
β”œβ”€β”€ Dockerfile # Docker image definition
β”œβ”€β”€ docker-entrypoint.sh # Container entrypoint script
β”œβ”€β”€ .dockerignore # Docker ignore rules
β”œβ”€β”€ storage/ # SQLite database files (auto-created)
β”œβ”€β”€ temp/ # Temporary files (gitignored)
β”œβ”€β”€ .env # Base configuration (tried first in development β€” highest priority)
β”œβ”€β”€ .env.dev # Development fallback (tried if .env not found)
β”œβ”€β”€ .env.test # Test configuration (port 3001, separate DB, no rate limit)
β”œβ”€β”€ .env.prod.example # Production template (copy to .env.prod)
β”œβ”€β”€ .env.example # Reference for all available variables
β”œβ”€β”€ package.json # Metadata and scripts
β”œβ”€β”€ LICENSE # MIT license
β”œβ”€β”€ README.md # Documentation (English)
β”œβ”€β”€ README.vi.md # Documentation (Vietnamese)
β”œβ”€β”€ .gitignore # Git ignore rules
└── vitest.config.js # Vitest test runner configuration
```
### Startup Flow
```
bin/start.js β†’ src/config/load-env.js (loads .env per NODE_ENV, skipped in production)
β†’ src/server/index.js
β”œβ”€β”€ src/config/index.js (centralized config, auto-loads dotenv)
β”œβ”€β”€ src/db/index.js (SQLite CRUD)
β”œβ”€β”€ src/redis/index.js (pure RESP + AUTH + URL)
└── src/middleware/rate-limiter.js (Redis || in-memory, circuit breaker, escalating blocks)
# Standalone scripts: npm run db:migrate / npm run db:seed / npm run db:setup (config loads dotenv automatically)
```
### Request Flow
```
HTTP Request β†’ CORS headers β†’ Rate limiter β†’ Route parser β†’ Handler β†’ JSON Response
```
---
## Implementation Notes
- **Minimal runtime dependencies** β€” only `argon2` for admin password hashing; everything else uses Node.js built-in modules (`http`, `url`, `fs`, `path`, `net`, `node:sqlite`). `dotenv` is a dev dependency.
- **Pure RESP protocol** β€” the Redis client in `src/redis/index.js` implements the Redis serialization protocol over raw TCP sockets without any third-party library. Supports `AUTH` password authentication and `REDIS_URL` connection strings.
- **Centralized config** β€” all environment variables are read in `src/config/index.js` and exported as camelCase (`port`, `dbPath`, `redisOpts`, `rateLimitMax`, `dbDebugSql`, `sensitiveKeys`, etc.) for use across the codebase.
- **Rate limiter** β€” `src/middleware/rate-limiter.js` features a circuit breaker for Redis failures, CIDR-based trusted proxy IP extraction, escalating block durations (5m β†’ 20m β†’ 1h), atomic Redis Lua scripts, and an in-memory fallback with LRU eviction. `createRateLimiter()` accepts options directly instead of reading config lazily.
- **Runtime configuration** β€” Patching rate-limit or Redis settings via the admin API immediately applies changes through `RuntimeConfig` (in-memory overrides), avoiding server restarts. Rate-limit hot-swapping uses `rateLimiter.updateConfig()`; Redis reconnection uses `Redis.reconnect()`.
- **Testable seed script** β€” `src/db/seed.js` accepts `database` and `fetch` parameters via dependency injection, enabling full unit testing without mocking `require()`.
- **SQL query logging** β€” `src/db/sql-logger.js` exports `wrapDb`/`wrapStmt` Proxy wrappers that log `exec`, `prepare`, `run`, `get`, and `all` calls to stderr. `src/db/index.js` uses them via `getWrappedDb()` when `DEBUG_SQL=true`.
- **Multi-environment** β€” `src/config/index.js` requires `src/config/load-env.js` at module level, which chain-loads all dotenv files in priority order with `override: false` β€” existing `process.env` values and earlier files take precedence over later ones. Every consumer (server, migrate, seed) simply requires `src/config/index.js` and gets correct env values. In production, dotenv is skipped entirely β€” env vars must come from the deployment environment.
- **SQL injection prevention** β€” `src/db/index.js` validates `_sort` against a whitelist of known columns and quotes identifiers with `""`; LIKE wildcards (`%`, `_`) are escaped to prevent injection through the `q` parameter
- **Argon2 auth cache** β€” `src/server/route.js` caches `ADMIN_KEY` verification results in a `Map` with 5-second TTL and 1,000-entry limit, avoiding repeated argon2 hashing on burst admin requests
- **CORS** enabled on all routes
- **Graceful shutdown** β€” handles `SIGINT` and `SIGTERM` to close the server and Redis connection cleanly