Nodejs-JSON-API-Server / docs /TECHNICAL.md
dangkhoa2016's picture
docs: extract technical content into bilingual docs
42e30c2
|
Raw History Blame Contribute Delete
9.96 kB

Technical Documentation

🌐 Language / NgΓ΄n ngα»―: English | TiαΊΏng Việt

Architecture, implementation notes, and internal design of json-api-server.

Architecture

json-api-server/
β”œβ”€β”€ bin/
β”‚   └── start.js                 # Entry point β€” loads .env via src/config/load-env.js, starts server
β”œβ”€β”€ src/
β”‚   β”œβ”€β”€ config/
β”‚   β”‚   β”œβ”€β”€ index.js              # Centralized config β€” auto-loads dotenv via load-env.js, exports camelCase
β”‚   β”‚   β”œβ”€β”€ load-env.js           # Shared dotenv loader β€” auto-run on require, skips in production
β”‚   β”‚   β”œβ”€β”€ runtime-config.js     # Thread-safe in-memory config overrides for runtime updates (rate-limit, Redis)
β”‚   β”‚   └── setting-defs.js       # Setting definitions for 14 env vars (NODE_ENV, PORT, ADMIN_KEY, etc.)
β”‚   β”œβ”€β”€ db/
β”‚   β”‚   β”œβ”€β”€ index.js              # SQLite layer (node:sqlite) β€” CRUD operations (reads config)
β”‚   β”‚   β”œβ”€β”€ migrate.js            # Table creation (standalone via npm run db:migrate)
β”‚   β”‚   β”œβ”€β”€ seed.js               # Fetches seed data from JSONPlaceholder API, auto-runs migrate
β”‚   β”‚   β”œβ”€β”€ seed-settings.js      # Seeds 14 env vars (NODE_ENV, PORT, ADMIN_KEY, etc.) into settings table
β”‚   β”‚   └── sql-logger.js         # Shared Proxy wrappers β€” logs exec/prepare/run/get/all to stderr
β”‚   β”œβ”€β”€ middleware/
β”‚   β”‚   └── rate-limiter.js       # Rate limiter (Redis/in-memory, circuit breaker, escalating blocks)
β”‚   β”œβ”€β”€ redis/
β”‚   β”‚   └── index.js              # Pure-Node Redis client via RESP protocol over TCP
β”‚   β”œβ”€β”€ public/
β”‚   β”‚   β”œβ”€β”€ favicon.ico           # Favicon (ICO format)
β”‚   β”‚   β”œβ”€β”€ favicon.png           # Favicon (PNG format)
β”‚   β”‚   └── license.md            # Flaticon license file
β”‚   └── server/
β”‚       β”œβ”€β”€ index.js              # HTTP server, graceful shutdown, startup orchestration
β”‚       └── route.js              # Route parser, request handlers, admin auth, favicon, runtime config updates
β”œβ”€β”€ tests/
β”‚   β”œβ”€β”€ config/
β”‚   β”‚   β”œβ”€β”€ config.test.js           # Config defaults and env var branches (9)
β”‚   β”‚   └── load-env.test.js         # Load-env file loading chain and error paths (7)
β”‚   β”œβ”€β”€ db/
β”‚   β”‚   β”œβ”€β”€ database.test.js         # Database CRUD, pagination, search, sort, SQL injection, cascade delete (11)
β”‚   β”‚   β”œβ”€β”€ migrate.test.js          # Migration success and failure paths (2)
β”‚   β”‚   β”œβ”€β”€ seed.test.js             # Seed with real DB + mocked deps, JSONPlaceholder fetch (5)
β”‚   β”‚   └── sql-logger.test.js       # SQL query logger wrapping (5)
β”‚   β”œβ”€β”€ middleware/
β”‚   β”‚   └── rate-limiter.test.js     # In-memory, Redis, circuit breaker, proxy IPs, updateConfig (57)
β”‚   β”œβ”€β”€ redis/
β”‚   β”‚   └── redis.test.js            # RESP protocol encoding/parsing, constructor options, eval, reconnect (33)
β”‚   β”œβ”€β”€ server/
β”‚   β”‚   β”œβ”€β”€ coverage-printlog.test.js # Printlog and server export V8 coverage (4)
β”‚   β”‚   β”œβ”€β”€ graceful-shutdown.test.js # SIGINT/SIGTERM handler coverage (1)
β”‚   β”‚   β”œβ”€β”€ index.test.js            # Server request handler, admin auth, graceful shutdown (13)
β”‚   β”‚   β”œβ”€β”€ integration.test.js      # API integration tests β€” real HTTP + SQLite, runtime PATCH (81)
β”‚   β”‚   └── route.test.js            # Route parsing, favicon, health, admin auth cache, runtime config (18)
β”‚   β”œβ”€β”€ README.md                    # Testing documentation (English)
β”‚   β”œβ”€β”€ README.vi.md                 # Testing documentation (Vietnamese)
β”‚   └── helpers/
β”‚       β”œβ”€β”€ coverage.js              # Test-coverage utilities (save/restore/setEnv/clearCjs/configMockFactory)
β”‚       β”œβ”€β”€ index.js                 # startServer / stopServer / request utilities
β”‚       β”œβ”€β”€ mock-factory.js          # Mock factory helpers (mkDb/mkReq/mkRes/mkRedis/mkSettingsTable)
β”‚       └── seed.js                  # Standalone script to create & seed temp DB
β”‚   └── seed-settings-coverage.test.js  # Seed-settings.js V8 coverage (4)
β”œβ”€β”€ manual/
β”‚   β”œβ”€β”€ admin.sh                 # Admin panel curl commands
β”‚   β”œβ”€β”€ albums.sh                # Albums endpoints
β”‚   β”œβ”€β”€ comments.sh              # Comments endpoints
β”‚   β”œβ”€β”€ curl.sh                  # Quick curl commands
β”‚   β”œβ”€β”€ health.sh                # Health endpoint
β”‚   β”œβ”€β”€ inspect-queries.sql      # SQL queries for database inspection
β”‚   β”œβ”€β”€ inspect.sh               # Database inspection script
β”‚   β”œβ”€β”€ inspect-docker-data.sh   # Docker data inspection script
β”‚   β”œβ”€β”€ photos.sh                # Photos endpoints
β”‚   β”œβ”€β”€ posts.sh                 # Posts endpoints
β”‚   β”œβ”€β”€ todos.sh                 # Todos endpoints
β”‚   └── users.sh                 # Users endpoints
β”œβ”€β”€ manual-test-coverage/
β”‚   β”œβ”€β”€ README.md                    # Coverage verification documentation
β”‚   └── verify-commit-coverage.sh    # Coverage verification script
β”œβ”€β”€ Dockerfile                   # Docker image definition
β”œβ”€β”€ docker-entrypoint.sh         # Container entrypoint script
β”œβ”€β”€ .dockerignore                # Docker ignore rules
β”œβ”€β”€ storage/                     # SQLite database files (auto-created)
β”œβ”€β”€ temp/                        # Temporary files (gitignored)
β”œβ”€β”€ .env                         # Base configuration (tried first in development β€” highest priority)
β”œβ”€β”€ .env.dev                     # Development fallback (tried if .env not found)
β”œβ”€β”€ .env.test                    # Test configuration (port 3001, separate DB, no rate limit)
β”œβ”€β”€ .env.prod.example            # Production template (copy to .env.prod)
β”œβ”€β”€ .env.example                 # Reference for all available variables
β”œβ”€β”€ package.json                 # Metadata and scripts
β”œβ”€β”€ LICENSE                      # MIT license
β”œβ”€β”€ README.md                    # Documentation (English)
β”œβ”€β”€ README.vi.md                 # Documentation (Vietnamese)
β”œβ”€β”€ .gitignore                   # Git ignore rules
└── vitest.config.js             # Vitest test runner configuration

Startup Flow

bin/start.js β†’ src/config/load-env.js (loads .env per NODE_ENV, skipped in production)
  β†’ src/server/index.js
      β”œβ”€β”€ src/config/index.js     (centralized config, auto-loads dotenv)
      β”œβ”€β”€ src/db/index.js         (SQLite CRUD)
      β”œβ”€β”€ src/redis/index.js      (pure RESP + AUTH + URL)
      └── src/middleware/rate-limiter.js (Redis || in-memory, circuit breaker, escalating blocks)

# Standalone scripts: npm run db:migrate / npm run db:seed / npm run db:setup (config loads dotenv automatically)

Request Flow

HTTP Request β†’ CORS headers β†’ Rate limiter β†’ Route parser β†’ Handler β†’ JSON Response

Implementation Notes

  • Minimal runtime dependencies β€” only argon2 for admin password hashing; everything else uses Node.js built-in modules (http, url, fs, path, net, node:sqlite). dotenv is a dev dependency.
  • Pure RESP protocol β€” the Redis client in src/redis/index.js implements the Redis serialization protocol over raw TCP sockets without any third-party library. Supports AUTH password authentication and REDIS_URL connection strings.
  • Centralized config β€” all environment variables are read in src/config/index.js and exported as camelCase (port, dbPath, redisOpts, rateLimitMax, dbDebugSql, sensitiveKeys, etc.) for use across the codebase.
  • Rate limiter β€” src/middleware/rate-limiter.js features a circuit breaker for Redis failures, CIDR-based trusted proxy IP extraction, escalating block durations (5m β†’ 20m β†’ 1h), atomic Redis Lua scripts, and an in-memory fallback with LRU eviction. createRateLimiter() accepts options directly instead of reading config lazily.
  • Runtime configuration β€” Patching rate-limit or Redis settings via the admin API immediately applies changes through RuntimeConfig (in-memory overrides), avoiding server restarts. Rate-limit hot-swapping uses rateLimiter.updateConfig(); Redis reconnection uses Redis.reconnect().
  • Testable seed script β€” src/db/seed.js accepts database and fetch parameters via dependency injection, enabling full unit testing without mocking require().
  • SQL query logging β€” src/db/sql-logger.js exports wrapDb/wrapStmt Proxy wrappers that log exec, prepare, run, get, and all calls to stderr. src/db/index.js uses them via getWrappedDb() when DEBUG_SQL=true.
  • Multi-environment β€” src/config/index.js requires src/config/load-env.js at module level, which chain-loads all dotenv files in priority order with override: false β€” existing process.env values and earlier files take precedence over later ones. Every consumer (server, migrate, seed) simply requires src/config/index.js and gets correct env values. In production, dotenv is skipped entirely β€” env vars must come from the deployment environment.
  • SQL injection prevention β€” src/db/index.js validates _sort against a whitelist of known columns and quotes identifiers with ""; LIKE wildcards (%, _) are escaped to prevent injection through the q parameter
  • Argon2 auth cache β€” src/server/route.js caches ADMIN_KEY verification results in a Map with 5-second TTL and 1,000-entry limit, avoiding repeated argon2 hashing on burst admin requests
  • CORS enabled on all routes
  • Graceful shutdown β€” handles SIGINT and SIGTERM to close the server and Redis connection cleanly