# Backend Design Document ## Habit Journal (Discrete PWA) — Hugging Face Spaces **Version:** 2.0.0 **Scope:** Backend API, storage, auth, server-side statistics, coach, traces, debugging, export, and operations. The frontend is separate. This is the repository specification. If implementation and this document conflict, this document wins until a human changes it. ## Implementation rules - Work in the vertical slices listed below and keep each diff small. - Prefer stdlib, FastAPI, httpx, Pydantic, and pydantic-settings. - Use no SQL database, Redis, Celery, DI framework, or repository hierarchy. - Every module has a short purpose docstring and public functions are typed. - Request and response bodies use Pydantic models. - Secrets come only from environment variables. Never log passwords or keys. - Keep generic Habit Journal naming and private biography out of git. - Disable OpenAPI, Swagger, and ReDoc when `ENV=prod`. - After each slice, the app must import and start with uvicorn. - Target at most about 15 Python files under `app/`; merge stores if needed. ## Runtime contract On a fresh Hugging Face Docker Space with `APP_PASSWORD` and `APP_SECRET_KEY`, startup creates `/data/loop_logger`, login works, entries and daily rows survive restart, statistics are computed in Python, coach always returns model or backup text, and every debug paste bundle is understandable without prior user context. The Docker image uses Python 3.11 slim and runs: ```text uvicorn app.main:app --host 0.0.0.0 --port ${PORT:-7860} ``` Required and optional environment: - `APP_PASSWORD`: bootstrap password when `config.json` does not exist - `APP_SECRET_KEY`: required session-signing secret - `DATA_ROOT=/data/loop_logger` - `ENV=prod`; `APP_NAME=Habit Journal` - `OPENROUTER_API_KEY`; `OPENROUTER_MODEL=openrouter/free` - `OPENROUTER_BASE_URL=https://openrouter.ai/api/v1` - `COACH_TIMEOUT_SEC=25`; `COACH_TEMPERATURE=0.3` - `COACH_MAX_TOKENS=400`; `COACH_HISTORY_K=10` - `COACH_TRACE_LIMIT=200` - `MIN_STATS_N=5`; `STATS_SHRINK_K=3`; `MATCH_ALPHA=0.5` - `SESSION_MAX_AGE_SEC=604800` - `DEBUG_INCLUDE_FULL_BRIEF=false` - `LOGIN_RATE_LIMIT=10`; `LOGIN_RATE_WINDOW_SEC=900` ## Flat architecture and durable files Routers call services, services call stores, and stores access the filesystem. Do not introduce circular imports. `main.py` owns wiring. `DATA_ROOT` contains `config.json`, `entries.jsonl`, `daily.jsonl`, `coach_brief.md`, and `traces.jsonl`. Each mutated path uses a lock. Entry creation appends one flushed JSONL line under lock. Updates and deletes read, map/filter, then write a flushed temporary file and `os.replace` it. If configuration is absent and `APP_PASSWORD` exists, create: ```json { "password_hash": "pbkdf2_sha256$260000$...", "session_version": 1, "created_at": "ISO-8601", "updated_at": "ISO-8601" } ``` PBKDF2 uses SHA-256, at least 200,000 iterations, a random salt, and `hmac.compare_digest`. Without config and bootstrap password, readiness is false and protected data routes return `setup_required`. `coach_brief.md` is operator-editable, read to at most 32 KiB, and seeded with generic policy only. Debug includes its hash and first 500 characters unless `DEBUG_INCLUDE_FULL_BRIEF=true`. ## API envelope and security Every success is `{"ok":true,"data":...,"error":null}`. Every failure is `{"ok":false,"data":null,"error":{"code":"...","message":"..."}}`. Codes are `unauthorized`, `forbidden`, `validation_error`, `not_found`, `setup_required`, `rate_limited`, and `internal`. Use Starlette signed cookie sessions containing `auth`, `sv`, and Unix `exp`. Reject expired sessions and sessions whose version differs from config. Cookies are HTTP-only, SameSite Lax, and secure in production. Rate-limit login attempts by IP in memory and return generic credential failures. Limit request bodies to 64 KiB. Do not mount `DATA_ROOT`. Best-effort strip emails and long digit runs from coach prompts. Never return stack traces. Unauthenticated routes are only `GET /api/health`, `GET /api/ready`, `POST /api/auth/login`, and `GET /api/auth/me`. Auth routes: - `POST /api/auth/login` with `{password}` - `POST /api/auth/logout` - `GET /api/auth/me` - `POST /api/auth/change-password` with old/new password; bump session version ## Entries and daily rows Entry fields are `id`, `ts`, timestamps, `activity`, `happened`, `emotions`, `intensity`, `remedy`, `result`, `tags`, `notes`, and nested coach metadata. Activity is 1–500 characters, happened 1–4000, emotions at most 12, intensity 1–10, remedy and notes at most 2000, and tags at most 20. Normalize emotions to lowercase stripped text; normalize tags likewise and replace spaces with underscores. Results are `worked`, `partial`, `failed`, or `pending`; pending never enters probability denominators. Entries support create, list/filter, get, patch, delete, and per-entry coach. List filters are start/end date, tag, result, emotion, substring `q`, limit (50 default, 200 max), and offset; sort newest first. Daily rows are unique by date and contain brick, brick completion, corn sessions, delay flag, daydream, rerun, court, points, note, and update time. Enums are brick `A|B|C|D|E|S|none`, daydream `none|done|fc`, rerun `clean|R`, and court `closed|court`. Corn sessions range from 0 to 50. Server-authoritative daily points: ```text 2 if brick_done +1 if corn_sessions == 0 or (corn_sessions <= 1 and delay_ok) +1 if daydream != "fc" +1 if rerun == "clean" +1 if court == "closed" ``` For a seven-calendar-day window, missing days produce `incomplete`; otherwise 28–42 is `strong`, 18–27 `mixed`, and 0–17 `escape_heavy`. ## Server statistics and evidence `stats_math.py` contains pure functions only. Excluding pending outcomes: - `p_worked(r) = N(worked,r) / N(r)` - `p_helped(r) = N(worked|partial,r) / N(r)` - `p_failed(t) = N(failed,t) / N(t)` - `rank(r) = p_helped(r) * n/(n+k)` - `match(r,T) = |T intersect helped_tags(r)| / max(|T|,1)` - `pick(r) = rank(r) * (1 + alpha*match(r,T))` Intensity buckets are 1–3, 4–6, and 7–10. Emotions are multi-label. Top coach tables hide remedies below `min_n`. Server picks are the top five by pick score. `DATA_THIN` means fewer than ten scored entries. Daily statistics include brick done, corn okay, no FC, clean rerun, closed court, and average points rates. `/api/stats` returns counts, outcomes, breakdowns, daily rates, formula strings, and generation time. `/api/stats/remedies` returns rank-sorted remedies. The model never computes statistics. ## Coach Coach is an optional OpenRouter selector/copywriter over server evidence and server picks. A hardcoded ordered rule table is mandatory and always available. A valid coach request always returns HTTP 200 with non-empty text and source `model`, `backup`, or `model_unparsed_fallback`. Pipeline: resolve current input; load last K truncated entries and today's daily row; compute evidence and picks; build system/user messages; start a trace; call OpenRouter; strictly parse the template; fall back on empty, failed, timed-out, or unparseable output; set flags; append/ring-trim trace; optionally persist coach metadata on the entry. OpenRouter receives `POST {BASE}/chat/completions`, Bearer authorization, JSON content type, `X-Title: Habit Journal`, configured model, messages, temperature, token limit, and timeout. Required model output: ```text LOOP: FEELING: INTENSITY_GUESS: <1-10|n/a> REMEDY: NEXT_BRICK: DO_NOT: LINE: <12 words or fewer> NOTE: DATA_THIN: ``` The prompt forbids invented numbers, canceling committed processes merely from fear, pity, and pep talks. Self-harm language triggers crisis redirection. Rules are ordered: crisis, court, rerun, rage_movie, daydream/fc, urge/corn, bully, shame, spain/admin, build/earn, home, loneliness, default. Non-crisis rules may substitute server pick number one. Flags include `DATA_THIN`, `PARSE_FAIL`, `TIMEOUT`, `EMPTY_MODEL`, `NO_API_KEY`, and `OUT_OF_EVIDENCE`. ## Traces, debug, export, and operations Every coach call stores request, current situation, truncated history, evidence, picks, brief hash/excerpt/full value according to config, both prompts, model settings, latency/status/error, raw output, parse, source, backup rule, flags, app version, and final text. Keep the newest configured N. Authenticated debug endpoints list traces, return full traces, return one or the newest context-free Markdown paste bundle, preview prompts without a model call, and test coach with optional forced backup. The paste bundle contains: how an external AI should help, authoritative formulas, evidence, picks, current input, history, prompts, raw response, parse/source/flags, final text, and a reviewer checklist for invented numbers, ignored picks, format, fallback quality, evidence length, and weight tuning. Authenticated exports provide entries JSONL, entries CSV, and daily JSONL. CSV has a fixed entry header and pipe-joins emotions and tags. Health is public. Readiness reports writable data plus config presence. Authenticated settings status reports app name, coach configured state, model, data status, environment, and app version—never an API key. Mount `./static` at `/` only when present, after `/api` routes. Never mount the data directory. ## Default generic policy - A: admin/critical checklist item - B: environment/hygiene one act - C: build/earn one ship - D: logistics one line - E: boundary one line then silence - S: stop spiral — water/shower/sleep Court is closed by default. Daydream requires a same-day brick. Urge policy is delay then brick, with a ceiling of one session per day expressed only in policy text. ## Ordered implementation slices 1. S0: Dockerfile, requirements, settings, paths, fs utilities, health/ready 2. S1: password hashing, config store, auth, session, dependencies 3. S2: entries CRUD JSONL 4. S3: daily upsert, points, range band 5. S4: pure statistics and stats routes 6. S5: evidence and server picks 7. S6: rule backup and offline coach 8. S7: OpenRouter, parser, flags 9. S8: traces and debug paste endpoints 10. S9: export and settings status 11. S10: seed brief, production checks, error polish Do not begin S7 before S6. Tests must cover daily point edge cases, hand-calculated helped/rank math, and backup rule precedence; an authenticated temporary-data API test is optional. ## Acceptance Storage survives restart; good login sets a cookie and bad login is 401; unauthenticated data access is 401; daily and probability fixtures match; coach without a key and failed model calls return backup text; evidence and debug paste contain probabilities and server picks; paste contains formulas; authenticated export works; production OpenAPI is absent; and no secret is committed. Explicit decisions are final: JSONL only, server-only math, probabilities and picks passed to the model, hardcoded backup, context-free debug bundles, single shared password, entry result as truth, and private life details only in the operator brief on disk.