docs/BACKEND.md

Backend Design Document

Habit Journal (Discrete PWA) — Hugging Face Spaces

Version: 2.0.0
Scope: Backend API, storage, auth, server-side statistics, coach, traces, debugging, export, and operations. The frontend is separate.

This is the repository specification. If implementation and this document conflict, this document wins until a human changes it.

Implementation rules

• Work in the vertical slices listed below and keep each diff small.
• Prefer stdlib, FastAPI, httpx, Pydantic, and pydantic-settings.
• Use no SQL database, Redis, Celery, DI framework, or repository hierarchy.
• Every module has a short purpose docstring and public functions are typed.
• Request and response bodies use Pydantic models.
• Secrets come only from environment variables. Never log passwords or keys.
• Keep generic Habit Journal naming and private biography out of git.
• Disable OpenAPI, Swagger, and ReDoc when ENV=prod.
• After each slice, the app must import and start with uvicorn.
• Target at most about 15 Python files under app/; merge stores if needed.

Runtime contract

On a fresh Hugging Face Docker Space with APP_PASSWORD and APP_SECRET_KEY, startup creates /data/loop_logger, login works, entries and daily rows survive restart, statistics are computed in Python, coach always returns model or backup text, and every debug paste bundle is understandable without prior user context.

The Docker image uses Python 3.11 slim and runs:

uvicorn app.main:app --host 0.0.0.0 --port ${PORT:-7860}

Required and optional environment:

• APP_PASSWORD: bootstrap password when config.json does not exist
• APP_SECRET_KEY: required session-signing secret
• DATA_ROOT=/data/loop_logger
• ENV=prod; APP_NAME=Habit Journal
• OPENROUTER_API_KEY; OPENROUTER_MODEL=openrouter/free
• OPENROUTER_BASE_URL=https://openrouter.ai/api/v1
• COACH_TIMEOUT_SEC=25; COACH_TEMPERATURE=0.3
• COACH_MAX_TOKENS=400; COACH_HISTORY_K=10
• COACH_TRACE_LIMIT=200
• MIN_STATS_N=5; STATS_SHRINK_K=3; MATCH_ALPHA=0.5
• SESSION_MAX_AGE_SEC=604800
• DEBUG_INCLUDE_FULL_BRIEF=false
• LOGIN_RATE_LIMIT=10; LOGIN_RATE_WINDOW_SEC=900

Flat architecture and durable files

Routers call services, services call stores, and stores access the filesystem. Do not introduce circular imports. main.py owns wiring.

DATA_ROOT contains config.json, entries.jsonl, daily.jsonl, coach_brief.md, and traces.jsonl. Each mutated path uses a lock. Entry creation appends one flushed JSONL line under lock. Updates and deletes read, map/filter, then write a flushed temporary file and os.replace it.

If configuration is absent and APP_PASSWORD exists, create:

{
  "password_hash": "pbkdf2_sha256$260000$...",
  "session_version": 1,
  "created_at": "ISO-8601",
  "updated_at": "ISO-8601"
}

PBKDF2 uses SHA-256, at least 200,000 iterations, a random salt, and hmac.compare_digest. Without config and bootstrap password, readiness is false and protected data routes return setup_required.

coach_brief.md is operator-editable, read to at most 32 KiB, and seeded with generic policy only. Debug includes its hash and first 500 characters unless DEBUG_INCLUDE_FULL_BRIEF=true.

API envelope and security

Every success is {"ok":true,"data":...,"error":null}. Every failure is {"ok":false,"data":null,"error":{"code":"...","message":"..."}}. Codes are unauthorized, forbidden, validation_error, not_found, setup_required, rate_limited, and internal.

Use Starlette signed cookie sessions containing auth, sv, and Unix exp. Reject expired sessions and sessions whose version differs from config. Cookies are HTTP-only, SameSite Lax, and secure in production. Rate-limit login attempts by IP in memory and return generic credential failures. Limit request bodies to 64 KiB. Do not mount DATA_ROOT. Best-effort strip emails and long digit runs from coach prompts. Never return stack traces.

Unauthenticated routes are only GET /api/health, GET /api/ready, POST /api/auth/login, and GET /api/auth/me.

Auth routes:

• POST /api/auth/login with {password}
• POST /api/auth/logout
• GET /api/auth/me
• POST /api/auth/change-password with old/new password; bump session version

Entries and daily rows

Entry fields are id, ts, timestamps, activity, happened, emotions, intensity, remedy, result, tags, notes, and nested coach metadata. Activity is 1–500 characters, happened 1–4000, emotions at most 12, intensity 1–10, remedy and notes at most 2000, and tags at most 20. Normalize emotions to lowercase stripped text; normalize tags likewise and replace spaces with underscores. Results are worked, partial, failed, or pending; pending never enters probability denominators.

Entries support create, list/filter, get, patch, delete, and per-entry coach. List filters are start/end date, tag, result, emotion, substring q, limit (50 default, 200 max), and offset; sort newest first.

Daily rows are unique by date and contain brick, brick completion, corn sessions, delay flag, daydream, rerun, court, points, note, and update time. Enums are brick A|B|C|D|E|S|none, daydream none|done|fc, rerun clean|R, and court closed|court. Corn sessions range from 0 to 50.

Server-authoritative daily points:

2 if brick_done
+1 if corn_sessions == 0 or (corn_sessions <= 1 and delay_ok)
+1 if daydream != "fc"
+1 if rerun == "clean"
+1 if court == "closed"

For a seven-calendar-day window, missing days produce incomplete; otherwise 28–42 is strong, 18–27 mixed, and 0–17 escape_heavy.

Server statistics and evidence

stats_math.py contains pure functions only. Excluding pending outcomes:

• p_worked(r) = N(worked,r) / N(r)
• p_helped(r) = N(worked|partial,r) / N(r)
• p_failed(t) = N(failed,t) / N(t)
• rank(r) = p_helped(r) * n/(n+k)
• match(r,T) = |T intersect helped_tags(r)| / max(|T|,1)
• pick(r) = rank(r) * (1 + alpha*match(r,T))

Intensity buckets are 1–3, 4–6, and 7–10. Emotions are multi-label. Top coach tables hide remedies below min_n. Server picks are the top five by pick score. DATA_THIN means fewer than ten scored entries.

Daily statistics include brick done, corn okay, no FC, clean rerun, closed court, and average points rates. /api/stats returns counts, outcomes, breakdowns, daily rates, formula strings, and generation time. /api/stats/remedies returns rank-sorted remedies. The model never computes statistics.

Coach

Coach is an optional OpenRouter selector/copywriter over server evidence and server picks. A hardcoded ordered rule table is mandatory and always available. A valid coach request always returns HTTP 200 with non-empty text and source model, backup, or model_unparsed_fallback.

Pipeline: resolve current input; load last K truncated entries and today's daily row; compute evidence and picks; build system/user messages; start a trace; call OpenRouter; strictly parse the template; fall back on empty, failed, timed-out, or unparseable output; set flags; append/ring-trim trace; optionally persist coach metadata on the entry.

OpenRouter receives POST {BASE}/chat/completions, Bearer authorization, JSON content type, X-Title: Habit Journal, configured model, messages, temperature, token limit, and timeout.

Required model output:

LOOP: <tag>
FEELING: <word>
INTENSITY_GUESS: <1-10|n/a>
REMEDY: <one concrete action>
NEXT_BRICK: <A|B|C|D|E|S|specific>
DO_NOT: <one thing>
LINE: <12 words or fewer>
NOTE: <optional 20 words or fewer>
DATA_THIN: <true|false>

The prompt forbids invented numbers, canceling committed processes merely from fear, pity, and pep talks. Self-harm language triggers crisis redirection. Rules are ordered: crisis, court, rerun, rage_movie, daydream/fc, urge/corn, bully, shame, spain/admin, build/earn, home, loneliness, default. Non-crisis rules may substitute server pick number one.

Flags include DATA_THIN, PARSE_FAIL, TIMEOUT, EMPTY_MODEL, NO_API_KEY, and OUT_OF_EVIDENCE.

Traces, debug, export, and operations

Every coach call stores request, current situation, truncated history, evidence, picks, brief hash/excerpt/full value according to config, both prompts, model settings, latency/status/error, raw output, parse, source, backup rule, flags, app version, and final text. Keep the newest configured N.

Authenticated debug endpoints list traces, return full traces, return one or the newest context-free Markdown paste bundle, preview prompts without a model call, and test coach with optional forced backup. The paste bundle contains: how an external AI should help, authoritative formulas, evidence, picks, current input, history, prompts, raw response, parse/source/flags, final text, and a reviewer checklist for invented numbers, ignored picks, format, fallback quality, evidence length, and weight tuning.

Authenticated exports provide entries JSONL, entries CSV, and daily JSONL. CSV has a fixed entry header and pipe-joins emotions and tags.

Health is public. Readiness reports writable data plus config presence. Authenticated settings status reports app name, coach configured state, model, data status, environment, and app version—never an API key.

Mount ./static at / only when present, after /api routes. Never mount the data directory.

Default generic policy

• A: admin/critical checklist item
• B: environment/hygiene one act
• C: build/earn one ship
• D: logistics one line
• E: boundary one line then silence
• S: stop spiral — water/shower/sleep

Court is closed by default. Daydream requires a same-day brick. Urge policy is delay then brick, with a ceiling of one session per day expressed only in policy text.

Ordered implementation slices

1. S0: Dockerfile, requirements, settings, paths, fs utilities, health/ready
2. S1: password hashing, config store, auth, session, dependencies
3. S2: entries CRUD JSONL
4. S3: daily upsert, points, range band
5. S4: pure statistics and stats routes
6. S5: evidence and server picks
7. S6: rule backup and offline coach
8. S7: OpenRouter, parser, flags
9. S8: traces and debug paste endpoints
10. S9: export and settings status
11. S10: seed brief, production checks, error polish

Do not begin S7 before S6. Tests must cover daily point edge cases, hand-calculated helped/rank math, and backup rule precedence; an authenticated temporary-data API test is optional.

Acceptance

Storage survives restart; good login sets a cookie and bad login is 401; unauthenticated data access is 401; daily and probability fixtures match; coach without a key and failed model calls return backup text; evidence and debug paste contain probabilities and server picks; paste contains formulas; authenticated export works; production OpenAPI is absent; and no secret is committed.

Explicit decisions are final: JSONL only, server-only math, probabilities and picks passed to the model, hardcoded backup, context-free debug bundles, single shared password, entry result as truth, and private life details only in the operator brief on disk.