docs/BACKEND.md

# Backend Design Document
## Habit Journal (Discrete PWA) — Hugging Face Spaces

**Version:** 2.0.0  
**Scope:** Backend API, storage, auth, server-side statistics, coach, traces,
debugging, export, and operations. The frontend is separate.

This is the repository specification. If implementation and this document
conflict, this document wins until a human changes it.

## Implementation rules

- Work in the vertical slices listed below and keep each diff small.
- Prefer stdlib, FastAPI, httpx, Pydantic, and pydantic-settings.
- Use no SQL database, Redis, Celery, DI framework, or repository hierarchy.
- Every module has a short purpose docstring and public functions are typed.
- Request and response bodies use Pydantic models.
- Secrets come only from environment variables. Never log passwords or keys.
- Keep generic Habit Journal naming and private biography out of git.
- Disable OpenAPI, Swagger, and ReDoc when `ENV=prod`.
- After each slice, the app must import and start with uvicorn.
- Target at most about 15 Python files under `app/`; merge stores if needed.

## Runtime contract

On a fresh Hugging Face Docker Space with `APP_PASSWORD` and
`APP_SECRET_KEY`, startup creates `/data/loop_logger`, login works, entries and
daily rows survive restart, statistics are computed in Python, coach always
returns model or backup text, and every debug paste bundle is understandable
without prior user context.

The Docker image uses Python 3.11 slim and runs:

```text
uvicorn app.main:app --host 0.0.0.0 --port ${PORT:-7860}
```

Required and optional environment:

- `APP_PASSWORD`: bootstrap password when `config.json` does not exist
- `APP_SECRET_KEY`: required session-signing secret
- `DATA_ROOT=/data/loop_logger`
- `ENV=prod`; `APP_NAME=Habit Journal`
- `OPENROUTER_API_KEY`; `OPENROUTER_MODEL=openrouter/free`
- `OPENROUTER_BASE_URL=https://openrouter.ai/api/v1`
- `COACH_TIMEOUT_SEC=25`; `COACH_TEMPERATURE=0.3`
- `COACH_MAX_TOKENS=400`; `COACH_HISTORY_K=10`
- `COACH_TRACE_LIMIT=200`
- `MIN_STATS_N=5`; `STATS_SHRINK_K=3`; `MATCH_ALPHA=0.5`
- `SESSION_MAX_AGE_SEC=604800`
- `DEBUG_INCLUDE_FULL_BRIEF=false`
- `LOGIN_RATE_LIMIT=10`; `LOGIN_RATE_WINDOW_SEC=900`

## Flat architecture and durable files

Routers call services, services call stores, and stores access the filesystem.
Do not introduce circular imports. `main.py` owns wiring.

`DATA_ROOT` contains `config.json`, `entries.jsonl`, `daily.jsonl`,
`coach_brief.md`, and `traces.jsonl`. Each mutated path uses a lock. Entry
creation appends one flushed JSONL line under lock. Updates and deletes read,
map/filter, then write a flushed temporary file and `os.replace` it.

If configuration is absent and `APP_PASSWORD` exists, create:

```json
{
  "password_hash": "pbkdf2_sha256$260000$...",
  "session_version": 1,
  "created_at": "ISO-8601",
  "updated_at": "ISO-8601"
}
```

PBKDF2 uses SHA-256, at least 200,000 iterations, a random salt, and
`hmac.compare_digest`. Without config and bootstrap password, readiness is
false and protected data routes return `setup_required`.

`coach_brief.md` is operator-editable, read to at most 32 KiB, and seeded with
generic policy only. Debug includes its hash and first 500 characters unless
`DEBUG_INCLUDE_FULL_BRIEF=true`.

## API envelope and security

Every success is `{"ok":true,"data":...,"error":null}`. Every failure is
`{"ok":false,"data":null,"error":{"code":"...","message":"..."}}`.
Codes are `unauthorized`, `forbidden`, `validation_error`, `not_found`,
`setup_required`, `rate_limited`, and `internal`.

Use Starlette signed cookie sessions containing `auth`, `sv`, and Unix `exp`.
Reject expired sessions and sessions whose version differs from config.
Cookies are HTTP-only, SameSite Lax, and secure in production. Rate-limit
login attempts by IP in memory and return generic credential failures.
Limit request bodies to 64 KiB. Do not mount `DATA_ROOT`. Best-effort strip
emails and long digit runs from coach prompts. Never return stack traces.

Unauthenticated routes are only `GET /api/health`, `GET /api/ready`,
`POST /api/auth/login`, and `GET /api/auth/me`.

Auth routes:

- `POST /api/auth/login` with `{password}`
- `POST /api/auth/logout`
- `GET /api/auth/me`
- `POST /api/auth/change-password` with old/new password; bump session version

## Entries and daily rows

Entry fields are `id`, `ts`, timestamps, `activity`, `happened`, `emotions`,
`intensity`, `remedy`, `result`, `tags`, `notes`, and nested coach metadata.
Activity is 1–500 characters, happened 1–4000, emotions at most 12,
intensity 1–10, remedy and notes at most 2000, and tags at most 20. Normalize
emotions to lowercase stripped text; normalize tags likewise and replace
spaces with underscores. Results are `worked`, `partial`, `failed`, or
`pending`; pending never enters probability denominators.

Entries support create, list/filter, get, patch, delete, and per-entry coach.
List filters are start/end date, tag, result, emotion, substring `q`, limit
(50 default, 200 max), and offset; sort newest first.

Daily rows are unique by date and contain brick, brick completion, corn
sessions, delay flag, daydream, rerun, court, points, note, and update time.
Enums are brick `A|B|C|D|E|S|none`, daydream `none|done|fc`, rerun `clean|R`,
and court `closed|court`. Corn sessions range from 0 to 50.

Server-authoritative daily points:

```text
2 if brick_done
+1 if corn_sessions == 0 or (corn_sessions <= 1 and delay_ok)
+1 if daydream != "fc"
+1 if rerun == "clean"
+1 if court == "closed"
```

For a seven-calendar-day window, missing days produce `incomplete`; otherwise
28–42 is `strong`, 18–27 `mixed`, and 0–17 `escape_heavy`.

## Server statistics and evidence

`stats_math.py` contains pure functions only. Excluding pending outcomes:

- `p_worked(r) = N(worked,r) / N(r)`
- `p_helped(r) = N(worked|partial,r) / N(r)`
- `p_failed(t) = N(failed,t) / N(t)`
- `rank(r) = p_helped(r) * n/(n+k)`
- `match(r,T) = |T intersect helped_tags(r)| / max(|T|,1)`
- `pick(r) = rank(r) * (1 + alpha*match(r,T))`

Intensity buckets are 1–3, 4–6, and 7–10. Emotions are multi-label.
Top coach tables hide remedies below `min_n`. Server picks are the top five by
pick score. `DATA_THIN` means fewer than ten scored entries.

Daily statistics include brick done, corn okay, no FC, clean rerun, closed
court, and average points rates. `/api/stats` returns counts, outcomes,
breakdowns, daily rates, formula strings, and generation time.
`/api/stats/remedies` returns rank-sorted remedies. The model never computes
statistics.

## Coach

Coach is an optional OpenRouter selector/copywriter over server evidence and
server picks. A hardcoded ordered rule table is mandatory and always available.
A valid coach request always returns HTTP 200 with non-empty text and source
`model`, `backup`, or `model_unparsed_fallback`.

Pipeline: resolve current input; load last K truncated entries and today's
daily row; compute evidence and picks; build system/user messages; start a
trace; call OpenRouter; strictly parse the template; fall back on empty,
failed, timed-out, or unparseable output; set flags; append/ring-trim trace;
optionally persist coach metadata on the entry.

OpenRouter receives `POST {BASE}/chat/completions`, Bearer authorization,
JSON content type, `X-Title: Habit Journal`, configured model, messages,
temperature, token limit, and timeout.

Required model output:

```text
LOOP: <tag>
FEELING: <word>
INTENSITY_GUESS: <1-10|n/a>
REMEDY: <one concrete action>
NEXT_BRICK: <A|B|C|D|E|S|specific>
DO_NOT: <one thing>
LINE: <12 words or fewer>
NOTE: <optional 20 words or fewer>
DATA_THIN: <true|false>
```

The prompt forbids invented numbers, canceling committed processes merely from
fear, pity, and pep talks. Self-harm language triggers crisis redirection.
Rules are ordered: crisis, court, rerun, rage_movie, daydream/fc, urge/corn,
bully, shame, spain/admin, build/earn, home, loneliness, default. Non-crisis
rules may substitute server pick number one.

Flags include `DATA_THIN`, `PARSE_FAIL`, `TIMEOUT`, `EMPTY_MODEL`,
`NO_API_KEY`, and `OUT_OF_EVIDENCE`.

## Traces, debug, export, and operations

Every coach call stores request, current situation, truncated history,
evidence, picks, brief hash/excerpt/full value according to config, both
prompts, model settings, latency/status/error, raw output, parse, source,
backup rule, flags, app version, and final text. Keep the newest configured N.

Authenticated debug endpoints list traces, return full traces, return one or
the newest context-free Markdown paste bundle, preview prompts without a model
call, and test coach with optional forced backup. The paste bundle contains:
how an external AI should help, authoritative formulas, evidence, picks,
current input, history, prompts, raw response, parse/source/flags, final text,
and a reviewer checklist for invented numbers, ignored picks, format,
fallback quality, evidence length, and weight tuning.

Authenticated exports provide entries JSONL, entries CSV, and daily JSONL.
CSV has a fixed entry header and pipe-joins emotions and tags.

Health is public. Readiness reports writable data plus config presence.
Authenticated settings status reports app name, coach configured state, model,
data status, environment, and app version—never an API key.

Mount `./static` at `/` only when present, after `/api` routes. Never mount the
data directory.

## Default generic policy

- A: admin/critical checklist item
- B: environment/hygiene one act
- C: build/earn one ship
- D: logistics one line
- E: boundary one line then silence
- S: stop spiral — water/shower/sleep

Court is closed by default. Daydream requires a same-day brick. Urge policy is
delay then brick, with a ceiling of one session per day expressed only in
policy text.

## Ordered implementation slices

1. S0: Dockerfile, requirements, settings, paths, fs utilities, health/ready
2. S1: password hashing, config store, auth, session, dependencies
3. S2: entries CRUD JSONL
4. S3: daily upsert, points, range band
5. S4: pure statistics and stats routes
6. S5: evidence and server picks
7. S6: rule backup and offline coach
8. S7: OpenRouter, parser, flags
9. S8: traces and debug paste endpoints
10. S9: export and settings status
11. S10: seed brief, production checks, error polish

Do not begin S7 before S6. Tests must cover daily point edge cases,
hand-calculated helped/rank math, and backup rule precedence; an authenticated
temporary-data API test is optional.

## Acceptance

Storage survives restart; good login sets a cookie and bad login is 401;
unauthenticated data access is 401; daily and probability fixtures match;
coach without a key and failed model calls return backup text; evidence and
debug paste contain probabilities and server picks; paste contains formulas;
authenticated export works; production OpenAPI is absent; and no secret is
committed.

Explicit decisions are final: JSONL only, server-only math, probabilities and
picks passed to the model, hardcoded backup, context-free debug bundles,
single shared password, entry result as truth, and private life details only
in the operator brief on disk.