Add files via upload

docs/ARCHITECTURE.md

@@ -7,73 +7,73 @@ This document goes one level deeper than the main [README](../README.md) into ho
 1. Telegram POSTs an update to `/webhook`, authenticated by a secret header (`X-Telegram-Bot-Api-Secret-Token`).
 2. `_handle_message_core` in `bot.py` classifies the message: is it a TikTok link, a `/draw` or `/tts` trigger phrase, a reply to earlier media, plain text, or an attachment?
 3. `_build_route` (`lumen_router_config.py`) turns that classification into an ordered list of `(provider, model_id)` candidates.
-4. `_run_route` tries the first candidate. If a whole provider's chain fails, it falls back to the other provider — except where that's physically impossible (only Gemini can read links or analyze video/audio).
+4. `_run_route` tries the first candidate. If a whole provider's chain fails, it falls back to the other provider, except where that's physically impossible (only Gemini can read links or analyze video/audio).
 5. The reply streams into the chat if the request qualifies (see [Streaming](#streaming--typing-pace) below), or is sent as a single message otherwise.
 
 ## Automatic model routing
 
-There is no `/model` or `/provider` command — the router builds a fresh candidate list for every message:
+There is no `/model` or `/provider` command. The router builds a fresh candidate list for every message:
 
 - **A YouTube or website link** → Gemini only. It's the only provider that can read page content (`url_context`) or analyze a video by URL.
 - **A video or audio attachment** → Gemini only. OpenRouter's multimodal models only accept images as base64.
 - **An image attachment, no live-info need** → free OpenRouter vision models first, Gemini as a reserve.
-- **Needs current information** (a lightweight keyword heuristic — "now," "today," "price," "who is currently...") → Gemini, prioritizing the models with a real search-grounding quota, then OpenRouter as a reserve.
-- **Plain text, no attachments, no freshness need** (the most common case) → straight to OpenRouter. Heavier requests (code, multi-step reasoning — another lightweight heuristic) get routed to the stronger free OpenRouter models first.
+- **Needs current information** (a lightweight keyword heuristic: "now," "today," "price," "who is currently...") → Gemini, prioritizing the models with a real search-grounding quota, then OpenRouter as a reserve.
+- **Plain text, no attachments, no freshness need** (the most common case) → straight to OpenRouter. Heavier requests (code, multi-step reasoning, caught by another lightweight heuristic) get routed to the stronger free OpenRouter models first.
 
-The reasoning: Gemini's free quota (roughly 20 requests/day for the flagship model) is the scarcest resource in the system, so it's only spent where a capability unique to Gemini is actually needed. Everything else — the bulk of ordinary messages — runs on OpenRouter's much more generous free tier.
+The reasoning: Gemini's free quota (roughly 20 requests per day for the flagship model) is the scarcest resource in the system, so it's only spent where a capability unique to Gemini is actually needed. Everything else, the bulk of ordinary messages, runs on OpenRouter's much more generous free tier.
 
-Models that the router should never pick are tracked in a single registry, `_OR_MODEL_HEALTH` (`lumen_router_config.py`), each with a dated reason (the provider dropped the free tier, or the model turned out to be uncensored and a poor fit for Lumen's persona). Nothing is removed on speculation — only on confirmed evidence from production logs (`HTTP 404`, `"no endpoints,"` etc.).
+Models that the router should never pick are tracked in a single registry, `_OR_MODEL_HEALTH` (`lumen_router_config.py`), each with a dated reason: the provider dropped the free tier, or the model turned out to be uncensored and a poor fit for Lumen's persona. Nothing is removed on speculation, only on confirmed evidence from production logs (`HTTP 404`, `"no endpoints,"` etc.).
 
-Each model is tried exactly once per route, with no retries — a single failure (timeout, `429`, `5xx`) moves straight to the next candidate. That keeps the worst case bounded by `len(route) × ROUTE_MODEL_TIMEOUT_SEC`, further capped by `ROUTE_TOTAL_BUDGET_SEC` for the whole route.
+Each model is tried exactly once per route, with no retries: a single failure (timeout, `429`, `5xx`) moves straight to the next candidate. That keeps the worst case bounded by `len(route) × ROUTE_MODEL_TIMEOUT_SEC`, further capped by `ROUTE_TOTAL_BUDGET_SEC` for the whole route.
 
 ### Image generation
 
-`/draw` follows the same philosophy on a smaller scale: `_pick_image_model` (`lumen_images.py`) picks a Pollinations.ai model from the prompt's content (anime/manga → `flux-anime`, fantasy → `dreamshaper`, "photorealistic" → `flux-realism`, "quick sketch" → `turbo`, otherwise a general-purpose default) and falls back through the rest of the catalog if the chosen model fails, all within `DRAW_TOTAL_BUDGET_SEC`.
+`/draw` follows the same philosophy on a smaller scale. `_pick_image_model` (`lumen_images.py`) picks a Pollinations.ai model from the prompt's content (anime/manga → `flux-anime`, fantasy → `dreamshaper`, "photorealistic" → `flux-realism`, "quick sketch" → `turbo`, otherwise a general-purpose default) and falls back through the rest of the catalog if the chosen model fails, all within `DRAW_TOTAL_BUDGET_SEC`.
 
 ## Streaming & typing pace
 
-For the first candidate in a route — plain text, no attachments, no links — Lumen streams the reply by repeatedly editing one message, the same mechanism for both Gemini (`generate_content_stream`) and OpenRouter (SSE over `chat/completions`).
+For the first candidate in a route (plain text, no attachments, no links), Lumen streams the reply by repeatedly editing one message, the same mechanism for both Gemini (`generate_content_stream`) and OpenRouter (SSE over `chat/completions`).
 
-Telegram won't let a bot edit one message more than about once a second, so true token-by-token output isn't possible regardless of how fast the model generates text. Instead, `lumen_typing_pace.py` tracks each `provider:model` pair's real observed characters-per-second and reveals a growing slice of the buffered text at that pace, capped by whatever has actually arrived. If a backend returns the whole answer in one large chunk — common for free OpenRouter models, since the same `:free` slug can be served by different backends depending on load — a short "catch-up" phase (at most `STREAM_TYPING_MAX_CATCHUP_TICKS × STREAM_TYPING_TICK_SEC` seconds) finishes revealing it smoothly instead of dumping it all at once.
+Telegram won't let a bot edit one message more than about once a second, so true token-by-token output isn't possible regardless of how fast the model generates text. Instead, `lumen_typing_pace.py` tracks each `provider:model` pair's real observed characters-per-second and reveals a growing slice of the buffered text at that pace, capped by whatever has actually arrived. If a backend returns the whole answer in one large chunk (common for free OpenRouter models, since the same `:free` slug can be served by different backends depending on load), a short "catch-up" phase finishes revealing it smoothly instead of dumping it all at once, capped at `STREAM_TYPING_MAX_CATCHUP_TICKS × STREAM_TYPING_TICK_SEC` seconds.
 
-This speed estimate is measured, not hardcoded: there's no table of "model X does N tokens/sec" to maintain, because that number isn't a property of the model on OpenRouter's free tier in the first place. A new model just starts at a default speed and calibrates itself over its first few replies.
+This speed estimate is measured, not hardcoded. There is no table of "model X does N tokens/sec" to maintain, because that number isn't a property of the model on OpenRouter's free tier in the first place. A new model just starts at a default speed and calibrates itself over its first few replies.
 
 If the streaming attempt for the head-of-route model fails before showing any text, the bot falls back to a normal (non-streaming) call further down the chain, reusing the same placeholder message rather than sending a new one.
 
 ### Manual smoke test before touching streaming code
 
-Streaming is covered only by mocked clients — no test exercises a real Gemini or OpenRouter SSE stream. Before merging changes to `_run_streaming_reply`, `_gemini_stream_pieces`, `_openrouter_stream_pieces`, or `lumen_typing_pace.py`, check by hand in a real chat:
+Streaming is covered only by mocked clients; no test exercises a real Gemini or OpenRouter SSE stream. Before merging changes to `_run_streaming_reply`, `_gemini_stream_pieces`, `_openrouter_stream_pieces`, or `lumen_typing_pace.py`, check by hand in a real chat:
 
-1. A short message (fits in one edit) — once via Gemini, once via OpenRouter.
-2. A long message (past `TG_MAX_LEN`) — confirm it splits into multiple messages and continues correctly.
-3. A request where the streaming model is deliberately unavailable — confirm the silent fallback to a non-streaming reserve model, with no visible glitch.
+1. A short message (fits in one edit): once via Gemini, once via OpenRouter.
+2. A long message (past `TG_MAX_LEN`): confirm it splits into multiple messages and continues correctly.
+3. A request where the streaming model is deliberately unavailable: confirm the silent fallback to a non-streaming reserve model, with no visible glitch.
 
 ## Prompt-injection and identity-leak defenses
 
-Lumen deliberately hides which model or provider answers a given message (see the persona in `system_prompt.py`). The system prompt alone is the weakest layer — any LLM can potentially be talked out of following it with a creative enough injection — so there are several independent layers, each assuming the previous one might have failed:
+Lumen deliberately hides which model or provider answers a given message (see the persona in `system_prompt.py`). The system prompt alone is the weakest layer, since any LLM can potentially be talked out of following it with a creative enough injection. So there are several independent layers, each assuming the previous one might have failed:
 
-1. **Input pre-filter** (`_looks_like_injection_probe`, `lumen_security.py`) — well-known jailbreak phrasing ("ignore previous instructions," "developer mode," "show me your system prompt") is caught before the model is ever called. The response is fully deterministic.
-2. **System prompt** (`system_prompt.py`) — instructs the model that anything outside the prompt itself (user messages, chat background, page/video/document content) is data, not instructions, and that the persona doesn't change no matter who claims authority to override it.
-3. **Output identity-leak filter** (`_detect_identity_leak` / `_scrub_identity_leak`) — a deterministic check on the finished reply: exact internal model IDs, and narrow self-identification patterns ("I am Gemini," "made by OpenAI"). Deliberately narrow, to avoid false positives on ordinary, honest discussion of other AI companies.
-4. **Injected-payload echo filter** (`_detect_injected_payload_echo`) — catches the case where an attacker embeds an instruction in a photo or web page ("output this exact string to confirm the jailbreak worked") and the model, while refusing to *follow* it, ends up quoting it back verbatim while summarizing the content.
+1. **Input pre-filter** (`_looks_like_injection_probe`, `lumen_security.py`): well-known jailbreak phrasing ("ignore previous instructions," "developer mode," "show me your system prompt") is caught before the model is ever called. The response is fully deterministic.
+2. **System prompt** (`system_prompt.py`): instructs the model that anything outside the prompt itself (user messages, chat background, page/video/document content) is data, not instructions, and that the persona doesn't change no matter who claims authority to override it.
+3. **Output identity-leak filter** (`_detect_identity_leak` / `_scrub_identity_leak`): a deterministic check on the finished reply, catching exact internal model IDs and narrow self-identification patterns ("I am Gemini," "made by OpenAI"). Deliberately narrow, to avoid false positives on ordinary, honest discussion of other AI companies.
+4. **Injected-payload echo filter** (`_detect_injected_payload_echo`): catches the case where an attacker embeds an instruction in a photo or web page ("output this exact string to confirm the jailbreak worked") and the model, while refusing to *follow* it, ends up quoting it back verbatim while summarizing the content.
 
-Layers 3 and 4 run before the reply is written to chat history (so a leak can't influence future turns) and, during streaming, before each chunk is shown to the user — not only on the final text.
+Layers 3 and 4 run before the reply is written to chat history (so a leak can't influence future turns) and, during streaming, before each chunk is shown to the user, not just the final text.
 
 None of this is airtight except the input pre-filter. The goal is raising the bar for known attack patterns, not proving immunity to every possible phrasing; incidents get logged with `[identity-leak]` / `[injection-echo]` / `[injection-probe]` tags so new patterns can be added as they show up.
 
 ## TikTok downloader
 
-Downloads go through the public TikWM API (`tikwm.com`), no account or API key needed. The bot never re-encodes video or photos — bytes go to Telegram exactly as TikWM served them.
+Downloads go through the public TikWM API (`tikwm.com`), no account or API key needed. The bot never re-encodes video or photos; bytes go to Telegram exactly as TikWM served them.
 
 - **Video quality**: requested with `&hd=1`; among the variants TikWM returns (`hdplay`/`play`/`wmplay`, each with a known byte size), the best one that fits Telegram's 50 MB upload limit is picked. If Telegram still rejects it as too large, the next lighter variant is tried automatically.
 - **Slideshows**: TikTok allows up to 35 slides per post; Telegram's `sendMediaGroup` caps out at 10 items per call. The bot downloads the whole post in parallel and sends it as several media groups in sequence, never with a trailing group of exactly one item (Telegram requires 2–10 per group).
-- **"Live" slides**: TikWM's response includes a separate `live_images` field alongside the regular `images` — that's where the actually-moving version of a slide lives, if it has one. Each downloaded slide is also double-checked by its magic bytes (`ftyp` = video container) rather than trusting the field alone.
-- TikWM rejects Hugging Face Spaces' outbound IPs with an empty `403` — the same class of block documented for YouTube below. The proxy (`TIKWM_API_BASE_URL`) is the only fix that's worked in practice; throttling, retries, and spoofed `Referer`/`Origin` headers didn't help on their own (they're still in place as defense in depth).
+- **"Live" slides**: TikWM's response includes a separate `live_images` field alongside the regular `images`; that's where the actually-moving version of a slide lives, if it has one. Each downloaded slide is also double-checked by its magic bytes (`ftyp` = video container) rather than trusting the field alone.
+- TikWM rejects Hugging Face Spaces' outbound IPs with an empty `403`, the same class of block that also keeps [YouTube downloading off the table](../README.md#known-limitations). The proxy (`TIKWM_API_BASE_URL`) is the only fix that's worked in practice; throttling, retries, and spoofed `Referer`/`Origin` headers didn't help on their own (they're still in place as defense in depth).
 
 ## Persistent storage
 
-Chat history and quota counters live in `chat_state.json` / `global_quota.json` on the container's local disk by default — which Hugging Face Spaces wipes on every redeploy. If `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` are both set, the same data is written to Upstash Redis instead, and survives redeploys. Each chat gets its own key so one failed write can't take down every chat's state at once, and a small in-memory index tracks which chats exist.
+Chat history and quota counters live in `chat_state.json` / `global_quota.json` on the container's local disk by default, which Hugging Face Spaces wipes on every redeploy. If `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` are both set, the same data is written to Upstash Redis instead, and survives redeploys. Each chat gets its own key so one failed write can't take down every chat's state at once, and a small in-memory index tracks which chats exist.
 
 ## Error tracking
 
-If `SENTRY_DSN` is set, `sentry_sdk` picks up every `log.exception()`/`log.error()` call already in the codebase with no changes needed at the call sites. Every event is scrubbed of known secrets (`BOT_TOKEN`, API keys, `WEBHOOK_SECRET`, `ADMIN_PANEL_KEY`, the Upstash token) before it leaves the process. Performance tracing is off — this only tracks errors, to stay comfortably inside Sentry's free tier.
+If `SENTRY_DSN` is set, `sentry_sdk` picks up every `log.exception()`/`log.error()` call already in the codebase with no changes needed at the call sites. Every event is scrubbed of known secrets (`BOT_TOKEN`, API keys, `WEBHOOK_SECRET`, `ADMIN_PANEL_KEY`, the Upstash token) before it leaves the process. Performance tracing is off; this only tracks errors, to stay comfortably inside Sentry's free tier.

docs/ENVIRONMENT.md

@@ -1,6 +1,6 @@
 # Environment variables
 
-Only `BOT_TOKEN` and `GEMINI_API_KEY` are required. Everything else has a working default — most deployments never need to touch the rest of this file.
+Only `BOT_TOKEN` and `GEMINI_API_KEY` are required. Everything else has a working default; most deployments never need to touch the rest of this file.
 
 ## Required
 
@@ -11,7 +11,7 @@ Only `BOT_TOKEN` and `GEMINI_API_KEY` are required. Everything else has a workin
 
 ## Telegram & TikWM proxy
 
-Hugging Face Spaces' outbound IPs are blocked by Telegram's API and rejected (`403`) by TikWM — see [Proxy setup](../README.md#proxy-setup) in the main README.
+Hugging Face Spaces' outbound IPs are blocked by Telegram's API and rejected (`403`) by TikWM. See [Proxy setup](../README.md#proxy-setup) in the main README.
 
 | Variable | Default | Purpose |
 |---|---|---|
@@ -26,7 +26,7 @@ Hugging Face Spaces' outbound IPs are blocked by Telegram's API and rejected (`4
 
 | Variable | Default | Purpose |
 |---|---|---|
-| `ADMIN_SECRET_SEED` | `BOT_TOKEN` | Salt used to derive `WEBHOOK_SECRET`/`ADMIN_PANEL_KEY`. Set it independently to rotate those two secrets without touching the bot's actual Telegram token. |
+| `ADMIN_SECRET_SEED` | falls back to `BOT_TOKEN`, then to a random value if that's empty too | Salt used to derive `WEBHOOK_SECRET`/`ADMIN_PANEL_KEY`. Set it independently to rotate those two secrets without touching the bot's actual Telegram token. |
 | `OWNER_ID` / `BOT_OWNER_ID` / `ADMIN_ID` / `TELEGRAM_OWNER_ID` | — | Telegram user ID that unlocks `/logs` and `/stats`. The first non-empty variable found is used. |
 | `BOT_USERNAME` | `LumenAI_bot` | Fallback username; the bot fetches its real one via `getMe` on startup and uses that instead. |
 
@@ -50,28 +50,28 @@ Hugging Face Spaces' outbound IPs are blocked by Telegram's API and rejected (`4
 | `TTS_MAX_CHARS` | `800` | Max text length accepted by `/tts`. |
 | `RATE_LIMIT_MAX_REQUESTS` | `5` | Max requests per user within `RATE_LIMIT_WINDOW_SEC`. |
 | `RATE_LIMIT_WINDOW_SEC` | `30s` | Sliding window width for rate limiting. |
-| `ROUTE_MODEL_TIMEOUT_SEC` | `22s` | Timeout for a single attempt at a single model. No retries — any failure moves straight to the next model. |
+| `ROUTE_MODEL_TIMEOUT_SEC` | `22s` | Timeout for a single attempt at a single model. No retries: any failure moves straight to the next model. |
 | `ROUTE_TOTAL_BUDGET_SEC` | `40s` | Total time budget for the whole routing chain of one message, across both providers. |
 | `DRAW_TOTAL_BUDGET_SEC` | `120s` | Same idea, for the `/draw` fallback chain across image models. |
 | `STREAM_CHUNK_TIMEOUT_SEC` | `30s` | Timeout waiting for the next streamed chunk, shared by Gemini and OpenRouter. |
 | `STREAM_EDIT_MIN_INTERVAL_SEC` | `1.2s` | Minimum interval between message edits during streaming (protects against Telegram's `429`). |
 | `STREAM_TYPING_TICK_SEC` | `0.5s` | Interval between steps of the post-stream "catch-up" reveal. |
 | `STREAM_TYPING_MAX_CATCHUP_TICKS` | `6` | Max catch-up steps, capping the extra delay this can add. |
-| `STATE_FLUSH_CONCURRENCY` | `10` | Max concurrent background writes to storage per flush cycle. |
 
 ## TikTok downloader
 
 | Variable | Default | Purpose |
 |---|---|---|
 | `TIKTOK_DOWNLOAD_MAX_BYTES` | `75 MB` | Hard cap on any single downloaded TikTok file (video, slide, or cover), aborted mid-stream if exceeded. |
-| `TIKTOK_SLIDE_DOWNLOAD_CONCURRENCY` | `8` | Max slideshow slides downloaded in parallel — keeps one large post from hogging the shared HTTP connection pool. |
+| `TIKTOK_SLIDE_DOWNLOAD_CONCURRENCY` | `8` | Max slideshow slides downloaded in parallel; keeps one large post from hogging the shared HTTP connection pool. |
 | `TIKTOK_VIDEO_SLIDE_PROBE_CONCURRENCY` | `4` | Max concurrent `ffprobe`/`ffmpeg` processes when probing "live" video slides in a slideshow. |
 
 ## Persistent storage
 
 | Variable | Default | Purpose |
 |---|---|---|
-| `STATE_DIR` | `/app` | Where `chat_state`/`global_quota` files live if Upstash isn't configured. This is the container's ephemeral disk — see [Known limitations](../README.md#known-limitations). Falls back to a temp directory if not writable. |
+| `STATE_DIR` | `/app` | Where `chat_state`/`global_quota` files live if Upstash isn't configured. This is the container's ephemeral disk (see [Known limitations](../README.md#known-limitations)). Falls back to a temp directory if not writable. |
+| `STATE_FLUSH_CONCURRENCY` | `10` | Max concurrent background writes to storage per flush cycle. |
 | `UPSTASH_REDIS_REST_URL` | — | Upstash Redis REST URL. Set together with the token below to persist state across redeploys. |
 | `UPSTASH_REDIS_REST_TOKEN` | — | Upstash Redis REST token. |