Root cause of the actual crash you hit: the local engine's true context
window is 4096 tokens, and unlike the server path (capped at the last
20 messages server-side), nothing bounded how much of `history` got
sent to the local engine each turn. A conversation with even one
document attachment kept growing every turn until it blew the budget
outright — the real error was ContextWindowSizeExceededError (4973
prompt tokens vs. a 4096 window), which then crashed the WebGPU device
("Device was lost"), which broke every later local message until a
full page reload since the dead engine reference was never cleared.
Fixes:
- trimHistoryForLocalEngine() keeps the system message plus as many
recent messages as fit a conservative token-budget estimate (no
tokenizer is loaded client-side, so chars/2.5 stands in, biased to
over-count so it under-counts what fits), always keeping at least
the message just sent even if it alone exceeds budget.
- MAX_ONE_SHOT_DOC_CHARS_LOCAL lowered 4000 -> …
Fix one-shot attachments being double-wrapped in the outgoing message
pendingOneShotDoc's text was being labeled "[Attached file: name]" at
file-select time, then wrapped in another "[Attached file: ...]" header
again at send time — affecting every one-shot attachment (single file
included, both server and local engine), not just the multi-file case.
For 2+ files this nested the labels further, producing a genuinely
confusing prompt.
Root-caused after a real local-mode reply to "summarize both files"
came back as unrelated gibberish. Verified the fix against the real
server model: single-file message now has exactly one wrapper, and a
two-file "summarize both" request now returns a correct, coherent
per-file summary.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Allow one-shot document attachments in offline Chat mode
Local (offline) Chat mode previously hard-blocked any document
attachment outright, on an earlier real test report that the full
12,000-char server-side cap produced garbled, unusable replies from
the tiny local 1B model. Re-enabled with a much smaller cap
(MAX_ONE_SHOT_DOC_CHARS_LOCAL, 4000 chars combined) instead of
removing the safeguard entirely, so a document can't crowd out the
system prompt and conversation history. Multi-file selection already
combines into one attachment for this path, so it works there too.
Images remain blocked in local mode — that's a real capability gap
(text-only engine), not a size-tunable limit.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Sidebar order is now Attachment mode > Chunks per answer > Indexed
documents. "Chunks per answer" only shows in RAG mode (it's meaningless
in Chat mode's one-shot attachment). "Indexed documents" stays hidden
until at least one document exists for the active conversation, and
hides again once the last one is removed — checked against the real
current list on every mode switch, conversation switch, upload, and
removal, not a locally-tracked count that could drift.
Also allows selecting multiple files at once in both modes: RAG mode
indexes each file individually with combined progress/summary
feedback; Chat mode's one-shot attachment concatenates all of them
into a single message. Only one image is still supported per message
(the vision endpoint takes exactly one) — extras are skipped with a
notice if several are selected alongside or instead of documents.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Show total chunks/embeddings in the indexed-documents popup
Adds a summary line above the per-document list (e.g. "2 documents —
9 chunks / 9 embeddings"), recomputed on every open and after each
remove so it always reflects what's actually indexed right now.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Show RAG source chunks with closeness scores, add adjustable top-K
Adds a sources badge under any reply that used retrieved document
excerpts, opening a popup with each chunk's document name, match
percentage, and preview text. Also adds a "chunks per answer" sidebar
control (1-8) shared by both server and offline/local RAG, persisted
to localStorage and sent to the worker as ragTopK.
Requires the paired devseis-chat-proxy worker deploy (X-Rag-Chunks
response header + ragTopK support), already live in production.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Add a documents panel to view/remove RAG-indexed documents
New "Indexed documents" button in the sidebar opens a modal listing
every document currently indexed for the active conversation, with its
real chunk count, and a Remove button per document — works for both
online (server KV) and offline (IndexedDB) RAG, following whichever
one is currently active.
Always does a fresh fetch/query when opened rather than trusting any
client-side cache, so what's shown can never drift from what
retrieval actually reads. Removing a document is the same single
write as "the index being recalculated" — there's no separate index
structure to rebuild, since retrieval always scans whatever's
currently stored; the modal just re-fetches after removal to show
that state, rather than optimistically dropping the row locally.
Also removes ragDocumentNames, an in-memory tracking array made fully
redundant by this fresh-fetch approach — it was already write-only
dead code after an earlier change, and would have been actively wrong
here (stale after a reload) if left…
Extends RAG mode to work with zero network calls when combined with
local (in-browser WebLLM) mode — documents are chunked, embedded, and
retrieved entirely on-device, so a visitor can do private document
Q&A with no internet connection and nothing ever leaving the browser.
- Embeddings: WebLLM (already used for local chat) doesn't do
embeddings, so this pulls in a second, much smaller model via
Transformers.js (Xenova/all-MiniLM-L6-v2, ~23MB quantized) — lazy-
loaded the same way the local chat model already is.
- Storage: IndexedDB instead of the server's KV — persists
indefinitely (unlike the server's 24h TTL, since this is the
visitor's own device with no server cost to bound), cleaned up only
when a saved conversation is deleted outright.
- Retrieval: the exact same chunking/cosine-similarity logic as the
server path, ported as plain dependency-free JS, injected into the
local model's prompt for that turn only (never persisted into
history).
- Local mode's exist…
Swap local AI fallback model from 3B to 1B for mobile speed
Real report: the 3B local model (WebLLM, in-browser) made things
noticeably slow, especially on mobile where WebGPU memory budgets are
often under 1-1.5GB. Llama-3.2-1B-Instruct is the same Meta family/
generation as the 3B model was, specifically built by Meta for on-
device/edge use, at roughly 1/2.6 the VRAM (879MB vs. 2,264MB,
verified directly against WebLLM's own prebuilt config) — a real but
smaller capability gap than the original 360M model had.
Splits document-attachment handling into two explicit, user-chosen
modes instead of one blended behavior:
- Chat mode (default): a document attached to a message is a one-shot
analysis for that single message only (the original behavior) —
never indexed, never used again afterward.
- RAG mode: a document is chunked, embedded, and indexed on upload,
and every later question in the conversation retrieves relevant
excerpts from it — today's behavior, now opt-in rather than the only
option.
Deliberately not retroactive: switching modes only affects documents
attached from that point forward, never re-indexes or un-indexes
anything already uploaded under the other mode.
Also adds a soft background tint (light and dark theme variants) while
RAG mode is active, as a visual cue that attachments behave
differently right now — not a full re-theme, just a small step off the
normal background.
Replaces the old design (the whole extracted document text, up to
12,000 characters, re-sent as part of every message for the rest of
the conversation) with real retrieval-augmented generation: a document
is chunked and embedded once at upload time, and every later question
in the same conversation retrieves only the most relevant few chunks
instead of resending the whole document.
- Documents are indexed immediately on file select via a new
`ingestDocument` request, scoped by the conversation's own id
(activeConvoId) rather than the device-wide visitor id, so switching
to a different conversation naturally stops using a previous one's
documents with no separate cleanup needed.
- Supports multiple documents per conversation (chunks accumulate,
don't replace) and much larger documents than the old single-prompt
budget allowed.
- Local (in-browser WebLLM) mode still declines documents outright —
it has no way to reach the server-side RAG index at all.
Backend: chunking + …
Upgrade local (in-browser) AI fallback model from 360M to 3B
Swaps the WebLLM local-mode model from SmolLM2-360M-Instruct to
Llama-3.2-3B-Instruct (same q4f16_1 quantization, same family as the
server's own "Standard" tier), trading a larger one-time download for
noticeably better coherence and reasoning when a visitor opts into
local mode during Neuron exhaustion. Model ID verified directly against
WebLLM's own prebuilt config source before shipping. Updated the
in-app tooltip copy and code comments describing the old 360M model to
match.
Fix local AI mode attempting to process document attachments
Local mode already declined image attachments but not text/PDF
documents — those get folded into the message as a "[Attached file: ...]"
block with the full extracted text, which the tiny 360M-parameter local
model has no realistic ability to process (real report: it returned
incoherent gibberish when handed one). Now declined with the same clear
message as images, before ever reaching the model.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016b3u4n81zwJ6rpE9NeCokb
Add PWA share target, client-side shareable read-only chat links, and QR codes
- PWA share target: manifest.json share_target lets Android send a
link/text from any app straight into Nox ("hey Nox, summarize this").
Pre-fills the composer with a summarize prompt rather than auto-sending
— the visitor reviews and hits send themselves.
- Shareable read-only chat links: the conversation is gzip-compressed
(native CompressionStream, base64 fallback for older browsers) into the
URL fragment itself (#shared=...) — fragments never reach any server, so
this needs zero backend storage. Opening a shared link shows the
conversation read-only with a banner offering to fork it into an
editable chat of the visitor's own, or discard it and start fresh.
- QR "beam to phone": new Share row in the sidebar (Copy link / QR code)
generates the same compressed link, either copied to clipboard or
rendered as a real scannable QR code (qrcodejs) in a modal.
Verified end-to-end: a link generated in one browser context, opened
fresh in a separate one, correctly renders…
Add hover/tap tooltip explaining what "limited" means in local AI mode
Hovering (desktop) or tapping (mobile) the word "limited" in the local-AI
banner now shows the 5 concrete capability gaps versus the server model:
smaller model, no images, no live data, no date awareness, and
device-dependent speed. Positioned via JS from the term's actual on-screen
rect and clamped to the viewport, rather than a fixed CSS offset, since
the term sits at an unpredictable horizontal spot in the sentence.
Fixed a real bug caught by testing on a touch-emulated device: the term
has tabindex=0, so a tap fires a native `focus` event before `click`,
which independently opened the tooltip — the click handler's toggle then
read "already open" and immediately closed it again on every tap. Fixed
by tracking the click-toggle's own open/closed state in a dedicated flag
instead of inferring it from the tooltip's current display state.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016b3u4n81zwJ6rpE9NeCokb
Add a manual Server/Local AI switch, independent of Neuron exhaustion
New "AI mode" row in the sidebar footer (Server / Local (beta) buttons,
only shown on WebGPU-capable browsers) lets a visitor switch to the
in-browser local model whenever they want, and back, regardless of
whether the shared Neuron budget is actually exhausted. Reuses the same
enableLocalMode()/exitLocalMode() functions as the exhaustion-triggered
flow — the persistent banner and all existing safety behavior (crisis
check, image-attachment rejection) apply identically either way.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016b3u4n81zwJ6rpE9NeCokb
Add opt-in in-browser AI fallback for when the Neuron budget is exhausted
When the real Cloudflare Neuron budget (not just this app's own daily
message cap) runs out, visitors on a WebGPU-capable browser now see a
"Try local AI (beta)" option in the capacity modal/banner. Clicking it
downloads a small model (SmolLM2-360M-Instruct, ~376MB, one-time — cached
by the browser after that) via @mlc-ai/web-llm and runs it entirely on
the visitor's own device, so the app keeps answering simple questions
instead of just showing a "come back later" wall. Clearly labeled as a
limited/beta mode everywhere it's active, with a one-click "Use server
again" exit.
Only ever offered when navigator.gpu exists — WebLLM has no CPU/WASM
fallback, so there's no dead button on unsupported browsers.
Critical safety note: local mode bypasses the server entirely, so the
crisis-detection check (CRISIS_RE/CRISIS_RESPONSE) is duplicated
client-side verbatim from worker.js and runs BEFORE the local model is
ever invoked — a crisis message never reaches the small unsupervised
model. Verif…
SpeechSynthesisUtterance was left to the browser's own default voice,
which on many systems (Chrome especially) is a low-quality offline
synthesizer rather than the natural-sounding voices actually installed.
Now explicitly picks a Google/Natural/Neural/Enhanced voice for the
current language when one is available, falling back sensibly otherwise.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016b3u4n81zwJ6rpE9NeCokb
Add voice input, read-aloud, and a backgrounded-tab reply flash
- Voice input: mic button (only shown when SpeechRecognition is supported)
transcribes speech straight into the composer, no AI/transcription call
- Read-aloud: speaker button on bot replies uses SpeechSynthesis, with
markdown stripped first so it doesn't read "asterisk asterisk" aloud
- Tab title + favicon flash when a reply lands while the tab is
backgrounded, clearing itself the moment the visitor switches back
All native browser APIs — zero Neuron cost, no worker changes.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016b3u4n81zwJ6rpE9NeCokb
Remove the copy button from user messages, keep it on bot replies only
- Conversations now persist to localStorage per-thread and survive reload/crash
- New sidebar lists saved chats (switch, delete); font-size (A-/A+) and
export-as-text/PDF (jsPDF) live in its footer
- Copy button added to user messages too (bot replies already had one)
- Cmd/Ctrl+K focuses the composer, Esc clears a draft or closes the sidebar
- PWA: manifest.json + a narrow offline app-shell service worker (never
touches the chat API or CDN libraries) + generated app icons, so Nox is
installable via "Add to Home Screen"
- Accessibility: aria-live on the thread, dialog role + focus handling on
the sidebar, visible focus rings, and a text-size control
All client-side/localStorage — no worker changes, zero Neuron cost.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016b3u4n81zwJ6rpE9NeCokb
Feedback posts to a new lightweight /feedback-shaped request the worker
handles before any of the chat/usage-cap/AI logic — a plain KV write, so
voting costs zero Neurons and never counts against the daily message cap.
Copy button copies the reply's raw text to the clipboard.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016b3u4n81zwJ6rpE9NeCokb
Give the message input bar a lighter gray fill in dark mode (WhatsApp-style), instead of blending into the page background
Replaces the inline-only banner experience with a prominent, centered
modal (responsive for any screen size, dismissible via OK) shown once
per distinct reset cause, on top of the existing inline banner and
disabled composer. Both the message-cap and Neuron-exhaustion states
now show a countdown — the cap's is exact (this app's own counter,
known UTC-midnight reset); the Neuron-exhaustion one is a best
estimate, worded honestly as such since the real reset schedule isn't
queryable. Also adds a background re-check while the composer is
disabled, so the app can recover early if the real Neuron budget frees
up before the estimated countdown expires, rather than being stuck
until the full estimate is exhausted.
Show real Neuron-exhaustion state in status bar and cap banner, distinct from the app's own message counter
Same fix as applied to Lex: the status bar and cap-reached banner
previously only knew about this app's own self-imposed message-count
cap, which could show plenty of remaining capacity while the actual
Cloudflare account-wide Neuron budget was fully exhausted and every
request was failing. Adds a dedicated banner state (no false countdown,
since the real reset schedule isn't queryable) and corrects the chat
bubble wording to not claim a message-cap explanation for a different
underlying cause.
Load pdf.js as a classic script instead of an ES module
The earlier ordering fix (await a load promise) didn't resolve the
report from mobile, pointing at ES module support itself as the
weaker link (some mobile browsers/in-app webviews have inconsistent
type=module support). Pins pdf.js 2.16.105, the last version with a
classic UMD build, loaded the same way as marked.js/DOMPurify already
are — synchronous, document-order, no async race possible.
Fix PDF attachment race condition on slow (mobile) connections
pdf.js loads via an async module import; a visitor could attach a PDF
before that import resolved, hitting the generic 'Couldn't read that
file' error. extractPdfText now awaits the library's actual load
promise instead of assuming it's ready.
Add 'Powered by Devseis' link next to Privacy & Terms in footer
