Umer359 Claude Sonnet 5 commited on
Commit
2cfd8cc
·
1 Parent(s): 4e13b44

Document OpenAI addition, scoped to RAG-grounded replies only

Browse files

Unlike Lex (OpenAI primary for everything), Nox only routes to OpenAI
when a reply is actually grounded in retrieved document excerpts —
plain chat and live-data questions stay on the free Cloudflare tiers.
Real streaming, same as the Cloudflare path.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Files changed (1) hide show
  1. README.md +12 -0
README.md CHANGED
@@ -132,6 +132,18 @@ with an 8s timeout so one slow provider can't stall the whole request. Scoped to
132
  text chat path only — the vision endpoint already degrades gracefully on its own and
133
  would need real separate per-provider multimodal handling to extend this to it.
134
 
 
 
 
 
 
 
 
 
 
 
 
 
135
  ### Everything else
136
  - **PWA**: installable, small app-shell cache via `sw.js` (network-first for
137
  `index.html` so a redeploy is visible immediately when online; falls back to cache
 
132
  text chat path only — the vision endpoint already degrades gracefully on its own and
133
  would need real separate per-provider multimodal handling to extend this to it.
134
 
135
+ **OpenAI, scoped narrowly, unlike Lex**: `OPENAI_API_KEY` is set on this worker too, but
136
+ tried first **only when the reply is genuinely grounded in retrieved document excerpts**
137
+ (`ragChunksMeta.length > 0` — a document was actually uploaded and something relevant
138
+ was found for this question), streamed via `callOpenAIStream` the same way Cloudflare's
139
+ own reply streams. Plain chat and live-data questions never touch it and stay entirely
140
+ on the free Cloudflare tiers regardless of which model tier the visitor picked — a
141
+ deliberate choice to keep OpenAI's real cost scoped to where it was shown to matter most
142
+ (synthesizing several retrieved excerpts faithfully) rather than all of Nox's traffic
143
+ the way it ended up scoped for Lex. Capped at `OPENAI_DAILY_CALL_CAP = 200`/day
144
+ (KV-tracked, separate from `DAILY_MESSAGE_CAP`), a smaller number than Lex's cap since
145
+ this only fires for document-grounded replies, not every message.
146
+
147
  ### Everything else
148
  - **PWA**: installable, small app-shell cache via `sw.js` (network-first for
149
  `index.html` so a redeploy is visible immediately when online; falls back to cache