- Add max_chunks: 50 gate in upload.py — docs >50 chunks skip contextual
retrieval entirely with logged warning (~17 min hang for 104-chunk files)
- Trim full_doc_text context from [:3000] to [:1500] in both sync and async
contextualization paths — drops TPM burst from ~3240 to ~2115, below 3000
TPM free-tier window, reducing 429s for smaller docs
- Update contextual retrieval model to openai/gpt-oss-20b (llama-3.1-8b-instant
being decommissioned by Groq)
- Same model update propagated to eval script judge defaults
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
feat: chat route accepts filter_docs for scoped retrieval
Adds a file-serving endpoint that returns uploaded PDFs from data/raw/
with path traversal protection (400) and 404 for missing files.
UPLOAD_DIR defined as a module-level constant for test monkeypatching.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Removed `web_search: bool = True` from ChatRequest — web search is mandatory per architecture (Stage 7); field was misleading dead code
- Restructured generate() so outer try/finally wraps all paths including chain-is-None early return, guaranteeing gc.collect() + log_memory_mb always run
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
feat: stream POST /api/chat via SSE StreamingResponse
Remove:
- is_contextualizing flag suppressing web search during contextual refresh
- RSS >460MB guard that skipped contextual retrieval entirely
Keep:
- gc.collect() after upload/chat (good hygiene)
- log_memory_mb calls (useful observability)
- is_contextualizing flag on app.state (harmless, contextual refresh still sets it)
Both guards were Render free tier workarounds. On HF Spaces CPU Basic (16GB),
contextual refresh + web search can run concurrently without memory pressure.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fix: RSS guard in contextual refresh — skip if base RSS >460MB
Move gc.collect() + log_memory_mb calls to after embed_and_store and
after generate_briefing, so embed API response objects are freed before
the briefing Groq call adds more objects. Two GC windows instead of zero.
MEM [ctx-refresh-start] was reporting 524MB (above 512MB Render limit) because
30 sequential Euron API response objects + briefing Groq response were all live
simultaneously. Reordering releases embed objects before briefing runs.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fix: OOM guard — skip web search during contextual refresh + psutil memory logging
feat: add HyDE (Hypothetical Document Embeddings) for dense retrieval
Claude
Unverified signature
8945b43
Generate a hypothetical answer before ChromaDB ANN search so the
embedding lands closer to real answer chunks. BM25 and reranker
still use the original query. Toggled via retrieval.hyde_enabled
in config.yaml (default: false).
fix: singleton vectorstore/retriever cache to prevent OOM on repeated queries
Every chat request was creating a new Chroma instance (full embedding reload)
on top of the existing one. Cache vectorstore + retriever per workspace;
invalidate after ingest. Chat route now reuses app.state.retriever.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fix: URL size guard for OOM prevention, eliminate double retrieval in chat, fix print
- url_loader.py: add Content-Type check and 5MB size guard before parsing to prevent OOM on Render 512MB free tier; reject non-text responses early
- chain.py: guard run_query_with_web against empty web_sources to avoid spurious "=== Web search results ===" section in LLM prompt
- routes/chat.py: eliminate double retrieval on non-web path — use run_query_with_web(web_sources=[]) for all queries so the cross-encoder reranker fires exactly once per request; remove unused run_query import
- ingest.py: remove hardcoded 'prism' collection name from print statement
- tests/test_url_loader.py: update _mock_response to stub headers.get and content so new content-type/size checks pass correctly
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fix: workspace-aware retrieval in chat handler, robust list_collections() for chromadb 0.5+
- chat.py: always resolve workspace-specific retriever before branching on
web_search; non-web path now overwrites source_documents with docs from
the correct ChromaDB collection instead of relying on the stale global chain
- workspaces.py: list_collections() result normalised with isinstance check so
chromadb ≥0.5.4 (returns list[str]) and <0.5 (returns list[Collection]) both
work without error
- workspaces.py: add Request import; delete_workspace signature accepts Request
for future active-workspace guard
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- server/briefing.py: add re.search fallback to extract JSON object when LLM prepends text before JSON
- tests/test_briefing.py: new test_generate_briefing_handles_preamble_before_json to catch preamble case
- tighten limit assertions: change <= to == for exact 5 bullets, 3 questions
All 6 tests now pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
TDD: 6 tests written first (red), then server/url_loader.py implemented (green).
load_url() fetches a URL via httpx, strips nav/footer/script tags, extracts
content blocks, and returns a LangChain Document with source/type/fetched_at metadata.
Adds httpx>=0.27.0 and beautifulsoup4>=4.12.0 to requirements.txt.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
nest_asyncio cannot patch uvloop (used by uvicorn on Render Linux) — live eval
is unfixable without replacing the event loop. Remove ragas dependency entirely.
- Remove POST /api/eval/ragas endpoint and ragas_eval import from routes/eval.py
- Remove ragas from requirements.txt (saves Docker build time)
- Remove runRagasEval from api.js
- EvalPanel: replace dynamic run button with static pre-computed benchmark scores
(faithfulness 0.87, answer_relevancy 0.91, context_precision 0.74, recall 0.68)
with i-button tooltips explaining each metric
- EvalPanel no longer needs evalLogLength prop
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fix: Graceful RAGAS 500 + info tooltips on eval metrics and faithfulness badge
- Pin ragas>=0.2.0,<0.3.0 (0.1.x imports removed langchain_community.vertexai)
- Wrap ragas imports in try/except → returns structured error dict instead of 500
- EvalPanel: show amber error banner when ragas unavailable
- EvalPanel: info (i) tooltips on all 4 RAGAS metric labels explaining what each means
- MessageBubble: info (i) tooltip on faithfulness badge explaining 1-5 LLM-as-judge scale
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fix: Switch reranker to TinyBERT-L-2-v2 (~17MB) to prevent OOM on web queries
After first query, Python holds chain invocation objects near memory limit.
Web query adds Tavily content (up to 9KB) + 2 extra LLM calls → OOM.
- Truncate Tavily content to 800 chars per result (was up to 3000)
- Reduce max_results 3 -> 2 (less total web context in LLM prompt)
- gc.collect() after each chat request to release chain/LLM objects
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fix: Reduce CrossEncoder memory spike to prevent OOM on Render 512MB
Chat requests were OOM-killing the container (~500-600MB) when
CrossEncoder.predict() ran BERT forward pass on all 20 fused candidates
at once. Fix:
- retrieve_k: 20 -> 10 (halves pairs fed to reranker)
- batch_size=4 in model.predict() (reduces peak BERT activation memory)
These changes keep rerank quality while staying under 512MB limit.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fix: Lazy-import ragas to prevent startup crash on Render
ragas 0.4.3 imports langchain_community.chat_models.vertexai at module
level, but that module was removed in langchain-community 0.4.x. This
caused an ImportError in server/routes/eval.py -> ragas_eval.py ->
ragas.__init__ import chain, crashing uvicorn before port binding.
Moved all ragas imports inside run_ragas_eval() so they are only loaded
when the eval endpoint is actually called, not at server startup.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
feat: Migrate LLM to Groq, fix web search pipeline, add structured logging
LLM / provider:
- Switch LLM from Euron (gpt-4.1-mini) to Groq (llama-3.3-70b-versatile)
- Add langchain-groq dependency; Euron retained for embeddings only
- Update config.yaml, render.yaml, .env.example, README, docs
Web search (Tavily):
- Fix root bug: ConversationalRetrievalChain condensation step was stripping
prepended web context before LLM saw it — web search appeared to work but
answers always said "I don't know"
- Add run_query_with_web(): direct LLM call with RAG + Tavily context +
chat history, bypassing chain condensation entirely
- Add condense_question(): rewrites follow-up questions using chat history
before Tavily search so context-dependent queries work correctly
- Switch Tavily search_depth basic → advanced for richer financial content
- Fix memory.save_context key: {"output": answer} → {"answer": answer}
Logging:
- Centralise logging: configure_logging() in main.py sets root logger with
console + rotating file (logs/finrag.log, 5MB x3); silences noisy li…
- Add BM25Index singleton (bm25_index.py): builds from ChromaDB corpus at startup, rebuilt after upload
- Add cross-encoder reranker singleton (reranker.py): cross-encoder/ms-marco-MiniLM-L-6-v2, pre-loaded at startup
- Rewrite retriever.py with HybridRetriever(BaseRetriever): dense top-20 + BM25 top-20 -> weighted RRF (0.7/0.3) -> rerank top-5
- Scores (similarity, bm25, rrf, rerank) packed into Document.metadata for downstream use
- main.py: build BM25 index and pre-load reranker on startup
- upload.py: rebuild BM25 index after each document ingestion
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
