Fetching metadata from the HF Docker repository...
Jul 5 15:27 docs: Stage 18 — free-tier stability config + live/blocked eval status benroshan +1: Claude Sonnet 4.6 - docs/evolution.md: Stage 18 section + updated Current State Snapshot
(HyDE/MQ disabled, contextual ON, Groq call budget table, upgrade path)
- docs/decisions.md: 2026-07-05 decision entries for HyDE+MQ disabling
and eval dashboard live/blocked markers; Cerebras + HyDE+MQ in rejected
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Jun 27 14:42 docs: Stage 17 metadata filtering — evolution, decisions, architecture 13:28 docs: add metadata filtering implementation plan 13:24 docs: add metadata filtering design spec (Stage 17) Jun 26 09:51 docs: update architecture.md to reflect current shipped state (v1.3.0 stack) 09:47 docs: audit learning.md — remove 6 non-GenAI concepts, update stale data, add Concept 21 contextual retrieval 09:38 docs: add Concept 20 — semantic chunking recall vs precision tradeoff 09:36 docs: update evolution, decisions for Stage 16 citation highlighting 05:30 fix: replace startswith with is_relative_to() for path traversal security 05:20 docs: add citation highlighting implementation plan 05:17 docs: add citation highlighting design spec Jun 24 17:12 feat: HyDE re-eval (recall +21pp), briefing JSON fix, eval run cleanup benroshan +1: Claude Sonnet 4.6 - server/briefing.py: strip control chars + ast.literal_eval fallback fixes JSONDecodeError crash
- config.yaml: hyde_enabled=true, multi_query_enabled=true, contextual_retrieval.enabled=false
- eval-dashboard: delete stale v1.0–v1.4 runs; add v1.1.0_20260624 (HyDE=true, 18 samples)
- recall 0.510→0.721 (+21pp), relevancy 0.845, P@5 0.911, p50 4018ms
- docs/evolution.md: Stage 14 + updated Current State Snapshot + Interview Story Arc
- docs/decisions.md: briefing fix + HyDE default decision entries
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> 17:03 docs: update evolution.md with streaming responses (Stage 14) 16:37 docs: streaming responses implementation plan 16:34 docs: streaming responses design spec Jun 23 16:18 docs: log Stage 13 async embed upload 15:51 docs: async embed upload design spec Jun 22 17:59 docs: update all references Render → HF Spaces (Stage 12) benroshan +1: Claude Sonnet 4.6 - README.md: deployment row, design decisions, architecture diagram
- docs/architecture.md: component table, key decisions, known limits, future work
- docs/decisions.md: Render decision superseded, HF Spaces decision added
- docs/evolution.md: Stage 12 added, Current State Snapshot updated
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Jun 20 09:56 docs: Stage 11 — contextual prod + dashboard polish + concepts 18-19 07:40 docs: Stage 10 contextual retrieval results, update current state snapshot 05:57 docs: contextual retrieval implementation plan 05:54 docs: contextual retrieval design spec (eval-only, v1.3.0 target) Jun 19 17:19 feat: multi-query retrieval + HyDE/MQ eval runs (v1.1.0, v1.2.0 Violet) benroshan +1: Claude Sonnet 4.6 - server/retriever.py: _multi_query_expand() generates 3 query phrasings via
Groq LLM; _get_relevant_documents() pools results across all phrasings,
deduplicates by best rank, RRF fuses wider pool before reranking
- config.yaml: multi_query_enabled toggle (default: false); reverted to
baseline (hyde=false, multi_query=false) after eval
- docs/learning.md: Concept 17 — Multi-Query Retrieval
- docs/evolution.md: Stage 9 + Phase 1 conclusion
Eval results:
v1.1.0 Violet (hyde=true): recall=0.545, p50=5883ms — 3× latency, not worth it
v1.2.0 Violet (multi_query=true): recall=0.517, p50=2620ms — negligible lift
Phase 1 exhausted. recall=0.51 is a chunking problem, not query formulation.
Next: Phase 2 — Contextual Retrieval (ingest-time LLM chunk augmentation).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Jun 18 17:31 chore: name changes on MAJOR version only — all v1.x.x = Violet 17:30 chore: adopt semver for eval runs, rename v1.0 → v1.0.0 Violet benroshan +1: Claude Sonnet 4.6 - Eval run renamed v1.0 → v1.0.0, tag Violet
- Versioning: v1.0.1=fix, v1.1.0=feature, v2.0.0=major arch change
- Names follow spectrum: Violet → Indigo → Azure → Amber → Scarlet
- docs/evolution.md: updated script example to v1.1.0 Indigo (next run)
- Next eval run after Multi-Query Retrieval: v1.1.0 Indigo
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> 17:16 fix: pass methodology prop to MetricCard, rename v2.0 → Prism v1.0, update docs benroshan +1: Claude Sonnet 4.6 - MetricCard i-button fix: methodology prop was missing from map call in App.jsx
- Renamed eval run v2.0 → v1.0 "Prism v1.0 baseline"
- evolution.md: updated baseline scores (50-sample run results)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> 16:42 docs: update evolution.md — Stage 8 TOC, current state URLs, HyDE done, next = multi-query 16:15 feat: Stage 8 — eval dashboard, mandatory web search, docs update benroshan +1: Claude Sonnet 4.6 - eval-dashboard/: standalone Vite+React static site (own Vercel project)
MetricCard, EvolutionChart (Recharts), RunTable, LatencyStats
reads versioned JSON runs from public/data/runs/ + index.json
- scripts/run_eval_versioned.py: offline eval script
metrics: answer_correctness, answer_relevancy, context_recall, precision@5, latency p50/p95/p99
writes versioned JSON + updates index.json
- data/ground_truth/eval_pairs.json: expanded 20 → 50 pairs
added multi-hop, comparative, negative, numeric, edge-case questions
- frontend: removed per-message faithfulness badge (MessageBubble.jsx)
removed web search toggle — hardcoded always-on (ChatArea.jsx)
- server/routes/chat.py: removed score_faithfulness() call; web_search default True
- docs: architecture, decisions, api-spec, evolution, structure updated for Stage 8
- .env.example: updated comments; LangSmith listed as optional
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Jun 13 13:42 docs: add Prism v3 implementation plan (8 tasks: rebrand + URL ingest + briefing + citations + multi-workspace) 13:12 docs: add Prism v3 design spec (domain-agnostic rebrand + URL ingestion + briefing + citations + multi-workspace) May 30 16:44 docs: update architecture and decisions for 2026-05-30 changes benroshan +1: Claude Sonnet 4.6 - architecture.md: TinyBERT-L-2-v2 reranker, RAGAS pre-computed JSON benchmark
- decisions.md: TinyBERT switch, RAGAS removal, 8B judge model, ground truth answers
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> May 24 06:38 feat: Migrate LLM to Groq, fix web search pipeline, add structured logging benroshan +1: Claude Sonnet 4.6 LLM / provider:
- Switch LLM from Euron (gpt-4.1-mini) to Groq (llama-3.3-70b-versatile)
- Add langchain-groq dependency; Euron retained for embeddings only
- Update config.yaml, render.yaml, .env.example, README, docs
Web search (Tavily):
- Fix root bug: ConversationalRetrievalChain condensation step was stripping
prepended web context before LLM saw it — web search appeared to work but
answers always said "I don't know"
- Add run_query_with_web(): direct LLM call with RAG + Tavily context +
chat history, bypassing chain condensation entirely
- Add condense_question(): rewrites follow-up questions using chat history
before Tavily search so context-dependent queries work correctly
- Switch Tavily search_depth basic → advanced for richer financial content
- Fix memory.save_context key: {"output": answer} → {"answer": answer}
Logging:
- Centralise logging: configure_logging() in main.py sets root logger with
console + rotating file (logs/finrag.log, 5MB x3); silences noisy li… May 23 14:59 feat: Add RAGAS eval, score passthrough, docs, and hybrid retrieval config benroshan +1: Claude Sonnet 4.6 - chain.py: pass similarity/bm25/rrf/rerank scores from metadata to API response
- routes/chat.py: store contexts in eval_log for RAGAS; surface retrieval_method
- routes/eval.py: wire POST /api/eval/ragas endpoint with RagasRequest schema
- server/eval/ragas_eval.py: RAGAS faithfulness + answer_relevancy via LangchainLLMWrapper
- main.py: add GET /health endpoint (status + version)
- config.yaml: add hybrid retrieval params (dense_weight, sparse_weight, retrieve_k, rerank_k)
- requirements.txt: add rank_bm25, sentence-transformers, ragas, datasets
- docs/: migrate architecture, decisions, api-spec, structure from finrag.md
- .gitignore: exclude CLAUDE.md and .claude/ (personal workstation files)
- finrag.md: deleted (content moved to docs/)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> 
