Fetching metadata from the HF Docker repository...
-
feat: Stage 8 β eval dashboard, mandatory web search, docs updateβ
benroshan +1: Claude Sonnet 4.6 - eval-dashboard/: standalone Vite+React static site (own Vercel project)
MetricCard, EvolutionChart (Recharts), RunTable, LatencyStats
reads versioned JSON runs from public/data/runs/ + index.json
- scripts/run_eval_versioned.py: offline eval script
metrics: answer_correctness, answer_relevancy, context_recall, precision@5, latency p50/p95/p99
writes versioned JSON + updates index.json
- data/ground_truth/eval_pairs.json: expanded 20 β 50 pairs
added multi-hop, comparative, negative, numeric, edge-case questions
- frontend: removed per-message faithfulness badge (MessageBubble.jsx)
removed web search toggle β hardcoded always-on (ChatArea.jsx)
- server/routes/chat.py: removed score_faithfulness() call; web_search default True
- docs: architecture, decisions, api-spec, evolution, structure updated for Stage 8
- .env.example: updated comments; LangSmith listed as optional
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
-
feat: Add RAGAS eval, score passthrough, docs, and hybrid retrieval configβ
benroshan +1: Claude Sonnet 4.6 - chain.py: pass similarity/bm25/rrf/rerank scores from metadata to API response
- routes/chat.py: store contexts in eval_log for RAGAS; surface retrieval_method
- routes/eval.py: wire POST /api/eval/ragas endpoint with RagasRequest schema
- server/eval/ragas_eval.py: RAGAS faithfulness + answer_relevancy via LangchainLLMWrapper
- main.py: add GET /health endpoint (status + version)
- config.yaml: add hybrid retrieval params (dense_weight, sparse_weight, retrieve_k, rerank_k)
- requirements.txt: add rank_bm25, sentence-transformers, ragas, datasets
- docs/: migrate architecture, decisions, api-spec, structure from finrag.md
- .gitignore: exclude CLAUDE.md and .claude/ (personal workstation files)
- finrag.md: deleted (content moved to docs/)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>