Fetching metadata from the HF Docker repository...
-
docs: Stage 18 — free-tier stability config + live/blocked eval status
benroshan +1: Claude Sonnet 4.6
-
docs: Stage 17 metadata filtering — evolution, decisions, architecture
-
docs: update evolution, decisions for Stage 16 citation highlighting
-
fix: replace startswith with is_relative_to() for path traversal security
-
feat: HyDE re-eval (recall +21pp), briefing JSON fix, eval run cleanup
benroshan +1: Claude Sonnet 4.6 - server/briefing.py: strip control chars + ast.literal_eval fallback fixes JSONDecodeError crash
- config.yaml: hyde_enabled=true, multi_query_enabled=true, contextual_retrieval.enabled=false
- eval-dashboard: delete stale v1.0–v1.4 runs; add v1.1.0_20260624 (HyDE=true, 18 samples)
- recall 0.510→0.721 (+21pp), relevancy 0.845, P@5 0.911, p50 4018ms
- docs/evolution.md: Stage 14 + updated Current State Snapshot + Interview Story Arc
- docs/decisions.md: briefing fix + HyDE default decision entries
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> -
docs: update evolution.md with streaming responses (Stage 14)
-
docs: log Stage 13 async embed upload
-
docs: update all references Render → HF Spaces (Stage 12)
benroshan +1: Claude Sonnet 4.6 - README.md: deployment row, design decisions, architecture diagram
- docs/architecture.md: component table, key decisions, known limits, future work
- docs/decisions.md: Render decision superseded, HF Spaces decision added
- docs/evolution.md: Stage 12 added, Current State Snapshot updated
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
-
docs: Stage 11 — contextual prod + dashboard polish + concepts 18-19
-
docs: Stage 10 contextual retrieval results, update current state snapshot
-
feat: multi-query retrieval + HyDE/MQ eval runs (v1.1.0, v1.2.0 Violet)
benroshan +1: Claude Sonnet 4.6 - server/retriever.py: _multi_query_expand() generates 3 query phrasings via
Groq LLM; _get_relevant_documents() pools results across all phrasings,
deduplicates by best rank, RRF fuses wider pool before reranking
- config.yaml: multi_query_enabled toggle (default: false); reverted to
baseline (hyde=false, multi_query=false) after eval
- docs/learning.md: Concept 17 — Multi-Query Retrieval
- docs/evolution.md: Stage 9 + Phase 1 conclusion
Eval results:
v1.1.0 Violet (hyde=true): recall=0.545, p50=5883ms — 3× latency, not worth it
v1.2.0 Violet (multi_query=true): recall=0.517, p50=2620ms — negligible lift
Phase 1 exhausted. recall=0.51 is a chunking problem, not query formulation.
Next: Phase 2 — Contextual Retrieval (ingest-time LLM chunk augmentation).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
-
chore: name changes on MAJOR version only — all v1.x.x = Violet
-
chore: adopt semver for eval runs, rename v1.0 → v1.0.0 Violet
benroshan +1: Claude Sonnet 4.6 - Eval run renamed v1.0 → v1.0.0, tag Violet
- Versioning: v1.0.1=fix, v1.1.0=feature, v2.0.0=major arch change
- Names follow spectrum: Violet → Indigo → Azure → Amber → Scarlet
- docs/evolution.md: updated script example to v1.1.0 Indigo (next run)
- Next eval run after Multi-Query Retrieval: v1.1.0 Indigo
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> -
fix: pass methodology prop to MetricCard, rename v2.0 → Prism v1.0, update docs
benroshan +1: Claude Sonnet 4.6 - MetricCard i-button fix: methodology prop was missing from map call in App.jsx
- Renamed eval run v2.0 → v1.0 "Prism v1.0 baseline"
- evolution.md: updated baseline scores (50-sample run results)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> -
docs: update evolution.md — Stage 8 TOC, current state URLs, HyDE done, next = multi-query
-
feat: Stage 8 — eval dashboard, mandatory web search, docs update
benroshan +1: Claude Sonnet 4.6 - eval-dashboard/: standalone Vite+React static site (own Vercel project)
MetricCard, EvolutionChart (Recharts), RunTable, LatencyStats
reads versioned JSON runs from public/data/runs/ + index.json
- scripts/run_eval_versioned.py: offline eval script
metrics: answer_correctness, answer_relevancy, context_recall, precision@5, latency p50/p95/p99
writes versioned JSON + updates index.json
- data/ground_truth/eval_pairs.json: expanded 20 → 50 pairs
added multi-hop, comparative, negative, numeric, edge-case questions
- frontend: removed per-message faithfulness badge (MessageBubble.jsx)
removed web search toggle — hardcoded always-on (ChatArea.jsx)
- server/routes/chat.py: removed score_faithfulness() call; web_search default True
- docs: architecture, decisions, api-spec, evolution, structure updated for Stage 8
- .env.example: updated comments; LangSmith listed as optional
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
