Spaces:
Sleeping
Sleeping
Commit History
-
Add Fine-Tuned benchmark results (85.3% accuracy) from checkpoint-25
Aditya- Update system3_inference.py to load LoRA adapter directly from checkpoint-25/ instead of requiring a merged finetuned_model/ dir - Add fill_r3.py: standalone script to populate r3 metrics in the existing benchmark cache without re-running API calls - Update data/benchmark_cache.json with real Fine-Tuned metrics for all 50 questions (accuracy 85.3%, ROUGE-L 0.283, HR 0.0%) - Update README benchmark table with real Fine-Tuned numbers09c4e61 -
Add pre-computed benchmark cache for all 50 questions
Aditya- data/benchmark_cache.json: real benchmark results from Baseline LLM and RAG across all 50 reference Q&A pairs (accuracy, ROUGE-L, groundedness, answer relevance, faithfulness, latency, cost). App loads this on startup so the analytics tab shows results instantly without re-running the benchmark. Summary: Baseline 81.5% accuracy @ 1.46s avg, RAG 79.9% @ 4.19s avg. - run_benchmark.py: standalone script to regenerate the cache. Imports rouge_score before heavy ML libs to avoid segfault. Saves incrementally after each question so it can resume from interruptions.78c912f -
Major upgrade: 50-question benchmark, persistent analytics, paper-style TABLE II
Adityademo.py changes: - BENCHMARK_QUESTIONS now dynamically loads all 50 keys from reference_answers.json instead of a hardcoded 6-item list - Quick Questions grid fixed: 3-column layout (i % 3) prevents button overflow - compute_auto_metrics() extended with answer_relevance and faithfulness metrics - _estimate_cost() helper added for per-query USD cost estimation - run_benchmark() stores 7 metric groups per row (rouge, sim/accuracy, groundedness, answer_relevance, faithfulness, cost, latency) - benchmark_kpis() computes all 8 TABLE II metrics including hallucination rate and weighted overall score (1–5) - Analytics always visible: zero-KPI fallback renders charts and table even before first benchmark run; empty-state annotation shown on Plotly charts - Benchmark persistence: results saved to data/benchmark_cache.json after run and loaded from cache on every app startup — run once, display permanently - Analytics table expanded to TABLE II format with Best and Unit columns data/r…900c573
-
Major upgrade: auto-metrics, Plotly charts, Qwen2.5, PDF support, expanded KB
Aditya- Add ROUGE-L and semantic similarity auto-metrics per answer card - Add Plotly bar charts for aggregate quality/speed comparison in Analytics section - Upgrade fine-tuned model from TinyLlama to Qwen/Qwen2.5-1.5B-Instruct - Update Colab notebook with Qwen2.5 model and ChatML prompt format - Add PDF ingestion to RAG pipeline (drop PDFs into data/pdfs/) - Add 5 new knowledge base docs: big_o_notation, heaps, sql_basics, typescript_basics, git_basics - Add data/reference_answers.json with 21 Q&A pairs for automated scoring - Add HuggingFace Spaces frontmatter and deploy guide to README - Add plotly, rouge-score, pypdf, langchain-huggingface, numpy to requirementsa52cc98
-
Add LLM vs RAG vs Fine-Tuning comparison project
Aditya4a3f117