Dhruvpandey1476 commited on
Commit
799cd4d
·
1 Parent(s): 9d7f196

FIX FAIRNESS: Remove strict 'ONLY use context' prompt from BasicRAG to allow parametric knowledge fallback

Browse files

- Changed system prompt from restrictive grounding ('Answer based ONLY on context') to flexible ('Use context when available, leverage knowledge if insufficient')
- This ensures BasicRAG scores fairly - previously it would refuse to answer and get 2.0 scores when FAISS didn't find documents
- Now both BasicRAG and GraphRAG have fair access to knowledge (FAISS + parametric), making comparison about retrieval strategy not knowledge access
- Fixes bias concern where BasicRAG judge scores (3.32 avg) appeared unfairly low vs GraphRAG (7.88 avg)

Files changed (1) hide show
  1. backend/rag/basic_rag.py +4 -3
backend/rag/basic_rag.py CHANGED
@@ -225,15 +225,16 @@ class BasicRAG:
225
 
226
  # 4. Build prompt
227
  system_prompt = (
228
- "You are an assistant grounded in provided documents. Answer based ONLY on the context. "
229
- "Cite sources from the context. If context is insufficient, acknowledge it."
 
230
  )
231
  user_prompt = f"""Context from documents:
232
  {context}
233
 
234
  Question: {question}
235
 
236
- Based on the context above, provide a comprehensive answer with citations:"""
237
 
238
  # 5. Call Gemini via shared client (accurate token counts)
239
  result = gemini_generate(
 
225
 
226
  # 4. Build prompt
227
  system_prompt = (
228
+ "You are an expert assistant. Use the provided context when available. "
229
+ "If context is insufficient, leverage your knowledge to provide a helpful answer. "
230
+ "Always prioritize context when it's available."
231
  )
232
  user_prompt = f"""Context from documents:
233
  {context}
234
 
235
  Question: {question}
236
 
237
+ Provide a comprehensive answer using the context if available, or general knowledge:"""
238
 
239
  # 5. Call Gemini via shared client (accurate token counts)
240
  result = gemini_generate(