ishan-25 commited on
Commit
2746075
Β·
verified Β·
1 Parent(s): d88114a

Tuned build_prompt further

Browse files
Files changed (1) hide show
  1. agents/scorer.py +9 -8
agents/scorer.py CHANGED
@@ -2,7 +2,6 @@
2
  agents/scorer.py
3
  ────────────────────────────────────────────────────────────────────────────
4
  ScorerAgent: Semantic, context-aware answer scoring β€” implements Agenda Item #4.
5
- Upgraded to leverage Qwen2.5-32B-Instruct for advanced conceptual grading.
6
  """
7
 
8
  import json
@@ -58,30 +57,32 @@ class ScorerAgent:
58
  "star_hint": star_hint,
59
  }
60
 
61
- # ── Advanced Prompt Builder ───────────────────────────────────────────────
 
62
  def _build_prompt(self, answer: str, question: str, industry: str,
63
  role_level: str, keywords: list) -> str:
64
  keywords_str = ", ".join(keywords) if keywords else "General Industry Domain"
65
 
66
- return f"""You are an elite corporate technical interviewer. Evaluate the candidate's response using the STAR (Situation, Task, Action, Result) methodology.
67
 
68
- Interview Question Asked: {question}
69
  Candidate's Answer: {answer}
70
  Target Domain Keywords: {keywords_str}
71
 
72
- CRITICAL KEYWORD EVALUATION RULE:
73
- Do NOT run a rigid character match. If the candidate naturally explains the underlying concept, uses functional synonyms, describes the operational setup, or discusses the exact methodology related to a keyword, classify it as a SUCCESSFUL semantic match.
 
 
74
 
75
  Provide your evaluation in a raw, clean JSON format. Do not include any markdown block formatting (like ```json). Use exactly this structure:
76
  {{
77
- "score": [Integer from 1 to 10 based on relevance and quality],
78
  "hit_keywords": [Array of string keywords from the Target list that were covered or conceptually explained],
79
  "missed_keywords": [Array of string keywords from the Target list completely missing in both text and concept],
80
  "strength": "One clear sentence highlighting what the candidate did well.",
81
  "weakness": "One clear sentence identifying the biggest structural or domain gap.",
82
  "fix": "One clear sentence containing actionable advice to improve the response."
83
  }}"""
84
-
85
  # ── Robust Safe JSON Parser ───────────────────────────────────────────────
86
  def _parse_json_response(self, raw_text: str, original_keywords: list) -> dict:
87
  """Safely parses LLM output and provides bulletproof defaults if parsing fails."""
 
2
  agents/scorer.py
3
  ────────────────────────────────────────────────────────────────────────────
4
  ScorerAgent: Semantic, context-aware answer scoring β€” implements Agenda Item #4.
 
5
  """
6
 
7
  import json
 
57
  "star_hint": star_hint,
58
  }
59
 
60
+
61
+ # ── Advanced Prompt Builder ───────────────────────────────────────────────
62
  def _build_prompt(self, answer: str, question: str, industry: str,
63
  role_level: str, keywords: list) -> str:
64
  keywords_str = ", ".join(keywords) if keywords else "General Industry Domain"
65
 
66
+ return f"""You are an elite technical interviewer. Evaluate the candidate's response using the STAR methodology.
67
 
68
+ Interview Question: {question}
69
  Candidate's Answer: {answer}
70
  Target Domain Keywords: {keywords_str}
71
 
72
+ 🚨 CRITICAL KEYWORD EVALUATION RULES (READ CAREFULLY):
73
+ 1. Do NOT look for exact text characters or verbatim matches.
74
+ 2. If the candidate describes the concept of a keyword, mentions a synonym, or explains the real-world task/workflow related to that keyword, you MUST count it as a SUCCESSFUL match.
75
+ 3. For example: If the keyword is "Database", and they discuss "storing text files" or "SQL optimization", that is a successful match. Be highly generous with conceptual matches!
76
 
77
  Provide your evaluation in a raw, clean JSON format. Do not include any markdown block formatting (like ```json). Use exactly this structure:
78
  {{
79
+ "score": [Integer from 1 to 10 based on content quality],
80
  "hit_keywords": [Array of string keywords from the Target list that were covered or conceptually explained],
81
  "missed_keywords": [Array of string keywords from the Target list completely missing in both text and concept],
82
  "strength": "One clear sentence highlighting what the candidate did well.",
83
  "weakness": "One clear sentence identifying the biggest structural or domain gap.",
84
  "fix": "One clear sentence containing actionable advice to improve the response."
85
  }}"""
 
86
  # ── Robust Safe JSON Parser ───────────────────────────────────────────────
87
  def _parse_json_response(self, raw_text: str, original_keywords: list) -> dict:
88
  """Safely parses LLM output and provides bulletproof defaults if parsing fails."""