grevy Claude Opus 4.7 commited on
Commit
15f4283
·
1 Parent(s): 6dc313e

Split into manager + 3 subagents + answer normalizer

Browse files

Architecture:
BasicAgent (manager)
├── web_research -> WebResearchAgent (Tavily + fetch_url)
├── file_analysis -> FileAnalysisAgent (PDF/XLSX/CSV/TXT/JSON)
├── media_analysis -> MediaAgent (VLM/Whisper/YouTube/video)
├── python_exec -> direct
├── calculator -> direct
└── AnswerNormalizer -> GAIA-style final formatting

agents.py: replace flat ResearchAgent with WebResearchAgent,
FileAnalysisAgent, MediaAgent (all via langchain.agents.create_agent
with topic-specific system prompts) and AnswerNormalizer (single LLM
call enforcing GAIA answer format).

tools.py: add read_text_file. Wrap each subagent as a @tool
(web_research / file_analysis / media_analysis). ALL_TOOLS now
exposes only the 5 manager-level tools; low-level tools stay
declared for the subagents to import.

app.py: BasicAgent gets a manager system prompt describing routing
rules, and answer() pipes the raw output through AnswerNormalizer
before returning.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019ryPUj8AtsFopnufGRSc4L

Files changed (3) hide show
  1. agents.py +139 -21
  2. app.py +31 -3
  3. tools.py +59 -23
agents.py CHANGED
@@ -1,40 +1,158 @@
1
  import os
2
 
3
- from langchain_groq import ChatGroq
4
  from langchain.agents import create_agent
 
 
 
5
 
 
 
 
 
 
 
6
 
7
- RESEARCH_SYSTEM_PROMPT = """You are a research specialist.
 
8
 
9
  For each question:
10
- 1. Mentally break it into sub-questions if it has multiple parts.
11
- 2. Use the search tool to find sources. When the fact is not obvious,
12
- cross-check across at least 2 independent sources before concluding.
13
- 3. Return a concise, factual answer. If the question asks for a specific
14
- value (number, name, date, list), give ONLY that value in the final
15
- answer. Otherwise, keep the answer under 3 sentences.
16
- 4. Never guess. If you cannot verify, say "unknown".
17
- """
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
18
 
 
 
 
 
 
 
 
 
 
19
 
20
- class ResearchAgent:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
21
  def __init__(self):
22
- llm = ChatGroq(
23
- model="openai/gpt-oss-120b",
24
- temperature=0.1,
25
- api_key=os.getenv("GROQ_API_KEY"),
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
26
  )
27
- from tools import search_tool # lazy: avoids circular import with tools.py
28
 
29
  self.graph = create_agent(
30
- llm,
31
- tools=[search_tool],
32
- system_prompt=RESEARCH_SYSTEM_PROMPT,
33
  )
34
 
35
  def __call__(self, question: str) -> str:
36
- return self.answer(question)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
37
 
38
- def answer(self, question: str) -> str:
39
  result = self.graph.invoke({"messages": [("user", question)]})
40
  return result["messages"][-1].content
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  import os
2
 
 
3
  from langchain.agents import create_agent
4
+ from langchain_core.messages import HumanMessage, SystemMessage
5
+ from langchain_groq import ChatGroq
6
+
7
 
8
+ def _make_llm(model: str = "openai/gpt-oss-120b", temperature: float = 0.1) -> ChatGroq:
9
+ return ChatGroq(
10
+ model=model,
11
+ temperature=temperature,
12
+ api_key=os.getenv("GROQ_API_KEY"),
13
+ )
14
 
15
+
16
+ WEB_RESEARCH_PROMPT = """You are a web research specialist.
17
 
18
  For each question:
19
+ 1. Break it into sub-questions if needed.
20
+ 2. Use `search_tool` to find candidate sources. When a search snippet is
21
+ ambiguous or truncated, call `fetch_url` on the most relevant hit to
22
+ read the actual page.
23
+ 3. Cross-check facts across at least 2 independent sources before concluding
24
+ when the answer is not obvious.
25
+ 4. Return a concise, factual answer. If the question asks for a specific
26
+ value (number, name, date, list), give ONLY that value. Otherwise, keep
27
+ the answer under 3 sentences.
28
+ 5. Never guess. If you cannot verify, say "unknown"."""
29
+
30
+
31
+ FILE_ANALYSIS_PROMPT = """You are a document analysis specialist.
32
+
33
+ You handle questions that reference a task_id, URL, or file path pointing to
34
+ a document (PDF, XLSX, CSV, TXT, JSON, MD).
35
+
36
+ Workflow:
37
+ 1. If given a task_id (no extension, no slash), call `download_task_file`
38
+ to fetch the file and get its local path.
39
+ 2. Route by extension:
40
+ - .pdf -> read_pdf
41
+ - .xlsx / .xls -> read_excel
42
+ - .csv -> read_csv
43
+ - anything else textual -> read_text_file
44
+ 3. Answer the user's question from the extracted content. Quote values
45
+ verbatim when possible. Do not speculate."""
46
+
47
+
48
+ MEDIA_ANALYSIS_PROMPT = """You are a media analysis specialist (image, audio, video).
49
+
50
+ You handle questions that reference a task_id, URL, or file path pointing to
51
+ media, or a YouTube URL.
52
 
53
+ Workflow:
54
+ 1. If given a task_id, call `download_task_file` first to get a local path.
55
+ 2. Route by content type:
56
+ - Image (.jpg/.png/.webp/...) -> analyze_image
57
+ - Audio (.mp3/.wav/.m4a/.flac/...) -> transcribe_audio, then reason on the text
58
+ - Video file or non-YouTube video URL -> analyze_video; if audio matters, also transcribe_audio on the same file
59
+ - YouTube URL -> try `youtube_transcript` first; if captions are missing or
60
+ the question is about visuals, also call `analyze_video`
61
+ 3. Give ONLY the answer the question asks for. No preamble."""
62
 
63
+
64
+ GAIA_NORMALIZER_PROMPT = """You are a strict answer normalizer for the GAIA benchmark.
65
+
66
+ You receive a QUESTION and a RAW answer produced by an agent. Return ONLY the
67
+ final answer in exactly the format the question requires:
68
+
69
+ - Numbers: digits only, no thousand separators, no units unless the question
70
+ explicitly asks for units. "3,141" -> "3141". "42 seconds" -> "42" unless
71
+ the question asked for a unit.
72
+ - Names / entities: just the name, no articles ("the"), no titles, no
73
+ explanatory prefix like "The answer is".
74
+ - Yes/No: exactly "Yes" or "No".
75
+ - Lists: comma-separated, in the order the question asks (alphabetical,
76
+ chronological, ...). No trailing "and".
77
+ - Dates: match the format the question asks for; if unspecified, use
78
+ YYYY-MM-DD.
79
+
80
+ Never explain, never apologize, never quote the answer.
81
+ Output ONLY the normalized answer, on a single line, nothing else."""
82
+
83
+
84
+ class WebResearchAgent:
85
  def __init__(self):
86
+ from tools import fetch_url, search_tool # lazy: avoids circular import
87
+
88
+ self.graph = create_agent(
89
+ _make_llm(),
90
+ tools=[search_tool, fetch_url],
91
+ system_prompt=WEB_RESEARCH_PROMPT,
92
+ )
93
+
94
+ def __call__(self, question: str) -> str:
95
+ result = self.graph.invoke({"messages": [("user", question)]})
96
+ return result["messages"][-1].content
97
+
98
+
99
+ class FileAnalysisAgent:
100
+ def __init__(self):
101
+ from tools import (
102
+ download_task_file,
103
+ read_csv,
104
+ read_excel,
105
+ read_pdf,
106
+ read_text_file,
107
  )
 
108
 
109
  self.graph = create_agent(
110
+ _make_llm(),
111
+ tools=[download_task_file, read_pdf, read_excel, read_csv, read_text_file],
112
+ system_prompt=FILE_ANALYSIS_PROMPT,
113
  )
114
 
115
  def __call__(self, question: str) -> str:
116
+ result = self.graph.invoke({"messages": [("user", question)]})
117
+ return result["messages"][-1].content
118
+
119
+
120
+ class MediaAgent:
121
+ def __init__(self):
122
+ from tools import (
123
+ analyze_image,
124
+ analyze_video,
125
+ download_task_file,
126
+ transcribe_audio,
127
+ youtube_transcript,
128
+ )
129
+
130
+ self.graph = create_agent(
131
+ _make_llm(),
132
+ tools=[
133
+ download_task_file,
134
+ analyze_image,
135
+ analyze_video,
136
+ transcribe_audio,
137
+ youtube_transcript,
138
+ ],
139
+ system_prompt=MEDIA_ANALYSIS_PROMPT,
140
+ )
141
 
142
+ def __call__(self, question: str) -> str:
143
  result = self.graph.invoke({"messages": [("user", question)]})
144
  return result["messages"][-1].content
145
+
146
+
147
+ class AnswerNormalizer:
148
+ def __init__(self):
149
+ self.llm = _make_llm(temperature=0.0)
150
+
151
+ def __call__(self, question: str, raw_answer: str) -> str:
152
+ response = self.llm.invoke([
153
+ SystemMessage(content=GAIA_NORMALIZER_PROMPT),
154
+ HumanMessage(
155
+ content=f"QUESTION:\n{question}\n\nRAW ANSWER:\n{raw_answer}"
156
+ ),
157
+ ])
158
+ return response.content.strip()
app.py CHANGED
@@ -8,11 +8,34 @@ from langchain_groq import ChatGroq
8
  from typing import TypedDict, Annotated
9
  from langgraph.graph.message import add_messages
10
  from langgraph.graph import START, StateGraph
11
- from langchain_core.messages import AnyMessage, HumanMessage
12
  from tools import ALL_TOOLS
 
13
  from langgraph.prebuilt import tools_condition
14
  from langgraph.prebuilt import ToolNode
15
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
16
  # (Keep Constants as is)
17
  # --- Constants ---
18
  DEFAULT_API_URL = "https://agents-course-unit4-scoring.hf.space"
@@ -45,6 +68,7 @@ class BasicAgent:
45
  )
46
  self.chat = self.chat.bind_tools(self.tools)
47
  self.agent = self.__init__agent__()
 
48
 
49
  def __call__(self, question: str) -> str:
50
  return self.answer(question)
@@ -59,14 +83,18 @@ class BasicAgent:
59
  return builder.compile()
60
 
61
  def assitant(self, state: AgentState):
 
 
 
62
  return {
63
- "messages": [self.chat.invoke(state["messages"])],
64
  }
65
 
66
  def answer(self, question):
67
  messages = [HumanMessage(content=question)]
68
  response = self.agent.invoke({"messages": messages})
69
- return response["messages"][-1].content
 
70
 
71
 
72
  def run_and_submit_all(profile: gr.OAuthProfile | None):
 
8
  from typing import TypedDict, Annotated
9
  from langgraph.graph.message import add_messages
10
  from langgraph.graph import START, StateGraph
11
+ from langchain_core.messages import AnyMessage, HumanMessage, SystemMessage
12
  from tools import ALL_TOOLS
13
+ from agents import AnswerNormalizer
14
  from langgraph.prebuilt import tools_condition
15
  from langgraph.prebuilt import ToolNode
16
 
17
+
18
+ MANAGER_SYSTEM_PROMPT = """You are a QA agent for the GAIA benchmark.
19
+
20
+ You have five tools:
21
+ - web_research(question): factual/current world knowledge (people, dates, events).
22
+ - file_analysis(question): question about an attached document
23
+ (pdf/xlsx/csv/txt/json). Include the task_id or URL in your question.
24
+ - media_analysis(question): question about an image, audio, or video (also
25
+ YouTube URLs). Include the task_id or URL in your question.
26
+ - python_exec(code): run Python for computation, list ops, date math,
27
+ string parsing. You must print() the result.
28
+ - calculator(expression): plain math expressions.
29
+
30
+ Rules:
31
+ 1. When a task has an attached file (mentioned by task_id), route to
32
+ file_analysis or media_analysis with the task_id embedded in the question.
33
+ 2. Ask each specialist ONE precise question at a time.
34
+ 3. Chain tools when needed (e.g. web_research to find a number, then
35
+ calculator to combine).
36
+ 4. Final answer: give ONLY the value the question asks for. No preamble,
37
+ no explanation."""
38
+
39
  # (Keep Constants as is)
40
  # --- Constants ---
41
  DEFAULT_API_URL = "https://agents-course-unit4-scoring.hf.space"
 
68
  )
69
  self.chat = self.chat.bind_tools(self.tools)
70
  self.agent = self.__init__agent__()
71
+ self.normalizer = AnswerNormalizer()
72
 
73
  def __call__(self, question: str) -> str:
74
  return self.answer(question)
 
83
  return builder.compile()
84
 
85
  def assitant(self, state: AgentState):
86
+ messages = state["messages"]
87
+ if not messages or not isinstance(messages[0], SystemMessage):
88
+ messages = [SystemMessage(content=MANAGER_SYSTEM_PROMPT)] + list(messages)
89
  return {
90
+ "messages": [self.chat.invoke(messages)],
91
  }
92
 
93
  def answer(self, question):
94
  messages = [HumanMessage(content=question)]
95
  response = self.agent.invoke({"messages": messages})
96
+ raw = response["messages"][-1].content
97
+ return self.normalizer(question, raw)
98
 
99
 
100
  def run_and_submit_all(profile: gr.OAuthProfile | None):
tools.py CHANGED
@@ -13,7 +13,7 @@ from langchain_core.tools import tool
13
  from langchain_groq import ChatGroq
14
  from tavily import TavilyClient
15
 
16
- from agents import ResearchAgent
17
 
18
  DEFAULT_API_URL = "https://agents-course-unit4-scoring.hf.space"
19
 
@@ -150,6 +150,16 @@ def read_csv(path: str) -> str:
150
  return pd.read_csv(path).to_string()
151
 
152
 
 
 
 
 
 
 
 
 
 
 
153
  @tool
154
  def python_exec(code: str) -> str:
155
  """Execute Python code in a subprocess and return stdout (and stderr).
@@ -368,37 +378,63 @@ def youtube_transcript(url_or_id: str) -> str:
368
  return " ".join(e["text"] for e in entries)
369
 
370
 
371
- research_agent = ResearchAgent()
 
 
372
 
373
 
374
  @tool
375
- def research(request: str) -> str:
376
- """Answer factual questions using web research.
377
 
378
- Use this when the user asks for information that requires up-to-date or
379
- external knowledge — names, dates, events, statistics, biographies, or
380
- anything that depends on the current state of the world. Handles query
381
- decomposition, web search, cross-checking and synthesis.
 
 
 
 
 
 
 
 
 
 
382
 
383
- Input: Natural language research question (e.g., 'who won the 2024 Nobel
384
- Prize in Physics?')
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
385
  """
386
- return research_agent(request)
387
 
388
 
389
  ALL_TOOLS = [
390
- search_tool,
391
- fetch_url,
392
- wikipedia_lookup,
393
- download_task_file,
394
- read_pdf,
395
- read_excel,
396
- read_csv,
397
  python_exec,
398
  calculator,
399
- analyze_image,
400
- analyze_video,
401
- transcribe_audio,
402
- youtube_transcript,
403
- search_tool,
404
  ]
 
13
  from langchain_groq import ChatGroq
14
  from tavily import TavilyClient
15
 
16
+ from agents import FileAnalysisAgent, MediaAgent, WebResearchAgent
17
 
18
  DEFAULT_API_URL = "https://agents-course-unit4-scoring.hf.space"
19
 
 
150
  return pd.read_csv(path).to_string()
151
 
152
 
153
+ @tool
154
+ def read_text_file(path: str) -> str:
155
+ """Read a plain-text file (txt, json, md, py, ...) and return its contents.
156
+
157
+ Args:
158
+ path: Local file path.
159
+ """
160
+ return Path(path).read_text(encoding="utf-8", errors="replace")
161
+
162
+
163
  @tool
164
  def python_exec(code: str) -> str:
165
  """Execute Python code in a subprocess and return stdout (and stderr).
 
378
  return " ".join(e["text"] for e in entries)
379
 
380
 
381
+ _web_agent = WebResearchAgent()
382
+ _file_agent = FileAnalysisAgent()
383
+ _media_agent = MediaAgent()
384
 
385
 
386
  @tool
387
+ def web_research(question: str) -> str:
388
+ """Answer a factual question using web search and page reading.
389
 
390
+ Delegates to a specialist that searches (Tavily), visits pages when
391
+ snippets are insufficient, and cross-checks. Use for anything requiring
392
+ up-to-date or external world knowledge: people, dates, events, statistics,
393
+ biographies, current affairs.
394
+
395
+ Args:
396
+ question: Natural-language research question.
397
+ """
398
+ return _web_agent(question)
399
+
400
+
401
+ @tool
402
+ def file_analysis(question: str) -> str:
403
+ """Answer a question about an attached document (PDF, XLSX, CSV, TXT, JSON).
404
 
405
+ Delegates to a specialist that downloads the file (given a task_id, URL,
406
+ or local path), routes to the right reader by extension, and answers.
407
+
408
+ Include the file reference in your question, e.g.:
409
+ "Given task_id abc-123, what is the total revenue in Q3?"
410
+
411
+ Args:
412
+ question: Natural-language question mentioning the task_id / URL / path.
413
+ """
414
+ return _file_agent(question)
415
+
416
+
417
+ @tool
418
+ def media_analysis(question: str) -> str:
419
+ """Answer a question about media (image, audio, or video).
420
+
421
+ Delegates to a specialist that downloads if needed, then routes to VLM /
422
+ Whisper / video frame sampling / YouTube captions.
423
+
424
+ Include the media reference in your question, e.g.:
425
+ "For task_id abc-123, what animal is shown?"
426
+ "Watch https://youtube.com/watch?v=... and tell me the speaker's main claim."
427
+
428
+ Args:
429
+ question: Natural-language question mentioning the task_id / URL / path.
430
  """
431
+ return _media_agent(question)
432
 
433
 
434
  ALL_TOOLS = [
435
+ web_research,
436
+ file_analysis,
437
+ media_analysis,
 
 
 
 
438
  python_exec,
439
  calculator,
 
 
 
 
 
440
  ]