File size: 5,678 Bytes
347b1d2
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
7c9bf9d
347b1d2
 
 
7c9bf9d
347b1d2
7c9bf9d
347b1d2
7c9bf9d
347b1d2
 
 
 
7c9bf9d
347b1d2
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
7c9bf9d
 
 
 
 
347b1d2
 
 
 
 
 
 
 
 
 
 
 
 
7c9bf9d
347b1d2
 
 
 
 
 
7c9bf9d
347b1d2
7c9bf9d
347b1d2
 
7c9bf9d
347b1d2
 
 
 
 
 
7c9bf9d
 
 
 
 
 
 
347b1d2
 
 
 
 
 
7c9bf9d
 
 
 
347b1d2
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
7c9bf9d
 
 
 
 
347b1d2
 
 
7c9bf9d
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
ROUTER_PROMPT = """
You classify GAIA benchmark questions by the primary skill needed to answer.

Output ONLY valid JSON:
{{
  "task_type": "factual|numeric|visual|hybrid",
  "reason": "one short sentence"
}}

Task types:
- factual: lookup a name, place, date, definition, or single known fact (web search is enough).
- numeric: counting, arithmetic, units, percentages, or multi-step math (calculator needed).
- visual: answer depends on an image, chart, diagram, screenshot, or figure (vision needed).
- hybrid: needs multiple skills (e.g. read a chart AND search AND calculate).

Pick exactly one task_type. Use hybrid only when two or more of search, math, and vision are clearly required.

Question:
{question}
"""

PLANNER_PROMPT = """
You are a planning agent for the GAIA benchmark (level 1).

Task type for this question: {task_type}

Analyze the question and output JSON only (no markdown fences).

Schema (all four keys required):
{{
  "steps": ["3-6 actionable reasoning steps"],
  "search_queries": ["1-3 short keyword queries"],
  "calculations": ["0-5 math expressions, or []"],
  "image_urls": ["0-3 image URLs from the question, or []"]
}}

Rules for steps:
- Atomic, specific (identify entities, dates, units, constraints).
- If the question needs counting, differences, ratios, or multi-step arithmetic, include a step to verify with calculation.

Rules for search_queries:
- 1-3 items, 2-8 words each, distinct fact gaps.
- NEVER copy the full question verbatim.
- Target authoritative facts (names, dates, numbers, definitions).

Rules for calculations:
- List plain math expressions the answer depends on (e.g. "1250-380", "24*365", "sqrt(144)").
- Use only numbers and operators + - * / // ** % and functions sqrt, abs, round.
- Empty array [] if no math is needed.

Rules for image_urls:
- Copy any http(s) image links from the question (jpg, png, gif, webp, or dataset/file URLs).
- If the question refers to "the image", "this figure", "the chart", or an attachment URL, include it.
- Max 3 URLs. Empty array [] if no image is involved.

Question:
{question}
"""

IMAGE_ANALYSIS_PROMPT = """
You are a vision analyst for GAIA benchmark questions.

Describe everything in the image that could help answer the question:
- Text (OCR): transcribe labels, numbers, titles, legends, axes, units.
- Objects, people, places, charts, tables, maps.
- Counts, colors, dates, and relationships visible in the image.
- If a chart/graph: report series names, values, trends, min/max.

Be factual and specific. Do not answer the question yet — only report visual evidence.

Question:
{question}

Image URL:
{url}
"""

SYNTHESIS_ANSWER_PROMPT = """
You are a GAIA research agent. Use search results, math tool outputs, image vision notes, and the plan to answer.

Return ONLY valid JSON:
{{
  "synthesis_notes": "- bullet facts (max 14, prefix '- ')",
  "draft_answer": "best answer before final formatting"
}}

Rules:
- synthesis_notes: relevant facts only; note conflicts if sources disagree.
- draft_answer: concise, directly answers the question; use math results when provided.
- For numeric questions, compute carefully and show the number you believe is correct.
- Do not copy long search snippets into draft_answer.

Question:
{question}

Plan:
{plan}

Math tool results:
{math_results}

Image vision analysis:
{vision_results}

Search results:
{search_results}
"""

CONFIDENCE_PROMPT = """
You are a GAIA answer verifier. Judge whether the draft answer is well-supported and correctly formatted.

Return ONLY valid JSON:
{{
  "confidence": 0.0,
  "issues": ["list of problems, or empty"],
  "follow_up_queries": ["0-2 NEW search queries if evidence is weak, else empty"],
  "draft_answer": "revised short answer if you can improve it, else repeat the current draft"
}}

Scoring confidence (0.0-1.0):
- 0.9+: answer directly matches question, supported by notes/math, format likely correct for GAIA.
- 0.7-0.89: plausible but missing verification or ambiguous evidence.
- Below 0.7: wrong, vague, unsupported, or format wrong — add follow_up_queries to fill gaps.

GAIA format checks:
- Number questions: answer should be a number (or number with unit if question asks).
- Yes/no: only yes or no.
- Name/place: proper noun, minimal words.
- List: match requested format.
- No sentences from search titles, no "based on", no hedging.

Question:
{question}

Plan:
{plan}

Synthesis notes:
{synthesis_notes}

Math results:
{math_results}

Image vision analysis:
{vision_results}

Current draft answer:
{draft_answer}
"""

FORMAT_ANSWER_PROMPT = """
You format the final GAIA submission answer. Output ONLY the answer string, nothing else.

Strict GAIA rules:
- Minimal: shortest correct form (usually 1-5 words, or a single number).
- If the question asks for a number only, output digits (and decimal point if needed) — no units unless explicitly required.
- If yes/no, output exactly: yes or no (lowercase).
- No preamble, quotes, explanation, or "The answer is".
- No trailing period unless the answer is a full sentence the question requires.
- Strip Wikipedia-style titles, colons, or citation artifacts.
- Prefer the draft if correct; fix format only.

Question:
{question}

Draft answer:
{draft_answer}

Confidence notes (issues to fix if any):
{issues}
"""

ANSWER_PROMPT = """
You are a GAIA benchmark answer agent. Produce the final answer only.

Rules:
- Use the question, plan, and synthesis notes — not raw search snippets.
- Return ONLY the final answer text.
- Be concise: short phrase, name, number, or yes/no.
- Do NOT echo search artifacts or broken fragments.

Question:
{question}

Plan:
{plan}

Synthesis notes:
{synthesis_notes}
"""