jm-vis commited on
Commit
5bfaa58
·
1 Parent(s): e22b104

Clean up Space: kept only app.py, requirements.txt, and README.md

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. README_original.md +0 -294
  2. SUPABASE_TEST_README.md +0 -104
  3. agent/__init__.py +0 -22
  4. agent/__pycache__/__init__.cpython-312.pyc +0 -0
  5. agent/__pycache__/__init__.cpython-313.pyc +0 -0
  6. agent/__pycache__/agent.cpython-312.pyc +0 -0
  7. agent/__pycache__/agent.cpython-313.pyc +0 -0
  8. agent/__pycache__/assessment_analyzer.cpython-313.pyc +0 -0
  9. agent/__pycache__/config.cpython-312.pyc +0 -0
  10. agent/__pycache__/config.cpython-313.pyc +0 -0
  11. agent/__pycache__/graph.cpython-312.pyc +0 -0
  12. agent/__pycache__/graph.cpython-313.pyc +0 -0
  13. agent/__pycache__/tool_registry.cpython-313.pyc +0 -0
  14. agent/agent.py +0 -291
  15. agent/assessment_analyzer.py +0 -306
  16. agent/config.py +0 -187
  17. agent/graph.py +0 -703
  18. agent/tool_registry.py +0 -732
  19. app.py +346 -346
  20. app_demo_mode_patch.py +0 -104
  21. app_fixed.py +0 -367
  22. app_no_oauth.py +0 -226
  23. backup_tests/minimal_supabase_test.py +0 -75
  24. backup_tests/test_supabase_tables_final.py +0 -167
  25. backup_tests/test_supabase_tables_fix.py +0 -276
  26. backup_tests/test_supabase_tables_production.py +0 -481
  27. build_error_fix_guide.md +0 -78
  28. check_app.py +0 -42
  29. cleanup_results.txt +0 -340
  30. complete_deployment_guide.md +0 -129
  31. complete_deployment_solution.md +0 -183
  32. create_env_and_test.py +0 -77
  33. debug_test.py +0 -113
  34. dependency_conflict_fix_guide.md +0 -84
  35. deploy_to_hf.py +0 -392
  36. deploy_to_huggingface.py +0 -333
  37. deploy_to_huggingface_credential.py +0 -320
  38. deploy_to_huggingface_dotenv.py +0 -227
  39. deploy_to_huggingface_env.py +0 -203
  40. deploy_with_token.py +0 -176
  41. deployment_checklist.md +0 -83
  42. deployment_readiness_report.json +0 -38
  43. deployment_readiness_report.md +0 -97
  44. deployment_verification.py +0 -327
  45. docs/all_apis_integration_testing.md +0 -170
  46. docs/configuration.md +0 -357
  47. docs/connection_verification.md +0 -114
  48. docs/deployment.md +0 -303
  49. docs/duckduckgo_integration.md +0 -156
  50. docs/enhanced_web_search.md +0 -299
README_original.md DELETED
@@ -1,294 +0,0 @@
1
- # GAIA Agent - Hugging Face Agents Course Final Project
2
-
3
- ## Project Overview
4
-
5
- This project implements an intelligent agent capable of answering questions from the GAIA (Grounded AI Alignment) benchmark as part of the Hugging Face Agents Course final assignment. The agent leverages LangChain/LangGraph frameworks and various tools to solve complex, multi-step reasoning tasks.
6
-
7
- ## What is GAIA?
8
-
9
- GAIA is a benchmark designed to evaluate AI assistants on real-world tasks that require a combination of core capabilities:
10
- - Reasoning across multiple steps
11
- - Multimodal understanding (text, images, etc.)
12
- - Web browsing and information retrieval
13
- - Proficient tool use
14
-
15
- The benchmark consists of 466 carefully curated questions that are conceptually simple for humans (~92% success rate) but challenging for current AI systems (GPT-4 with plugins: ~15% success rate).
16
-
17
- ## Challenge Requirements
18
-
19
- - **Task**: Create an agent that answers questions from a subset of GAIA (20 questions from level 1)
20
- - **Success Criteria**: Score at least 30% on the benchmark
21
- - **Evaluation Method**: Exact match comparison with ground truth answers
22
- - **Response Format**: Answers should be concise (number, few words, or comma-separated list)
23
-
24
- ## Technical Architecture
25
-
26
- Our implementation uses:
27
-
28
- 1. **LangChain/LangGraph**: For creating a flexible, modular agent architecture
29
- 2. **Supabase**: For memory management and persistence
30
- 3. **Web Tools**: Integration with search engines (DuckDuckGo, Serper), academic resources (arXiv), and AI assistants (Perplexity)
31
- 4. **Gradio**: For the user interface and submission process
32
-
33
- ## Project Structure
34
-
35
- ```
36
- ├── README.md # Project documentation
37
- ├── app.py # Main application file with Gradio interface
38
- ├── requirements.txt # Project dependencies
39
- ├── agent/ # Agent implementation
40
- │ ├── __init__.py # Package initialization
41
- │ ├── agent.py # Main agent class
42
- │ ├── config.py # Configuration settings
43
- │ ├── graph.py # LangGraph workflow definition
44
- │ └── tool_registry.py # Tool registration and management
45
- ├── tools/ # Tool implementations
46
- │ ├── __init__.py # Package initialization
47
- │ ├── web_tools.py # Web browsing and search tools
48
- │ ├── reasoning_tools.py # Reasoning and analysis tools
49
- │ ├── multimodal_tools.py # Tools for handling images and other media
50
- │ ├── arxiv_tool.py # Tools for searching academic papers
51
- │ └── perplexity_tool.py # Integration with Perplexity AI
52
- ├── memory/ # Memory management
53
- │ ├── __init__.py # Package initialization
54
- │ ├── supabase_memory.py # Supabase integration for memory
55
- │ ├── supabase_memory_patch.py # Patches for Supabase memory
56
- │ └── simplified_memory_fix.py # Simplified memory implementation
57
- └── utils/ # Utility functions
58
- ├── __init__.py # Package initialization
59
- ├── parsing.py # Functions for parsing questions and answers
60
- └── formatting.py # Formatting utilities
61
- ```
62
-
63
- ## Implementation Approach
64
-
65
- Our agent follows a multi-step process to answer GAIA questions:
66
-
67
- 1. **Question Analysis**: Parse and understand the question, identifying required tools and information sources
68
- 2. **Planning**: Develop a step-by-step plan to answer the question
69
- 3. **Tool Execution**: Use appropriate tools in the correct sequence
70
- 4. **Answer Formulation**: Extract and format the precise answer from the results
71
- 5. **Verification**: Validate the answer format and content before submission
72
-
73
- ## Key Features
74
-
75
- - **Modular Architecture**: Easily extensible with new tools and capabilities
76
- - **Robust Error Handling**: Graceful recovery from tool failures or unexpected results
77
- - **Efficient Tool Selection**: Smart selection of the most appropriate tools for each question
78
- - **Answer Precision**: Focus on extracting exact answers that match the ground truth format
79
- - **Memory Integration**: Persistent memory across sessions using Supabase
80
-
81
- ## Getting Started
82
-
83
- ### Prerequisites
84
-
85
- - Python 3.8 or higher
86
- - OpenAI API key
87
- - Supabase account and API key (for memory features)
88
- - Optional: Serper API key (for enhanced search)
89
- - Optional: Perplexity API key (for enhanced reasoning)
90
-
91
- ### Installation
92
-
93
- 1. Clone this repository
94
- 2. Install dependencies:
95
- ```bash
96
- pip install -r requirements.txt
97
- ```
98
-
99
- > **Note**: If you encounter issues with the `langchain_openai` dependency, make sure you have the latest version installed:
100
- > ```bash
101
- > pip install --upgrade langchain_openai
102
- > ```
103
-
104
- 3. Set up environment variables (create a `.env` file in the project root):
105
- ```
106
- OPENAI_API_KEY=your_openai_api_key
107
- SUPABASE_KEY=your_supabase_api_key
108
- SERPER_API_KEY=your_serper_api_key (optional)
109
- PERPLEXITY_API_KEY=your_perplexity_api_key (optional)
110
- ```
111
-
112
- 4. Run the application:
113
- ```bash
114
- python app.py
115
- ```
116
-
117
- 5. Log in with your Hugging Face account
118
- 6. Click "Run Evaluation & Submit All Answers" to test the agent
119
-
120
- ## Testing
121
-
122
- The project includes comprehensive unit tests for all components:
123
-
124
- - **Agent Tests**: Test the GAIAAgent class and its workflow
125
- - **Tools Tests**: Test the web, reasoning, and multimodal tools
126
- - **Memory Tests**: Test the memory components and Supabase integration
127
- - **Utils Tests**: Test utility functions for parsing and formatting
128
- - **Integration Tests**: Test integration with external APIs like OpenAI and Perplexity
129
-
130
- To run all tests:
131
- ```bash
132
- python -m unittest discover tests
133
- ```
134
-
135
- To run specific test files:
136
- ```bash
137
- python -m unittest tests.test_agent
138
- python -m unittest tests.test_tools
139
- python -m unittest tests.test_memory
140
- python -m unittest tests.test_utils
141
- ```
142
-
143
- ### OpenAI Integration Testing
144
-
145
- The project includes specialized tests for OpenAI integration with newer models:
146
-
147
- ```bash
148
- # Test OpenAI integration with newer models
149
- python tests/test_openai_integration.py
150
-
151
- # Test OpenAI integration with GAIA agent
152
- python tests/test_openai_integration.py --test-agent
153
-
154
- # Test GAIA agent with multiple AI providers (OpenAI and Perplexity)
155
- python tests/test_gaia_multi_provider.py
156
- ```
157
-
158
- For convenience, you can run all OpenAI integration tests at once:
159
-
160
- ```bash
161
- # Run all OpenAI integration tests
162
- python tests/run_openai_tests.py
163
-
164
- # Run all tests with verbose logging
165
- python tests/run_openai_tests.py --verbose
166
-
167
- # Run all tests and save results to a custom directory
168
- python tests/run_openai_tests.py --output my_test_results
169
-
170
- # Stop running tests after the first failure
171
- python tests/run_openai_tests.py --stop-on-failure
172
- ```
173
-
174
- These tests verify that:
175
- - The OpenAI API connection works correctly
176
- - Newer models like GPT-4o can be used successfully
177
- - The GAIA agent can use both OpenAI and Perplexity tools together
178
-
179
- ### API Integration Testing
180
-
181
- To test all API integrations together:
182
-
183
- ```bash
184
- # Run all API integration tests
185
- python tests/run_all_apis_integration_test.py
186
-
187
- # Run with verbose logging
188
- python tests/run_all_apis_integration_test.py --verbose
189
-
190
- # Specify a custom output file
191
- python tests/run_all_apis_integration_test.py --report-file my_report.json
192
- ```
193
-
194
- ### Supabase Integration Testing
195
-
196
- To test the Supabase memory integration:
197
-
198
- ```bash
199
- # Run all Supabase integration tests
200
- python tests/run_supabase_integration_tests.py
201
-
202
- # Run with verbose logging
203
- python tests/run_supabase_integration_tests.py --verbose
204
-
205
- # Specify a custom output directory
206
- python tests/run_supabase_integration_tests.py --output-dir custom_docs
207
- ```
208
-
209
- ## Evaluation
210
-
211
- The agent is evaluated on its ability to provide exact matches to the ground truth answers for the GAIA benchmark questions. The submission process includes:
212
-
213
- - Fetching questions from the course API
214
- - Running the agent on each question
215
- - Submitting answers for scoring
216
- - Viewing results on the leaderboard
217
-
218
- ## Benchmark Testing
219
-
220
- The project includes a benchmark testing framework to evaluate the agent's performance:
221
-
222
- ```bash
223
- # Run benchmark tests with default settings
224
- python -m tests.benchmark --output benchmark_results.json
225
-
226
- # Run benchmark tests with custom questions
227
- python -m tests.benchmark --questions custom_questions.json --output benchmark_results.json
228
-
229
- # Run benchmark tests with a specific number of questions
230
- python -m tests.benchmark --max-questions 5 --output benchmark_results.json
231
- ```
232
-
233
- The benchmark testing framework:
234
- - Tests the agent against sample questions
235
- - Measures performance metrics (accuracy, latency, resource usage)
236
- - Generates detailed reports of benchmark results
237
- - Allows comparison of different agent configurations
238
-
239
- ## Troubleshooting
240
-
241
- ### Common Issues
242
-
243
- #### 1. langchain_openai Dependency Issues
244
-
245
- If you encounter errors related to the `langchain_openai` package, such as:
246
- - `ImportError: No module named 'langchain_openai'`
247
- - `AttributeError: module 'langchain_openai' has no attribute 'ChatOpenAI'`
248
-
249
- Try the following solutions:
250
- - Ensure you have the latest version installed: `pip install --upgrade langchain_openai`
251
- - Make sure you're using the correct import: `from langchain_openai import ChatOpenAI`
252
- - Check for conflicts with older langchain versions: `pip uninstall langchain-openai` (note the hyphen)
253
-
254
- #### 2. API Connection Issues
255
-
256
- If you're having trouble connecting to external APIs:
257
- - Verify that your API keys are correctly set in the environment variables
258
- - Check that the API service is not experiencing downtime
259
- - Ensure you have a stable internet connection
260
- - Check for rate limiting or quota issues
261
-
262
- #### 3. Supabase Connection Issues
263
-
264
- If you're having trouble with Supabase:
265
- - Verify that your Supabase URL and API key are correctly set
266
- - Ensure that the required tables exist in your Supabase project
267
- - Check that your API key has the necessary permissions
268
- - Verify that your Supabase project is active
269
-
270
- #### 4. Memory Issues
271
-
272
- If you're experiencing issues with the agent's memory:
273
- - Check that memory is enabled in the configuration
274
- - Verify that the table names are correctly set
275
- - Ensure that the TTL and cache size settings are appropriate
276
- - Check for errors in the Supabase logs
277
-
278
- ## Documentation
279
-
280
- Comprehensive documentation is available in the `docs/` directory:
281
-
282
- - [Usage Examples](docs/examples.md): Examples of how to use the GAIA agent in different scenarios
283
- - [Configuration Guide](docs/configuration.md): Detailed information about configuration options
284
- - [Deployment Guide](docs/deployment.md): Instructions for deploying the agent in various environments
285
- - [OpenAI Integration Testing](docs/openai_integration_testing.md): Guide for testing OpenAI integration with newer models
286
- - [Connection Verification](docs/connection_verification.md): Guide for verifying connections to external services
287
- - [Supabase Memory Integration](docs/supabase_memory_integration.md): Guide for setting up and using Supabase memory
288
- - [All APIs Integration Testing](docs/all_apis_integration_testing.md): Guide for testing all API integrations together
289
- - [Multi-Search Integration](docs/multi_search_integration.md): Guide for using multiple search providers
290
- - [GAIA Memory Integration](docs/gaia_memory_integration.md): Guide for using memory in the GAIA agent
291
-
292
- ## License
293
-
294
- This project is part of the Hugging Face Agents Course and follows its licensing terms.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
SUPABASE_TEST_README.md DELETED
@@ -1,104 +0,0 @@
1
- # Standalone Supabase Connection Test
2
-
3
- This directory contains scripts for testing the Supabase connection without relying on any project modules.
4
-
5
- ## Main Script: `supabase_direct_test.py`
6
-
7
- This script tests the connection to Supabase, verifies access to all required tables, and performs basic CRUD operations on the test_connection table.
8
-
9
- ### Prerequisites
10
-
11
- - Python 3.6+
12
- - Required packages:
13
- - supabase
14
- - python-dotenv
15
-
16
- You can install the required packages with:
17
-
18
- ```bash
19
- pip install supabase python-dotenv
20
- ```
21
-
22
- ### Usage
23
-
24
- You can run the script in several ways:
25
-
26
- 1. Using environment variables:
27
-
28
- ```bash
29
- # Set environment variables
30
- export SUPABASE_URL=https://your-project-id.supabase.co
31
- export SUPABASE_KEY=your-supabase-api-key
32
-
33
- # Run the test
34
- python supabase_direct_test.py
35
- ```
36
-
37
- 2. Using command-line arguments:
38
-
39
- ```bash
40
- python supabase_direct_test.py --key your-supabase-api-key
41
- ```
42
-
43
- 3. Using a .env file:
44
-
45
- Create a `.env` file in the same directory with the following content:
46
-
47
- ```
48
- SUPABASE_URL=https://your-project-id.supabase.co
49
- SUPABASE_KEY=your-supabase-api-key
50
- ```
51
-
52
- Then run the script:
53
-
54
- ```bash
55
- python supabase_direct_test.py
56
- ```
57
-
58
- ### Options
59
-
60
- - `--verbose`: Enable verbose logging
61
- - `--url`: Specify the Supabase URL (overrides environment variable)
62
- - `--key`: Specify the Supabase API key (overrides environment variable)
63
-
64
- Example with verbose logging:
65
-
66
- ```bash
67
- python supabase_direct_test.py --verbose --key your-supabase-api-key
68
- ```
69
-
70
- ## Helper Scripts
71
-
72
- The repository also includes several helper scripts for troubleshooting:
73
-
74
- - `check_supabase_env.py`: Checks if the required Supabase environment variables are set
75
- - `supabase_import_test.py`: Tests importing the Supabase library
76
- - `supabase_init_test.py`: Tests initializing the Supabase client
77
- - `supabase_minimal_test.py`: Minimal test for Supabase connection
78
- - `create_env_and_test.py`: Creates a .env file and tests the connection
79
-
80
- ## Tables Tested
81
-
82
- The script verifies connection to the following tables:
83
-
84
- 1. `gaia_memory`: Main memory table
85
- 2. `gaia_memory_conversations`: Conversation history table
86
- 3. `gaia_memory_cache`: Cache table
87
- 4. `gaia_memory_working`: Working memory table
88
- 5. `test_connection`: Test table for connection verification
89
-
90
- If any of these tables don't exist, you can create them by running the SQL statements in `supabase_tables.sql`.
91
-
92
- ## Troubleshooting
93
-
94
- If you encounter issues:
95
-
96
- 1. Check that your Supabase API key is correct
97
- 2. Verify that the Supabase URL is correct
98
- 3. Ensure that the required tables exist in your Supabase project
99
- 4. Check network connectivity to Supabase
100
- 5. Run with the `--verbose` flag for more detailed error information
101
-
102
- ## Security Note
103
-
104
- Never hardcode your Supabase API key in your scripts. Always use environment variables, .env files, or command-line arguments to provide the key securely.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
agent/__init__.py DELETED
@@ -1,22 +0,0 @@
1
- """
2
- Agent module for GAIA implementation.
3
- Contains the core agent architecture and configuration.
4
- """
5
-
6
- from agent.agent import GAIAAgent
7
- from agent.config import (
8
- get_logging_config,
9
- get_model_config,
10
- get_tool_config,
11
- get_memory_config,
12
- get_agent_config
13
- )
14
-
15
- __all__ = [
16
- 'GAIAAgent',
17
- 'get_logging_config',
18
- 'get_model_config',
19
- 'get_tool_config',
20
- 'get_memory_config',
21
- 'get_agent_config'
22
- ]
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
agent/__pycache__/__init__.cpython-312.pyc DELETED
Binary file (635 Bytes)
 
agent/__pycache__/__init__.cpython-313.pyc DELETED
Binary file (635 Bytes)
 
agent/__pycache__/agent.cpython-312.pyc DELETED
Binary file (12.9 kB)
 
agent/__pycache__/agent.cpython-313.pyc DELETED
Binary file (12.8 kB)
 
agent/__pycache__/assessment_analyzer.cpython-313.pyc DELETED
Binary file (11.8 kB)
 
agent/__pycache__/config.cpython-312.pyc DELETED
Binary file (6.05 kB)
 
agent/__pycache__/config.cpython-313.pyc DELETED
Binary file (5.95 kB)
 
agent/__pycache__/graph.cpython-312.pyc DELETED
Binary file (22.8 kB)
 
agent/__pycache__/graph.cpython-313.pyc DELETED
Binary file (23.2 kB)
 
agent/__pycache__/tool_registry.cpython-313.pyc DELETED
Binary file (27.6 kB)
 
agent/agent.py DELETED
@@ -1,291 +0,0 @@
1
- """
2
- Main agent implementation for the GAIA benchmark.
3
-
4
- This module contains the GAIAAgent class which is responsible for:
5
- - Processing questions from the GAIA benchmark
6
- - Selecting and executing appropriate tools
7
- - Formulating precise answers
8
- - Handling errors and logging
9
-
10
- The agent uses LangGraph for workflow management and Supabase for memory,
11
- with configuration from the config module. It can be extended with
12
- additional capabilities as needed.
13
- """
14
-
15
- import logging
16
- import time
17
- import traceback
18
- import uuid
19
- from typing import Dict, Any, Optional, List, Union, Tuple
20
-
21
- from agent.config import (
22
- get_logging_config,
23
- get_model_config,
24
- get_tool_config,
25
- get_memory_config,
26
- get_agent_config,
27
- VERBOSE
28
- )
29
-
30
- # Import LangGraph workflow
31
- from agent.graph import run_agent_graph
32
-
33
- from memory import SupabaseMemory, ConversationMemory, ResultCache, WorkingMemory
34
-
35
- logging_config = get_logging_config()
36
- logging.basicConfig(
37
- level=logging_config["level"],
38
- format=logging_config["format"],
39
- filename=logging_config["filename"]
40
- )
41
- logger = logging.getLogger("gaia_agent")
42
-
43
- class GAIAAgent:
44
- """
45
- Agent for answering questions from the GAIA benchmark.
46
-
47
- This agent processes questions, selects appropriate tools,
48
- executes a reasoning process, and formulates precise answers.
49
- """
50
-
51
- def __init__(self, config: Optional[Dict[str, Any]] = None):
52
- """
53
- Initialize the GAIA agent with configuration.
54
-
55
- Args:
56
- config (Optional[Dict[str, Any]]): Custom configuration dictionary.
57
- If None, the default configuration from config.py is used.
58
- """
59
- logger.info("Initializing GAIAAgent")
60
-
61
- self.config = config or {
62
- "model": get_model_config(),
63
- "tools": get_tool_config(),
64
- "memory": get_memory_config(),
65
- "agent": get_agent_config()
66
- }
67
-
68
- if VERBOSE:
69
- logger.setLevel(logging.DEBUG)
70
- if not any(isinstance(h, logging.StreamHandler) for h in logger.handlers):
71
- console_handler = logging.StreamHandler()
72
- console_handler.setLevel(logging.DEBUG)
73
- console_handler.setFormatter(logging.Formatter(logging_config["format"]))
74
- logger.addHandler(console_handler)
75
-
76
- self._initialize_memory()
77
-
78
- self.reset_state()
79
-
80
- logger.info("GAIAAgent initialized successfully")
81
-
82
- def _initialize_memory(self) -> None:
83
- """Initialize memory components."""
84
-
85
- try:
86
- self.memory = SupabaseMemory(self.config["memory"])
87
-
88
- self.session_id = str(uuid.uuid4())
89
-
90
- self.conversation_memory = ConversationMemory(self.memory, self.session_id)
91
- self.result_cache = ResultCache(self.memory)
92
- self.working_memory = WorkingMemory(self.memory, self.session_id)
93
-
94
- except Exception as e:
95
- logger.error(f"Error initializing memory components: {str(e)}")
96
- logger.warning("Continuing without persistent memory")
97
-
98
- self.memory = None
99
- self.conversation_memory = None
100
- self.result_cache = None
101
- self.working_memory = None
102
-
103
- def reset_state(self) -> None:
104
- """Reset the agent's internal state."""
105
- self.state = {
106
- "current_question": None,
107
- "tools_used": [],
108
- "intermediate_results": [],
109
- "start_time": None,
110
- "end_time": None,
111
- "error": None
112
- }
113
-
114
- def __call__(self, question: str) -> str:
115
- """
116
- Process a question and return an answer.
117
-
118
- This is the main entry point for the agent. It takes a question,
119
- processes it, and returns a concise answer.
120
-
121
- Args:
122
- question (str): The question to answer
123
-
124
- Returns:
125
- str: The answer to the question
126
-
127
- Raises:
128
- Exception: If an error occurs during processing
129
- """
130
- self.reset_state()
131
- self.state["current_question"] = question
132
- self.state["start_time"] = time.time()
133
-
134
- logger.info(f"Processing question: {question[:100]}...")
135
-
136
- try:
137
- max_retries = self.config["agent"]["max_retries"]
138
- retry_delay = self.config["agent"]["retry_delay"]
139
-
140
- for attempt in range(max_retries):
141
- try:
142
- answer = self._process_question(question)
143
-
144
- self.state["end_time"] = time.time()
145
- processing_time = self.state["end_time"] - self.state["start_time"]
146
- logger.info(f"Question processed in {processing_time:.2f} seconds. Answer: {answer[:100]}...")
147
-
148
- return answer
149
-
150
- except Exception as e:
151
- if attempt < max_retries - 1:
152
- logger.warning(f"Attempt {attempt + 1}/{max_retries} failed: {str(e)}. Retrying in {retry_delay} seconds...")
153
- time.sleep(retry_delay)
154
- else:
155
- raise
156
-
157
- except Exception as e:
158
- self.state["error"] = str(e)
159
- self.state["end_time"] = time.time()
160
- processing_time = self.state["end_time"] - self.state["start_time"]
161
-
162
- logger.error(f"Error processing question after {processing_time:.2f} seconds: {str(e)}")
163
- logger.error(traceback.format_exc())
164
-
165
- if self.config["agent"].get("return_error_message", False):
166
- return f"Error: {str(e)}"
167
- else:
168
- raise
169
-
170
- def _process_question(self, question: str) -> str:
171
- """
172
- Internal method to process a question and generate an answer.
173
-
174
- This method implements the core logic of the agent using LangGraph:
175
- 1. Analyze the question
176
- 2. Plan the approach
177
- 3. Execute tools
178
- 4. Formulate the answer
179
-
180
- Args:
181
- question (str): The question to process
182
-
183
- Returns:
184
- str: The answer to the question
185
- """
186
-
187
- if self.result_cache:
188
- import hashlib
189
- cache_key = f"answer:{hashlib.md5(question.encode()).hexdigest()}"
190
-
191
- cached_answer = self.result_cache.get_result(cache_key)
192
- if cached_answer:
193
- logger.info(f"Using cached answer for question: {question[:30]}...")
194
- return cached_answer
195
-
196
- graph_config = {
197
- "recursion_limit": self.config["agent"].get("max_iterations", 10),
198
- "timeout": self.config["agent"].get("timeout", 300)
199
- }
200
-
201
- try:
202
- result = run_agent_graph(question, graph_config)
203
-
204
- if result.get("answer"):
205
- answer = result["answer"]
206
- elif result.get("error"):
207
- raise Exception(result["error"])
208
- else:
209
- answer = f"Unable to answer: {question[:30]}..."
210
-
211
- if self.result_cache:
212
- self.result_cache.cache_result(cache_key, answer)
213
-
214
- if self.working_memory:
215
- if result.get("analysis"):
216
- self.working_memory.store_intermediate_result(
217
- "analysis", result["analysis"]
218
- )
219
-
220
- if result.get("plan"):
221
- self.working_memory.store_intermediate_result(
222
- "plan", result["plan"]
223
- )
224
-
225
- if result.get("tool_results"):
226
- self.working_memory.store_intermediate_result(
227
- "tool_results", result["tool_results"]
228
- )
229
-
230
- if self.conversation_memory:
231
- self.conversation_memory.add_message("user", question)
232
- self.conversation_memory.add_message("assistant", answer)
233
-
234
- return answer
235
- except Exception as e:
236
- logger.error(f"Error running LangGraph workflow: {str(e)}")
237
- raise
238
-
239
- def get_memory_snapshot(self) -> Dict[str, Any]:
240
- """
241
- Get a snapshot of the agent's memory.
242
-
243
- This method returns a dictionary containing the current state of the
244
- agent's memory, including conversation history, cached results, and
245
- intermediate results.
246
-
247
- Returns:
248
- Dict[str, Any]: Memory snapshot
249
- """
250
- snapshot = {
251
- "conversation": [],
252
- "cached_results": {},
253
- "intermediate_results": {}
254
- }
255
-
256
- if self.conversation_memory:
257
- snapshot["conversation"] = self.conversation_memory.get_messages()
258
-
259
- if self.result_cache and self.memory:
260
- cache_keys = self.memory.list_keys("cache:")
261
- for key in cache_keys:
262
- original_key = key[6:] # Remove "cache:" prefix
263
-
264
- result = self.memory.retrieve(key)
265
-
266
- snapshot["cached_results"][original_key] = result
267
-
268
- if self.working_memory:
269
- snapshot["intermediate_results"] = self.working_memory.list_intermediate_results()
270
-
271
- return snapshot
272
-
273
- def clear_memory(self) -> None:
274
- """
275
- Clear the agent's memory.
276
-
277
- This method clears all memory components, including conversation history,
278
- cached results, and intermediate results.
279
- """
280
- logger.info("Clearing agent memory")
281
-
282
- if self.conversation_memory:
283
- self.conversation_memory.clear()
284
-
285
- if self.result_cache:
286
- self.result_cache.clear()
287
-
288
- if self.working_memory:
289
- self.working_memory.clear()
290
-
291
- logger.info("Agent memory cleared")
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
agent/assessment_analyzer.py DELETED
@@ -1,306 +0,0 @@
1
- """
2
- Assessment analyzer for the GAIA agent.
3
-
4
- This module provides functionality to analyze assessment results,
5
- categorize questions, calculate success rates, identify error patterns,
6
- and generate recommendations for further improvements.
7
- """
8
-
9
- import logging
10
- from typing import Dict, Any, List, Optional, Set, Tuple
11
- import re
12
- from collections import Counter, defaultdict
13
-
14
- logger = logging.getLogger("gaia_agent.assessment_analyzer")
15
-
16
- class AnalysisResult:
17
- """Class to store assessment analysis results."""
18
-
19
- def __init__(
20
- self,
21
- category: str,
22
- success_rate: float,
23
- successful_questions: Dict[str, Dict[str, Any]],
24
- failed_questions: Dict[str, Dict[str, Any]],
25
- error_patterns: Dict[str, int],
26
- recommendations: List[str]
27
- ):
28
- """
29
- Initialize the analysis result.
30
-
31
- Args:
32
- category: The category of questions analyzed
33
- success_rate: The success rate (0.0 to 1.0)
34
- successful_questions: Dictionary of successfully answered questions
35
- failed_questions: Dictionary of failed questions
36
- error_patterns: Dictionary of error patterns and their frequencies
37
- recommendations: List of recommendations for improvement
38
- """
39
- self.category = category
40
- self.success_rate = success_rate
41
- self.successful_questions = successful_questions
42
- self.failed_questions = failed_questions
43
- self.error_patterns = error_patterns
44
- self.recommendations = recommendations
45
-
46
- def __str__(self) -> str:
47
- """Return a string representation of the analysis result."""
48
- return (
49
- f"Analysis Result for {self.category} questions:\n"
50
- f"Success Rate: {self.success_rate:.2%}\n"
51
- f"Successful Questions: {len(self.successful_questions)}\n"
52
- f"Failed Questions: {len(self.failed_questions)}\n"
53
- f"Top Error Patterns: {dict(sorted(self.error_patterns.items(), key=lambda x: x[1], reverse=True)[:3])}\n"
54
- f"Recommendations: {self.recommendations}"
55
- )
56
-
57
- def categorize_question(question_id: str, question_data: Dict[str, Any]) -> str:
58
- """
59
- Categorize a question based on its content and ID.
60
-
61
- Args:
62
- question_id: The ID of the question
63
- question_data: The question data
64
-
65
- Returns:
66
- The category of the question
67
- """
68
- question_text = question_data.get("question", "").lower()
69
-
70
- # Web search questions
71
- web_search_keywords = [
72
- "mercedes sosa", "wikipedia", "featured article", "malko competition",
73
- "yankees", "olympics", "universe today", "polish tv actor"
74
- ]
75
-
76
- # Multimodal questions
77
- multimodal_keywords = [
78
- "video", "youtube", "watch?v=", "audio", "listen", "chess position", "image"
79
- ]
80
-
81
- # File processing questions
82
- file_processing_keywords = [
83
- "excel", "spreadsheet", "python code", "execute", "run this code"
84
- ]
85
-
86
- # Academic paper questions
87
- academic_paper_keywords = [
88
- "arxiv", "paper", "scientific", "research", "publication", "journal"
89
- ]
90
-
91
- # Mathematical reasoning questions
92
- math_keywords = [
93
- "math", "calculate", "commutativity", "table", "operation", "subset"
94
- ]
95
-
96
- # Check for web search questions
97
- for keyword in web_search_keywords:
98
- if keyword in question_text:
99
- return "web_search"
100
-
101
- # Check for multimodal questions
102
- for keyword in multimodal_keywords:
103
- if keyword in question_text:
104
- return "multimodal"
105
-
106
- # Check for file processing questions
107
- for keyword in file_processing_keywords:
108
- if keyword in question_text:
109
- return "file_processing"
110
-
111
- # Check for academic paper questions
112
- for keyword in academic_paper_keywords:
113
- if keyword in question_text:
114
- return "academic_paper"
115
-
116
- # Check for mathematical reasoning questions
117
- for keyword in math_keywords:
118
- if keyword in question_text:
119
- return "mathematical_reasoning"
120
-
121
- # Default to "other" if no category matches
122
- return "other"
123
-
124
- def identify_error_patterns(failed_questions: Dict[str, Dict[str, Any]]) -> Dict[str, int]:
125
- """
126
- Identify common error patterns in failed questions.
127
-
128
- Args:
129
- failed_questions: Dictionary of failed questions
130
-
131
- Returns:
132
- Dictionary of error patterns and their frequencies
133
- """
134
- error_patterns = Counter()
135
-
136
- for question_id, question_data in failed_questions.items():
137
- error = question_data.get("error", "")
138
- answer = question_data.get("answer", "")
139
- expected_answer = question_data.get("expected_answer", "")
140
-
141
- # Check for specific error types
142
- if error:
143
- # Check for timeout-related errors
144
- if any(term in error.lower() for term in ["timeout", "timed out", "time limit"]):
145
- error_patterns["timeout"] += 1
146
- # Check for API-related errors
147
- elif any(term in error.lower() for term in ["api", "rate limit", "quota"]):
148
- error_patterns["api_error"] += 1
149
- # Check for unhashable type errors
150
- elif "unhashable" in error.lower():
151
- error_patterns["unhashable_type"] += 1
152
- else:
153
- error_patterns["other_error"] += 1
154
- # Check for incorrect answers
155
- elif answer and expected_answer and answer != expected_answer:
156
- # Check for partial matches
157
- if any(word in answer.lower() for word in expected_answer.lower().split()):
158
- error_patterns["partial_match"] += 1
159
- else:
160
- error_patterns["completely_wrong_answer"] += 1
161
- else:
162
- error_patterns["unknown_failure"] += 1
163
-
164
- return error_patterns
165
-
166
- def generate_recommendations(
167
- category: str,
168
- success_rate: float,
169
- error_patterns: Dict[str, int],
170
- failed_questions: Dict[str, Dict[str, Any]]
171
- ) -> List[str]:
172
- """
173
- Generate recommendations for improvement based on analysis.
174
-
175
- Args:
176
- category: The category of questions
177
- success_rate: The success rate
178
- error_patterns: Dictionary of error patterns and their frequencies
179
- failed_questions: Dictionary of failed questions
180
-
181
- Returns:
182
- List of recommendations
183
- """
184
- recommendations = []
185
-
186
- # General recommendations based on success rate
187
- if success_rate < 0.3:
188
- recommendations.append(f"Major improvements needed for {category} questions")
189
- elif success_rate < 0.7:
190
- recommendations.append(f"Moderate improvements needed for {category} questions")
191
- elif success_rate < 1.0:
192
- recommendations.append(f"Minor improvements needed for {category} questions")
193
- else:
194
- recommendations.append(f"All {category} questions are answered correctly")
195
-
196
- # Specific recommendations based on error patterns
197
- if error_patterns.get("timeout", 0) > 0:
198
- recommendations.append("Optimize search performance to prevent timeouts")
199
-
200
- if error_patterns.get("api_error", 0) > 0:
201
- recommendations.append("Improve error handling for API failures")
202
-
203
- if error_patterns.get("unhashable_type", 0) > 0:
204
- recommendations.append("Fix unhashable type errors in the agent implementation")
205
-
206
- if error_patterns.get("partial_match", 0) > 0:
207
- recommendations.append("Improve answer extraction precision")
208
-
209
- if error_patterns.get("completely_wrong_answer", 0) > 0:
210
- recommendations.append("Enhance search result relevance filtering")
211
-
212
- # Category-specific recommendations
213
- if category == "web_search":
214
- if success_rate < 1.0:
215
- recommendations.append("Implement specialized search for specific domains (Wikipedia, news, etc.)")
216
- recommendations.append("Improve result ranking based on query relevance")
217
- recommendations.append("Add better error handling for failed searches")
218
-
219
- elif category == "multimodal":
220
- if success_rate < 1.0:
221
- recommendations.append("Enhance video content extraction capabilities")
222
- recommendations.append("Improve audio transcription accuracy")
223
- recommendations.append("Add specialized image analysis for specific domains")
224
-
225
- elif category == "file_processing":
226
- if success_rate < 1.0:
227
- recommendations.append("Improve Excel file parsing and data extraction")
228
- recommendations.append("Enhance code execution sandbox for better isolation")
229
-
230
- elif category == "academic_paper":
231
- if success_rate < 1.0:
232
- recommendations.append("Improve arXiv paper content extraction")
233
- recommendations.append("Add support for more academic paper sources")
234
-
235
- elif category == "mathematical_reasoning":
236
- if success_rate < 1.0:
237
- recommendations.append("Enhance mathematical operation analysis")
238
- recommendations.append("Improve structured data parsing for tables and matrices")
239
-
240
- return recommendations
241
-
242
- def analyze_assessment_results(
243
- assessment_results: Dict[str, Dict[str, Any]],
244
- category: Optional[str] = None
245
- ) -> AnalysisResult:
246
- """
247
- Analyze assessment results and generate insights.
248
-
249
- Args:
250
- assessment_results: Dictionary of assessment results
251
- category: Optional category to filter questions by
252
-
253
- Returns:
254
- AnalysisResult object with analysis results
255
- """
256
- # Categorize questions
257
- categorized_questions = defaultdict(dict)
258
-
259
- for question_id, question_data in assessment_results.items():
260
- question_category = categorize_question(question_id, question_data)
261
- categorized_questions[question_category][question_id] = question_data
262
-
263
- # If a specific category is requested, filter questions
264
- if category:
265
- questions_to_analyze = categorized_questions.get(category, {})
266
- category_name = category
267
- else:
268
- questions_to_analyze = assessment_results
269
- category_name = "all"
270
-
271
- # Separate successful and failed questions
272
- successful_questions = {}
273
- failed_questions = {}
274
-
275
- for question_id, question_data in questions_to_analyze.items():
276
- if question_data.get("correct", False):
277
- successful_questions[question_id] = question_data
278
- else:
279
- failed_questions[question_id] = question_data
280
-
281
- # Calculate success rate
282
- total_questions = len(questions_to_analyze)
283
- successful_count = len(successful_questions)
284
-
285
- success_rate = successful_count / total_questions if total_questions > 0 else 0.0
286
-
287
- # Identify error patterns
288
- error_patterns = identify_error_patterns(failed_questions)
289
-
290
- # Generate recommendations
291
- recommendations = generate_recommendations(
292
- category_name,
293
- success_rate,
294
- error_patterns,
295
- failed_questions
296
- )
297
-
298
- # Create and return analysis result
299
- return AnalysisResult(
300
- category=category_name,
301
- success_rate=success_rate,
302
- successful_questions=successful_questions,
303
- failed_questions=failed_questions,
304
- error_patterns=error_patterns,
305
- recommendations=recommendations
306
- )
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
agent/config.py DELETED
@@ -1,187 +0,0 @@
1
- """
2
- Configuration module for the GAIA agent.
3
-
4
- This module centralizes all configuration settings for the agent, including:
5
- - API endpoints
6
- - Model selection
7
- - Tool parameters
8
- - Memory settings
9
- - Logging configuration
10
- - Web browsing settings
11
- - Multimodal processing parameters
12
-
13
- It uses python-dotenv to load environment variables for sensitive credentials.
14
- """
15
-
16
- import os
17
- import logging
18
- from typing import Dict, Any, Optional, List, Union, Tuple
19
-
20
- try:
21
- from dotenv import load_dotenv
22
- load_dotenv(verbose=False)
23
- except ImportError:
24
- def load_dotenv(*args, **kwargs):
25
- pass # No-op implementation when dotenv is not available
26
-
27
- OPENAI_API_KEY = os.getenv("OPENAI_API_KEY", "")
28
- SERPER_API_KEY = os.getenv("SERPER_API_KEY", "")
29
- PERPLEXITY_API_KEY = os.getenv("PERPLEXITY_API_KEY", "")
30
- SUPABASE_KEY = os.getenv("SUPABASE_KEY", "")
31
- HF_TOKEN = os.getenv("HF_TOKEN", "")
32
-
33
- # Supabase Configuration (partially hardcoded)
34
- SUPABASE_URL = "https://tjamxhvvtnypbadvrkjq.supabase.co"
35
- SUPABASE_PROJECT_ID = "tjamxhvvtnypbadvrkjq"
36
- SUPABASE_PROJECT_NAME = "HF_GAIA_Assessment_AGT"
37
-
38
- OPENAI_API_BASE = "https://api.openai.com/v1"
39
- SERPER_API_URL = "https://google.serper.dev/search"
40
- PERPLEXITY_API_URL = "https://api.perplexity.ai"
41
- HUGGINGFACE_API_URL = "https://api-inference.huggingface.co/models"
42
-
43
- # Model Configuration (hardcoded with updated models)
44
- DEFAULT_MODEL = "gpt-4o" # Updated from gpt-4-turbo
45
- EMBEDDING_MODEL = "text-embedding-3-large" # Updated from text-embedding-ada-002
46
- FALLBACK_MODEL = "gpt-3.5-turbo"
47
- VISION_MODEL = "gpt-4o" # Updated from gpt-4-vision-preview
48
- REASONING_MODEL = "gpt-4o" # Updated from gpt-4-turbo
49
- MODEL_TEMPERATURE = 0.1
50
- MODEL_MAX_TOKENS = 4096
51
-
52
- # Tool Parameters (hardcoded)
53
- WEB_SEARCH_RESULT_COUNT = 5
54
- WEB_SEARCH_TIMEOUT = 10
55
- ARXIV_MAX_RESULTS = 3
56
- DUCKDUCKGO_TIMEOUT = 10
57
- DUCKDUCKGO_MAX_RESULTS = 5
58
- PERPLEXITY_TIMEOUT = 30
59
- PERPLEXITY_MODEL = "sonar-reasoning"
60
- WEB_SCRAPING_TIMEOUT = 15
61
- WEB_SCRAPING_MAX_LINKS = 3
62
- WEB_SCRAPING_MAX_CONTENT_LENGTH = 10000
63
-
64
- # Memory Settings (hardcoded)
65
- MEMORY_ENABLED = True
66
- MEMORY_TABLE_NAME = "gaia_memory"
67
- MEMORY_TTL = 86400 # 24 hours in seconds
68
- MEMORY_CACHE_SIZE = 100
69
- MEMORY_SIMILARITY_THRESHOLD = 0.85
70
- MEMORY_MAX_ENTRIES = 1000
71
-
72
- # Agent Settings (hardcoded)
73
- MAX_ITERATIONS = 10
74
- VERBOSE = False
75
- TIMEOUT = 300 # 5 minutes in seconds
76
- MAX_RETRIES = 3
77
- RETRY_DELAY = 2 # seconds
78
- PARALLEL_TOOL_EXECUTION = False
79
-
80
- # Logging Configuration (hardcoded)
81
- LOG_LEVEL = "INFO"
82
- LOG_FILE = "gaia_agent.log"
83
- LOG_FORMAT = "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
84
-
85
- # Multimodal Settings (hardcoded)
86
- IMAGE_MAX_SIZE = 1024 # pixels
87
- IMAGE_FORMAT = "JPEG"
88
- VISION_DETAIL_LEVEL = "high" # low, medium, high
89
-
90
- # Web Browsing Settings (hardcoded)
91
- USER_AGENT = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36"
92
- BROWSER_TIMEOUT = 30
93
- BROWSER_HEADLESS = True
94
- BROWSER_IMAGES_ENABLED = True
95
-
96
- def get_logging_config() -> Dict[str, Any]:
97
- """
98
- Returns a dictionary with logging configuration.
99
-
100
- Returns:
101
- Dict[str, Any]: Logging configuration dictionary
102
- """
103
- return {
104
- "level": getattr(logging, LOG_LEVEL.upper(), logging.INFO),
105
- "format": LOG_FORMAT,
106
- "filename": LOG_FILE if LOG_FILE else None,
107
- }
108
-
109
- def get_model_config() -> Dict[str, Any]:
110
- """
111
- Returns a dictionary with model configuration.
112
-
113
- Returns:
114
- Dict[str, Any]: Model configuration dictionary
115
- """
116
- return {
117
- "default_model": DEFAULT_MODEL,
118
- "embedding_model": EMBEDDING_MODEL,
119
- "fallback_model": FALLBACK_MODEL,
120
- "vision_model": VISION_MODEL,
121
- "reasoning_model": REASONING_MODEL,
122
- "temperature": MODEL_TEMPERATURE,
123
- "max_tokens": MODEL_MAX_TOKENS,
124
- }
125
-
126
- def get_tool_config() -> Dict[str, Dict[str, Any]]:
127
- """
128
- Returns a dictionary with tool configurations.
129
-
130
- Returns:
131
- Dict[str, Dict[str, Any]]: Tool configuration dictionary
132
- """
133
- return {
134
- "web_search": {
135
- "result_count": WEB_SEARCH_RESULT_COUNT,
136
- "timeout": WEB_SEARCH_TIMEOUT,
137
- },
138
- "arxiv": {
139
- "max_results": ARXIV_MAX_RESULTS,
140
- },
141
- "duckduckgo": {
142
- "timeout": DUCKDUCKGO_TIMEOUT,
143
- "max_results": DUCKDUCKGO_MAX_RESULTS,
144
- },
145
- "perplexity": {
146
- "timeout": PERPLEXITY_TIMEOUT,
147
- "model": PERPLEXITY_MODEL,
148
- "api_url": PERPLEXITY_API_URL,
149
- },
150
- "web_scraping": {
151
- "timeout": WEB_SCRAPING_TIMEOUT,
152
- "max_links": WEB_SCRAPING_MAX_LINKS,
153
- "max_content_length": WEB_SCRAPING_MAX_CONTENT_LENGTH,
154
- },
155
- }
156
-
157
- def get_memory_config() -> Dict[str, Any]:
158
- """
159
- Returns a dictionary with memory configuration.
160
-
161
- Returns:
162
- Dict[str, Any]: Memory configuration dictionary
163
- """
164
- return {
165
- "enabled": MEMORY_ENABLED,
166
- "table_name": MEMORY_TABLE_NAME,
167
- "ttl": MEMORY_TTL,
168
- "cache_size": MEMORY_CACHE_SIZE,
169
- "similarity_threshold": MEMORY_SIMILARITY_THRESHOLD,
170
- "max_entries": MEMORY_MAX_ENTRIES,
171
- }
172
-
173
- def get_agent_config() -> Dict[str, Any]:
174
- """
175
- Returns a dictionary with agent configuration.
176
-
177
- Returns:
178
- Dict[str, Any]: Agent configuration dictionary
179
- """
180
- return {
181
- "max_iterations": MAX_ITERATIONS,
182
- "verbose": VERBOSE,
183
- "timeout": TIMEOUT,
184
- "max_retries": MAX_RETRIES,
185
- "retry_delay": RETRY_DELAY,
186
- "parallel_tool_execution": PARALLEL_TOOL_EXECUTION,
187
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
agent/graph.py DELETED
@@ -1,703 +0,0 @@
1
- """
2
- LangGraph workflow definition for the GAIA agent.
3
-
4
- This module defines the workflow graph for the GAIA agent using LangGraph.
5
- It implements nodes for question analysis, planning, tool selection,
6
- tool execution, and answer formulation, as well as the edges between them.
7
-
8
- The graph is designed to be integrated with the GAIAAgent class and can be
9
- extended with additional nodes and edges as needed.
10
- """
11
-
12
- import logging
13
- import json
14
- import re
15
- import traceback
16
- import openai
17
- from typing import Dict, Any, List, Tuple, Optional, Annotated, TypedDict, Literal, Union, cast
18
-
19
- from langchain.prompts import PromptTemplate
20
- from langchain_core.messages import HumanMessage, AIMessage
21
- from langchain_core.runnables import RunnableConfig
22
- from langchain_core.output_parsers import StrOutputParser
23
-
24
- from langgraph.graph import StateGraph, END
25
-
26
- from agent.config import get_model_config, get_agent_config, VERBOSE
27
-
28
- logger = logging.getLogger("gaia_agent.graph")
29
-
30
- class QuestionAnalysis(TypedDict):
31
- """Type for question analysis results."""
32
- question_type: str
33
- required_tools: List[str]
34
- information_sources: List[str]
35
- complexity: str
36
-
37
- class PlanStep(TypedDict):
38
- """Type for a single step in the plan."""
39
- step_number: int
40
- description: str
41
- tool: Optional[str]
42
- tool_input: Optional[Dict[str, Any]]
43
-
44
- class ToolResult(TypedDict):
45
- """Type for tool execution results."""
46
- tool_name: str
47
- success: bool
48
- result: Any
49
- error: Optional[str]
50
-
51
- class AgentState(TypedDict):
52
- """Type for the agent state."""
53
- question: str
54
- analysis: Optional[QuestionAnalysis]
55
- plan: Optional[List[PlanStep]]
56
- current_step: int
57
- tools_to_use: Optional[List[str]]
58
- tool_results: List[ToolResult]
59
- intermediate_thoughts: List[str]
60
- answer: Optional[str]
61
- error: Optional[str]
62
- check_count: int # Counter to prevent infinite loops
63
-
64
- def extract_json_from_text(text: str) -> str:
65
- """
66
- Extract JSON from text that might contain additional content.
67
-
68
- Args:
69
- text: Text that might contain JSON
70
-
71
- Returns:
72
- Extracted JSON string or original text if no JSON found
73
- """
74
- # Try to find JSON-like content using regex
75
- json_pattern = r'(\{[\s\S]*\})'
76
- json_matches = re.findall(json_pattern, text)
77
-
78
- if json_matches:
79
- # Return the first match that is valid JSON
80
- for match in json_matches:
81
- try:
82
- json.loads(match)
83
- return match
84
- except:
85
- continue
86
-
87
- # If no valid JSON found in matches, return the original text
88
- return text
89
-
90
- def call_openai_api(prompt: str, model_config: Dict[str, Any]) -> str:
91
- """
92
- Call the OpenAI API directly instead of using langchain_openai.ChatOpenAI.
93
-
94
- Args:
95
- prompt: The prompt to send to the API
96
- model_config: Configuration for the model
97
-
98
- Returns:
99
- The response from the API
100
- """
101
- try:
102
- response = openai.ChatCompletion.create(
103
- model=model_config["reasoning_model"],
104
- messages=[{"role": "user", "content": prompt}],
105
- temperature=model_config["temperature"],
106
- max_tokens=model_config["max_tokens"]
107
- )
108
- return response.choices[0].message.content
109
- except Exception as e:
110
- logger.error(f"Error calling OpenAI API: {str(e)}")
111
- raise
112
-
113
- def analyze_question(state: AgentState) -> AgentState:
114
- """
115
- Analyze the question to determine its requirements.
116
-
117
- Args:
118
- state: The current agent state
119
-
120
- Returns:
121
- Updated agent state with question analysis
122
- """
123
-
124
- model_config = get_model_config()
125
-
126
- analysis_prompt = PromptTemplate.from_template(
127
- """You are an expert question analyzer for the GAIA benchmark.
128
-
129
- Your task is to analyze the following question and determine:
130
- 1. The question type (factual, mathematical, reasoning, etc.)
131
- 2. The tools required to answer it (web_search, calculator, reasoning, etc.)
132
- 3. The information sources needed (web, knowledge base, computation, etc.)
133
- 4. The complexity level (simple, medium, complex)
134
-
135
- Question: {question}
136
-
137
- Provide your analysis in the following JSON format:
138
- {{
139
- "question_type": "type of question",
140
- "required_tools": ["tool1", "tool2", ...],
141
- "information_sources": ["source1", "source2", ...],
142
- "complexity": "complexity level"
143
- }}
144
-
145
- IMPORTANT: Your response must be valid JSON. Do not include any text before or after the JSON.
146
-
147
- Analysis:"""
148
- )
149
-
150
- try:
151
- # Use the prompt template to format the prompt
152
- formatted_prompt = analysis_prompt.format(question=state["question"])
153
-
154
- # Call OpenAI API directly
155
- analysis_str = call_openai_api(formatted_prompt, model_config)
156
-
157
- # Try to extract JSON from the response
158
- json_str = extract_json_from_text(analysis_str)
159
-
160
- # If we still can't parse the JSON, provide a default analysis
161
- try:
162
- analysis = json.loads(json_str)
163
- except json.JSONDecodeError:
164
- # Fallback to a default analysis if JSON parsing fails
165
- analysis = {
166
- "question_type": "unknown",
167
- "required_tools": ["web_search"],
168
- "information_sources": ["web"],
169
- "complexity": "medium"
170
- }
171
- logger.warning(f"Failed to parse analysis JSON, using default analysis. Response was: {analysis_str}")
172
-
173
- new_state = state.copy()
174
- new_state["analysis"] = analysis
175
- new_state["check_count"] = 0 # Initialize check_count
176
-
177
- return new_state
178
- except Exception as e:
179
- logger.error(f"Error analyzing question: {str(e)}")
180
-
181
- # Create a default analysis instead of setting an error
182
- default_analysis = {
183
- "question_type": "unknown",
184
- "required_tools": ["web_search"],
185
- "information_sources": ["web"],
186
- "complexity": "medium"
187
- }
188
-
189
- new_state = state.copy()
190
- new_state["analysis"] = default_analysis
191
- new_state["check_count"] = 0 # Initialize check_count
192
-
193
- # Add the error as a thought rather than stopping the workflow
194
- thought = f"Error during question analysis: {str(e)}. Using default analysis."
195
- new_state["intermediate_thoughts"] = state.get("intermediate_thoughts", []) + [thought]
196
-
197
- return new_state
198
-
199
- def create_plan(state: AgentState) -> AgentState:
200
- """
201
- Create a plan to answer the question based on the analysis.
202
-
203
- Args:
204
- state: The current agent state
205
-
206
- Returns:
207
- Updated agent state with plan
208
- """
209
-
210
- if state.get("error"):
211
- return state
212
-
213
- model_config = get_model_config()
214
-
215
- planning_prompt = PromptTemplate.from_template(
216
- """You are an expert planner for the GAIA benchmark.
217
-
218
- Your task is to create a step-by-step plan to answer the following question:
219
-
220
- Question: {question}
221
-
222
- Question Analysis:
223
- {analysis}
224
-
225
- Create a detailed plan with the following steps:
226
- 1. Break down the question into logical steps
227
- 2. For each step, specify which tool to use (if any)
228
- 3. Specify the inputs for each tool
229
- 4. Describe how to combine the results to formulate the final answer
230
-
231
- Provide your plan in the following JSON format:
232
- [
233
- {{
234
- "step_number": 1,
235
- "description": "description of step",
236
- "tool": "tool_name or null if no tool needed",
237
- "tool_input": {{ "param1": "value1", ... }} or null if no tool needed
238
- }},
239
- ...
240
- ]
241
-
242
- IMPORTANT: Your response must be valid JSON. Do not include any text before or after the JSON.
243
-
244
- Plan:"""
245
- )
246
-
247
- try:
248
- # Use the prompt template to format the prompt
249
- formatted_prompt = planning_prompt.format(
250
- question=state["question"],
251
- analysis=state["analysis"]
252
- )
253
-
254
- # Call OpenAI API directly
255
- plan_str = call_openai_api(formatted_prompt, model_config)
256
-
257
- # Try to extract JSON from the response
258
- json_str = extract_json_from_text(plan_str)
259
-
260
- # If we still can't parse the JSON, provide a default plan
261
- try:
262
- plan = json.loads(json_str)
263
- except json.JSONDecodeError:
264
- # Fallback to a default plan if JSON parsing fails
265
- plan = [
266
- {
267
- "step_number": 1,
268
- "description": "Search for information related to the question",
269
- "tool": "web_search",
270
- "tool_input": {"query": state["question"]}
271
- },
272
- {
273
- "step_number": 2,
274
- "description": "Formulate an answer based on search results",
275
- "tool": None,
276
- "tool_input": None
277
- }
278
- ]
279
- logger.warning(f"Failed to parse plan JSON, using default plan. Response was: {plan_str}")
280
-
281
- new_state = state.copy()
282
- new_state["plan"] = plan
283
- new_state["current_step"] = 0
284
-
285
- tools_to_use = []
286
- for step in plan:
287
- if step.get("tool"):
288
- tools_to_use.append(step["tool"])
289
-
290
- new_state["tools_to_use"] = list(set(tools_to_use)) # Remove duplicates
291
-
292
- return new_state
293
- except Exception as e:
294
- logger.error(f"Error creating plan: {str(e)}")
295
-
296
- # Create a default plan instead of setting an error
297
- default_plan = [
298
- {
299
- "step_number": 1,
300
- "description": "Search for information related to the question",
301
- "tool": "web_search",
302
- "tool_input": {"query": state["question"]}
303
- },
304
- {
305
- "step_number": 2,
306
- "description": "Formulate an answer based on search results",
307
- "tool": None,
308
- "tool_input": None
309
- }
310
- ]
311
-
312
- new_state = state.copy()
313
- new_state["plan"] = default_plan
314
- new_state["current_step"] = 0
315
- new_state["tools_to_use"] = ["web_search"]
316
-
317
- # Add the error as a thought rather than stopping the workflow
318
- thought = f"Error during plan creation: {str(e)}. Using default plan."
319
- new_state["intermediate_thoughts"] = state.get("intermediate_thoughts", []) + [thought]
320
-
321
- return new_state
322
-
323
- def check_next_step(state: AgentState) -> AgentState:
324
- """
325
- Check the current state and update it to prepare for the next step.
326
-
327
- Args:
328
- state: The current agent state
329
-
330
- Returns:
331
- Updated agent state
332
- """
333
- if state.get("error"):
334
- return state
335
-
336
- if state.get("answer"):
337
- return state
338
-
339
- # Increment check count to prevent infinite loops
340
- new_state = state.copy()
341
- new_state["check_count"] = state.get("check_count", 0) + 1
342
-
343
- # If we've checked too many times, force move to formulate_answer
344
- if new_state["check_count"] > 3:
345
- thought = "Too many checks without progress, moving to formulate answer."
346
- new_state["intermediate_thoughts"] = state.get("intermediate_thoughts", []) + [thought]
347
- return new_state
348
-
349
- if state["current_step"] >= len(state["plan"]):
350
- # All steps completed, move to formulate answer
351
- return new_state
352
-
353
- current_step = state["plan"][state["current_step"]]
354
-
355
- if not current_step.get("tool"):
356
- # No tool needed for this step, increment step counter and continue
357
- new_state["current_step"] += 1
358
-
359
- thought = f"Step {current_step['step_number']}: {current_step['description']} - No tool needed, moving to next step."
360
- new_state["intermediate_thoughts"] = state.get("intermediate_thoughts", []) + [thought]
361
-
362
- return new_state
363
-
364
- def route_next_step(state: AgentState) -> str:
365
- """
366
- Determine the next step in the workflow based on the current state.
367
-
368
- Args:
369
- state: The current agent state
370
-
371
- Returns:
372
- String indicating the next node
373
- """
374
- if state.get("error"):
375
- return "end"
376
-
377
- if state.get("answer"):
378
- return "end"
379
-
380
- # If we've checked too many times, force move to formulate_answer
381
- if state.get("check_count", 0) > 3:
382
- return "formulate_answer"
383
-
384
- if state["current_step"] >= len(state["plan"]):
385
- return "formulate_answer"
386
-
387
- current_step = state["plan"][state["current_step"]]
388
-
389
- if current_step.get("tool"):
390
- return "execute_tool"
391
- else:
392
- return "check_next_step"
393
-
394
- def execute_tool(state: AgentState) -> AgentState:
395
- """
396
- Execute the selected tool based on the current plan step.
397
-
398
- Args:
399
- state: The current agent state
400
-
401
- Returns:
402
- Updated agent state with tool execution results
403
- """
404
- if state.get("error"):
405
- return state
406
-
407
- current_step = state["plan"][state["current_step"]]
408
- tool_name = current_step.get("tool")
409
- tool_input = current_step.get("tool_input", {})
410
-
411
- try:
412
- # Import the tool registry here to avoid circular imports
413
- from agent.tool_registry import create_default_registry, search, robust_search, merged_search
414
-
415
- # Create the tool registry
416
- registry = create_default_registry()
417
-
418
- # Execute the appropriate tool based on the tool name
419
- if tool_name == "web_search":
420
- query = tool_input.get("query")
421
- if not query:
422
- raise ValueError("Query is required for web search")
423
-
424
- try:
425
- # Get working memory if available
426
- working_memory = None
427
- if hasattr(state, "get") and callable(state.get):
428
- from memory import WorkingMemory
429
- working_memory_instance = state.get("working_memory")
430
- if isinstance(working_memory_instance, WorkingMemory):
431
- working_memory = working_memory_instance
432
-
433
- # Use the unified search wrapper with working memory
434
- result = search(registry, query, format_type="unified", working_memory=working_memory)
435
- logger.info(f"Web search successful using unified search")
436
- except Exception as search_error:
437
- logger.error(f"Unified search failed: {str(search_error)}")
438
- # If unified search fails, try robust search as a fallback
439
- try:
440
- result = robust_search(registry, query)
441
- logger.info(f"Web search successful using fallback robust search")
442
- except Exception as robust_error:
443
- logger.error(f"Robust search failed: {str(robust_error)}")
444
- raise ValueError(f"All search methods failed: {str(search_error)}")
445
-
446
- elif tool_name in ["duckduckgo_search", "serper_search", "perplexity_search", "arxiv_search"]:
447
- query = tool_input.get("query")
448
- if not query:
449
- raise ValueError("Query is required for search tools")
450
-
451
- tool = registry.get_tool(tool_name)
452
- if not tool:
453
- raise ValueError(f"Tool not found: {tool_name}")
454
-
455
- result = registry.execute_tool(tool_name, query=query)
456
-
457
- elif tool_name == "arxiv_get_paper":
458
- paper_id = tool_input.get("paper_id")
459
- if not paper_id:
460
- raise ValueError("Paper ID is required for arXiv paper retrieval")
461
-
462
- tool = registry.get_tool("arxiv_search")
463
- if not tool:
464
- raise ValueError("arXiv tool not found")
465
-
466
- result = registry.execute_tool("arxiv_get_paper", paper_id=paper_id)
467
-
468
- elif tool_name == "extract_web_content":
469
- url = tool_input.get("url")
470
- if not url:
471
- raise ValueError("URL is required for web content extraction")
472
-
473
- # Import the web content extractor
474
- from tools.web_tools import create_web_content_extractor
475
- extractor = create_web_content_extractor()
476
- result = extractor.extract_content(url)
477
-
478
- elif tool_name == "youtube_video":
479
- video_id_or_url = tool_input.get("video_id_or_url")
480
- language = tool_input.get("language")
481
- if not video_id_or_url:
482
- raise ValueError("Video ID or URL is required for YouTube video analysis")
483
-
484
- tool = registry.get_tool("youtube_video")
485
- if not tool:
486
- raise ValueError("YouTube tool not found")
487
-
488
- result = registry.execute_tool("youtube_video", video_id_or_url=video_id_or_url, language=language)
489
-
490
- else:
491
- # Fallback to simulated result for unknown tools
492
- result = f"Simulated result for {tool_name} with input {tool_input}"
493
- logger.warning(f"Using simulated result for unknown tool: {tool_name}")
494
-
495
- tool_result = {
496
- "tool_name": tool_name,
497
- "success": True,
498
- "result": result,
499
- "error": None
500
- }
501
-
502
- new_state = state.copy()
503
- new_state["tool_results"] = state.get("tool_results", []) + [tool_result]
504
- new_state["current_step"] += 1
505
- new_state["check_count"] = 0 # Reset check count after successful tool execution
506
-
507
- thought = f"Step {current_step['step_number']}: {current_step['description']} - Executed {tool_name} and got result: {str(result)[:200]}..."
508
- new_state["intermediate_thoughts"] = state.get("intermediate_thoughts", []) + [thought]
509
-
510
- return new_state
511
- except Exception as e:
512
- logger.error(f"Error executing tool {tool_name}: {str(e)}")
513
- logger.error(f"Stack trace: {traceback.format_exc()}")
514
-
515
- tool_result = {
516
- "tool_name": tool_name,
517
- "success": False,
518
- "result": None,
519
- "error": str(e)
520
- }
521
-
522
- new_state = state.copy()
523
- new_state["tool_results"] = state.get("tool_results", []) + [tool_result]
524
- new_state["current_step"] += 1
525
- new_state["check_count"] = 0 # Reset check count after tool execution (even if failed)
526
-
527
- thought = f"Step {current_step['step_number']}: {current_step['description']} - Failed to execute {tool_name}: {str(e)}"
528
- new_state["intermediate_thoughts"] = state.get("intermediate_thoughts", []) + [thought]
529
-
530
- return new_state
531
-
532
- def formulate_answer(state: AgentState) -> AgentState:
533
- """
534
- Formulate the final answer based on tool results.
535
-
536
- Args:
537
- state: The current agent state
538
-
539
- Returns:
540
- Updated agent state with answer
541
- """
542
-
543
- if state.get("error"):
544
- return state
545
-
546
- model_config = get_model_config()
547
-
548
- answer_prompt = PromptTemplate.from_template(
549
- """You are an expert answer formulator for the GAIA benchmark.
550
-
551
- Your task is to formulate a precise, concise answer to the following question based on the tool results:
552
-
553
- Question: {question}
554
-
555
- Question Analysis:
556
- {analysis}
557
-
558
- Plan:
559
- {plan}
560
-
561
- Tool Results:
562
- {tool_results}
563
-
564
- Intermediate Thoughts:
565
- {thoughts}
566
-
567
- Remember, GAIA benchmark answers should be very concise - either a number, a few words, or a comma-separated list.
568
- Do not provide explanations or additional context in your answer.
569
-
570
- Concise Answer:"""
571
- )
572
-
573
- try:
574
- # Use the prompt template to format the prompt
575
- formatted_prompt = answer_prompt.format(
576
- question=state["question"],
577
- analysis=state["analysis"],
578
- plan=state["plan"],
579
- tool_results=state["tool_results"],
580
- thoughts=state.get("intermediate_thoughts", [])
581
- )
582
-
583
- # Call OpenAI API directly
584
- answer = call_openai_api(formatted_prompt, model_config)
585
-
586
- new_state = state.copy()
587
- new_state["answer"] = answer.strip()
588
-
589
- return new_state
590
- except Exception as e:
591
- logger.error(f"Error formulating answer: {str(e)}")
592
-
593
- # Provide a default answer instead of setting an error
594
- default_answer = "Unable to provide a specific answer due to technical limitations."
595
-
596
- new_state = state.copy()
597
- new_state["answer"] = default_answer
598
-
599
- # Add the error as a thought rather than stopping the workflow
600
- thought = f"Error during answer formulation: {str(e)}. Using default answer."
601
- new_state["intermediate_thoughts"] = state.get("intermediate_thoughts", []) + [thought]
602
-
603
- return new_state
604
-
605
- def create_agent_graph() -> Tuple[StateGraph, Dict[str, Any]]:
606
- """
607
- Create the agent workflow graph.
608
-
609
- Returns:
610
- Tuple containing the StateGraph and the initial state
611
- """
612
- workflow = StateGraph(AgentState)
613
-
614
- # Add nodes
615
- workflow.add_node("analyze_question", analyze_question)
616
- workflow.add_node("create_plan", create_plan)
617
- workflow.add_node("check_next_step", check_next_step)
618
- workflow.add_node("execute_tool", execute_tool)
619
- workflow.add_node("formulate_answer", formulate_answer)
620
-
621
- # Set entry point
622
- workflow.set_entry_point("analyze_question")
623
-
624
- # Add standard edges
625
- workflow.add_edge("analyze_question", "create_plan")
626
- workflow.add_edge("formulate_answer", END)
627
-
628
- # Add conditional edges
629
- workflow.add_conditional_edges(
630
- "create_plan",
631
- route_next_step,
632
- {
633
- "check_next_step": "check_next_step",
634
- "execute_tool": "execute_tool",
635
- "formulate_answer": "formulate_answer",
636
- "end": END
637
- }
638
- )
639
-
640
- workflow.add_conditional_edges(
641
- "check_next_step",
642
- route_next_step,
643
- {
644
- "check_next_step": "check_next_step",
645
- "execute_tool": "execute_tool",
646
- "formulate_answer": "formulate_answer",
647
- "end": END
648
- }
649
- )
650
-
651
- workflow.add_conditional_edges(
652
- "execute_tool",
653
- route_next_step,
654
- {
655
- "check_next_step": "check_next_step",
656
- "execute_tool": "execute_tool",
657
- "formulate_answer": "formulate_answer",
658
- "end": END
659
- }
660
- )
661
-
662
- initial_state = {
663
- "question": "",
664
- "analysis": None,
665
- "plan": None,
666
- "current_step": 0,
667
- "tools_to_use": None,
668
- "tool_results": [],
669
- "intermediate_thoughts": [],
670
- "answer": None,
671
- "error": None,
672
- "check_count": 0 # Initialize check_count
673
- }
674
-
675
- return workflow, initial_state
676
-
677
- def run_agent_graph(question: str, config: Optional[Dict[str, Any]] = None) -> Dict[str, Any]:
678
- """
679
- Run the agent workflow graph with a question.
680
-
681
- Args:
682
- question: The question to answer
683
- config: Optional configuration for the graph
684
-
685
- Returns:
686
- The final state of the graph
687
- """
688
- workflow, initial_state = create_agent_graph()
689
-
690
- app = workflow.compile()
691
-
692
- state = initial_state.copy()
693
- state["question"] = question
694
-
695
- # Set a higher recursion limit to prevent errors
696
- config_with_recursion = (config or {}).copy()
697
- config_with_recursion["recursion_limit"] = 25
698
-
699
- runnable_config = RunnableConfig(config_with_recursion)
700
-
701
- result = app.invoke(state, config=runnable_config)
702
-
703
- return result
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
agent/tool_registry.py DELETED
@@ -1,732 +0,0 @@
1
- """
2
- Tool registry for the GAIA agent.
3
-
4
- This module provides a registry for tools that can be used by the GAIA agent.
5
- It includes factory functions for creating tool instances and a registry class
6
- for managing tool instances.
7
-
8
- The registry is designed to be used with the LangGraph workflow in agent/graph.py.
9
- """
10
-
11
- import logging
12
- import os
13
- import re
14
- import time
15
- from typing import Dict, Any, Optional, List, Callable
16
-
17
- from tools.web_tools import (
18
- DuckDuckGoSearchTool,
19
- SerperSearchTool,
20
- EnhancedWebSearchTool,
21
- LibrarySearchTool,
22
- ApiSearchTool,
23
- create_duckduckgo_search,
24
- create_serper_search,
25
- create_wikipedia_search,
26
- create_enhanced_web_search,
27
- create_library_search,
28
- create_api_search
29
- )
30
- from tools.perplexity_tool import PerplexityTool, create_perplexity_tool
31
- from tools.arxiv_tool import ArxivSearchTool, create_arxiv_search
32
- from tools.multimodal_tools import YouTubeVideoTool, create_youtube_video_tool, BrowserSearchTool, create_browser_search_tool
33
-
34
- logger = logging.getLogger("gaia_agent.tool_registry")
35
-
36
- class ToolRegistry:
37
- """Registry for tools used by the GAIA agent."""
38
-
39
- def __init__(self):
40
- """Initialize an empty tool registry."""
41
- self.tools = {}
42
-
43
- def register_tool(self, name: str, tool: Any) -> None:
44
- """
45
- Register a tool in the registry.
46
-
47
- Args:
48
- name: The name of the tool
49
- tool: The tool instance
50
- """
51
- self.tools[name] = tool
52
-
53
- def get_tool(self, name: str) -> Optional[Any]:
54
- """
55
- Get a tool from the registry.
56
-
57
- Args:
58
- name: The name of the tool
59
-
60
- Returns:
61
- The tool instance, or None if not found
62
- """
63
- tool = self.tools.get(name)
64
- if not tool:
65
- logger.warning(f"Tool not found in registry: {name}")
66
- return tool
67
-
68
- def list_tools(self) -> List[str]:
69
- """
70
- List all tools in the registry.
71
-
72
- Returns:
73
- List of tool names
74
- """
75
- return list(self.tools.keys())
76
-
77
- def execute_tool(self, name: str, **kwargs) -> Any:
78
- """
79
- Execute a tool from the registry.
80
-
81
- Args:
82
- name: The name of the tool
83
- **kwargs: Arguments to pass to the tool
84
-
85
- Returns:
86
- The result of the tool execution
87
-
88
- Raises:
89
- Exception: If the tool is not found or execution fails
90
- """
91
- tool = self.get_tool(name)
92
- if not tool:
93
- raise Exception(f"Tool not found in registry: {name}")
94
-
95
-
96
- try:
97
- if name in ["duckduckgo_search", "serper_search", "wikipedia_search", "enhanced_web_search",
98
- "library_search", "api_search"]:
99
- query = kwargs.get("query")
100
- if not query:
101
- raise ValueError("Query is required for search tools")
102
- return tool.search(query)
103
-
104
- elif name == "browser_search":
105
- query = kwargs.get("query")
106
- source = kwargs.get("source")
107
- if not query:
108
- raise ValueError("Query is required for browser search")
109
- return tool.search(query, source)
110
-
111
- elif name == "perplexity_search":
112
- query = kwargs.get("query")
113
- if not query:
114
- raise ValueError("Query is required for Perplexity search")
115
- return tool.search(query)
116
-
117
- elif name == "arxiv_search":
118
- query = kwargs.get("query")
119
- max_results = kwargs.get("max_results")
120
- if not query:
121
- raise ValueError("Query is required for arXiv search")
122
- return tool.search(query, max_results)
123
-
124
- elif name == "arxiv_get_paper":
125
- paper_id = kwargs.get("paper_id")
126
- if not paper_id:
127
- raise ValueError("Paper ID is required for arXiv paper retrieval")
128
- return tool.get_paper_by_id(paper_id)
129
-
130
- elif name == "arxiv_search_category":
131
- category = kwargs.get("category")
132
- max_results = kwargs.get("max_results")
133
- if not category:
134
- raise ValueError("Category is required for arXiv category search")
135
- return tool.search_by_category(category, max_results)
136
-
137
- elif name == "wikipedia_extract_page":
138
- url = kwargs.get("url")
139
- if not url:
140
- raise ValueError("URL is required for Wikipedia page extraction")
141
- return tool.extract_page_content(url)
142
-
143
- elif name == "wikipedia_featured_articles":
144
- topic = kwargs.get("topic")
145
- return tool.find_featured_articles(topic)
146
-
147
- elif name == "youtube_video":
148
- video_id_or_url = kwargs.get("video_id_or_url")
149
- language = kwargs.get("language")
150
- if not video_id_or_url:
151
- raise ValueError("Video ID or URL is required for YouTube video analysis")
152
- return tool.extract_transcript(video_id_or_url, language)
153
-
154
- else:
155
- return tool.run(**kwargs)
156
-
157
- except Exception as e:
158
- logger.error(f"Error executing tool {name}: {str(e)}")
159
- raise
160
-
161
- def create_default_registry() -> ToolRegistry:
162
- """
163
- Create a default tool registry with all available tools.
164
-
165
- Returns:
166
- ToolRegistry: A registry with all available tools
167
- """
168
- registry = ToolRegistry()
169
-
170
- # Register Enhanced Web Search tool (handles GAIA assessment questions)
171
- try:
172
- enhanced_web_tool = create_enhanced_web_search()
173
- registry.register_tool("enhanced_web_search", enhanced_web_tool)
174
- logger.info("Registered Enhanced Web Search tool")
175
- except Exception as e:
176
- logger.warning(f"Failed to create Enhanced Web Search tool: {str(e)}")
177
-
178
- # Register DuckDuckGo search tool (no API key required)
179
- try:
180
- duckduckgo_tool = create_duckduckgo_search()
181
- registry.register_tool("duckduckgo_search", duckduckgo_tool)
182
- except Exception as e:
183
- logger.warning(f"Failed to create DuckDuckGo search tool: {str(e)}")
184
-
185
- # Register Serper search tool if API key is available
186
- serper_api_key = os.environ.get("SERPER_API_KEY")
187
- if serper_api_key:
188
- try:
189
- serper_tool = create_serper_search()
190
- registry.register_tool("serper_search", serper_tool)
191
- except Exception as e:
192
- logger.warning(f"Failed to create Serper search tool: {str(e)}")
193
- else:
194
- logger.warning("Serper API key not available, skipping Serper search tool")
195
-
196
- # Register Perplexity tool if API key is available
197
- perplexity_api_key = os.environ.get("PERPLEXITY_API_KEY")
198
- if perplexity_api_key:
199
- try:
200
- perplexity_tool = create_perplexity_tool()
201
- registry.register_tool("perplexity_search", perplexity_tool)
202
- except Exception as e:
203
- logger.warning(f"Failed to create Perplexity tool: {str(e)}")
204
- else:
205
- logger.warning("Perplexity API key not available, skipping Perplexity tool")
206
-
207
- try:
208
- arxiv_tool = create_arxiv_search()
209
- registry.register_tool("arxiv_search", arxiv_tool)
210
- registry.register_tool("arxiv_get_paper", arxiv_tool)
211
- registry.register_tool("arxiv_search_category", arxiv_tool)
212
- except Exception as e:
213
- logger.warning(f"Failed to create arXiv search tool: {str(e)}")
214
-
215
- # Register Wikipedia search tool
216
- try:
217
- wikipedia_tool = create_wikipedia_search()
218
- registry.register_tool("wikipedia_search", wikipedia_tool)
219
- except Exception as e:
220
- logger.warning(f"Failed to create Wikipedia search tool: {str(e)}")
221
-
222
- # Register YouTube video tool
223
- try:
224
- youtube_tool = create_youtube_video_tool()
225
- registry.register_tool("youtube_video", youtube_tool)
226
- except Exception as e:
227
- logger.warning(f"Failed to create YouTube video tool: {str(e)}")
228
-
229
- # Register Browser Search tool
230
- try:
231
- browser_search_tool = create_browser_search_tool()
232
- registry.register_tool("browser_search", browser_search_tool)
233
- logger.info("Registered Browser Search tool")
234
- except Exception as e:
235
- logger.warning(f"Failed to create Browser Search tool: {str(e)}")
236
-
237
- # Register Library Search tool
238
- try:
239
- library_search_tool = create_library_search()
240
- registry.register_tool("library_search", library_search_tool)
241
- logger.info("Registered Library Search tool")
242
- except Exception as e:
243
- logger.warning(f"Failed to create Library Search tool: {str(e)}")
244
-
245
- # Register API Search tool if API keys are available
246
- if os.environ.get("PERPLEXITY_API_KEY") or os.environ.get("SERPER_API_KEY"):
247
- try:
248
- api_search_tool = create_api_search()
249
- registry.register_tool("api_search", api_search_tool)
250
- logger.info("Registered API Search tool")
251
- except Exception as e:
252
- logger.warning(f"Failed to create API Search tool: {str(e)}")
253
- else:
254
- logger.warning("Neither Perplexity nor Serper API keys available, skipping API Search tool")
255
-
256
- logger.info(f"Created default tool registry with {len(registry.list_tools())} tools")
257
- return registry
258
-
259
- def analyze_query(query: str) -> Dict[str, Any]:
260
- """
261
- Analyze a query to determine the best search strategy.
262
-
263
- This function examines the query to identify:
264
- - Source-specific keywords (Wikipedia, YouTube, arXiv)
265
- - Question type (factual, research, multimedia)
266
- - Information depth needed
267
-
268
- Args:
269
- query: The search query
270
-
271
- Returns:
272
- Dict with analysis results
273
- """
274
- analysis = {
275
- "source_specific": False,
276
- "preferred_sources": [],
277
- "question_type": "factual", # Default to factual
278
- "depth_needed": "medium", # Default to medium depth
279
- "is_multimedia": False
280
- }
281
-
282
- # Check for source-specific keywords
283
- query_lower = query.lower()
284
-
285
- # Wikipedia specific
286
- if "wikipedia" in query_lower or "featured article" in query_lower:
287
- analysis["source_specific"] = True
288
- analysis["preferred_sources"].append("wikipedia")
289
-
290
- # YouTube specific
291
- if "youtube" in query_lower or "video" in query_lower:
292
- analysis["source_specific"] = True
293
- analysis["preferred_sources"].append("youtube")
294
- analysis["is_multimedia"] = True
295
-
296
- # arXiv specific
297
- if "arxiv" in query_lower or "paper" in query_lower or "research paper" in query_lower:
298
- analysis["source_specific"] = True
299
- analysis["preferred_sources"].append("arxiv")
300
- analysis["question_type"] = "research"
301
- analysis["depth_needed"] = "high"
302
-
303
- # Determine question type if not already set
304
- if "how" in query_lower or "why" in query_lower or "explain" in query_lower:
305
- analysis["question_type"] = "explanatory"
306
- analysis["depth_needed"] = "high"
307
- elif "when" in query_lower or "where" in query_lower or "who" in query_lower:
308
- analysis["question_type"] = "factual"
309
- elif "compare" in query_lower or "difference" in query_lower:
310
- analysis["question_type"] = "comparative"
311
- analysis["depth_needed"] = "high"
312
-
313
- # Check for indicators of needed depth
314
- if "detailed" in query_lower or "comprehensive" in query_lower or "in depth" in query_lower:
315
- analysis["depth_needed"] = "high"
316
- elif "brief" in query_lower or "summary" in query_lower or "overview" in query_lower:
317
- analysis["depth_needed"] = "low"
318
-
319
- return analysis
320
-
321
- def unified_search(registry: ToolRegistry, query: str, working_memory=None) -> Dict[str, Any]:
322
- """
323
- Perform a unified search using an intelligent routing approach.
324
-
325
- This function:
326
- 1. Analyzes the query to determine the best search strategy
327
- 2. Routes to appropriate tools based on the analysis
328
- 3. Executes tools in parallel when appropriate
329
- 4. Stores intermediate results in working_memory
330
- 5. Combines and ranks results
331
-
332
- Args:
333
- registry: The tool registry
334
- query: The search query
335
- working_memory: Optional working memory instance for storing results
336
-
337
- Returns:
338
- Dict with search results and metadata
339
- """
340
- from tools.web_tools import calculate_query_relevance
341
- import concurrent.futures
342
-
343
- # Analyze the query
344
- analysis = analyze_query(query)
345
-
346
- # Store the analysis in working memory if available
347
- if working_memory:
348
- working_memory.store_intermediate_result("query_analysis", analysis)
349
-
350
- # Initialize results container
351
- all_results = []
352
- metadata = {
353
- "providers_used": [],
354
- "analysis": analysis,
355
- "execution_times": {}
356
- }
357
-
358
- # Check if we should use the enhanced web search tool
359
- # This tool is especially useful for GAIA assessment questions
360
- if registry.get_tool("enhanced_web_search"):
361
- # First try the enhanced web search tool directly (not in parallel)
362
- # This is more efficient for GAIA assessment questions
363
- try:
364
- logger.info(f"Using enhanced web search for query: {query}")
365
- start_time = time.time()
366
- enhanced_results = registry.execute_tool("enhanced_web_search", query=query)
367
- end_time = time.time()
368
-
369
- metadata["execution_times"]["enhanced_web_search"] = end_time - start_time
370
- metadata["providers_used"].append("enhanced_web_search")
371
-
372
- # If we got good results from the enhanced tool, return them directly
373
- if enhanced_results and len(enhanced_results) > 0:
374
- # Check if we have a high-quality result (like from Perplexity)
375
- has_high_quality = False
376
- for result in enhanced_results:
377
- if result.get("source") == "perplexity" or result.get("relevance_score", 0) > 8.0:
378
- has_high_quality = True
379
- break
380
-
381
- if has_high_quality:
382
- logger.info("Enhanced web search returned high-quality results, skipping other tools")
383
-
384
- # Store results in working memory
385
- if working_memory:
386
- working_memory.store_intermediate_result("enhanced_search_results", enhanced_results)
387
-
388
- return {
389
- "results": enhanced_results,
390
- "metadata": metadata
391
- }
392
- except Exception as e:
393
- logger.warning(f"Enhanced web search failed: {str(e)}")
394
-
395
- # Determine which tools to use based on analysis
396
- tools_to_use = []
397
-
398
- # If source-specific, prioritize those sources
399
- if analysis["source_specific"]:
400
- for source in analysis["preferred_sources"]:
401
- if source == "wikipedia" and registry.get_tool("wikipedia_search"):
402
- tools_to_use.append("wikipedia_search")
403
- elif source == "youtube" and registry.get_tool("youtube_video"):
404
- tools_to_use.append("youtube_video")
405
- elif source == "arxiv" and registry.get_tool("arxiv_search"):
406
- tools_to_use.append("arxiv_search")
407
-
408
- # For high depth questions, always include Perplexity if available
409
- if analysis["depth_needed"] == "high" and registry.get_tool("perplexity_search"):
410
- if "perplexity_search" not in tools_to_use:
411
- tools_to_use.append("perplexity_search")
412
-
413
- # Check for API search tool (combines Perplexity and Serper)
414
- if registry.get_tool("api_search") and "api_search" not in tools_to_use:
415
- tools_to_use.append("api_search")
416
-
417
- # Check for library search tool (combines DuckDuckGo and arXiv)
418
- if registry.get_tool("library_search") and "library_search" not in tools_to_use:
419
- tools_to_use.append("library_search")
420
-
421
- # Always include general search tools as fallbacks
422
- if registry.get_tool("duckduckgo_search") and "duckduckgo_search" not in tools_to_use:
423
- tools_to_use.append("duckduckgo_search")
424
-
425
- if registry.get_tool("serper_search") and "serper_search" not in tools_to_use:
426
- tools_to_use.append("serper_search")
427
-
428
- # For visual or interactive queries, include browser search tool
429
- if analysis["is_multimedia"] and registry.get_tool("browser_search") and "browser_search" not in tools_to_use:
430
- tools_to_use.append("browser_search")
431
-
432
- # Execute tools in parallel
433
- results_dict = {}
434
- with concurrent.futures.ThreadPoolExecutor() as executor:
435
- future_to_tool = {}
436
-
437
- for tool_name in tools_to_use:
438
- if tool_name == "youtube_video":
439
- # YouTube tool requires different handling
440
- continue
441
-
442
- future = executor.submit(registry.execute_tool, tool_name, query=query)
443
- future_to_tool[future] = tool_name
444
-
445
- for future in concurrent.futures.as_completed(future_to_tool):
446
- tool_name = future_to_tool[future]
447
- try:
448
- start_time = time.time()
449
- result = future.result()
450
- end_time = time.time()
451
-
452
- metadata["execution_times"][tool_name] = end_time - start_time
453
- metadata["providers_used"].append(tool_name)
454
-
455
- results_dict[tool_name] = result
456
-
457
- # Store intermediate results in working memory
458
- if working_memory:
459
- working_memory.store_intermediate_result(f"search_result_{tool_name}", result)
460
-
461
- except Exception as e:
462
- logger.warning(f"{tool_name} search failed: {str(e)}")
463
- metadata["execution_times"][tool_name] = -1 # Indicate failure
464
-
465
- # Process and merge results
466
- seen_urls = set()
467
-
468
- # Process source-specific results first
469
- for source in analysis["preferred_sources"]:
470
- tool_name = None
471
- if source == "wikipedia":
472
- tool_name = "wikipedia_search"
473
- elif source == "arxiv":
474
- tool_name = "arxiv_search"
475
-
476
- if tool_name and tool_name in results_dict:
477
- results = results_dict[tool_name]
478
-
479
- # Format results if needed
480
- formatted_results = []
481
- if tool_name == "arxiv_search":
482
- for result in results:
483
- if "url" in result and result["url"] not in seen_urls:
484
- title = result.get("title", "")
485
- summary = result.get("summary", "")
486
-
487
- # Calculate relevance
488
- title_relevance = calculate_query_relevance(title, query)
489
- summary_relevance = calculate_query_relevance(summary, query)
490
- relevance_score = (title_relevance * 2 + summary_relevance) / 3
491
-
492
- formatted_result = {
493
- "title": title,
494
- "link": result.get("url", ""),
495
- "snippet": summary[:200] + "..." if summary else "",
496
- "relevance_score": relevance_score * 1.2, # Boost source-specific results
497
- "source": "arxiv"
498
- }
499
-
500
- formatted_results.append(formatted_result)
501
- seen_urls.add(result["url"])
502
- else:
503
- # For other tools, just add source and boost relevance
504
- for result in results:
505
- if result["link"] not in seen_urls:
506
- if "relevance_score" not in result:
507
- title_relevance = calculate_query_relevance(result.get("title", ""), query)
508
- snippet_relevance = calculate_query_relevance(result.get("snippet", ""), query)
509
- result["relevance_score"] = (title_relevance * 2 + snippet_relevance) / 3
510
-
511
- # Boost source-specific results
512
- result["relevance_score"] = result["relevance_score"] * 1.2
513
- result["source"] = source
514
-
515
- formatted_results.append(result)
516
- seen_urls.add(result["link"])
517
-
518
- all_results.extend(formatted_results)
519
-
520
- # Process general search results
521
- for tool_name in ["duckduckgo_search", "serper_search", "library_search", "api_search"]:
522
- if tool_name in results_dict:
523
- for result in results_dict[tool_name]:
524
- if "link" in result and result["link"] not in seen_urls:
525
- if "relevance_score" not in result:
526
- title_relevance = calculate_query_relevance(result.get("title", ""), query)
527
- snippet_relevance = calculate_query_relevance(result.get("snippet", ""), query)
528
- result["relevance_score"] = (title_relevance * 2 + snippet_relevance) / 3
529
-
530
- # If source is not already set, derive it from the tool name
531
- if "source" not in result:
532
- result["source"] = tool_name.replace("_search", "")
533
-
534
- all_results.append(result)
535
- seen_urls.add(result["link"])
536
-
537
- # Process Perplexity results
538
- if "perplexity_search" in results_dict:
539
- perplexity_result = results_dict["perplexity_search"]
540
- perplexity_content = None
541
-
542
- if isinstance(perplexity_result, dict) and "content" in perplexity_result:
543
- perplexity_content = perplexity_result["content"]
544
-
545
- # Add perplexity as a result if it's not empty
546
- if perplexity_content and perplexity_content.strip():
547
- relevance_score = calculate_query_relevance(perplexity_content, query)
548
-
549
- # For high depth questions, boost Perplexity even more
550
- if analysis["depth_needed"] == "high":
551
- relevance_score = relevance_score * 1.5
552
-
553
- formatted_result = {
554
- "title": "Perplexity AI Search Result",
555
- "link": "https://perplexity.ai/",
556
- "snippet": perplexity_content[:200] + "..." if len(perplexity_content) > 200 else perplexity_content,
557
- "relevance_score": relevance_score,
558
- "source": "perplexity"
559
- }
560
- all_results.append(formatted_result)
561
-
562
- # Store the full perplexity content in metadata
563
- metadata["perplexity_content"] = perplexity_content
564
-
565
- # Process Browser Search results
566
- if "browser_search" in results_dict:
567
- browser_results = results_dict["browser_search"]
568
- if browser_results and isinstance(browser_results, list):
569
- for result in browser_results:
570
- if "link" in result and result["link"] not in seen_urls:
571
- # Browser search results already have high relevance scores
572
- if "relevance_score" not in result:
573
- result["relevance_score"] = 9.0 # High default score for browser results
574
-
575
- all_results.append(result)
576
- seen_urls.add(result["link"])
577
-
578
- # Sort all results by relevance score
579
- all_results.sort(key=lambda x: x.get("relevance_score", 0), reverse=True)
580
-
581
- # Store final merged results in working memory
582
- if working_memory:
583
- working_memory.store_intermediate_result("merged_search_results", all_results)
584
- working_memory.store_intermediate_result("search_metadata", metadata)
585
-
586
- return {
587
- "results": all_results[:10], # Return top 10 results
588
- "metadata": metadata
589
- }
590
-
591
- def search(registry: ToolRegistry, query: str, format_type: str = "unified", working_memory=None) -> Dict[str, Any]:
592
- """
593
- Unified wrapper function for all search types.
594
-
595
- This function serves as a single entry point for all search operations,
596
- eliminating redundancy while maintaining backward compatibility with
597
- different output formats.
598
-
599
- Args:
600
- registry: The tool registry
601
- query: The search query
602
- format_type: The desired output format ("unified", "robust", or "merged")
603
- working_memory: Optional working memory instance for storing results
604
-
605
- Returns:
606
- Dict with search results formatted according to format_type
607
- """
608
- # Execute the unified search
609
- search_result = unified_search(registry, query, working_memory)
610
-
611
- # Return results in the requested format
612
- if format_type == "robust":
613
- # Format as robust_search result
614
- providers = [result.get("source", "unknown") for result in search_result["results"]]
615
- unique_providers = list(set(providers))
616
-
617
- return {
618
- "provider": ",".join(unique_providers),
619
- "results": search_result["results"]
620
- }
621
-
622
- elif format_type == "merged":
623
- # Format as merged_search result
624
- perplexity_content = search_result["metadata"].get("perplexity_content")
625
-
626
- # Extract arxiv results if available
627
- arxiv_results = []
628
- browser_results = []
629
- library_results = []
630
- api_results = []
631
-
632
- for result in search_result["results"]:
633
- source = result.get("source", "")
634
-
635
- if source == "arxiv":
636
- # Try to reconstruct original arxiv result format
637
- arxiv_result = {
638
- "title": result.get("title", ""),
639
- "url": result.get("link", ""),
640
- "summary": result.get("snippet", "")
641
- }
642
- arxiv_results.append(arxiv_result)
643
-
644
- elif source == "browser":
645
- browser_results.append({
646
- "title": result.get("title", ""),
647
- "url": result.get("link", ""),
648
- "snippet": result.get("snippet", "")
649
- })
650
-
651
- elif source == "library":
652
- library_results.append({
653
- "title": result.get("title", ""),
654
- "url": result.get("link", ""),
655
- "snippet": result.get("snippet", "")
656
- })
657
-
658
- elif source == "api":
659
- api_results.append({
660
- "title": result.get("title", ""),
661
- "url": result.get("link", ""),
662
- "snippet": result.get("snippet", "")
663
- })
664
-
665
- return {
666
- "merged_results": search_result["results"],
667
- "perplexity_context": perplexity_content,
668
- "arxiv_context": arxiv_results,
669
- "browser_context": browser_results,
670
- "library_context": library_results,
671
- "api_context": api_results
672
- }
673
-
674
- else: # "unified" or any other value
675
- # Return the unified search result directly
676
- return search_result
677
-
678
- def robust_search(registry: ToolRegistry, query: str) -> Dict[str, Any]:
679
- """
680
- Legacy robust search function - now uses the unified search wrapper.
681
-
682
- This function is maintained for backward compatibility.
683
- New code should use the 'search' function with format_type="robust".
684
-
685
- Args:
686
- registry: The tool registry
687
- query: The search query
688
-
689
- Returns:
690
- Dict with provider name and search results
691
- """
692
- return search(registry, query, format_type="robust")
693
-
694
- def merged_search(registry: ToolRegistry, query: str, working_memory=None) -> Dict[str, Any]:
695
- """
696
- Legacy merged search function - now uses the unified search wrapper.
697
-
698
- This function is maintained for backward compatibility.
699
- New code should use the 'search' function with format_type="merged".
700
-
701
- Args:
702
- registry: The tool registry
703
- query: The search query
704
- working_memory: Optional working memory instance
705
-
706
- Returns:
707
- Dict with merged results and context
708
- """
709
- return search(registry, query, format_type="merged", working_memory=working_memory)
710
-
711
- if __name__ == "__main__":
712
- logging.basicConfig(level=logging.INFO)
713
-
714
- registry = create_default_registry()
715
-
716
- try:
717
- query = "latest Python version"
718
- result = robust_search(registry, query)
719
- print(f"Robust search found {len(result.get('results', []))} results")
720
- except Exception as e:
721
- print(f"Robust search failed: {str(e)}")
722
-
723
- try:
724
- query = "latest Python version"
725
- result = merged_search(registry, query)
726
- if result["perplexity_context"]:
727
- print("Perplexity context available")
728
- if result.get("arxiv_context"):
729
- print(f"arXiv context available with {len(result.get('arxiv_context', []))} results")
730
- print(f"Merged search found {len(result.get('merged_results', []))} results")
731
- except Exception as e:
732
- print(f"Merged search failed: {str(e)}")
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
app.py CHANGED
@@ -1,347 +1,347 @@
1
- import os
2
- import gradio as gr
3
- import requests
4
- import inspect
5
- import pandas as pd
6
- import logging
7
- import time
8
- from typing import Dict, Any, List, Optional, Tuple, Union
9
-
10
- DEFAULT_API_URL = "https://agents-course-unit4-scoring.hf.space"
11
-
12
- from agent import GAIAAgent
13
- from agent.config import VERBOSE
14
-
15
- logger = logging.getLogger("gaia_app")
16
- if VERBOSE:
17
- logging.basicConfig(level=logging.DEBUG, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')
18
- else:
19
- logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')
20
-
21
- def run_and_submit_all(username: str = "anonymous", progress=gr.Progress()) -> Tuple[str, Optional[pd.DataFrame]]:
22
- """
23
- Fetches all questions, runs the GAIA Agent on them, submits all answers,
24
- and displays the results with progress tracking.
25
-
26
- Args:
27
- username: Username for submission (default: anonymous)
28
- progress: Gradio progress tracker
29
-
30
- Returns:
31
- Tuple containing status message and results dataframe
32
- """
33
- space_id = os.getenv("SPACE_ID") # Get the SPACE_ID for sending link to the code
34
- logger.info(f"Running with username: {username}")
35
-
36
- api_url = DEFAULT_API_URL
37
- questions_url = f"{api_url}/questions"
38
- submit_url = f"{api_url}/submit"
39
-
40
- progress(0, desc="Initializing GAIA Agent...")
41
- try:
42
- start_time = time.time()
43
- agent = GAIAAgent()
44
- init_time = time.time() - start_time
45
- logger.info(f"GAIA Agent initialized successfully in {init_time:.2f} seconds.")
46
- progress(0.1, desc=f"Agent initialized in {init_time:.2f}s")
47
- except Exception as e:
48
- error_msg = f"Error initializing GAIA agent: {e}"
49
- logger.error(error_msg)
50
- return error_msg, None
51
-
52
- agent_code = f"https://huggingface.co/spaces/{space_id}/tree/main"
53
- logger.info(f"Agent code URL: {agent_code}")
54
-
55
- progress(0.15, desc="Fetching questions...")
56
- logger.info(f"Fetching questions from: {questions_url}")
57
- try:
58
- response = requests.get(questions_url, timeout=15)
59
- response.raise_for_status()
60
- questions_data = response.json()
61
- if not questions_data:
62
- error_msg = "Fetched questions list is empty or invalid format."
63
- logger.warning(error_msg)
64
- return error_msg, None
65
-
66
- question_count = len(questions_data)
67
- logger.info(f"Fetched {question_count} questions.")
68
- progress(0.2, desc=f"Fetched {question_count} questions")
69
- except requests.exceptions.RequestException as e:
70
- error_msg = f"Error fetching questions: {e}"
71
- logger.error(error_msg)
72
- return error_msg, None
73
- except requests.exceptions.JSONDecodeError as e:
74
- error_msg = f"Error decoding JSON response from questions endpoint: {e}"
75
- logger.error(f"{error_msg}\nResponse text: {response.text[:500]}")
76
- return error_msg, None
77
- except Exception as e:
78
- error_msg = f"An unexpected error occurred fetching questions: {e}"
79
- logger.error(error_msg)
80
- return error_msg, None
81
-
82
- results_log = []
83
- answers_payload = []
84
- total_questions = len(questions_data)
85
-
86
- logger.info(f"Running agent on {total_questions} questions...")
87
-
88
- question_progress_increment = 0.6 / total_questions # 60% of progress bar for processing questions
89
-
90
- for i, item in enumerate(questions_data):
91
- task_id = item.get("task_id")
92
- question_text = item.get("question")
93
-
94
- if not task_id or question_text is None:
95
- logger.warning(f"Skipping item with missing task_id or question: {item}")
96
- continue
97
-
98
- progress_value = 0.2 + (i * question_progress_increment)
99
- progress(progress_value, desc=f"Processing question {i+1}/{total_questions}...")
100
-
101
- logger.info(f"Processing question {i+1}/{total_questions}: {question_text[:50]}...")
102
- question_start_time = time.time()
103
-
104
- try:
105
- submitted_answer = agent(question_text)
106
- processing_time = time.time() - question_start_time
107
-
108
- answers_payload.append({"task_id": task_id, "submitted_answer": submitted_answer})
109
- results_log.append({
110
- "Task ID": task_id,
111
- "Question": question_text,
112
- "Submitted Answer": submitted_answer,
113
- "Processing Time": f"{processing_time:.2f}s",
114
- "Status": "Success"
115
- })
116
-
117
- logger.info(f"Question {i+1} processed in {processing_time:.2f}s. Answer: {submitted_answer[:50]}...")
118
-
119
- except Exception as e:
120
- error_msg = f"Error running agent on task {task_id}: {e}"
121
- logger.error(error_msg)
122
-
123
- results_log.append({
124
- "Task ID": task_id,
125
- "Question": question_text,
126
- "Submitted Answer": f"AGENT ERROR: {e}",
127
- "Processing Time": f"{time.time() - question_start_time:.2f}s",
128
- "Status": "Error"
129
- })
130
-
131
- if not answers_payload:
132
- error_msg = "Agent did not produce any answers to submit."
133
- logger.warning(error_msg)
134
- return error_msg, pd.DataFrame(results_log)
135
-
136
- # Only proceed with submission if not in demo mode
137
- progress(0.8, desc="Preparing submission...")
138
- submission_data = {"username": username.strip(), "agent_code": agent_code, "answers": answers_payload}
139
- status_update = f"Agent finished. Submitting {len(answers_payload)} answers for user '{username}'..."
140
- logger.info(status_update)
141
-
142
- progress(0.9, desc=f"Submitting {len(answers_payload)} answers...")
143
- logger.info(f"Submitting {len(answers_payload)} answers to: {submit_url}")
144
- try:
145
- response = requests.post(submit_url, json=submission_data, timeout=60)
146
- response.raise_for_status()
147
- result_data = response.json()
148
-
149
- correct_count = result_data.get('correct_count', 0)
150
- total_attempted = result_data.get('total_attempted', 0)
151
- score = result_data.get('score', 'N/A')
152
-
153
- final_status = (
154
- f"[SUCCESS] Submission Successful!\n"
155
- f"User: {result_data.get('username')}\n"
156
- f"Overall Score: {score}% "
157
- f"({correct_count}/{total_attempted} correct)\n"
158
- f"Message: {result_data.get('message', 'No message received.')}"
159
- )
160
- logger.info("Submission successful.")
161
-
162
- results_df = pd.DataFrame(results_log)
163
-
164
- if not results_df.empty:
165
- success_count = sum(1 for status in results_df.get("Status", []) if status == "Success")
166
- error_count = sum(1 for status in results_df.get("Status", []) if status == "Error")
167
-
168
- try:
169
- avg_time = sum(float(time.strip('s')) for time in results_df["Processing Time"]) / len(results_df)
170
- avg_time_str = f"{avg_time:.2f}s"
171
- except:
172
- avg_time_str = "N/A"
173
-
174
- logger.info(f"Processed {len(results_df)} questions: {success_count} successes, {error_count} errors, avg time: {avg_time_str}")
175
-
176
- progress(1.0, desc="Evaluation complete!")
177
- return final_status, results_df
178
- except requests.exceptions.HTTPError as e:
179
- progress(1.0, desc="Submission failed!")
180
- error_detail = f"Server responded with status {e.response.status_code}."
181
- try:
182
- error_json = e.response.json()
183
- error_detail += f" Detail: {error_json.get('detail', e.response.text)}"
184
- except requests.exceptions.JSONDecodeError:
185
- error_detail += f" Response: {e.response.text[:500]}"
186
- status_message = f"[ERROR] Submission Failed: {error_detail}"
187
- logger.error(status_message)
188
- results_df = pd.DataFrame(results_log)
189
- return status_message, results_df
190
- except requests.exceptions.Timeout:
191
- progress(1.0, desc="Submission timed out!")
192
- status_message = "[ERROR] Submission Failed: The request timed out."
193
- logger.error(status_message)
194
- results_df = pd.DataFrame(results_log)
195
- return status_message, results_df
196
- except requests.exceptions.RequestException as e:
197
- progress(1.0, desc="Network error!")
198
- status_message = f"[ERROR] Submission Failed: Network error - {e}"
199
- logger.error(status_message)
200
- results_df = pd.DataFrame(results_log)
201
- return status_message, results_df
202
- except Exception as e:
203
- progress(1.0, desc="Error during submission!")
204
- status_message = f"[ERROR] An unexpected error occurred during submission: {e}"
205
- logger.error(status_message)
206
- results_df = pd.DataFrame(results_log)
207
- return status_message, results_df
208
-
209
-
210
- # Check if we're running in a HF Space environment
211
- space_host = os.getenv("SPACE_HOST")
212
-
213
- # Create Gradio interface
214
- with gr.Blocks(theme=gr.themes.Soft()) as demo:
215
- gr.Markdown("# GAIA Agent Evaluation Runner")
216
- with gr.Tabs():
217
- with gr.TabItem("Run Evaluation"):
218
- gr.Markdown(
219
- """
220
- **Instructions:**
221
-
222
- 1. Enter your username (optional) for submission tracking.
223
- 2. Click 'Run Evaluation & Submit All Answers' to fetch questions, run the GAIA agent, submit answers, and see the score.
224
- 3. The progress bar will show you the current status of the evaluation.
225
- 4. Results will appear in the table below once processing is complete.
226
-
227
- **Note:** Running the evaluation may take some time as the agent processes all questions. Please be patient.
228
- """
229
- )
230
-
231
- with gr.Row():
232
- with gr.Column(scale=1):
233
- username_input = gr.Textbox(
234
- label="Username (optional)",
235
- placeholder="Enter username for submission",
236
- value="anonymous"
237
- )
238
- with gr.Column(scale=2):
239
- run_button = gr.Button("Run Evaluation & Submit All Answers",
240
- variant="primary", size="lg")
241
-
242
- status_output = gr.Textbox(
243
- label="Run Status / Submission Result",
244
- lines=6,
245
- interactive=False,
246
- elem_id="status_output"
247
- )
248
-
249
- results_accordion = gr.Accordion("Results", open=False, elem_id="results_accordion")
250
- with results_accordion:
251
- results_table = gr.DataFrame(
252
- label="Questions and Agent Answers",
253
- wrap=True,
254
- elem_id="results_table"
255
- )
256
-
257
- with gr.Row():
258
- with gr.Column():
259
- gr.Markdown("### Result Statistics")
260
- stats_output = gr.JSON(label="Statistics", visible=True)
261
-
262
- with gr.TabItem("About GAIA Agent"):
263
- gr.Markdown(
264
- """
265
- ## About GAIA Agent
266
-
267
- This agent is designed to answer questions from the GAIA (Grounded AI Alignment) benchmark.
268
-
269
- ### Key Features:
270
-
271
- - **Architecture**: Built with LangChain/LangGraph for workflow management
272
- - **Tools**: Integrates multiple tools for web search, reasoning, and multimodal understanding
273
- - **Memory**: Implements memory components for context retention
274
- - **Structured Approach**: Uses a systematic process for question analysis and answer formulation
275
-
276
- ### Components:
277
-
278
- 1. **Web Tools**: Search engines, content extraction, and web navigation
279
- 2. **Reasoning Tools**: Step-by-step reasoning, mathematical calculations, fact verification
280
- 3. **Multimodal Tools**: Image analysis, chart interpretation, document parsing
281
- 4. **Memory Components**: Conversation history, working memory, result caching
282
-
283
- ### Performance Metrics:
284
-
285
- The agent's performance is evaluated based on:
286
- - Accuracy of answers
287
- - Reasoning quality
288
- - Ability to use appropriate tools
289
- - Handling of edge cases
290
- """
291
- )
292
-
293
- with gr.TabItem("Help & Troubleshooting"):
294
- gr.Markdown(
295
- """
296
- ## Help & Troubleshooting
297
-
298
- ### Common Issues:
299
-
300
- 1. **Timeout Errors**: If the agent takes too long to process a question, try running the evaluation again.
301
- 2. **API Errors**: If you encounter API errors, check your internet connection and try again.
302
-
303
- ### Error Messages:
304
-
305
- - **"Error initializing GAIA agent"**: There was a problem setting up the agent. Check the logs for details.
306
- - **"Error fetching questions"**: The system couldn't retrieve questions from the API.
307
- - **"Submission Failed"**: There was a problem submitting your answers. Check the error details.
308
-
309
- ### Contact:
310
-
311
- If you continue to experience issues, please open an issue on the GitHub repository or contact the course instructors.
312
- """
313
- )
314
-
315
- run_button.click(
316
- fn=run_and_submit_all,
317
- inputs=[username_input],
318
- outputs=[status_output, results_table]
319
- ).then(
320
- fn=lambda df: {
321
- "Total Questions": len(df) if df is not None else 0,
322
- "Successful": sum(1 for status in df.get("Status", []) if status == "Success") if df is not None else 0,
323
- "Errors": sum(1 for status in df.get("Status", []) if status == "Error") if df is not None else 0,
324
- "Average Processing Time": f"{sum(float(time.strip('s')) for time in df['Processing Time']) / len(df):.2f}s" if df is not None and len(df) > 0 else "N/A"
325
- } if df is not None else {},
326
- inputs=[results_table],
327
- outputs=[stats_output]
328
- ).then(
329
- fn=lambda: gr.update(open=True, visible=True),
330
- outputs=[results_accordion]
331
- )
332
-
333
- if __name__ == "__main__":
334
- space_host_startup = os.getenv("SPACE_HOST")
335
- space_id_startup = os.getenv("SPACE_ID") # Get SPACE_ID at startup
336
-
337
- # Handle Space host configurations
338
- if space_host_startup:
339
- logger.info(f"Running on Hugging Face Space host: {space_host_startup}")
340
-
341
- # Print repository URLs if SPACE_ID is available
342
- if space_id_startup:
343
- repo_url = f"https://huggingface.co/spaces/{space_id_startup}"
344
- logger.info(f"Space repository URL: {repo_url}")
345
-
346
- # Launch the Gradio interface
347
  demo.launch(debug=True, share=False)
 
1
+ import os
2
+ import gradio as gr
3
+ import requests
4
+ import inspect
5
+ import pandas as pd
6
+ import logging
7
+ import time
8
+ from typing import Dict, Any, List, Optional, Tuple, Union
9
+
10
+ DEFAULT_API_URL = "https://agents-course-unit4-scoring.hf.space"
11
+
12
+ from agent import GAIAAgent
13
+ from agent.config import VERBOSE
14
+
15
+ logger = logging.getLogger("gaia_app")
16
+ if VERBOSE:
17
+ logging.basicConfig(level=logging.DEBUG, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')
18
+ else:
19
+ logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')
20
+
21
+ def run_and_submit_all(username: str = "anonymous", progress=gr.Progress()) -> Tuple[str, Optional[pd.DataFrame]]:
22
+ """
23
+ Fetches all questions, runs the GAIA Agent on them, submits all answers,
24
+ and displays the results with progress tracking.
25
+
26
+ Args:
27
+ username: Username for submission (default: anonymous)
28
+ progress: Gradio progress tracker
29
+
30
+ Returns:
31
+ Tuple containing status message and results dataframe
32
+ """
33
+ space_id = os.getenv("SPACE_ID") # Get the SPACE_ID for sending link to the code
34
+ logger.info(f"Running with username: {username}")
35
+
36
+ api_url = DEFAULT_API_URL
37
+ questions_url = f"{api_url}/questions"
38
+ submit_url = f"{api_url}/submit"
39
+
40
+ progress(0, desc="Initializing GAIA Agent...")
41
+ try:
42
+ start_time = time.time()
43
+ agent = GAIAAgent()
44
+ init_time = time.time() - start_time
45
+ logger.info(f"GAIA Agent initialized successfully in {init_time:.2f} seconds.")
46
+ progress(0.1, desc=f"Agent initialized in {init_time:.2f}s")
47
+ except Exception as e:
48
+ error_msg = f"Error initializing GAIA agent: {e}"
49
+ logger.error(error_msg)
50
+ return error_msg, None
51
+
52
+ agent_code = f"https://huggingface.co/spaces/{space_id}/tree/main"
53
+ logger.info(f"Agent code URL: {agent_code}")
54
+
55
+ progress(0.15, desc="Fetching questions...")
56
+ logger.info(f"Fetching questions from: {questions_url}")
57
+ try:
58
+ response = requests.get(questions_url, timeout=15)
59
+ response.raise_for_status()
60
+ questions_data = response.json()
61
+ if not questions_data:
62
+ error_msg = "Fetched questions list is empty or invalid format."
63
+ logger.warning(error_msg)
64
+ return error_msg, None
65
+
66
+ question_count = len(questions_data)
67
+ logger.info(f"Fetched {question_count} questions.")
68
+ progress(0.2, desc=f"Fetched {question_count} questions")
69
+ except requests.exceptions.RequestException as e:
70
+ error_msg = f"Error fetching questions: {e}"
71
+ logger.error(error_msg)
72
+ return error_msg, None
73
+ except requests.exceptions.JSONDecodeError as e:
74
+ error_msg = f"Error decoding JSON response from questions endpoint: {e}"
75
+ logger.error(f"{error_msg}\nResponse text: {response.text[:500]}")
76
+ return error_msg, None
77
+ except Exception as e:
78
+ error_msg = f"An unexpected error occurred fetching questions: {e}"
79
+ logger.error(error_msg)
80
+ return error_msg, None
81
+
82
+ results_log = []
83
+ answers_payload = []
84
+ total_questions = len(questions_data)
85
+
86
+ logger.info(f"Running agent on {total_questions} questions...")
87
+
88
+ question_progress_increment = 0.6 / total_questions # 60% of progress bar for processing questions
89
+
90
+ for i, item in enumerate(questions_data):
91
+ task_id = item.get("task_id")
92
+ question_text = item.get("question")
93
+
94
+ if not task_id or question_text is None:
95
+ logger.warning(f"Skipping item with missing task_id or question: {item}")
96
+ continue
97
+
98
+ progress_value = 0.2 + (i * question_progress_increment)
99
+ progress(progress_value, desc=f"Processing question {i+1}/{total_questions}...")
100
+
101
+ logger.info(f"Processing question {i+1}/{total_questions}: {question_text[:50]}...")
102
+ question_start_time = time.time()
103
+
104
+ try:
105
+ submitted_answer = agent(question_text)
106
+ processing_time = time.time() - question_start_time
107
+
108
+ answers_payload.append({"task_id": task_id, "submitted_answer": submitted_answer})
109
+ results_log.append({
110
+ "Task ID": task_id,
111
+ "Question": question_text,
112
+ "Submitted Answer": submitted_answer,
113
+ "Processing Time": f"{processing_time:.2f}s",
114
+ "Status": "Success"
115
+ })
116
+
117
+ logger.info(f"Question {i+1} processed in {processing_time:.2f}s. Answer: {submitted_answer[:50]}...")
118
+
119
+ except Exception as e:
120
+ error_msg = f"Error running agent on task {task_id}: {e}"
121
+ logger.error(error_msg)
122
+
123
+ results_log.append({
124
+ "Task ID": task_id,
125
+ "Question": question_text,
126
+ "Submitted Answer": f"AGENT ERROR: {e}",
127
+ "Processing Time": f"{time.time() - question_start_time:.2f}s",
128
+ "Status": "Error"
129
+ })
130
+
131
+ if not answers_payload:
132
+ error_msg = "Agent did not produce any answers to submit."
133
+ logger.warning(error_msg)
134
+ return error_msg, pd.DataFrame(results_log)
135
+
136
+ # Only proceed with submission if not in demo mode
137
+ progress(0.8, desc="Preparing submission...")
138
+ submission_data = {"username": username.strip(), "agent_code": agent_code, "answers": answers_payload}
139
+ status_update = f"Agent finished. Submitting {len(answers_payload)} answers for user '{username}'..."
140
+ logger.info(status_update)
141
+
142
+ progress(0.9, desc=f"Submitting {len(answers_payload)} answers...")
143
+ logger.info(f"Submitting {len(answers_payload)} answers to: {submit_url}")
144
+ try:
145
+ response = requests.post(submit_url, json=submission_data, timeout=60)
146
+ response.raise_for_status()
147
+ result_data = response.json()
148
+
149
+ correct_count = result_data.get('correct_count', 0)
150
+ total_attempted = result_data.get('total_attempted', 0)
151
+ score = result_data.get('score', 'N/A')
152
+
153
+ final_status = (
154
+ f"[SUCCESS] Submission Successful!\n"
155
+ f"User: {result_data.get('username')}\n"
156
+ f"Overall Score: {score}% "
157
+ f"({correct_count}/{total_attempted} correct)\n"
158
+ f"Message: {result_data.get('message', 'No message received.')}"
159
+ )
160
+ logger.info("Submission successful.")
161
+
162
+ results_df = pd.DataFrame(results_log)
163
+
164
+ if not results_df.empty:
165
+ success_count = sum(1 for status in results_df.get("Status", []) if status == "Success")
166
+ error_count = sum(1 for status in results_df.get("Status", []) if status == "Error")
167
+
168
+ try:
169
+ avg_time = sum(float(time.strip('s')) for time in results_df["Processing Time"]) / len(results_df)
170
+ avg_time_str = f"{avg_time:.2f}s"
171
+ except:
172
+ avg_time_str = "N/A"
173
+
174
+ logger.info(f"Processed {len(results_df)} questions: {success_count} successes, {error_count} errors, avg time: {avg_time_str}")
175
+
176
+ progress(1.0, desc="Evaluation complete!")
177
+ return final_status, results_df
178
+ except requests.exceptions.HTTPError as e:
179
+ progress(1.0, desc="Submission failed!")
180
+ error_detail = f"Server responded with status {e.response.status_code}."
181
+ try:
182
+ error_json = e.response.json()
183
+ error_detail += f" Detail: {error_json.get('detail', e.response.text)}"
184
+ except requests.exceptions.JSONDecodeError:
185
+ error_detail += f" Response: {e.response.text[:500]}"
186
+ status_message = f"[ERROR] Submission Failed: {error_detail}"
187
+ logger.error(status_message)
188
+ results_df = pd.DataFrame(results_log)
189
+ return status_message, results_df
190
+ except requests.exceptions.Timeout:
191
+ progress(1.0, desc="Submission timed out!")
192
+ status_message = "[ERROR] Submission Failed: The request timed out."
193
+ logger.error(status_message)
194
+ results_df = pd.DataFrame(results_log)
195
+ return status_message, results_df
196
+ except requests.exceptions.RequestException as e:
197
+ progress(1.0, desc="Network error!")
198
+ status_message = f"[ERROR] Submission Failed: Network error - {e}"
199
+ logger.error(status_message)
200
+ results_df = pd.DataFrame(results_log)
201
+ return status_message, results_df
202
+ except Exception as e:
203
+ progress(1.0, desc="Error during submission!")
204
+ status_message = f"[ERROR] An unexpected error occurred during submission: {e}"
205
+ logger.error(status_message)
206
+ results_df = pd.DataFrame(results_log)
207
+ return status_message, results_df
208
+
209
+
210
+ # Check if we're running in a HF Space environment
211
+ space_host = os.getenv("SPACE_HOST")
212
+
213
+ # Create Gradio interface
214
+ with gr.Blocks(theme=gr.themes.Soft()) as demo:
215
+ gr.Markdown("# GAIA Agent Evaluation Runner")
216
+ with gr.Tabs():
217
+ with gr.TabItem("Run Evaluation"):
218
+ gr.Markdown(
219
+ """
220
+ **Instructions:**
221
+
222
+ 1. Enter your username (optional) for submission tracking.
223
+ 2. Click 'Run Evaluation & Submit All Answers' to fetch questions, run the GAIA agent, submit answers, and see the score.
224
+ 3. The progress bar will show you the current status of the evaluation.
225
+ 4. Results will appear in the table below once processing is complete.
226
+
227
+ **Note:** Running the evaluation may take some time as the agent processes all questions. Please be patient.
228
+ """
229
+ )
230
+
231
+ with gr.Row():
232
+ with gr.Column(scale=1):
233
+ username_input = gr.Textbox(
234
+ label="Username (optional)",
235
+ placeholder="Enter username for submission",
236
+ value="anonymous"
237
+ )
238
+ with gr.Column(scale=2):
239
+ run_button = gr.Button("Run Evaluation & Submit All Answers",
240
+ variant="primary", size="lg")
241
+
242
+ status_output = gr.Textbox(
243
+ label="Run Status / Submission Result",
244
+ lines=6,
245
+ interactive=False,
246
+ elem_id="status_output"
247
+ )
248
+
249
+ results_accordion = gr.Accordion("Results", open=False, elem_id="results_accordion")
250
+ with results_accordion:
251
+ results_table = gr.DataFrame(
252
+ label="Questions and Agent Answers",
253
+ wrap=True,
254
+ elem_id="results_table"
255
+ )
256
+
257
+ with gr.Row():
258
+ with gr.Column():
259
+ gr.Markdown("### Result Statistics")
260
+ stats_output = gr.JSON(label="Statistics", visible=True)
261
+
262
+ with gr.TabItem("About GAIA Agent"):
263
+ gr.Markdown(
264
+ """
265
+ ## About GAIA Agent
266
+
267
+ This agent is designed to answer questions from the GAIA (Grounded AI Alignment) benchmark.
268
+
269
+ ### Key Features:
270
+
271
+ - **Architecture**: Built with LangChain/LangGraph for workflow management
272
+ - **Tools**: Integrates multiple tools for web search, reasoning, and multimodal understanding
273
+ - **Memory**: Implements memory components for context retention
274
+ - **Structured Approach**: Uses a systematic process for question analysis and answer formulation
275
+
276
+ ### Components:
277
+
278
+ 1. **Web Tools**: Search engines, content extraction, and web navigation
279
+ 2. **Reasoning Tools**: Step-by-step reasoning, mathematical calculations, fact verification
280
+ 3. **Multimodal Tools**: Image analysis, chart interpretation, document parsing
281
+ 4. **Memory Components**: Conversation history, working memory, result caching
282
+
283
+ ### Performance Metrics:
284
+
285
+ The agent's performance is evaluated based on:
286
+ - Accuracy of answers
287
+ - Reasoning quality
288
+ - Ability to use appropriate tools
289
+ - Handling of edge cases
290
+ """
291
+ )
292
+
293
+ with gr.TabItem("Help & Troubleshooting"):
294
+ gr.Markdown(
295
+ """
296
+ ## Help & Troubleshooting
297
+
298
+ ### Common Issues:
299
+
300
+ 1. **Timeout Errors**: If the agent takes too long to process a question, try running the evaluation again.
301
+ 2. **API Errors**: If you encounter API errors, check your internet connection and try again.
302
+
303
+ ### Error Messages:
304
+
305
+ - **"Error initializing GAIA agent"**: There was a problem setting up the agent. Check the logs for details.
306
+ - **"Error fetching questions"**: The system couldn't retrieve questions from the API.
307
+ - **"Submission Failed"**: There was a problem submitting your answers. Check the error details.
308
+
309
+ ### Contact:
310
+
311
+ If you continue to experience issues, please open an issue on the GitHub repository or contact the course instructors.
312
+ """
313
+ )
314
+
315
+ run_button.click(
316
+ fn=run_and_submit_all,
317
+ inputs=[username_input],
318
+ outputs=[status_output, results_table]
319
+ ).then(
320
+ fn=lambda df: {
321
+ "Total Questions": len(df) if df is not None else 0,
322
+ "Successful": sum(1 for status in df.get("Status", []) if status == "Success") if df is not None else 0,
323
+ "Errors": sum(1 for status in df.get("Status", []) if status == "Error") if df is not None else 0,
324
+ "Average Processing Time": f"{sum(float(time.strip('s')) for time in df['Processing Time']) / len(df):.2f}s" if df is not None and len(df) > 0 else "N/A"
325
+ } if df is not None else {},
326
+ inputs=[results_table],
327
+ outputs=[stats_output]
328
+ ).then(
329
+ fn=lambda: gr.update(open=True, visible=True),
330
+ outputs=[results_accordion]
331
+ )
332
+
333
+ if __name__ == "__main__":
334
+ space_host_startup = os.getenv("SPACE_HOST")
335
+ space_id_startup = os.getenv("SPACE_ID") # Get SPACE_ID at startup
336
+
337
+ # Handle Space host configurations
338
+ if space_host_startup:
339
+ logger.info(f"Running on Hugging Face Space host: {space_host_startup}")
340
+
341
+ # Print repository URLs if SPACE_ID is available
342
+ if space_id_startup:
343
+ repo_url = f"https://huggingface.co/spaces/{space_id_startup}"
344
+ logger.info(f"Space repository URL: {repo_url}")
345
+
346
+ # Launch the Gradio interface
347
  demo.launch(debug=True, share=False)
app_demo_mode_patch.py DELETED
@@ -1,104 +0,0 @@
1
- """
2
- Patch for app.py to add demo mode support
3
- This allows the app to run without OAuth for testing/development
4
- """
5
- import os
6
- import sys
7
-
8
- def apply_demo_mode_patch():
9
- """
10
- Applies a patch to app.py to add demo mode support
11
- Returns True if successful, False otherwise
12
- """
13
- # Check if app.py exists
14
- if not os.path.exists('app.py'):
15
- print("Error: app.py not found in current directory")
16
- return False
17
-
18
- # Read the original file
19
- with open('app.py', 'r') as f:
20
- content = f.read()
21
-
22
- # Check if already patched
23
- if 'DEMO_MODE' in content:
24
- print("app.py already has demo mode support")
25
- return True
26
-
27
- # Make the modifications
28
- # 1. Add demo mode imports and configuration after the existing imports
29
- import_section_end = "from agent.config import VERBOSE"
30
- demo_mode_code = """from agent.config import VERBOSE
31
-
32
- # Set to True to enable demo mode (skip OAuth requirement)
33
- DEMO_MODE = os.getenv("DEMO_MODE", "").lower() in ("true", "1", "yes")
34
- """
35
-
36
- # Replace the import section
37
- content = content.replace(import_section_end, demo_mode_code)
38
-
39
- # 2. Modify the OAuth check in run_and_submit_all function
40
- oauth_check = """ # OAuth is REQUIRED for assessment submission
41
- if not profile:
42
- logger.warning("User not logged in.")
43
- return "Please Login to Hugging Face with the button. OAuth login is required for assessment submission.", None"""
44
-
45
- demo_mode_oauth_check = """ # OAuth is REQUIRED for assessment submission (unless in demo mode)
46
- if not profile and not DEMO_MODE:
47
- logger.warning("User not logged in.")
48
- return "Please Login to Hugging Face with the button. OAuth login is required for assessment submission.", None
49
-
50
- # Use demo username in demo mode
51
- if DEMO_MODE and not profile:
52
- logger.warning("Running in DEMO MODE - using demo username")
53
- username = "demo_user"
54
- else:
55
- username = f"{profile.username}\""""
56
-
57
- # Replace the OAuth check
58
- content = content.replace(oauth_check, demo_mode_oauth_check)
59
-
60
- # 3. Modify the blocks initialization to add a demo mode indicator
61
- blocks_init = "with gr.Blocks(theme=gr.themes.Soft()) as demo:"
62
- demo_mode_blocks_init = """# Add demo mode indicator if enabled
63
- if DEMO_MODE:
64
- logger.warning("Running in DEMO MODE - OAuth requirement disabled")
65
-
66
- with gr.Blocks(theme=gr.themes.Soft()) as demo:
67
- if DEMO_MODE:
68
- gr.Markdown("# GAIA Agent Evaluation Runner (DEMO MODE)")
69
- else:
70
- gr.Markdown("# GAIA Agent Evaluation Runner")"""
71
-
72
- # Replace blocks initialization
73
- content = content.replace(blocks_init, demo_mode_blocks_init)
74
-
75
- # 4. Add demo mode warning at the end
76
- demo_warning = """
77
- # Add demo mode warning
78
- if DEMO_MODE:
79
- gr.Markdown(
80
- \"\"\"
81
- ## 🚨 DEMO MODE ACTIVE 🚨
82
-
83
- Running in demo mode - OAuth login is disabled.
84
- Submissions will not be recorded to the leaderboard.
85
-
86
- For actual assessment submission, set DEMO_MODE=false and enable OAuth in Space settings.
87
- \"\"\"
88
- )
89
- """
90
-
91
- # Add the warning before the if __name__ section
92
- name_main_section = "if __name__ == \"__main__\":"
93
- content = content.replace(name_main_section, demo_warning + "\n" + name_main_section)
94
-
95
- # Write the modified content back to app.py
96
- with open('app.py', 'w') as f:
97
- f.write(content)
98
-
99
- print("Successfully applied demo mode patch to app.py")
100
- return True
101
-
102
- if __name__ == "__main__":
103
- success = apply_demo_mode_patch()
104
- sys.exit(0 if success else 1)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
app_fixed.py DELETED
@@ -1,367 +0,0 @@
1
- import os
2
- import gradio as gr
3
- import requests
4
- import inspect
5
- import pandas as pd
6
- import logging
7
- import time
8
- from typing import Dict, Any, List, Optional, Tuple, Union
9
-
10
- DEFAULT_API_URL = "https://agents-course-unit4-scoring.hf.space"
11
-
12
- from agent import GAIAAgent
13
- from agent.config import VERBOSE
14
-
15
- logger = logging.getLogger("gaia_app")
16
- if VERBOSE:
17
- logging.basicConfig(level=logging.DEBUG, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')
18
- else:
19
- logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')
20
-
21
- def run_and_submit_all(profile: Optional[gr.OAuthProfile] = None, progress=gr.Progress()) -> Tuple[str, Optional[pd.DataFrame]]:
22
- """
23
- Fetches all questions, runs the GAIA Agent on them, submits all answers,
24
- and displays the results with progress tracking.
25
-
26
- Args:
27
- profile: Hugging Face user profile for authentication
28
- progress: Gradio progress tracker
29
-
30
- Returns:
31
- Tuple containing status message and results dataframe
32
- """
33
- space_id = os.getenv("SPACE_ID") # Get the SPACE_ID for sending link to the code
34
- # OAuth is REQUIRED for assessment submission
35
- if not profile:
36
- logger.warning("User not logged in.")
37
- return "Please Login to Hugging Face with the button. OAuth login is required for assessment submission.", None
38
-
39
- username = f"{profile.username}"
40
- logger.info(f"User logged in: {username}")
41
-
42
- api_url = DEFAULT_API_URL
43
- questions_url = f"{api_url}/questions"
44
- submit_url = f"{api_url}/submit"
45
-
46
- progress(0, desc="Initializing GAIA Agent...")
47
- try:
48
- start_time = time.time()
49
- agent = GAIAAgent()
50
- init_time = time.time() - start_time
51
- logger.info(f"GAIA Agent initialized successfully in {init_time:.2f} seconds.")
52
- progress(0.1, desc=f"Agent initialized in {init_time:.2f}s")
53
- except Exception as e:
54
- error_msg = f"Error initializing GAIA agent: {e}"
55
- logger.error(error_msg)
56
- return error_msg, None
57
-
58
- agent_code = f"https://huggingface.co/spaces/{space_id}/tree/main"
59
- logger.info(f"Agent code URL: {agent_code}")
60
-
61
- progress(0.15, desc="Fetching questions...")
62
- logger.info(f"Fetching questions from: {questions_url}")
63
- try:
64
- response = requests.get(questions_url, timeout=15)
65
- response.raise_for_status()
66
- questions_data = response.json()
67
- if not questions_data:
68
- error_msg = "Fetched questions list is empty or invalid format."
69
- logger.warning(error_msg)
70
- return error_msg, None
71
-
72
- question_count = len(questions_data)
73
- logger.info(f"Fetched {question_count} questions.")
74
- progress(0.2, desc=f"Fetched {question_count} questions")
75
- except requests.exceptions.RequestException as e:
76
- error_msg = f"Error fetching questions: {e}"
77
- logger.error(error_msg)
78
- return error_msg, None
79
- except requests.exceptions.JSONDecodeError as e:
80
- error_msg = f"Error decoding JSON response from questions endpoint: {e}"
81
- logger.error(f"{error_msg}\nResponse text: {response.text[:500]}")
82
- return error_msg, None
83
- except Exception as e:
84
- error_msg = f"An unexpected error occurred fetching questions: {e}"
85
- logger.error(error_msg)
86
- return error_msg, None
87
-
88
- results_log = []
89
- answers_payload = []
90
- total_questions = len(questions_data)
91
-
92
- logger.info(f"Running agent on {total_questions} questions...")
93
-
94
- question_progress_increment = 0.6 / total_questions # 60% of progress bar for processing questions
95
-
96
- for i, item in enumerate(questions_data):
97
- task_id = item.get("task_id")
98
- question_text = item.get("question")
99
-
100
- if not task_id or question_text is None:
101
- logger.warning(f"Skipping item with missing task_id or question: {item}")
102
- continue
103
-
104
- progress_value = 0.2 + (i * question_progress_increment)
105
- progress(progress_value, desc=f"Processing question {i+1}/{total_questions}...")
106
-
107
- logger.info(f"Processing question {i+1}/{total_questions}: {question_text[:50]}...")
108
- question_start_time = time.time()
109
-
110
- try:
111
- submitted_answer = agent(question_text)
112
- processing_time = time.time() - question_start_time
113
-
114
- answers_payload.append({"task_id": task_id, "submitted_answer": submitted_answer})
115
- results_log.append({
116
- "Task ID": task_id,
117
- "Question": question_text,
118
- "Submitted Answer": submitted_answer,
119
- "Processing Time": f"{processing_time:.2f}s",
120
- "Status": "Success"
121
- })
122
-
123
- logger.info(f"Question {i+1} processed in {processing_time:.2f}s. Answer: {submitted_answer[:50]}...")
124
-
125
- except Exception as e:
126
- error_msg = f"Error running agent on task {task_id}: {e}"
127
- logger.error(error_msg)
128
-
129
- results_log.append({
130
- "Task ID": task_id,
131
- "Question": question_text,
132
- "Submitted Answer": f"AGENT ERROR: {e}",
133
- "Processing Time": f"{time.time() - question_start_time:.2f}s",
134
- "Status": "Error"
135
- })
136
-
137
- if not answers_payload:
138
- error_msg = "Agent did not produce any answers to submit."
139
- logger.warning(error_msg)
140
- return error_msg, pd.DataFrame(results_log)
141
-
142
- # Only proceed with submission if not in demo mode
143
- progress(0.8, desc="Preparing submission...")
144
- submission_data = {"username": username.strip(), "agent_code": agent_code, "answers": answers_payload}
145
- status_update = f"Agent finished. Submitting {len(answers_payload)} answers for user '{username}'..."
146
- logger.info(status_update)
147
-
148
- progress(0.9, desc=f"Submitting {len(answers_payload)} answers...")
149
- logger.info(f"Submitting {len(answers_payload)} answers to: {submit_url}")
150
- try:
151
- response = requests.post(submit_url, json=submission_data, timeout=60)
152
- response.raise_for_status()
153
- result_data = response.json()
154
-
155
- correct_count = result_data.get('correct_count', 0)
156
- total_attempted = result_data.get('total_attempted', 0)
157
- score = result_data.get('score', 'N/A')
158
-
159
- final_status = (
160
- f"[SUCCESS] Submission Successful!\n"
161
- f"User: {result_data.get('username')}\n"
162
- f"Overall Score: {score}% "
163
- f"({correct_count}/{total_attempted} correct)\n"
164
- f"Message: {result_data.get('message', 'No message received.')}"
165
- )
166
- logger.info("Submission successful.")
167
-
168
- results_df = pd.DataFrame(results_log)
169
-
170
- if not results_df.empty:
171
- success_count = sum(1 for status in results_df.get("Status", []) if status == "Success")
172
- error_count = sum(1 for status in results_df.get("Status", []) if status == "Error")
173
-
174
- try:
175
- avg_time = sum(float(time.strip('s')) for time in results_df["Processing Time"]) / len(results_df)
176
- avg_time_str = f"{avg_time:.2f}s"
177
- except:
178
- avg_time_str = "N/A"
179
-
180
- logger.info(f"Processed {len(results_df)} questions: {success_count} successes, {error_count} errors, avg time: {avg_time_str}")
181
-
182
- progress(1.0, desc="Evaluation complete!")
183
- return final_status, results_df
184
- except requests.exceptions.HTTPError as e:
185
- progress(1.0, desc="Submission failed!")
186
- error_detail = f"Server responded with status {e.response.status_code}."
187
- try:
188
- error_json = e.response.json()
189
- error_detail += f" Detail: {error_json.get('detail', e.response.text)}"
190
- except requests.exceptions.JSONDecodeError:
191
- error_detail += f" Response: {e.response.text[:500]}"
192
- status_message = f"[ERROR] Submission Failed: {error_detail}"
193
- logger.error(status_message)
194
- results_df = pd.DataFrame(results_log)
195
- return status_message, results_df
196
- except requests.exceptions.Timeout:
197
- progress(1.0, desc="Submission timed out!")
198
- status_message = "[ERROR] Submission Failed: The request timed out."
199
- logger.error(status_message)
200
- results_df = pd.DataFrame(results_log)
201
- return status_message, results_df
202
- except requests.exceptions.RequestException as e:
203
- progress(1.0, desc="Network error!")
204
- status_message = f"[ERROR] Submission Failed: Network error - {e}"
205
- logger.error(status_message)
206
- results_df = pd.DataFrame(results_log)
207
- return status_message, results_df
208
- except Exception as e:
209
- progress(1.0, desc="Error during submission!")
210
- status_message = f"[ERROR] An unexpected error occurred during submission: {e}"
211
- logger.error(status_message)
212
- results_df = pd.DataFrame(results_log)
213
- return status_message, results_df
214
-
215
-
216
- # Check if we're running in a HF Space environment
217
- space_host = os.getenv("SPACE_HOST")
218
-
219
- # Import HfApi for OAuth functionality
220
- try:
221
- from huggingface_hub import HfApi
222
- except ImportError:
223
- logger.warning("huggingface_hub not available - please install it for OAuth functionality")
224
- except Exception as e:
225
- logger.warning(f"Error importing huggingface_hub: {e}")
226
-
227
- with gr.Blocks(theme=gr.themes.Soft()) as demo:
228
- gr.Markdown("# GAIA Agent Evaluation Runner")
229
- with gr.Tabs():
230
- with gr.TabItem("Run Evaluation"):
231
- gr.Markdown(
232
- """
233
- **Instructions:**
234
-
235
- 1. Log in to your Hugging Face account using the button below. This uses your HF username for submission.
236
- 2. Click 'Run Evaluation & Submit All Answers' to fetch questions, run the GAIA agent, submit answers, and see the score.
237
- 3. The progress bar will show you the current status of the evaluation.
238
- 4. Results will appear in the table below once processing is complete.
239
-
240
- **Note:** Running the evaluation may take some time as the agent processes all questions. Please be patient.
241
- """
242
- )
243
-
244
- with gr.Row():
245
- with gr.Column(scale=1):
246
- gr.LoginButton()
247
- with gr.Column(scale=2):
248
- run_button = gr.Button("Run Evaluation & Submit All Answers",
249
- variant="primary", size="lg")
250
-
251
- status_output = gr.Textbox(
252
- label="Run Status / Submission Result",
253
- lines=6,
254
- interactive=False,
255
- elem_id="status_output"
256
- )
257
-
258
- results_accordion = gr.Accordion("Results", open=False, elem_id="results_accordion")
259
- with results_accordion:
260
- results_table = gr.DataFrame(
261
- label="Questions and Agent Answers",
262
- wrap=True,
263
- elem_id="results_table"
264
- )
265
-
266
- with gr.Row():
267
- with gr.Column():
268
- gr.Markdown("### Result Statistics")
269
- stats_output = gr.JSON(label="Statistics", visible=True)
270
-
271
- with gr.TabItem("About GAIA Agent"):
272
- gr.Markdown(
273
- """
274
- ## About GAIA Agent
275
-
276
- This agent is designed to answer questions from the GAIA (Grounded AI Alignment) benchmark.
277
-
278
- ### Key Features:
279
-
280
- - **Architecture**: Built with LangChain/LangGraph for workflow management
281
- - **Tools**: Integrates multiple tools for web search, reasoning, and multimodal understanding
282
- - **Memory**: Implements memory components for context retention
283
- - **Structured Approach**: Uses a systematic process for question analysis and answer formulation
284
-
285
- ### Components:
286
-
287
- 1. **Web Tools**: Search engines, content extraction, and web navigation
288
- 2. **Reasoning Tools**: Step-by-step reasoning, mathematical calculations, fact verification
289
- 3. **Multimodal Tools**: Image analysis, chart interpretation, document parsing
290
- 4. **Memory Components**: Conversation history, working memory, result caching
291
-
292
- ### Performance Metrics:
293
-
294
- The agent's performance is evaluated based on:
295
- - Accuracy of answers
296
- - Reasoning quality
297
- - Ability to use appropriate tools
298
- - Handling of edge cases
299
- """
300
- )
301
-
302
- with gr.TabItem("Help & Troubleshooting"):
303
- gr.Markdown(
304
- """
305
- ## Help & Troubleshooting
306
-
307
- ### Common Issues:
308
-
309
- 1. **Login Problems**: Make sure you're logged in to Hugging Face before running the evaluation.
310
- 2. **Timeout Errors**: If the agent takes too long to process a question, try running the evaluation again.
311
- 3. **API Errors**: If you encounter API errors, check your internet connection and try again.
312
-
313
- ### Error Messages:
314
-
315
- - **"Please Login to Hugging Face"**: You need to log in using the login button.
316
- - **"Error initializing GAIA agent"**: There was a problem setting up the agent. Check the logs for details.
317
- - **"Error fetching questions"**: The system couldn't retrieve questions from the API.
318
- - **"Submission Failed"**: There was a problem submitting your answers. Check the error details.
319
-
320
- ### Contact:
321
-
322
- If you continue to experience issues, please open an issue on the GitHub repository or contact the course instructors.
323
- """
324
- )
325
-
326
- run_button.click(
327
- fn=run_and_submit_all,
328
- outputs=[status_output, results_table]
329
- ).then(
330
- fn=lambda df: {
331
- "Total Questions": len(df) if df is not None else 0,
332
- "Successful": sum(1 for status in df.get("Status", []) if status == "Success") if df is not None else 0,
333
- "Errors": sum(1 for status in df.get("Status", []) if status == "Error") if df is not None else 0,
334
- "Average Processing Time": f"{sum(float(time.strip('s')) for time in df['Processing Time']) / len(df):.2f}s" if df is not None and len(df) > 0 else "N/A"
335
- } if df is not None else {},
336
- inputs=[results_table],
337
- outputs=[stats_output]
338
- ).then(
339
- fn=lambda: gr.update(open=True, visible=True),
340
- outputs=[results_accordion]
341
- )
342
-
343
- # Add note that OAuth is required
344
- gr.Markdown(
345
- """
346
- ## [WARNING] Important Note
347
-
348
- This GAIA Assessment requires OAuth login for submission to the leaderboard.
349
- Please make sure to log in with your Hugging Face account before running the evaluation.
350
- """
351
- )
352
-
353
- if __name__ == "__main__":
354
- space_host_startup = os.getenv("SPACE_HOST")
355
- space_id_startup = os.getenv("SPACE_ID") # Get SPACE_ID at startup
356
-
357
- # Handle Space host configurations
358
- if space_host_startup:
359
- logger.info(f"Running on Hugging Face Space host: {space_host_startup}")
360
-
361
- # Print repository URLs if SPACE_ID is available
362
- if space_id_startup:
363
- repo_url = f"https://huggingface.co/spaces/{space_id_startup}"
364
- logger.info(f"Space repository URL: {repo_url}")
365
-
366
- # Launch the Gradio interface
367
- demo.launch(debug=True, share=False)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
app_no_oauth.py DELETED
@@ -1,226 +0,0 @@
1
- import os
2
- import gradio as gr
3
- import requests
4
- import inspect
5
- import pandas as pd
6
- import logging
7
- import time
8
- from typing import Dict, Any, List, Optional, Tuple, Union
9
-
10
- DEFAULT_API_URL = "https://agents-course-unit4-scoring.hf.space"
11
-
12
- from agent import GAIAAgent
13
- from agent.config import VERBOSE
14
-
15
- logger = logging.getLogger("gaia_app")
16
- if VERBOSE:
17
- logging.basicConfig(level=logging.DEBUG, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')
18
- else:
19
- logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')
20
-
21
- def run_and_submit_all(username: str = "anonymous", progress=gr.Progress()) -> Tuple[str, Optional[pd.DataFrame]]:
22
- """
23
- Fetches all questions, runs the GAIA Agent on them, submits all answers,
24
- and displays the results with progress tracking.
25
-
26
- Args:
27
- username: Username for submission (default: anonymous)
28
- progress: Gradio progress tracker
29
-
30
- Returns:
31
- Tuple of (status message, results dataframe)
32
- """
33
- api_url = os.environ.get("GAIA_API_URL", DEFAULT_API_URL)
34
-
35
- try:
36
- # Step 1: Fetch questions
37
- progress(0, desc="Fetching questions...")
38
- questions_response = requests.get(f"{api_url}/questions")
39
- questions_response.raise_for_status()
40
- questions = questions_response.json()
41
-
42
- if not questions:
43
- return "Error: No questions received from the API.", None
44
-
45
- # Step 2: Initialize agent and prepare for processing
46
- progress(0.1, desc="Initializing GAIA Agent...")
47
- agent = GAIAAgent()
48
-
49
- # Step 3: Process each question
50
- results = []
51
- total_questions = len(questions)
52
-
53
- for i, question in enumerate(questions):
54
- question_id = question.get("id", f"q{i+1}")
55
- question_text = question.get("question", "")
56
-
57
- progress_value = 0.1 + (0.7 * (i / total_questions))
58
- progress(progress_value, desc=f"Processing question {i+1}/{total_questions}: {question_text[:30]}...")
59
-
60
- # Run the agent on the question
61
- try:
62
- start_time = time.time()
63
- answer = agent.run(question_text)
64
- end_time = time.time()
65
- processing_time = end_time - start_time
66
-
67
- results.append({
68
- "question_id": question_id,
69
- "question": question_text,
70
- "answer": answer,
71
- "processing_time": f"{processing_time:.2f}s"
72
- })
73
-
74
- logger.info(f"Processed question {question_id} in {processing_time:.2f}s")
75
- except Exception as e:
76
- error_msg = f"Error processing question {question_id}: {str(e)}"
77
- logger.error(error_msg)
78
- results.append({
79
- "question_id": question_id,
80
- "question": question_text,
81
- "answer": f"ERROR: {str(e)}",
82
- "processing_time": "N/A"
83
- })
84
-
85
- # Step 4: Submit all answers
86
- progress(0.8, desc="Submitting answers...")
87
-
88
- answers_payload = {
89
- "username": username,
90
- "answers": {item["question_id"]: item["answer"] for item in results}
91
- }
92
-
93
- submission_response = requests.post(
94
- f"{api_url}/submit",
95
- json=answers_payload
96
- )
97
-
98
- # Step 5: Process submission response
99
- progress(0.9, desc="Processing submission results...")
100
-
101
- if submission_response.status_code == 200:
102
- submission_data = submission_response.json()
103
- score = submission_data.get("score", "N/A")
104
- feedback = submission_data.get("feedback", {})
105
-
106
- # Add score and feedback to results
107
- for item in results:
108
- question_id = item["question_id"]
109
- item["score"] = feedback.get(question_id, {}).get("score", "N/A")
110
- item["feedback"] = feedback.get(question_id, {}).get("feedback", "")
111
-
112
- # Create DataFrame for display
113
- results_df = pd.DataFrame(results)
114
-
115
- status_message = f"Submission successful! Overall score: {score}"
116
- logger.info(status_message)
117
-
118
- progress(1.0, desc="Complete!")
119
- return status_message, results_df
120
- else:
121
- error_message = f"Submission failed with status code {submission_response.status_code}: {submission_response.text}"
122
- logger.error(error_message)
123
- progress(1.0, desc="Failed!")
124
- return error_message, pd.DataFrame(results)
125
-
126
- except Exception as e:
127
- error_message = f"Error during evaluation process: {str(e)}"
128
- logger.error(error_message)
129
- progress(1.0, desc="Failed!")
130
- return error_message, None
131
-
132
- # Create Gradio interface without OAuth
133
- with gr.Blocks(theme=gr.themes.Soft()) as demo:
134
- gr.Markdown("# GAIA Agent Evaluation Runner (No OAuth)")
135
-
136
- with gr.Tabs():
137
- with gr.TabItem("Run Evaluation"):
138
- gr.Markdown(
139
- """
140
- **Instructions:**
141
-
142
- 1. Enter your username (optional) for submission tracking.
143
- 2. Click 'Run Evaluation & Submit All Answers' to fetch questions, run the GAIA agent, submit answers, and see the score.
144
- 3. The progress bar will show you the current status of the evaluation.
145
- 4. Results will appear in the table below once processing is complete.
146
-
147
- **Note:** Running the evaluation may take some time as the agent processes all questions. Please be patient.
148
- """
149
- )
150
-
151
- with gr.Row():
152
- with gr.Column(scale=1):
153
- username_input = gr.Textbox(
154
- label="Username (optional)",
155
- placeholder="Enter username for submission",
156
- value="anonymous"
157
- )
158
- with gr.Column(scale=2):
159
- run_button = gr.Button("Run Evaluation & Submit All Answers",
160
- variant="primary", size="lg")
161
-
162
- status_output = gr.Textbox(
163
- label="Run Status / Submission Result",
164
- lines=6,
165
- interactive=False,
166
- elem_id="status_output"
167
- )
168
-
169
- results_accordion = gr.Accordion("Results", open=False, elem_id="results_accordion")
170
- with results_accordion:
171
- results_table = gr.DataFrame(
172
- label="Questions and Agent Answers",
173
- wrap=True,
174
- elem_id="results_table"
175
- )
176
-
177
- with gr.Row():
178
- with gr.Column():
179
- score_chart = gr.Plot(
180
- label="Score Distribution",
181
- elem_id="score_chart"
182
- )
183
- with gr.Column():
184
- time_chart = gr.Plot(
185
- label="Processing Time",
186
- elem_id="time_chart"
187
- )
188
-
189
- with gr.TabItem("About"):
190
- gr.Markdown(
191
- """
192
- ## About GAIA Agent Evaluation
193
-
194
- This application runs the GAIA (Generative AI Assistant) agent on a set of evaluation questions
195
- and submits the answers for scoring. The agent uses various tools and APIs to answer questions
196
- about factual information.
197
-
198
- ### Features
199
-
200
- - Fetches evaluation questions from the scoring API
201
- - Runs the GAIA agent on each question
202
- - Submits answers for scoring
203
- - Displays results with feedback
204
-
205
- ### Technical Details
206
-
207
- The GAIA agent is built using:
208
-
209
- - LangChain for agent orchestration
210
- - Various search tools and APIs
211
- - Supabase for memory storage
212
-
213
- For more information, see the project documentation.
214
- """
215
- )
216
-
217
- # Set up event handlers
218
- run_button.click(
219
- fn=run_and_submit_all,
220
- inputs=[username_input],
221
- outputs=[status_output, results_table]
222
- )
223
-
224
- # Launch the app
225
- if __name__ == "__main__":
226
- demo.launch()
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
backup_tests/minimal_supabase_test.py DELETED
@@ -1,75 +0,0 @@
1
- """
2
- Minimal Supabase Connection Test
3
-
4
- This script performs a minimal test of the Supabase connection.
5
- """
6
-
7
- import os
8
- import sys
9
- import logging
10
- from dotenv import load_dotenv
11
-
12
- logging.basicConfig(level=logging.DEBUG)
13
- logger = logging.getLogger("minimal_supabase_test")
14
-
15
- dotenv_path = os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), '.env')
16
- logger.info(f"Loading environment variables from {dotenv_path}")
17
- load_dotenv(dotenv_path)
18
-
19
- supabase_url = os.environ.get("SUPABASE_URL")
20
- supabase_key = os.environ.get("SUPABASE_KEY")
21
-
22
- logger.info(f"SUPABASE_URL: {supabase_url}")
23
- logger.info(f"SUPABASE_KEY: {'***' if supabase_key else None}")
24
-
25
- if not supabase_url or not supabase_key:
26
- logger.error("Supabase URL or key not provided")
27
- logger.error("Please set SUPABASE_URL and SUPABASE_KEY environment variables")
28
- sys.exit(1)
29
-
30
- try:
31
- from supabase import create_client, Client
32
- logger.info("Supabase package imported successfully")
33
- except ImportError as e:
34
- logger.error(f"Error importing supabase: {str(e)}")
35
- logger.error("Please install the supabase package with: pip install supabase")
36
- sys.exit(1)
37
-
38
- try:
39
- logger.info(f"Connecting to Supabase at {supabase_url}")
40
- client = create_client(supabase_url, supabase_key)
41
- logger.info("Supabase client created successfully")
42
-
43
- import time
44
- from datetime import datetime
45
-
46
- test_id = int(time.time())
47
- test_name = f"Test Record {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}"
48
-
49
- logger.info(f"Inserting test record with ID {test_id} and name '{test_name}'...")
50
- insert_result = client.table("test_connection").insert({
51
- "id": test_id,
52
- "name": test_name
53
- }).execute()
54
-
55
- if insert_result.data:
56
- logger.info("Test record inserted successfully")
57
- else:
58
- logger.warning("Failed to insert test record")
59
-
60
- logger.info("Querying test_connection table...")
61
- result = client.table("test_connection").select("*").limit(5).execute()
62
-
63
- if result.data is not None:
64
- logger.info(f"Query successful, found {len(result.data)} records")
65
- for record in result.data:
66
- else:
67
- logger.warning("Query successful but no data returned")
68
-
69
- logger.info("Connection to Supabase successful")
70
- except Exception as e:
71
- logger.error(f"Error connecting to Supabase: {str(e)}")
72
- import traceback
73
- logger.error(f"Traceback: {traceback.format_exc()}")
74
- sys.exit(1)
75
-
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
backup_tests/test_supabase_tables_final.py DELETED
@@ -1,167 +0,0 @@
1
- """
2
- Final test script for Supabase tables in GAIA Agent.
3
-
4
- This script applies the fix to ensure all Supabase tables are properly utilized,
5
- then runs a simple test to verify the fix works correctly.
6
-
7
- Usage:
8
- python -m tests.test_supabase_tables_final
9
- """
10
-
11
- import os
12
- import sys
13
- import json
14
- import logging
15
- import time
16
- import uuid
17
- import datetime
18
- from typing import Dict, Any, List, Optional
19
-
20
- logging.basicConfig(
21
- level=logging.INFO,
22
- format="%(asctime)s - %(name)s - %(levelname)s - %(message)s"
23
- )
24
- logger = logging.getLogger("test_supabase_tables_final")
25
-
26
- sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
27
-
28
- try:
29
- from memory.supabase_memory import (
30
- SupabaseMemory,
31
- WorkingMemory
32
- )
33
- from agent.config import get_memory_config
34
-
35
- from memory.fix_supabase_tables import apply_monkey_patch, ensure_tables_exist
36
- except ImportError as e:
37
- logger.error(f"Error importing required components: {str(e)}")
38
- sys.exit(1)
39
-
40
- def apply_fix_and_verify():
41
- """Apply the fix and verify it works correctly."""
42
- logger.info("Applying fix for Supabase tables...")
43
-
44
- if not ensure_tables_exist():
45
- logger.error("Failed to ensure tables exist")
46
- return False
47
-
48
- if not apply_monkey_patch():
49
- logger.error("Failed to apply monkey patch")
50
- return False
51
-
52
- logger.info("Fix applied successfully")
53
- return True
54
-
55
- def test_working_memory():
56
- """Test if working memory is using the correct table."""
57
- logger.info("Testing working memory...")
58
-
59
- memory_config = get_memory_config()
60
- memory_config["enabled"] = True
61
-
62
- base_memory = SupabaseMemory(config=memory_config)
63
- session_id = f"test-session-{uuid.uuid4()}"
64
- working_memory = WorkingMemory(base_memory, session_id)
65
-
66
- logger.info(f"Base memory table: {base_memory.table_name}")
67
- logger.info(f"Working memory table: {working_memory.table_name}")
68
-
69
- test_key = f"test_working_{int(time.time())}"
70
- test_value = {
71
- "test_data": "This is test data for working memory",
72
- "timestamp": time.time(),
73
- "random_id": str(uuid.uuid4())
74
- }
75
-
76
- logger.info(f"Storing data in working memory: {test_key}")
77
- store_result = working_memory.store_intermediate_result(test_key, test_value)
78
-
79
- if not store_result:
80
- logger.error("Failed to store test value")
81
- return False
82
-
83
- logger.info(f"Retrieving data from working memory: {test_key}")
84
- retrieved_value = working_memory.get_intermediate_result(test_key)
85
-
86
- if retrieved_value != test_value:
87
- logger.error(f"Retrieved value does not match test value")
88
- logger.error(f"Expected: {test_value}")
89
- logger.error(f"Got: {retrieved_value}")
90
- return False
91
-
92
- logger.info("✅ Store and retrieve operations successful")
93
-
94
- logger.info("Verifying data was stored in the working memory table...")
95
-
96
- if base_memory.client and base_memory.initialized:
97
- try:
98
- working_key = f"working:{session_id}:{test_key}"
99
- response = base_memory.client.table(working_memory.table_name).select("*").eq("key", working_key).execute()
100
-
101
- if not response.data or len(response.data) == 0:
102
- logger.error("No data found in working memory table")
103
-
104
- response = base_memory.client.table(base_memory.table_name).select("*").eq("key", working_key).execute()
105
-
106
- if response.data and len(response.data) > 0:
107
- logger.error("❌ Data was stored in base memory table instead of working memory table")
108
- return False
109
- else:
110
- logger.error("❌ Data was not stored in any table")
111
- return False
112
-
113
- logger.info(f"✅ Data found in working memory table: {len(response.data)} entries")
114
-
115
- entry_data = response.data[0]
116
- stored_value = entry_data.get("value")
117
-
118
- if isinstance(stored_value, str):
119
- try:
120
- stored_value = json.loads(stored_value)
121
- except json.JSONDecodeError:
122
- pass
123
-
124
- if stored_value == test_value or stored_value == json.dumps(test_value):
125
- logger.info("✅ Stored data matches expected value")
126
- else:
127
- logger.error("❌ Stored data does not match expected value")
128
- logger.error(f"Expected: {test_value}")
129
- logger.error(f"Got: {stored_value}")
130
- return False
131
-
132
- logger.info("Cleaning up test data...")
133
- working_memory.clear()
134
-
135
- return True
136
- except Exception as e:
137
- logger.error(f"Error verifying data storage: {str(e)}")
138
- return False
139
-
140
- return False
141
-
142
- def main():
143
- """Main entry point for the test script."""
144
-
145
- if not apply_fix_and_verify():
146
- sys.exit(1)
147
-
148
- if test_working_memory():
149
-
150
- timestamp = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
151
- filename = f"gaia_memory_test_results_{timestamp}.json"
152
-
153
- results = {
154
- "timestamp": timestamp,
155
- "success": True,
156
- "message": "Working memory table is now being used correctly"
157
- }
158
-
159
- with open(filename, "w") as f:
160
- json.dump(results, f, indent=2)
161
-
162
- sys.exit(0)
163
- else:
164
- sys.exit(1)
165
-
166
- if __name__ == "__main__":
167
- main()
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
backup_tests/test_supabase_tables_fix.py DELETED
@@ -1,276 +0,0 @@
1
- """
2
- Test script for verifying the Supabase tables fix.
3
-
4
- This script tests if the working memory table (gaia_memory_working) is properly
5
- utilized by the GAIA agent. It verifies that data is correctly stored in and
6
- retrieved from the working memory table.
7
-
8
- Usage:
9
- python -m tests.test_supabase_tables_fix
10
- """
11
-
12
- import os
13
- import sys
14
- import json
15
- import logging
16
- import time
17
- import uuid
18
- from typing import Dict, Any, List, Optional
19
-
20
- logging.basicConfig(
21
- level=logging.INFO,
22
- format="%(asctime)s - %(name)s - %(levelname)s - %(message)s"
23
- )
24
- logger = logging.getLogger("test_supabase_tables")
25
-
26
- sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
27
-
28
- try:
29
- from memory.supabase_memory import (
30
- SupabaseMemory,
31
- WorkingMemory
32
- )
33
- from agent.config import get_memory_config
34
-
35
- from memory.simplified_memory_fix import apply_fix, create_working_memory_table
36
- except ImportError as e:
37
- logger.error(f"Error importing required components: {str(e)}")
38
- sys.exit(1)
39
-
40
- class SupabaseTablesTest:
41
- """Test class for verifying Supabase tables."""
42
-
43
- def __init__(self):
44
- """Initialize the test class."""
45
- self.memory_config = get_memory_config()
46
- self.memory_config["enabled"] = True
47
-
48
- self.base_memory = None
49
- self.working_memory = None
50
- self.session_id = str(uuid.uuid4())
51
-
52
- self.table_names = {
53
- "base": "gaia_memory",
54
- "working": "gaia_memory_working"
55
- }
56
-
57
- self.results = {
58
- "initialization": False,
59
- "table_existence": {},
60
- "working_memory": {}
61
- }
62
-
63
- def initialize_memory(self) -> bool:
64
- """Initialize memory components."""
65
- logger.info("Initializing memory components...")
66
-
67
- try:
68
- self.base_memory = SupabaseMemory(self.memory_config)
69
-
70
- self.working_memory = WorkingMemory(self.base_memory, self.session_id)
71
-
72
- if self.base_memory and self.base_memory.initialized:
73
- logger.info("✅ Memory components initialized successfully")
74
- self.results["initialization"] = True
75
- return True
76
- else:
77
- logger.error("❌ Memory components initialization failed")
78
- self.results["initialization"] = False
79
- return False
80
-
81
- except Exception as e:
82
- logger.error(f"Error initializing memory components: {str(e)}")
83
- self.results["initialization"] = False
84
- return False
85
-
86
- def check_table_existence(self) -> Dict[str, bool]:
87
- """Check if the required tables exist."""
88
- logger.info("Checking table existence...")
89
-
90
- results = {}
91
-
92
- if not self.base_memory or not self.base_memory.initialized:
93
- logger.error("Memory components not initialized")
94
- return {"error": "Memory components not initialized"}
95
-
96
- for table_type, table_name in self.table_names.items():
97
- try:
98
- response = self.base_memory.client.table(table_name).select("*").limit(1).execute()
99
-
100
- if hasattr(response, "data"):
101
- logger.info(f"✅ Table {table_name} exists")
102
- results[table_name] = True
103
- else:
104
- logger.warning(f"❌ Table {table_name} query failed")
105
- results[table_name] = False
106
-
107
- except Exception as e:
108
- logger.error(f"Error checking table {table_name}: {str(e)}")
109
- results[table_name] = False
110
-
111
- self.results["table_existence"] = results
112
- return results
113
-
114
- def test_working_memory(self) -> Dict[str, bool]:
115
- """Test working memory operations."""
116
- logger.info("Testing working memory operations...")
117
-
118
- results = {}
119
-
120
- if not self.working_memory:
121
- logger.error("Working memory not initialized")
122
- return {"error": "Working memory not initialized"}
123
-
124
- try:
125
- test_key = f"test_working_{int(time.time())}"
126
- test_value = {
127
- "test_data": "This is test data for working memory",
128
- "timestamp": time.time(),
129
- "random_id": str(uuid.uuid4())
130
- }
131
-
132
- logger.info(f"Storing data in working memory: {test_key}")
133
- store_result = self.working_memory.store_intermediate_result(test_key, test_value)
134
-
135
- if store_result:
136
- logger.info("✅ Store operation successful")
137
- results["store"] = True
138
- else:
139
- logger.warning("❌ Store operation failed")
140
- results["store"] = False
141
-
142
- logger.info(f"Retrieving data from working memory: {test_key}")
143
- retrieved_value = self.working_memory.get_intermediate_result(test_key)
144
-
145
- if retrieved_value and retrieved_value == test_value:
146
- logger.info("✅ Retrieve operation successful")
147
- results["retrieve"] = True
148
- else:
149
- logger.warning(f"❌ Retrieve operation failed or data mismatch")
150
- logger.warning(f"Expected: {test_value}")
151
- logger.warning(f"Got: {retrieved_value}")
152
- results["retrieve"] = False
153
-
154
- logger.info("Verifying data was stored in the correct table...")
155
-
156
- if self.base_memory.client and self.base_memory.initialized:
157
- try:
158
- working_key = f"working:{self.session_id}:{test_key}"
159
- response = self.base_memory.client.table(self.table_names["working"]).select("*").eq("key", working_key).execute()
160
-
161
- if response.data and len(response.data) > 0:
162
- logger.info(f"✅ Data found in working memory table: {len(response.data)} entries")
163
- results["correct_table"] = True
164
-
165
- entry_data = response.data[0]
166
- stored_value = entry_data.get("value")
167
-
168
- if isinstance(stored_value, str):
169
- try:
170
- stored_value = json.loads(stored_value)
171
- except json.JSONDecodeError:
172
- pass
173
-
174
- if stored_value == test_value or stored_value == json.dumps(test_value):
175
- logger.info("✅ Stored data matches expected value")
176
- results["data_match"] = True
177
- else:
178
- logger.warning("❌ Stored data does not match expected value")
179
- logger.warning(f"Expected: {test_value}")
180
- logger.warning(f"Got: {stored_value}")
181
- results["data_match"] = False
182
- else:
183
- logger.warning("❌ No data found in working memory table")
184
- results["correct_table"] = False
185
-
186
- response = self.base_memory.client.table(self.table_names["base"]).select("*").eq("key", working_key).execute()
187
-
188
- if response.data and len(response.data) > 0:
189
- logger.warning("⚠️ Data was stored in base memory table instead of working memory table")
190
- results["stored_in_base"] = True
191
- else:
192
- logger.warning("❌ Data was not stored in any table")
193
- results["stored_in_base"] = False
194
-
195
- except Exception as e:
196
- logger.error(f"Error verifying data storage: {str(e)}")
197
- results["correct_table"] = False
198
-
199
- logger.info("Cleaning up test data...")
200
- self.working_memory.clear()
201
-
202
- except Exception as e:
203
- logger.error(f"Error testing working memory: {str(e)}")
204
- results["error"] = str(e)
205
-
206
- self.results["working_memory"] = results
207
- return results
208
-
209
- def run_all_tests(self) -> Dict[str, Any]:
210
- """Run all tests."""
211
- logger.info("Running all tests...")
212
-
213
- if not self.initialize_memory():
214
- logger.error("Memory initialization failed, aborting tests")
215
- return self.results
216
-
217
- self.check_table_existence()
218
-
219
- if self.results["table_existence"].get(self.table_names["working"], False):
220
- logger.info("Working memory table exists, applying fix...")
221
- apply_fix()
222
- else:
223
- logger.info("Working memory table does not exist, creating it...")
224
- create_working_memory_table()
225
-
226
- self.check_table_existence()
227
-
228
- self.test_working_memory()
229
-
230
- return self.results
231
-
232
- def print_test_summary(self) -> None:
233
- """Print a summary of the test results."""
234
-
235
- if self.results["initialization"]:
236
- else:
237
-
238
- for table_name, exists in self.results["table_existence"].items():
239
- if exists:
240
- else:
241
-
242
- working_results = self.results["working_memory"]
243
-
244
- if working_results.get("store", False):
245
- else:
246
-
247
- if working_results.get("retrieve", False):
248
- else:
249
-
250
- if working_results.get("correct_table", False):
251
- else:
252
-
253
- if working_results.get("stored_in_base", False):
254
-
255
- if working_results.get("data_match", False):
256
- else:
257
-
258
- if (self.results["initialization"] and
259
- all(self.results["table_existence"].values()) and
260
- working_results.get("store", False) and
261
- working_results.get("retrieve", False) and
262
- working_results.get("correct_table", False) and
263
- working_results.get("data_match", False)):
264
- else:
265
-
266
-
267
- def main():
268
- """Main entry point for the test script."""
269
-
270
- test = SupabaseTablesTest()
271
- test.run_all_tests()
272
-
273
- test.print_test_summary()
274
-
275
- if __name__ == "__main__":
276
- main()
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
backup_tests/test_supabase_tables_production.py DELETED
@@ -1,481 +0,0 @@
1
- """
2
- Comprehensive Unit Tests for Supabase Tables
3
-
4
- This script tests all Supabase tables used by the GAIA agent:
5
- - gaia_memory: Main memory table
6
- - gaia_memory_working: Working memory table
7
- - gaia_memory_conversations: Conversation history table
8
- - gaia_memory_cache: Result cache table
9
-
10
- The tests verify:
11
- 1. Each table is created correctly
12
- 2. Each memory class uses its dedicated table
13
- 3. Data is properly isolated between tables
14
- 4. Performance metrics are captured
15
- 5. All test data is cleaned up after tests
16
-
17
- Usage:
18
- python tests/test_supabase_tables_production.py
19
- """
20
-
21
- import os
22
- import sys
23
- import json
24
- import time
25
- import uuid
26
- import logging
27
- import unittest
28
- from datetime import datetime, timedelta
29
- from typing import Dict, Any, List, Optional, Union, Tuple
30
-
31
- logging.basicConfig(
32
- level=logging.INFO,
33
- format="%(asctime)s - %(name)s - %(levelname)s - %(message)s"
34
- )
35
- logger = logging.getLogger("test_supabase_tables_production")
36
-
37
- parent_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
38
- sys.path.insert(0, parent_dir)
39
-
40
- try:
41
- from supabase import create_client, Client
42
- except ImportError as e:
43
- logger.error(f"Error importing supabase: {str(e)}")
44
- sys.exit(1)
45
-
46
- try:
47
- from agent.config import (
48
- SUPABASE_URL,
49
- SUPABASE_KEY,
50
- SUPABASE_PROJECT_ID,
51
- SUPABASE_PROJECT_NAME
52
- )
53
- except ImportError as e:
54
- logger.error(f"Error importing configuration: {str(e)}")
55
- sys.exit(1)
56
-
57
- def get_table_schema(client: Client, table_name: str) -> List[str]:
58
- """Get the schema of a table."""
59
- try:
60
- result = client.table(table_name).select("*").limit(1).execute()
61
-
62
- if result.data and len(result.data) > 0:
63
- return list(result.data[0].keys())
64
- else:
65
- schema_result = client.from_("information_schema.columns").select("column_name").eq("table_name", table_name).execute()
66
- if schema_result.data:
67
- return [col["column_name"] for col in schema_result.data]
68
- else:
69
- return []
70
- except Exception as e:
71
- logger.error(f"Error getting schema for table {table_name}: {str(e)}")
72
- return []
73
-
74
- def ensure_table_exists(client: Client, table_name: str) -> bool:
75
- """Ensure a table exists in Supabase."""
76
- try:
77
- client.table(table_name).select("*").limit(1).execute()
78
- logger.info(f"Table {table_name} exists")
79
- return True
80
- except Exception as e:
81
- logger.warning(f"Error checking table {table_name}: {str(e)}")
82
- logger.info(f"Creating table {table_name}")
83
-
84
- try:
85
- sql = f"""
86
- CREATE TABLE IF NOT EXISTS {table_name} (
87
- id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
88
- key TEXT NOT NULL UNIQUE,
89
- value TEXT NOT NULL,
90
- metadata JSONB,
91
- created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
92
- expires_at TIMESTAMPTZ
93
- );
94
-
95
- CREATE INDEX IF NOT EXISTS idx_{table_name}_key ON {table_name}(key);
96
- CREATE INDEX IF NOT EXISTS idx_{table_name}_expires_at ON {table_name}(expires_at);
97
- """
98
-
99
- client.query(sql).execute()
100
-
101
- client.table(table_name).select("*").limit(1).execute()
102
-
103
- logger.info(f"Table {table_name} created successfully")
104
- return True
105
- except Exception as e:
106
- logger.error(f"Error creating table {table_name}: {str(e)}")
107
- return False
108
-
109
- def store_value(client: Client, table_name: str, key: str, value: Any, metadata: Optional[Dict[str, Any]] = None, ttl: Optional[int] = None, session_id: Optional[str] = None) -> bool:
110
- """Store a value in a table."""
111
- try:
112
- entry = {
113
- "key": key,
114
- "value": json.dumps(value) if not isinstance(value, str) else value,
115
- "metadata": metadata or {}
116
- }
117
-
118
- if ttl:
119
- entry["expires_at"] = (datetime.now() + timedelta(seconds=ttl)).isoformat()
120
-
121
- if session_id and table_name.endswith("_working"):
122
- entry["session_id"] = session_id
123
-
124
- client.table(table_name).upsert(entry).execute()
125
-
126
- logger.info(f"Stored value in table {table_name}: {key}")
127
- return True
128
- except Exception as e:
129
- logger.error(f"Error storing value in table {table_name}: {key} - {str(e)}")
130
- return False
131
-
132
- def retrieve_value(client: Client, table_name: str, key: str, default: Any = None) -> Any:
133
- """Retrieve a value from a table."""
134
- try:
135
- result = client.table(table_name).select("*").eq("key", key).execute()
136
-
137
- if result.data and len(result.data) > 0:
138
- entry = result.data[0]
139
-
140
- value = entry.get("value")
141
- if isinstance(value, str):
142
- try:
143
- value = json.loads(value)
144
- except json.JSONDecodeError:
145
- pass
146
-
147
- expires_at = entry.get("expires_at")
148
- if expires_at:
149
- try:
150
- expiry = datetime.fromisoformat(expires_at)
151
- if expiry < datetime.now():
152
- delete_value(client, table_name, key)
153
- return default
154
- except (ValueError, TypeError):
155
- pass
156
-
157
- logger.info(f"Retrieved value from table {table_name}: {key}")
158
- return value
159
- except Exception as e:
160
- logger.error(f"Error retrieving value from table {table_name}: {key} - {str(e)}")
161
-
162
- return default
163
-
164
- def delete_value(client: Client, table_name: str, key: str) -> bool:
165
- """Delete a value from a table."""
166
- try:
167
- client.table(table_name).delete().eq("key", key).execute()
168
-
169
- logger.info(f"Deleted value from table {table_name}: {key}")
170
- return True
171
- except Exception as e:
172
- logger.error(f"Error deleting value from table {table_name}: {key} - {str(e)}")
173
- return False
174
-
175
- def list_keys(client: Client, table_name: str, prefix: Optional[str] = None) -> List[str]:
176
- """List keys in a table."""
177
- keys = []
178
-
179
- try:
180
- query = client.table(table_name).select("key")
181
-
182
- if prefix:
183
- query = query.like("key", f"{prefix}%")
184
-
185
- result = query.execute()
186
-
187
- if result.data:
188
- for entry in result.data:
189
- key = entry.get("key")
190
- if key:
191
- keys.append(key)
192
-
193
- logger.info(f"Listed keys from table {table_name}: {len(keys)} keys")
194
- except Exception as e:
195
- logger.error(f"Error listing keys from table {table_name}: {str(e)}")
196
-
197
- return keys
198
-
199
- def clear_table(client: Client, table_name: str, prefix: Optional[str] = None) -> bool:
200
- """Clear a table."""
201
- try:
202
- if prefix:
203
- keys = list_keys(client, table_name, prefix)
204
-
205
- for key in keys:
206
- delete_value(client, table_name, key)
207
- else:
208
- client.table(table_name).delete().execute()
209
-
210
- logger.info(f"Cleared table {table_name}")
211
- return True
212
- except Exception as e:
213
- logger.error(f"Error clearing table {table_name}: {str(e)}")
214
- return False
215
-
216
- def run_tests():
217
- """Run the tests."""
218
- logger.info("Running Supabase tables tests...")
219
-
220
- if not SUPABASE_URL or not SUPABASE_KEY:
221
- logger.error("Supabase URL or key not provided")
222
- logger.error("Please set SUPABASE_URL and SUPABASE_KEY environment variables")
223
- return False
224
-
225
- # Initialize Supabase client
226
- client = create_client(SUPABASE_URL, SUPABASE_KEY)
227
-
228
- test_run_id = str(uuid.uuid4())
229
- session_id = f"test-session-{test_run_id}"
230
- conversation_id = f"test-conversation-{test_run_id}"
231
-
232
- base_table = "gaia_memory"
233
- tables = {
234
- "base": base_table,
235
- "working": f"{base_table}_working",
236
- "conversations": f"{base_table}_conversations",
237
- "cache": f"{base_table}_cache"
238
- }
239
-
240
- for table_type, table_name in tables.items():
241
- if not ensure_table_exists(client, table_name):
242
- logger.error(f"Failed to ensure table {table_name} exists")
243
- return False
244
-
245
- schema = get_table_schema(client, table_name)
246
- logger.info(f"Table {table_name} schema: {schema}")
247
-
248
- if table_type == "conversations":
249
- if "conversation_id" in schema and "message_id" in schema and "content" in schema:
250
- logger.info(f"Table {table_name} has a different schema for conversations")
251
-
252
- logger.info("Testing base table operations...")
253
-
254
- test_key = f"test-{test_run_id}-base"
255
- test_value = {
256
- "test_data": f"This is test data for {test_key}",
257
- "timestamp": datetime.now().isoformat(),
258
- "random_id": str(uuid.uuid4())
259
- }
260
-
261
- if not store_value(client, tables["base"], test_key, test_value):
262
- logger.error("Failed to store test value in base table")
263
- return False
264
-
265
- retrieved_value = retrieve_value(client, tables["base"], test_key)
266
- if retrieved_value != test_value:
267
- logger.error("Retrieved value does not match test value")
268
- logger.error(f"Expected: {test_value}")
269
- logger.error(f"Got: {retrieved_value}")
270
- return False
271
-
272
- logger.info("[PASS] Base table operations successful")
273
-
274
- logger.info("Testing working memory table operations...")
275
-
276
- working_key = f"working:{session_id}:test-{test_run_id}-working"
277
- working_value = {
278
- "test_data": f"This is test data for {working_key}",
279
- "timestamp": datetime.now().isoformat(),
280
- "random_id": str(uuid.uuid4())
281
- }
282
-
283
- if not store_value(client, tables["working"], working_key, working_value, session_id=session_id):
284
- logger.error("Failed to store test value in working table")
285
- return False
286
-
287
- retrieved_value = retrieve_value(client, tables["working"], working_key)
288
- if retrieved_value != working_value:
289
- logger.error("Retrieved value does not match test value")
290
- logger.error(f"Expected: {working_value}")
291
- logger.error(f"Got: {retrieved_value}")
292
- return False
293
-
294
- base_value = retrieve_value(client, tables["base"], working_key)
295
- if base_value is not None:
296
- logger.error("Working memory data was incorrectly stored in base table")
297
- return False
298
-
299
- logger.info("[PASS] Working memory table operations successful")
300
-
301
- logger.info("Testing conversation memory table operations...")
302
-
303
- schema = get_table_schema(client, tables["conversations"])
304
-
305
- if "key" in schema and "value" in schema:
306
- conversation_key = f"conversation:{conversation_id}:message:{datetime.now().isoformat()}"
307
- conversation_value = {
308
- "role": "user",
309
- "content": "Hello, how are you?",
310
- "timestamp": datetime.now().isoformat()
311
- }
312
-
313
- if not store_value(client, tables["conversations"], conversation_key, conversation_value):
314
- logger.error("Failed to store test value in conversations table")
315
- return False
316
-
317
- retrieved_value = retrieve_value(client, tables["conversations"], conversation_key)
318
- if retrieved_value != conversation_value:
319
- logger.error("Retrieved value does not match test value")
320
- logger.error(f"Expected: {conversation_value}")
321
- logger.error(f"Got: {retrieved_value}")
322
- return False
323
-
324
- base_value = retrieve_value(client, tables["base"], conversation_key)
325
- if base_value is not None:
326
- logger.error("Conversation memory data was incorrectly stored in base table")
327
- return False
328
- else:
329
- logger.info("Using different schema for conversations table")
330
-
331
- try:
332
- message_id = f"message-{datetime.now().isoformat()}"
333
- entry = {
334
- "conversation_id": conversation_id,
335
- "message_id": message_id,
336
- "role": "user",
337
- "content": "Hello, how are you?",
338
- "created_at": datetime.now().isoformat()
339
- }
340
-
341
- client.table(tables["conversations"]).insert(entry).execute()
342
- logger.info(f"Inserted message in conversations table: {message_id}")
343
-
344
- result = client.table(tables["conversations"]).select("*").eq("conversation_id", conversation_id).execute()
345
-
346
- if not result.data or len(result.data) == 0:
347
- logger.error("No messages found in conversations table")
348
- return False
349
-
350
- logger.info(f"Found {len(result.data)} messages in conversations table")
351
- except Exception as e:
352
- logger.error(f"Error testing conversations table: {str(e)}")
353
- return False
354
-
355
- logger.info("[PASS] Conversation memory table operations successful")
356
-
357
- logger.info("Testing result cache table operations...")
358
-
359
- cache_key = f"cache:test-{test_run_id}-cache"
360
- cache_value = {
361
- "test_data": f"This is test data for {cache_key}",
362
- "timestamp": datetime.now().isoformat(),
363
- "random_id": str(uuid.uuid4())
364
- }
365
-
366
- if not store_value(client, tables["cache"], cache_key, cache_value, ttl=3600):
367
- logger.error("Failed to store test value in cache table")
368
- return False
369
-
370
- retrieved_value = retrieve_value(client, tables["cache"], cache_key)
371
- if retrieved_value != cache_value:
372
- logger.error("Retrieved value does not match test value")
373
- logger.error(f"Expected: {cache_value}")
374
- logger.error(f"Got: {retrieved_value}")
375
- return False
376
-
377
- base_value = retrieve_value(client, tables["base"], cache_key)
378
- if base_value is not None:
379
- logger.error("Result cache data was incorrectly stored in base table")
380
- return False
381
-
382
- logger.info("[PASS] Result cache table operations successful")
383
-
384
- logger.info("Testing data isolation between tables...")
385
-
386
- base_keys = list_keys(client, tables["base"])
387
- working_keys = list_keys(client, tables["working"])
388
- cache_keys = list_keys(client, tables["cache"])
389
-
390
- schema = get_table_schema(client, tables["conversations"])
391
- if "key" in schema:
392
- conversation_keys = list_keys(client, tables["conversations"])
393
-
394
- if conversation_key not in conversation_keys:
395
- logger.error(f"Conversation key {conversation_key} not found in conversations table")
396
- return False
397
- else:
398
- try:
399
- result = client.table(tables["conversations"]).select("*").eq("conversation_id", conversation_id).execute()
400
- if not result.data or len(result.data) == 0:
401
- logger.error(f"No messages found for conversation ID {conversation_id}")
402
- return False
403
- logger.info(f"Found {len(result.data)} messages for conversation ID {conversation_id}")
404
- except Exception as e:
405
- logger.error(f"Error checking conversations table: {str(e)}")
406
- return False
407
-
408
- if test_key not in base_keys:
409
- logger.error(f"Test key {test_key} not found in base table")
410
- return False
411
-
412
- if working_key not in working_keys:
413
- logger.error(f"Working key {working_key} not found in working table")
414
- return False
415
-
416
- if cache_key not in cache_keys:
417
- logger.error(f"Cache key {cache_key} not found in cache table")
418
- return False
419
-
420
- logger.info("[PASS] Data isolation between tables verified")
421
-
422
- logger.info("Testing performance metrics...")
423
-
424
- num_operations = 10
425
-
426
- base_store_times = []
427
- base_retrieve_times = []
428
-
429
- for i in range(num_operations):
430
- key = f"test-{test_run_id}-perf-base-{i}"
431
- value = {"index": i, "data": f"Performance test data {i}"}
432
-
433
- start_time = time.time()
434
- store_value(client, tables["base"], key, value)
435
- base_store_times.append(time.time() - start_time)
436
-
437
- start_time = time.time()
438
- retrieve_value(client, tables["base"], key)
439
- base_retrieve_times.append(time.time() - start_time)
440
-
441
- working_store_times = []
442
- working_retrieve_times = []
443
-
444
- for i in range(num_operations):
445
- key = f"working:{session_id}:test-{test_run_id}-perf-working-{i}"
446
- value = {"index": i, "data": f"Performance test data {i}"}
447
-
448
- start_time = time.time()
449
- store_value(client, tables["working"], key, value, session_id=session_id)
450
- working_store_times.append(time.time() - start_time)
451
-
452
- start_time = time.time()
453
- retrieve_value(client, tables["working"], key)
454
- working_retrieve_times.append(time.time() - start_time)
455
-
456
- logger.info(f"Base table: avg store {sum(base_store_times)/num_operations:.4f}s, avg retrieve {sum(base_retrieve_times)/num_operations:.4f}s")
457
- logger.info(f"Working table: avg store {sum(working_store_times)/num_operations:.4f}s, avg retrieve {sum(working_retrieve_times)/num_operations:.4f}s")
458
-
459
- logger.info("[PASS] Performance metrics collected")
460
-
461
- logger.info("Cleaning up test data...")
462
-
463
- clear_table(client, tables["base"], f"test-{test_run_id}")
464
- clear_table(client, tables["working"], f"working:{session_id}")
465
- clear_table(client, tables["conversations"], f"conversation:{conversation_id}")
466
- clear_table(client, tables["cache"], f"cache:test-{test_run_id}")
467
-
468
- logger.info("[PASS] Test data cleaned up")
469
-
470
- return True
471
-
472
- def main():
473
- """Main entry point for the test script."""
474
-
475
- if run_tests():
476
- sys.exit(0)
477
- else:
478
- sys.exit(1)
479
-
480
- if __name__ == "__main__":
481
- main()
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
build_error_fix_guide.md DELETED
@@ -1,78 +0,0 @@
1
- # Fixing the GAIA Agent Build Error on Hugging Face Space
2
-
3
- ## Problem
4
-
5
- The build process is failing with an "Unexpected job error" on Hugging Face Space. This is likely due to:
6
-
7
- 1. Incompatible package versions
8
- 2. Dependency conflicts
9
- 3. Version resolution issues during installation
10
-
11
- ## Solution
12
-
13
- Update the `requirements.txt` file with specific version pins to ensure all packages are compatible with each other and with the Hugging Face Space environment.
14
-
15
- ## Steps to Fix the Build Error
16
-
17
- 1. Navigate to your Space on Hugging Face
18
- 2. Click on the "Files" tab
19
- 3. Open `requirements.txt`
20
- 4. Click "Edit" (pencil icon)
21
- 5. Replace the contents with the following fixed requirements:
22
-
23
- ```
24
- # Fixed requirements file with specific version pins
25
- # Main dependencies
26
- gradio==3.50.2 # Downgraded to ensure compatibility with Hugging Face Spaces
27
- requests==2.31.0
28
- langchain==0.0.292
29
- langchain_openai==0.0.2.post1
30
- langgraph==0.0.19
31
- supabase==1.2.0
32
- duckduckgo-search==3.9.5
33
- beautifulsoup4==4.12.2
34
- pillow==10.0.1
35
- arxiv==1.4.8
36
- python-dotenv==1.0.0
37
-
38
- # Benchmark testing dependencies (optional for production)
39
- psutil==5.9.5
40
- matplotlib==3.7.3
41
- tabulate==0.9.0
42
-
43
- # Additional dependencies to ensure compatibility
44
- openai==1.3.7
45
- pydantic==1.10.13 # Using older version for compatibility
46
- httpx==0.24.1
47
- numpy==1.24.4
48
- pandas==2.0.3
49
- ```
50
-
51
- 6. Click "Commit changes" at the bottom of the page
52
- 7. Enter an optional commit message like "Fix requirements with specific version pins"
53
- 8. Click "Commit" to save the changes
54
- 9. Restart your Space (Settings tab → Factory reboot → Restart Space)
55
-
56
- ## Why This Works
57
-
58
- 1. **Specific Version Pinning**: Instead of allowing ranges or latest versions, specific versions ensure consistent builds
59
- 2. **Downgraded Gradio**: Using Gradio 3.50.2 instead of 5.x ensures better compatibility with Hugging Face Spaces
60
- 3. **Compatible Dependencies**: The specified versions are known to work together without conflicts
61
- 4. **Explicit Secondary Dependencies**: Including dependencies like openai and pydantic directly prevents version resolution issues
62
-
63
- ## Additional Troubleshooting
64
-
65
- If you continue to encounter build errors:
66
-
67
- 1. Check the Space logs for specific error messages
68
- 2. Consider removing optional testing dependencies (psutil, matplotlib, tabulate)
69
- 3. Try a minimal requirements set first, then add additional packages as needed
70
- 4. Check for any custom package installations in your code that might need to be added to requirements.txt
71
-
72
- ## Testing After Fix
73
-
74
- After updating requirements.txt and restarting your Space:
75
-
76
- 1. Monitor the build logs to confirm successful installation
77
- 2. Verify the app loads correctly in the browser
78
- 3. Test core functionality to ensure all components work properly
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
check_app.py DELETED
@@ -1,42 +0,0 @@
1
- """
2
- Simple script to check app.py file length and syntax
3
- """
4
- import os
5
- import sys
6
- import ast
7
-
8
- def check_file(file_path):
9
- try:
10
- with open(file_path, 'r', encoding='utf-8') as f:
11
- content = f.read()
12
- lines = content.splitlines()
13
- print(f"Total lines in {file_path}: {len(lines)}")
14
-
15
- # Try to parse the content to check for syntax errors
16
- try:
17
- ast.parse(content)
18
- print(f"✓ No syntax errors found in {file_path}")
19
- except SyntaxError as e:
20
- print(f"✗ Syntax error in {file_path}:")
21
- print(f" Line {e.lineno}, Column {e.offset}")
22
- print(f" {e.text.strip()}")
23
- print(f" {e}")
24
-
25
- # Print context around the error
26
- start_line = max(0, e.lineno - 5)
27
- end_line = min(len(lines), e.lineno + 5)
28
- print("\nContext around error:")
29
- for i in range(start_line, end_line):
30
- line_marker = ">" if i + 1 == e.lineno else " "
31
- print(f"{line_marker} {i+1:4d} | {lines[i]}")
32
-
33
- except Exception as e:
34
- print(f"Error reading or parsing {file_path}: {e}")
35
-
36
- if __name__ == "__main__":
37
- file_path = "app.py"
38
- if not os.path.exists(file_path):
39
- print(f"File not found: {file_path}")
40
- sys.exit(1)
41
-
42
- check_file(file_path)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
cleanup_results.txt DELETED
@@ -1,340 +0,0 @@
1
- 2025-04-30 22:18:05,446 - INFO - Finding temporary files to remove...
2
- 2025-04-30 22:18:05,563 - INFO - Found 39 temporary files to remove
3
- 2025-04-30 22:18:05,603 - INFO - Removed: gaia_assessment_results_20250430_205848.json
4
- 2025-04-30 22:18:05,615 - INFO - Removed: gaia_assessment_results_20250430_210047.json
5
- 2025-04-30 22:18:05,616 - INFO - Removed: gaia_assessment_results_20250430_210252.json
6
- 2025-04-30 22:18:05,619 - INFO - Removed: gaia_assessment_results_20250430_210521.json
7
- 2025-04-30 22:18:05,621 - INFO - Removed: gaia_assessment_results_20250430_211034.json
8
- 2025-04-30 22:18:05,622 - INFO - Removed: gaia_assessment_results_20250430_211508.json
9
- 2025-04-30 22:18:05,624 - INFO - Removed: gaia_assessment_results_20250430_213743.json
10
- 2025-04-30 22:18:05,627 - INFO - Removed: gaia_assessment_results_20250430_214034.json
11
- 2025-04-30 22:18:05,629 - INFO - Removed: gaia_assessment_results_20250430_214159.json
12
- 2025-04-30 22:18:05,631 - INFO - Removed: gaia_assessment_results_20250430_214711.json
13
- 2025-04-30 22:18:05,634 - INFO - Removed: gaia_assessment_results_20250430_220729.json
14
- 2025-04-30 22:18:05,636 - INFO - Removed: gaia_memory_test_results_20250430_185256.json
15
- 2025-04-30 22:18:05,639 - INFO - Removed: gaia_memory_test_results_20250430_185420.json
16
- 2025-04-30 22:18:05,641 - INFO - Removed: gaia_memory_test_results_20250430_185453.json
17
- 2025-04-30 22:18:05,644 - INFO - Removed: gaia_memory_test_results_20250430_185550.json
18
- 2025-04-30 22:18:05,648 - INFO - Removed: gaia_memory_test_results_20250430_185641.json
19
- 2025-04-30 22:18:05,671 - INFO - Removed: cleanup_script.bat
20
- 2025-04-30 22:18:05,672 - INFO - Removed: check_env_vars.py
21
- 2025-04-30 22:18:05,675 - INFO - Removed: check_imports.py
22
- 2025-04-30 22:18:05,678 - INFO - Removed: check_serper_config.py
23
- 2025-04-30 22:18:05,679 - INFO - Removed: check_supabase_env.py
24
- 2025-04-30 22:18:05,681 - INFO - Removed: direct_config_check.py
25
- 2025-04-30 22:18:05,683 - INFO - Removed: direct_serper_test.py
26
- 2025-04-30 22:18:05,685 - INFO - Removed: minimal_serper_test.py
27
- 2025-04-30 22:18:05,686 - INFO - Removed: simple_serper_test.py
28
- 2025-04-30 22:18:05,689 - INFO - Removed: simple_serper_test_final.py
29
- 2025-04-30 22:18:05,693 - INFO - Removed: supabase_direct_test.py
30
- 2025-04-30 22:18:05,695 - INFO - Removed: supabase_dotenv_test.py
31
- 2025-04-30 22:18:05,697 - INFO - Removed: supabase_import_test.py
32
- 2025-04-30 22:18:05,698 - INFO - Removed: supabase_init_test.py
33
- 2025-04-30 22:18:05,700 - INFO - Removed: supabase_minimal_test.py
34
- 2025-04-30 22:18:05,702 - INFO - Removed: test_serper_direct.py
35
- 2025-04-30 22:18:05,704 - INFO - Removed: verify_python.py
36
- 2025-04-30 22:18:05,706 - INFO - Removed: run_gaia_assessment.py
37
- 2025-04-30 22:18:05,707 - INFO - Removed: run_gaia_assessment_simple.py
38
- 2025-04-30 22:18:05,709 - INFO - Removed: run_gaia_fixed.py
39
- 2025-04-30 22:18:05,713 - INFO - Removed: run_gaia_minimal.py
40
- 2025-04-30 22:18:05,714 - INFO - Removed: run_gaia_with_fix.py
41
- 2025-04-30 22:18:05,717 - INFO - Removed: setup_serper_env.py
42
- 2025-04-30 22:18:05,717 - INFO - Finding duplicate test files...
43
- 2025-04-30 22:18:05,720 - INFO - Found 7 duplicate test files
44
- 2025-04-30 22:18:05,720 - INFO - Duplicate tests for test_config.py:
45
- 2025-04-30 22:18:05,720 - INFO - - tests\test_config.py
46
- 2025-04-30 22:18:05,720 - INFO - - tests\test_config_simple.py
47
- 2025-04-30 22:18:05,721 - INFO - Keeping: tests\test_config.py
48
- 2025-04-30 22:18:05,721 - INFO - Removing: tests\test_config_simple.py
49
- 2025-04-30 22:18:05,722 - INFO - Removed: tests\test_config_simple.py
50
- 2025-04-30 22:18:05,722 - INFO - Duplicate tests for test_serper_api.py:
51
- 2025-04-30 22:18:05,723 - INFO - - tests\test_serper_api.py
52
- 2025-04-30 22:18:05,723 - INFO - - tests\test_serper_api_final.py
53
- 2025-04-30 22:18:05,723 - INFO - Keeping: tests\test_serper_api_final.py
54
- 2025-04-30 22:18:05,723 - INFO - Removing: tests\test_serper_api.py
55
- 2025-04-30 22:18:05,724 - INFO - Removed: tests\test_serper_api.py
56
- 2025-04-30 22:18:05,726 - INFO - Duplicate tests for test_supabase_connection.py:
57
- 2025-04-30 22:18:05,726 - INFO - - tests\test_supabase_connection.py
58
- 2025-04-30 22:18:05,726 - INFO - - tests\test_supabase_connection_simple.py
59
- 2025-04-30 22:18:05,727 - INFO - Keeping: tests\test_supabase_connection.py
60
- 2025-04-30 22:18:05,727 - INFO - Removing: tests\test_supabase_connection_simple.py
61
- 2025-04-30 22:18:05,728 - INFO - Removed: tests\test_supabase_connection_simple.py
62
- 2025-04-30 22:18:05,729 - INFO - Duplicate tests for test_supabase_tables.py:
63
- 2025-04-30 22:18:05,729 - INFO - - tests\test_supabase_tables.py
64
- 2025-04-30 22:18:05,729 - INFO - - tests\test_supabase_tables_comprehensive.py
65
- 2025-04-30 22:18:05,729 - INFO - - tests\test_supabase_tables_final.py
66
- 2025-04-30 22:18:05,729 - INFO - - tests\test_supabase_tables_production.py
67
- 2025-04-30 22:18:05,729 - INFO - - tests\test_supabase_tables_simple.py
68
- 2025-04-30 22:18:05,730 - INFO - Keeping: tests\test_supabase_tables_comprehensive.py
69
- 2025-04-30 22:18:05,730 - INFO - Removing: tests\test_supabase_tables.py, tests\test_supabase_tables_simple.py
70
- 2025-04-30 22:18:05,731 - INFO - Removed: tests\test_supabase_tables.py
71
- 2025-04-30 22:18:05,733 - INFO - Removed: tests\test_supabase_tables_simple.py
72
- 2025-04-30 22:18:05,733 - INFO - Checking for debug statements and commented-out code...
73
- 2025-04-30 22:18:05,803 - INFO - Found 248 instances of debug statements or commented-out code
74
- 2025-04-30 22:18:05,803 - INFO - Issues in agent\agent.py:
75
- 2025-04-30 22:18:05,803 - INFO - Line 72 (Commented code): # Set up verbose logging if enabled
76
- 2025-04-30 22:18:05,803 - INFO - Line 75 (Commented code): # Add console handler if not already present
77
- 2025-04-30 22:18:05,803 - INFO - Line 92 (Debug statement): logger.debug("Initializing memory components")
78
- 2025-04-30 22:18:05,803 - INFO - Line 98 (Commented code): # Create a unique session ID for this agent instance
79
- 2025-04-30 22:18:05,803 - INFO - Line 106 (Debug statement): logger.debug("Memory components initialized successfully")
80
- 2025-04-30 22:18:05,804 - INFO - Line 161 (Commented code): # Record end time and return the answer
81
- 2025-04-30 22:18:05,804 - INFO - Line 207 (Debug statement): logger.debug(f"Processing question: {question}")
82
- 2025-04-30 22:18:05,804 - INFO - Line 209 (Commented code): # Check if we have a cached answer
83
- 2025-04-30 22:18:05,804 - INFO - Line 215 (Commented code): # Check if we have a cached answer
84
- 2025-04-30 22:18:05,804 - INFO - Line 221 (Commented code): # Create a config for the graph
85
- 2025-04-30 22:18:05,804 - INFO - Line 237 (Commented code): # Fallback answer if no answer or error is provided
86
- 2025-04-30 22:18:05,804 - INFO - Line 240 (Commented code): # Store the answer in the result cache if available
87
- 2025-04-30 22:18:05,804 - INFO - Line 244 (Commented code): # Store the intermediate results in working memory if available
88
- 2025-04-30 22:18:05,804 - INFO - Line 264 (Commented code): # Add the question and answer to conversation memory if available
89
- 2025-04-30 22:18:05,804 - INFO - Line 291 (Commented code): # Get conversation history if available
90
- 2025-04-30 22:18:05,804 - INFO - Line 295 (Commented code): # Get cached results if available
91
- 2025-04-30 22:18:05,805 - INFO - Line 308 (Commented code): # Get intermediate results if available
92
- 2025-04-30 22:18:05,805 - INFO - Line 323 (Commented code): # Clear conversation memory if available
93
- 2025-04-30 22:18:05,805 - INFO - Line 327 (Commented code): # Clear result cache if available
94
- 2025-04-30 22:18:05,805 - INFO - Line 331 (Commented code): # Clear working memory if available
95
- 2025-04-30 22:18:05,805 - INFO - Issues in agent\config.py:
96
- 2025-04-30 22:18:05,805 - INFO - Line 20 (Commented code): # Try to import dotenv, but don't fail if it's not available
97
- 2025-04-30 22:18:05,805 - INFO - Line 23 (Commented code): # Load environment variables from .env file if it exists
98
- 2025-04-30 22:18:05,805 - INFO - Line 28 (Debug statement): print("Warning: python-dotenv not installed. Environment variables will not be loaded from .env file.")
99
- 2025-04-30 22:18:05,806 - INFO - Line 30 (Commented code): # API Keys and Credentials (kept as environment variables)
100
- 2025-04-30 22:18:05,806 - INFO - Line 37 (Commented code): # Supabase Configuration (partially hardcoded)
101
- 2025-04-30 22:18:05,806 - INFO - Line 42 (Commented code): # API Endpoints (hardcoded)
102
- 2025-04-30 22:18:05,806 - INFO - Line 48 (Commented code): # Model Configuration (hardcoded with updated models)
103
- 2025-04-30 22:18:05,806 - INFO - Line 57 (Commented code): # Tool Parameters (hardcoded)
104
- 2025-04-30 22:18:05,806 - INFO - Line 69 (Commented code): # Memory Settings (hardcoded)
105
- 2025-04-30 22:18:05,806 - INFO - Line 77 (Commented code): # Agent Settings (hardcoded)
106
- 2025-04-30 22:18:05,806 - INFO - Line 85 (Commented code): # Logging Configuration (hardcoded)
107
- 2025-04-30 22:18:05,807 - INFO - Line 90 (Commented code): # Multimodal Settings (hardcoded)
108
- 2025-04-30 22:18:05,807 - INFO - Line 95 (Commented code): # Web Browsing Settings (hardcoded)
109
- 2025-04-30 22:18:05,807 - INFO - Line 101 (Commented code): # Helper functions for configuration
110
- 2025-04-30 22:18:05,807 - INFO - Issues in agent\graph.py:
111
- 2025-04-30 22:18:05,807 - INFO - Line 30 (Commented code): # Type definitions for state
112
- 2025-04-30 22:18:05,807 - INFO - Line 75 (Debug statement): logger.debug(f"Analyzing question: {state['question']}")
113
- 2025-04-30 22:18:05,807 - INFO - Line 117 (Commented code): # Parse the analysis (in a real implementation, use a proper JSON parser)
114
- 2025-04-30 22:18:05,807 - INFO - Line 118 (Commented code): # This is a simplified version for demonstration
115
- 2025-04-30 22:18:05,807 - INFO - Line 126 (Debug statement): logger.debug(f"Question analysis completed: {analysis}")
116
- 2025-04-30 22:18:05,807 - INFO - Line 148 (Debug statement): logger.debug("Creating plan based on question analysis")
117
- 2025-04-30 22:18:05,808 - INFO - Line 150 (Commented code): # If there's an error, return the state unchanged
118
- 2025-04-30 22:18:05,808 - INFO - Line 205 (Commented code): # Parse the plan (in a real implementation, use a proper JSON parser)
119
- 2025-04-30 22:18:05,808 - INFO - Line 222 (Debug statement): logger.debug(f"Plan created with {len(plan)} steps")
120
- 2025-04-30 22:18:05,809 - INFO - Line 244 (Debug statement): logger.debug(f"Selecting tool for step {state['current_step'] + 1}")
121
- 2025-04-30 22:18:05,809 - INFO - Line 246 (Commented code): # If there's an error, return END
122
- 2025-04-30 22:18:05,809 - INFO - Line 281 (Commented code): # If there's an error, return the state unchanged
123
- 2025-04-30 22:18:05,809 - INFO - Line 290 (Debug statement): logger.debug(f"Executing tool {tool_name} for step {current_step['step_number']}")
124
- 2025-04-30 22:18:05,809 - INFO - Line 315 (Debug statement): logger.debug(f"Tool {tool_name} executed successfully")
125
- 2025-04-30 22:18:05,810 - INFO - Line 350 (Debug statement): logger.debug("Formulating answer based on tool results")
126
- 2025-04-30 22:18:05,810 - INFO - Line 352 (Commented code): # If there's an error, return the state unchanged
127
- 2025-04-30 22:18:05,810 - INFO - Line 409 (Debug statement): logger.debug(f"Answer formulated: {answer.strip()}")
128
- 2025-04-30 22:18:05,810 - INFO - Line 431 (Commented code): # If there's an error, check if we should retry
129
- 2025-04-30 22:18:05,810 - INFO - Issues in agent\tool_registry.py:
130
- 2025-04-30 22:18:05,810 - INFO - Line 34 (Debug statement): logger.debug("Initialized empty tool registry")
131
- 2025-04-30 22:18:05,811 - INFO - Line 45 (Debug statement): logger.debug(f"Registered tool: {name}")
132
- 2025-04-30 22:18:05,811 - INFO - Line 89 (Debug statement): logger.debug(f"Executing tool: {name} with args: {kwargs}")
133
- 2025-04-30 22:18:05,811 - INFO - Line 147 (Commented code): # Register DuckDuckGo search tool (no API key required)
134
- 2025-04-30 22:18:05,811 - INFO - Line 154 (Commented code): # Register Serper search tool if API key is available
135
- 2025-04-30 22:18:05,811 - INFO - Line 165 (Commented code): # Register Perplexity tool if API key is available
136
- 2025-04-30 22:18:05,811 - INFO - Line 176 (Commented code): # Register arXiv tool (no API key required)
137
- 2025-04-30 22:18:05,812 - INFO - Line 180 (Commented code): # Register the same tool instance for different operations
138
- 2025-04-30 22:18:05,812 - INFO - Line 207 (Commented code): # Try DuckDuckGo first (no API key required)
139
- 2025-04-30 22:18:05,812 - INFO - Line 334 (Debug statement): print(f"Available tools: {registry.list_tools()}")
140
- 2025-04-30 22:18:05,812 - INFO - Line 340 (Debug statement): print(f"Robust search using {result['provider']} found {len(result['results'])} results")
141
- 2025-04-30 22:18:05,812 - INFO - Line 342 (Debug statement): print(f"Robust search failed: {str(e)}")
142
- 2025-04-30 22:18:05,812 - INFO - Line 348 (Debug statement): print(f"Merged search found {len(result['merged_results'])} results")
143
- 2025-04-30 22:18:05,813 - INFO - Line 350 (Debug statement): print(f"Perplexity context: {result['perplexity_context'][:100]}...")
144
- 2025-04-30 22:18:05,813 - INFO - Line 352 (Debug statement): print(f"arXiv papers: {len(result['arxiv_context'])}")
145
- 2025-04-30 22:18:05,813 - INFO - Line 354 (Debug statement): print(f"Merged search failed: {str(e)}")
146
- 2025-04-30 22:18:05,813 - INFO - Issues in memory\fix_supabase_tables.py:
147
- 2025-04-30 22:18:05,813 - INFO - Line 55 (Commented code): # Check if the client is initialized
148
- 2025-04-30 22:18:05,813 - INFO - Line 71 (Commented code): # Check if the table exists
149
- 2025-04-30 22:18:05,813 - INFO - Line 79 (Commented code): # SQL for creating the table
150
- 2025-04-30 22:18:05,814 - INFO - Line 130 (Commented code): # Trim cache if it's too large
151
- 2025-04-30 22:18:05,814 - INFO - Line 141 (Commented code): # Store in Supabase if initialized
152
- 2025-04-30 22:18:05,814 - INFO - Line 150 (Debug statement): logger.debug(f"Stored entry in table {current_table}: {key}")
153
- 2025-04-30 22:18:05,814 - INFO - Line 170 (Commented code): # Check if the entry has expired
154
- 2025-04-30 22:18:05,814 - INFO - Line 187 (Commented code): # Check if any entries were found
155
- 2025-04-30 22:18:05,814 - INFO - Line 196 (Commented code): # Check if the entry has expired
156
- 2025-04-30 22:18:05,814 - INFO - Line 208 (Debug statement): logger.debug(f"Retrieved entry from table {current_table}: {key}")
157
- 2025-04-30 22:18:05,814 - INFO - Line 227 (Commented code): # Remove from Supabase if initialized
158
- 2025-04-30 22:18:05,814 - INFO - Line 233 (Debug statement): logger.debug(f"Deleted entry from table {current_table}: {key}")
159
- 2025-04-30 22:18:05,814 - INFO - Line 256 (Commented code): # Get keys from Supabase if initialized
160
- 2025-04-30 22:18:05,815 - INFO - Line 262 (Commented code): # Filter by prefix if provided
161
- 2025-04-30 22:18:05,815 - INFO - Line 278 (Debug statement): logger.debug(f"Listed keys from table {current_table}: {len(keys)} keys")
162
- 2025-04-30 22:18:05,815 - INFO - Line 350 (Commented code): # Create a key for the message
163
- 2025-04-30 22:18:05,815 - INFO - Line 360 (Commented code): # Add metadata if provided
164
- 2025-04-30 22:18:05,816 - INFO - Line 384 (Commented code): # Filter by before/after if provided
165
- 2025-04-30 22:18:05,816 - INFO - Line 391 (Commented code): # Limit the number of messages if provided
166
- 2025-04-30 22:18:05,816 - INFO - Line 526 (Commented code): # Check if any entries were found
167
- 2025-04-30 22:18:05,816 - INFO - Line 546 (Debug statement): print("\nFixing Supabase Tables for GAIA Agent")
168
- 2025-04-30 22:18:05,816 - INFO - Line 547 (Debug statement): print("====================================\n")
169
- 2025-04-30 22:18:05,837 - INFO - Line 562 (Debug statement): print("All Supabase tables are now being used correctly.")
170
- 2025-04-30 22:18:05,839 - INFO - Line 566 (Debug statement): print("Please check the logs for more information")
171
- 2025-04-30 22:18:05,839 - INFO - Issues in memory\simplified_memory_fix.py:
172
- 2025-04-30 22:18:05,839 - INFO - Line 50 (Commented code): # Check if the client is initialized
173
- 2025-04-30 22:18:05,840 - INFO - Line 59 (Commented code): # Check if the table exists
174
- 2025-04-30 22:18:05,840 - INFO - Line 68 (Commented code): # SQL for creating the table
175
- 2025-04-30 22:18:05,840 - INFO - Line 118 (Commented code): # Trim cache if it's too large
176
- 2025-04-30 22:18:05,840 - INFO - Line 129 (Commented code): # Store in Supabase if initialized
177
- 2025-04-30 22:18:05,840 - INFO - Line 138 (Debug statement): logger.debug(f"Stored entry in table {current_table}: {key}")
178
- 2025-04-30 22:18:05,840 - INFO - Line 158 (Commented code): # Check if the entry has expired
179
- 2025-04-30 22:18:05,840 - INFO - Line 175 (Commented code): # Check if any entries were found
180
- 2025-04-30 22:18:05,840 - INFO - Line 184 (Commented code): # Check if the entry has expired
181
- 2025-04-30 22:18:05,841 - INFO - Line 196 (Debug statement): logger.debug(f"Retrieved entry from table {current_table}: {key}")
182
- 2025-04-30 22:18:05,841 - INFO - Line 215 (Commented code): # Remove from Supabase if initialized
183
- 2025-04-30 22:18:05,841 - INFO - Line 221 (Debug statement): logger.debug(f"Deleted entry from table {current_table}: {key}")
184
- 2025-04-30 22:18:05,841 - INFO - Line 244 (Commented code): # Get keys from Supabase if initialized
185
- 2025-04-30 22:18:05,841 - INFO - Line 250 (Commented code): # Filter by prefix if provided
186
- 2025-04-30 22:18:05,841 - INFO - Line 266 (Debug statement): logger.debug(f"Listed keys from table {current_table}: {len(keys)} keys")
187
- 2025-04-30 22:18:05,842 - INFO - Line 388 (Commented code): # Check if any entries were found
188
- 2025-04-30 22:18:05,842 - INFO - Line 408 (Debug statement): print("\nSimplified Fix for Supabase Tables in GAIA Agent")
189
- 2025-04-30 22:18:05,842 - INFO - Line 409 (Debug statement): print("==============================================\n")
190
- 2025-04-30 22:18:05,843 - INFO - Line 411 (Commented code): # Create the working memory table if it doesn't exist
191
- 2025-04-30 22:18:05,845 - INFO - Line 424 (Debug statement): print("The working memory table is now being used correctly.")
192
- 2025-04-30 22:18:05,846 - INFO - Line 425 (Debug statement): print("This should be sufficient for the GAIA benchmark test.")
193
- 2025-04-30 22:18:05,848 - INFO - Line 429 (Debug statement): print("Please check the logs for more information")
194
- 2025-04-30 22:18:05,848 - INFO - Issues in memory\supabase_memory.py:
195
- 2025-04-30 22:18:05,849 - INFO - Line 15 (Commented code): # Try to import supabase, but don't fail if it's not available
196
- 2025-04-30 22:18:05,849 - INFO - Line 19 (Commented code): # Define dummy types for type checking
197
- 2025-04-30 22:18:05,849 - INFO - Line 60 (Commented code): # Set expiration time if TTL is provided
198
- 2025-04-30 22:18:05,849 - INFO - Line 147 (Commented code): # Check if the main table exists
199
- 2025-04-30 22:18:05,849 - INFO - Line 181 (Commented code): # Check if the table exists by querying it
200
- 2025-04-30 22:18:05,850 - INFO - Line 183 (Debug statement): logger.debug(f"Table {actual_table} exists")
201
- 2025-04-30 22:18:05,850 - INFO - Line 249 (Commented code): # Trim cache if it's too large
202
- 2025-04-30 22:18:05,850 - INFO - Line 260 (Commented code): # Store in Supabase if initialized
203
- 2025-04-30 22:18:05,850 - INFO - Line 300 (Commented code): # Check if the entry has expired
204
- 2025-04-30 22:18:05,850 - INFO - Line 316 (Commented code): # Check if any entries were found
205
- 2025-04-30 22:18:05,850 - INFO - Line 324 (Commented code): # Check if the entry has expired
206
- 2025-04-30 22:18:05,851 - INFO - Line 359 (Commented code): # Remove from Supabase if initialized
207
- 2025-04-30 22:18:05,851 - INFO - Line 388 (Commented code): # Clear Supabase if initialized
208
- 2025-04-30 22:18:05,851 - INFO - Line 422 (Commented code): # Get keys from Supabase if initialized
209
- 2025-04-30 22:18:05,851 - INFO - Line 428 (Commented code): # Filter by prefix if provided
210
- 2025-04-30 22:18:05,851 - INFO - Line 484 (Commented code): # Create a key for the message
211
- 2025-04-30 22:18:05,851 - INFO - Line 494 (Commented code): # Add metadata if provided
212
- 2025-04-30 22:18:05,851 - INFO - Line 531 (Commented code): # Filter by before/after if provided
213
- 2025-04-30 22:18:05,851 - INFO - Line 538 (Commented code): # Limit the number of messages if provided
214
- 2025-04-30 22:18:05,852 - INFO - Issues in memory\supabase_memory_patch.py:
215
- 2025-04-30 22:18:05,852 - INFO - Line 60 (Commented code): # Check if the client is initialized
216
- 2025-04-30 22:18:05,852 - INFO - Line 73 (Commented code): # Check if the tables exist
217
- 2025-04-30 22:18:05,852 - INFO - Line 90 (Commented code): # SQL for creating the table
218
- 2025-04-30 22:18:05,853 - INFO - Line 151 (Debug statement): print("All tables are now properly utilized:")
219
- 2025-04-30 22:18:05,854 - INFO - Line 152 (Debug statement): print("- gaia_memory: Main memory table")
220
- 2025-04-30 22:18:05,854 - INFO - Line 153 (Debug statement): print("- gaia_memory_conversations: Conversation history table")
221
- 2025-04-30 22:18:05,854 - INFO - Line 154 (Debug statement): print("- gaia_memory_cache: Result cache table")
222
- 2025-04-30 22:18:05,854 - INFO - Line 155 (Debug statement): print("- gaia_memory_working: Working memory table")
223
- 2025-04-30 22:18:05,856 - INFO - Line 159 (Debug statement): print("Please check the logs for more information")
224
- 2025-04-30 22:18:05,856 - INFO - Issues in tools\arxiv_tool.py:
225
- 2025-04-30 22:18:05,856 - INFO - Line 60 (Debug statement): logger.debug(f"Searching arXiv for: {query}")
226
- 2025-04-30 22:18:05,856 - INFO - Line 82 (Debug statement): logger.debug(f"Found {len(results)} results from arXiv")
227
- 2025-04-30 22:18:05,856 - INFO - Line 134 (Debug statement): logger.debug(f"Getting arXiv paper with ID: {paper_id}")
228
- 2025-04-30 22:18:05,856 - INFO - Line 143 (Commented code): # Create a search for the specific paper
229
- 2025-04-30 22:18:05,856 - INFO - Line 201 (Debug statement): logger.debug(f"Searching arXiv for papers in category: {category}")
230
- 2025-04-30 22:18:05,856 - INFO - Line 203 (Commented code): # Create a query for the category
231
- 2025-04-30 22:18:05,856 - INFO - Line 222 (Commented code): # Search for papers
232
- 2025-04-30 22:18:05,857 - INFO - Line 225 (Debug statement): print(f"Found {len(results)} papers")
233
- 2025-04-30 22:18:05,857 - INFO - Line 227 (Debug statement): print(f"\nPaper {i+1}:")
234
- 2025-04-30 22:18:05,857 - INFO - Line 228 (Debug statement): print(f"Title: {paper['title']}")
235
- 2025-04-30 22:18:05,857 - INFO - Line 229 (Debug statement): print(f"Authors: {paper['authors']}")
236
- 2025-04-30 22:18:05,857 - INFO - Line 230 (Debug statement): print(f"Published: {paper['published']}")
237
- 2025-04-30 22:18:05,857 - INFO - Line 231 (Debug statement): print(f"URL: {paper['url']}")
238
- 2025-04-30 22:18:05,857 - INFO - Line 232 (Debug statement): print(f"Summary: {paper['summary'][:200]}...")
239
- 2025-04-30 22:18:05,857 - INFO - Line 234 (Debug statement): print(f"Error: {str(e)}")
240
- 2025-04-30 22:18:05,857 - INFO - Issues in tools\multimodal_tools.py:
241
- 2025-04-30 22:18:05,858 - INFO - Line 81 (Debug statement): logger.debug(f"Analyzing image: {image_path}")
242
- 2025-04-30 22:18:05,858 - INFO - Line 126 (Debug statement): logger.debug("Image analysis completed successfully")
243
- 2025-04-30 22:18:05,858 - INFO - Line 129 (Commented code): # If the result is not valid JSON, return it as plain text
244
- 2025-04-30 22:18:05,858 - INFO - Line 157 (Debug statement): logger.debug(f"Detecting objects in image: {image_path}")
245
- 2025-04-30 22:18:05,858 - INFO - Line 206 (Debug statement): logger.debug(f"Object detection completed, found {parsed_result.get('object_count', 0)} objects")
246
- 2025-04-30 22:18:05,859 - INFO - Line 255 (Debug statement): logger.debug(f"Interpreting chart: {chart_path}")
247
- 2025-04-30 22:18:05,859 - INFO - Line 306 (Debug statement): logger.debug(f"Chart interpretation completed: {parsed_result.get('chart_type', 'unknown')} chart")
248
- 2025-04-30 22:18:05,859 - INFO - Line 338 (Debug statement): logger.debug(f"Extracting data from chart: {chart_path}")
249
- 2025-04-30 22:18:05,859 - INFO - Line 386 (Debug statement): logger.debug(f"Chart data extraction completed with {len(parsed_result.get('data', []))} data points")
250
- 2025-04-30 22:18:05,859 - INFO - Line 422 (Commented code): # Check for required dependencies
251
- 2025-04-30 22:18:05,860 - INFO - Line 443 (Debug statement): logger.debug(f"Parsing document: {document_path}")
252
- 2025-04-30 22:18:05,860 - INFO - Line 468 (Commented code): # Truncate if too long
253
- 2025-04-30 22:18:05,860 - INFO - Line 504 (Debug statement): logger.debug(f"Extracting structured data from document: {document_path}")
254
- 2025-04-30 22:18:05,860 - INFO - Line 536 (Debug statement): logger.debug("Structured data extraction completed successfully")
255
- 2025-04-30 22:18:05,860 - INFO - Issues in tools\perplexity_tool.py:
256
- 2025-04-30 22:18:05,860 - INFO - Line 20 (Commented code): # Set up basic logging if not already configured
257
- 2025-04-30 22:18:05,860 - INFO - Line 40 (Commented code): # Get API key from config or environment (only configuration from environment)
258
- 2025-04-30 22:18:05,860 - INFO - Line 45 (Commented code): # Hardcoded values for API URL and model
259
- 2025-04-30 22:18:05,860 - INFO - Line 50 (Commented code): # Endpoint for chat completions
260
- 2025-04-30 22:18:05,861 - INFO - Line 53 (Debug statement): logger.debug(f"Initialized Perplexity tool with model {self.model}")
261
- 2025-04-30 22:18:05,861 - INFO - Line 73 (Debug statement): logger.debug(f"Querying Perplexity API: {question[:50]}...")
262
- 2025-04-30 22:18:05,861 - INFO - Line 82 (Commented code): # Add system prompt if provided
263
- 2025-04-30 22:18:05,861 - INFO - Line 134 (Debug statement): logger.debug(f"Perplexity API query successful, received {len(content) if content else 0} characters")
264
- 2025-04-30 22:18:05,862 - INFO - Line 169 (Debug statement): logger.debug(f"Searching with Perplexity API: {query[:50]}...")
265
- 2025-04-30 22:18:05,862 - INFO - Issues in tools\reasoning_tools.py:
266
- 2025-04-30 22:18:05,862 - INFO - Line 65 (Debug statement): logger.debug(f"Performing step-by-step reasoning on: {question}")
267
- 2025-04-30 22:18:05,862 - INFO - Line 93 (Commented code): # Add context if provided
268
- 2025-04-30 22:18:05,862 - INFO - Line 111 (Debug statement): logger.debug(f"Reasoning completed with {len(parsed_result.get('steps', []))} steps")
269
- 2025-04-30 22:18:05,862 - INFO - Line 134 (Debug statement): logger.debug(f"Performing chain-of-thought reasoning on: {question}")
270
- 2025-04-30 22:18:05,862 - INFO - Line 148 (Commented code): # Add context if provided
271
- 2025-04-30 22:18:05,862 - INFO - Line 163 (Debug statement): logger.debug("Chain-of-thought reasoning completed")
272
- 2025-04-30 22:18:05,862 - INFO - Line 203 (Debug statement): logger.debug(f"Calculating expression: {expression}")
273
- 2025-04-30 22:18:05,863 - INFO - Line 209 (Commented code): # Evaluate using sympy for symbolic math
274
- 2025-04-30 22:18:05,863 - INFO - Line 241 (Debug statement): logger.debug(f"Solving equation: {equation} for {variable}")
275
- 2025-04-30 22:18:05,863 - INFO - Line 262 (Commented code): # Convert to float or int if possible
276
- 2025-04-30 22:18:05,863 - INFO - Line 317 (Commented code): # Only replace if it's a whole word (not part of another word)
277
- 2025-04-30 22:18:05,863 - INFO - Line 354 (Debug statement): logger.debug(f"Verifying claim: {claim}")
278
- 2025-04-30 22:18:05,863 - INFO - Line 395 (Debug statement): logger.debug(f"Verification completed: {parsed_result.get('verification_result', 'unknown')}")
279
- 2025-04-30 22:18:05,863 - INFO - Line 418 (Debug statement): logger.debug(f"Comparing claims: '{claim1}' and '{claim2}'")
280
- 2025-04-30 22:18:05,863 - INFO - Line 455 (Debug statement): logger.debug(f"Comparison completed: {parsed_result.get('relationship', 'unknown')}")
281
- 2025-04-30 22:18:05,863 - INFO - Issues in tools\web_tools.py:
282
- 2025-04-30 22:18:05,864 - INFO - Line 124 (Debug statement): logger.debug(f"Searching DuckDuckGo for: {query}")
283
- 2025-04-30 22:18:05,864 - INFO - Line 137 (Debug statement): logger.debug(f"Found {len(results)} results from DuckDuckGo")
284
- 2025-04-30 22:18:05,864 - INFO - Line 187 (Debug statement): logger.debug(f"Searching Serper for: {query}")
285
- 2025-04-30 22:18:05,864 - INFO - Line 217 (Debug statement): logger.debug(f"Found {len(organic_results)} results from Serper")
286
- 2025-04-30 22:18:05,864 - INFO - Line 270 (Debug statement): logger.debug(f"Extracting content from: {url}")
287
- 2025-04-30 22:18:05,864 - INFO - Line 306 (Commented code): # Truncate if too long
288
- 2025-04-30 22:18:05,864 - INFO - Line 392 (Debug statement): logger.debug(f"Following links from: {url}")
289
- 2025-04-30 22:18:05,864 - INFO - Line 401 (Commented code): # Filter links by pattern if provided
290
- 2025-04-30 22:18:05,864 - INFO - Issues in tools\__init__.py:
291
- 2025-04-30 22:18:05,865 - INFO - Line 129 (Debug statement): logger.debug(f"Registered tool: {metadata.name} in category {metadata.category}")
292
- 2025-04-30 22:18:05,865 - INFO - Line 189 (Commented code): # Check for direct tool name match
293
- 2025-04-30 22:18:05,865 - INFO - Line 194 (Commented code): # Check for keyword matches
294
- 2025-04-30 22:18:05,865 - INFO - Line 659 (Commented code): # Convenience functions for accessing tools
295
- 2025-04-30 22:18:05,865 - INFO - Issues in utils\formatting.py:
296
- 2025-04-30 22:18:05,865 - INFO - Line 38 (Debug statement): logger.debug(f"Extracting answer from response (expected format: {expected_format})")
297
- 2025-04-30 22:18:05,865 - INFO - Line 40 (Commented code): # If the response is already concise, return it directly
298
- 2025-04-30 22:18:05,865 - INFO - Line 44 (Commented code): # Look for explicit answer indicators
299
- 2025-04-30 22:18:05,865 - INFO - Line 75 (Commented code): # Default: return the last sentence as the answer
300
- 2025-04-30 22:18:05,866 - INFO - Line 92 (Commented code): # Look for numbers with optional decimal points and commas
301
- 2025-04-30 22:18:05,866 - INFO - Line 108 (Commented code): # If no answer-indicating phrase, find all numbers and return the last one
302
- 2025-04-30 22:18:05,866 - INFO - Line 147 (Commented code): # If no answer-indicating phrase, find all dates and return the first one
303
- 2025-04-30 22:18:05,866 - INFO - Line 167 (Commented code): # Check for yes/no indicators at the beginning of the text or after answer indicators
304
- 2025-04-30 22:18:05,866 - INFO - Line 198 (Commented code): # If no clear yes/no, look for sentiment
305
- 2025-04-30 22:18:05,866 - INFO - Line 213 (Commented code): # Default to "Unknown" if can't determine
306
- 2025-04-30 22:18:05,866 - INFO - Line 226 (Commented code): # Look for explicit list indicators
307
- 2025-04-30 22:18:05,866 - INFO - Line 244 (Commented code): # Look for bullet points or numbered lists
308
- 2025-04-30 22:18:05,866 - INFO - Line 250 (Commented code): # Look for any comma-separated sequence
309
- 2025-04-30 22:18:05,866 - INFO - Line 268 (Commented code): # Look for entity indicators
310
- 2025-04-30 22:18:05,866 - INFO - Line 281 (Commented code): # Look for capitalized proper nouns
311
- 2025-04-30 22:18:05,867 - INFO - Line 306 (Commented code): # Look for JSON blocks
312
- 2025-04-30 22:18:05,867 - INFO - Line 318 (Commented code): # Look for JSON without code blocks
313
- 2025-04-30 22:18:05,867 - INFO - Line 341 (Debug statement): logger.debug(f"Formatting answer to {format_type} format")
314
- 2025-04-30 22:18:05,867 - INFO - Line 401 (Commented code): # Default: return as is
315
- 2025-04-30 22:18:05,867 - INFO - Line 415 (Debug statement): logger.debug(f"Validating answer format (expected: {expected_format})")
316
- 2025-04-30 22:18:05,867 - INFO - Line 435 (Commented code): # Validate date format (simple check)
317
- 2025-04-30 22:18:05,867 - INFO - Line 469 (Commented code): # Validate entity format (simple check for proper noun)
318
- 2025-04-30 22:18:05,867 - INFO - Line 502 (Debug statement): logger.debug(f"Processing answer (expected format: {expected_format})")
319
- 2025-04-30 22:18:05,867 - INFO - Line 504 (Commented code): # Default to text format if none specified
320
- 2025-04-30 22:18:05,867 - INFO - Issues in utils\parsing.py:
321
- 2025-04-30 22:18:05,868 - INFO - Line 34 (Debug statement): logger.debug(f"Extracting key information from question: {question}")
322
- 2025-04-30 22:18:05,868 - INFO - Line 47 (Commented code): # Extract entities (proper nouns, potentially)
323
- 2025-04-30 22:18:05,868 - INFO - Line 52 (Commented code): # Extract keywords (important nouns, verbs, adjectives)
324
- 2025-04-30 22:18:05,868 - INFO - Line 143 (Debug statement): logger.debug(f"Identifying required tools for question: {question}")
325
- 2025-04-30 22:18:05,868 - INFO - Line 197 (Debug statement): logger.debug(f"Determining complexity for question: {question}")
326
- 2025-04-30 22:18:05,868 - INFO - Line 293 (Debug statement): logger.debug(f"Parsing question: {question}")
327
- 2025-04-30 22:18:05,868 - INFO - Issues in utils\__init__.py:
328
- 2025-04-30 22:18:05,868 - INFO - Line 96 (Commented code): # Check for number format
329
- 2025-04-30 22:18:05,868 - INFO - Line 100 (Commented code): # Check for date format
330
- 2025-04-30 22:18:05,868 - INFO - Line 104 (Commented code): # Check for boolean format
331
- 2025-04-30 22:18:05,869 - INFO - Line 109 (Commented code): # Check for list format
332
- 2025-04-30 22:18:05,869 - INFO - Line 113 (Commented code): # Check for entity format
333
- 2025-04-30 22:18:05,869 - INFO - Checking for potential hardcoded credentials...
334
- 2025-04-30 22:18:05,990 - INFO - No potential hardcoded credentials found
335
- 2025-04-30 22:18:05,990 - INFO -
336
- Cleanup Summary:
337
- 2025-04-30 22:18:05,991 - INFO - - Temporary files to remove: 39
338
- 2025-04-30 22:18:05,991 - INFO - - Duplicate test files to remove: 7
339
- 2025-04-30 22:18:05,991 - INFO - - Files with debug statements or commented code: 17
340
- 2025-04-30 22:18:05,992 - INFO - - Files with potential hardcoded credentials: 0
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
complete_deployment_guide.md DELETED
@@ -1,129 +0,0 @@
1
- # Complete GAIA Agent Deployment Guide for Hugging Face Space
2
-
3
- This guide addresses all issues encountered when deploying the GAIA agent to Hugging Face Space, including OAuth configuration and dependency conflicts.
4
-
5
- ## Step 1: Fix OAuth Configuration
6
-
7
- The OAuth error occurs because Hugging Face Spaces requires OAuth for user identification.
8
-
9
- 1. Navigate to your Space on Hugging Face
10
- 2. Click on the "Files" tab
11
- 3. Open **README.md**
12
- 4. Edit the file to include this YAML frontmatter at the top:
13
-
14
- ```yaml
15
- ---
16
- title: GAIA Agent Assessment
17
- emoji: 🤖
18
- colorFrom: blue
19
- colorTo: green
20
- sdk: gradio
21
- sdk_version: 3.50.2
22
- app_file: app.py
23
- pinned: false
24
- license: mit
25
- hf_oauth: true # This enables OAuth!
26
- ---
27
- ```
28
-
29
- 5. Click "Commit changes"
30
-
31
- ## Step 2: Fix Dependency Conflicts
32
-
33
- The build error is caused by conflicts between package versions, particularly around httpx:
34
-
35
- - supabase 1.2.0 requires httpx<0.25.0 and >=0.24.0
36
- - openai 1.6.1 requires httpx<1 and >=0.23.0
37
- - duckduckgo-search 3.9.5 requires httpx>=0.25.1
38
-
39
- To resolve this:
40
-
41
- 1. Navigate to your Space's "Files" tab
42
- 2. Open **requirements.txt**
43
- 3. Replace with this compatible configuration:
44
-
45
- ```
46
- # Fixed requirements with compatible versions
47
- gradio==3.50.2
48
- requests==2.31.0
49
- langchain==0.0.292
50
-
51
- # OpenAI and related dependencies
52
- openai==1.6.1
53
- langchain_openai==0.0.2.post1
54
- langgraph==0.0.19
55
-
56
- # HTTP Libraries - key conflict area
57
- httpx==0.24.1 # Compatible with supabase and openai
58
-
59
- # Database and storage
60
- supabase==1.2.0
61
-
62
- # Search tools - downgraded to avoid httpx conflict
63
- duckduckgo-search==3.8.3
64
-
65
- # Parsing and content handling
66
- beautifulsoup4==4.12.2
67
- pillow==10.0.1
68
- arxiv==1.4.8
69
- python-dotenv==1.0.0
70
-
71
- # Additional dependencies
72
- pydantic==1.10.13
73
- numpy==1.24.4
74
- pandas==2.0.3
75
- ```
76
-
77
- 4. Click "Commit changes"
78
-
79
- ## Step 3: Configure Environment Variables
80
-
81
- 1. Go to the "Settings" tab of your Space
82
- 2. Scroll to "Repository secrets"
83
- 3. Add the following environment variables:
84
- - `OPENAI_API_KEY`: Your OpenAI API key
85
- - `SUPABASE_URL`: Your Supabase project URL
86
- - `SUPABASE_KEY`: Your Supabase API key
87
- - `SERPER_API_KEY`: (Optional) Your Serper API key
88
-
89
- ## Step 4: Restart Your Space
90
-
91
- 1. Still in "Settings", scroll to "Factory reboot"
92
- 2. Click "Restart Space" to apply all changes
93
-
94
- ## Verification Steps
95
-
96
- After restarting:
97
- 1. Check the build logs to confirm successful installation
98
- 2. Verify the app loads correctly in the browser
99
- 3. Test the OAuth login functionality
100
- 4. Run a test evaluation to ensure all components work
101
-
102
- ## Troubleshooting Common Issues
103
-
104
- ### If OAuth Still Doesn't Work
105
- - Verify the `hf_oauth: true` line is correctly added to the README.md frontmatter
106
- - Make sure you've restarted the Space after changes
107
-
108
- ### If Dependency Conflicts Persist
109
- Try these alternative approaches:
110
-
111
- 1. **Further simplify dependencies**
112
- - Remove optional packages temporarily
113
- - Add them back one by one to identify conflicts
114
-
115
- 2. **Use separate requirements files**
116
- - Create a base requirements.txt with core dependencies
117
- - Use post-installation scripts to install conflicting packages separately
118
-
119
- 3. **Check for custom code dependencies**
120
- - Review the codebase for any imports that might require additional packages
121
- - Look for conditional imports that might need to be modified
122
-
123
- ## Post-Deployment Monitoring
124
-
125
- After successful deployment:
126
- 1. Monitor the application logs for any runtime errors
127
- 2. Test all major functionality paths
128
- 3. Verify Supabase connections and memory functionality
129
- 4. Test search tool integrations
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
complete_deployment_solution.md DELETED
@@ -1,183 +0,0 @@
1
- # GAIA Agent - Complete HuggingFace Space Deployment Solution
2
-
3
- This comprehensive solution addresses all deployment issues encountered with the GAIA Agent on Hugging Face Space:
4
-
5
- ## Issues Identified & Fixed
6
-
7
- 1. **OAuth Configuration Error**
8
- 2. **Dependency Conflicts** (OpenAI/langchain_openai incompatibility)
9
- 3. **Runtime Import Error** with missing module
10
-
11
- ## Step 1: Fix OAuth Configuration
12
-
13
- 1. Navigate to your Space's "Files" tab
14
- 2. Open **README.md**
15
- 3. Add this YAML frontmatter at the top:
16
-
17
- ```yaml
18
- ---
19
- title: GAIA Agent Assessment
20
- emoji: 🤖
21
- colorFrom: blue
22
- colorTo: green
23
- sdk: gradio
24
- sdk_version: 3.50.2
25
- app_file: app.py
26
- pinned: false
27
- license: mit
28
- hf_oauth: true # This enables OAuth!
29
- ---
30
- ```
31
-
32
- ## Step 2: Fix Dependency Conflicts
33
-
34
- 1. Navigate to your Space's "Files" tab
35
- 2. Open **requirements.txt**
36
- 3. Replace with these compatible versions:
37
-
38
- ```
39
- # Final fixed requirements
40
- gradio==3.50.2
41
- requests==2.31.0
42
-
43
- # Langchain ecosystem with compatible versions
44
- langchain==0.1.0
45
- langchain-core==0.1.17
46
- openai==1.10.0 # Specific version required by langchain_openai
47
- langchain_openai==0.0.5
48
- langgraph==0.0.24
49
-
50
- # HTTP Libraries
51
- httpx==0.24.1 # Compatible with supabase
52
-
53
- # Database and storage
54
- supabase==1.2.0
55
-
56
- # Search tools - downgraded to avoid httpx conflict
57
- duckduckgo-search==3.8.3
58
-
59
- # Parsing and content handling
60
- beautifulsoup4==4.12.2
61
- pillow==10.0.1
62
- arxiv==1.4.8
63
- python-dotenv==1.0.0
64
-
65
- # Additional dependencies
66
- pydantic==1.10.13
67
- numpy==1.24.4
68
- pandas==2.0.3
69
- ```
70
-
71
- ## Step 3: Fix Runtime Import Error
72
-
73
- You need to modify **agent/graph.py**:
74
-
75
- 1. Navigate to your Space's "Files" tab
76
- 2. Open **agent/graph.py**
77
- 3. Find line 22: `from langgraph.prebuilt import ToolNode`
78
- 4. Replace with: `# Removed problematic import: from langgraph.prebuilt import ToolNode`
79
-
80
- Alternatively, upload and run the **fix_graph_imports.py** script:
81
-
82
- ```python
83
- #!/usr/bin/env python
84
- """
85
- Fix for the ModuleNotFoundError: No module named 'langchain.tools.render' error.
86
- """
87
- import os
88
- import re
89
- import sys
90
- import logging
91
-
92
- logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')
93
- logger = logging.getLogger(__name__)
94
-
95
- def patch_graph_file():
96
- """Patch the agent/graph.py file to remove the problematic import."""
97
- graph_file = "agent/graph.py"
98
-
99
- if not os.path.exists(graph_file):
100
- logger.error(f"Could not find {graph_file}")
101
- return False
102
-
103
- try:
104
- with open(graph_file, "r") as f:
105
- content = f.read()
106
-
107
- # Create a backup
108
- backup_file = f"{graph_file}.bak"
109
- with open(backup_file, "w") as f:
110
- f.write(content)
111
- logger.info(f"Created backup at {backup_file}")
112
-
113
- # Remove the problematic import
114
- pattern = r"from langgraph\.prebuilt import ToolNode\n"
115
- updated_content = re.sub(pattern, "# Removed problematic import: from langgraph.prebuilt import ToolNode\n", content)
116
-
117
- with open(graph_file, "w") as f:
118
- f.write(updated_content)
119
-
120
- logger.info(f"Successfully patched {graph_file}")
121
- return True
122
-
123
- except Exception as e:
124
- logger.error(f"Error patching file: {str(e)}")
125
- return False
126
-
127
- if __name__ == "__main__":
128
- logger.info("Starting graph.py import patch...")
129
-
130
- if patch_graph_file():
131
- logger.info("Patch applied successfully!")
132
- else:
133
- logger.error("Failed to apply patch")
134
- sys.exit(1)
135
- ```
136
-
137
- ## Step 4: Configure Environment Variables
138
-
139
- 1. Go to the "Settings" tab of your Space
140
- 2. Scroll to "Repository secrets"
141
- 3. Add the following environment variables:
142
- - `OPENAI_API_KEY`: Your OpenAI API key
143
- - `SUPABASE_URL`: Your Supabase project URL
144
- - `SUPABASE_KEY`: Your Supabase API key
145
- - `SERPER_API_KEY`: (Optional) Your Serper API key
146
-
147
- ## Step 5: Restart Your Space
148
-
149
- 1. Still in "Settings", scroll to "Factory reboot"
150
- 2. Click "Restart Space" to apply all changes
151
-
152
- ## Detailed Error Analysis
153
-
154
- ### 1. OAuth Configuration Issue
155
- Required for Hugging Face Spaces to properly identify users during the GAIA assessment.
156
-
157
- ### 2. Dependency Conflict
158
- The error was:
159
- ```
160
- Cannot install langchain-openai 0.0.5 and openai==1.6.1 because these package versions have conflicting dependencies.
161
- The conflict is caused by:
162
- The user requested openai==1.6.1
163
- langchain-openai 0.0.5 depends on openai<2.0.0 and >=1.10.0
164
- ```
165
-
166
- Our solution updates openai to 1.10.0 to satisfy langchain_openai's requirements.
167
-
168
- ### 3. Runtime Import Error
169
- The error was:
170
- ```
171
- ModuleNotFoundError: No module named 'langchain.tools.render'
172
- ```
173
- Caused by import statement at line 22 in agent/graph.py that tried to use a module from older versions of langchain. Since this import isn't used in the code, removing it solves the issue.
174
-
175
- ## Verification
176
-
177
- After deploying:
178
- 1. Check build logs - verify dependencies installed correctly
179
- 2. Confirm the app loads in browser
180
- 3. Test OAuth login
181
- 4. Verify agent functionality works correctly
182
-
183
- All these fixes together should resolve the deployment issues and get your GAIA agent running successfully on Hugging Face Space.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
create_env_and_test.py DELETED
@@ -1,77 +0,0 @@
1
- """
2
- Create .env File and Test Supabase Connection
3
-
4
- This script creates a .env file with hardcoded values and then tests the connection to Supabase.
5
- """
6
-
7
- import os
8
- import sys
9
- import time
10
-
11
- def create_env_file(api_key):
12
- """
13
- Create a .env file with the provided API key.
14
-
15
- Args:
16
- api_key: The Supabase API key
17
- """
18
-
19
- env_content = f"""
20
- # Supabase Configuration
21
- SUPABASE_URL=https://tjamxhvvtnypbadvrkjq.supabase.co
22
- SUPABASE_KEY={api_key}
23
- """
24
-
25
- with open(".env", "w") as f:
26
- f.write(env_content)
27
-
28
-
29
- def test_supabase():
30
- """Test the connection to Supabase."""
31
-
32
- try:
33
- from supabase import create_client
34
- from dotenv import load_dotenv
35
-
36
- load_dotenv(override=True)
37
-
38
- # Get Supabase URL and key
39
- supabase_url = os.getenv("SUPABASE_URL")
40
- supabase_key = os.getenv("SUPABASE_KEY")
41
-
42
- if supabase_key:
43
- masked_key = supabase_key[:4] + "*" * (len(supabase_key) - 8) + supabase_key[-4:] if len(supabase_key) > 8 else "****"
44
- else:
45
- return False
46
-
47
- client = create_client(supabase_url, supabase_key)
48
-
49
- result = client.table("test_connection").select("*").limit(1).execute()
50
-
51
- return True
52
- except Exception as e:
53
- import traceback
54
- traceback.print_exc()
55
- return False
56
-
57
- def main():
58
- """Main function."""
59
-
60
- api_key = input("> ").strip()
61
-
62
- if not api_key:
63
- return
64
-
65
- create_env_file(api_key)
66
-
67
- time.sleep(1)
68
-
69
- # Test Supabase connection
70
- success = test_supabase()
71
-
72
- if success:
73
- else:
74
-
75
-
76
- if __name__ == "__main__":
77
- main()
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
debug_test.py DELETED
@@ -1,113 +0,0 @@
1
- """
2
- Simple debug script to identify component issues
3
- """
4
- import os
5
- import sys
6
- import logging
7
- import traceback
8
-
9
- # Set up logging
10
- logging.basicConfig(
11
- level=logging.DEBUG,
12
- format="%(asctime)s - %(name)s - %(levelname)s - %(message)s"
13
- )
14
- logger = logging.getLogger("debug_test")
15
-
16
- def check_environment():
17
- """Check environment variables"""
18
- logger.info("Checking environment variables...")
19
- vars_to_check = ["OPENAI_API_KEY", "SUPABASE_URL", "SUPABASE_KEY", "SERPER_API_KEY"]
20
- for var in vars_to_check:
21
- value = os.environ.get(var)
22
- if value:
23
- logger.info(f"{var}: ✓ (Set)")
24
- else:
25
- logger.warning(f"{var}: ✗ (Not set)")
26
-
27
- def test_imports():
28
- """Test importing all required modules"""
29
- logger.info("Testing imports...")
30
-
31
- import_tests = [
32
- ("OpenAI", "from openai import OpenAI; client = OpenAI()"),
33
- ("Supabase", "from supabase import create_client"),
34
- ("Memory modules", "from memory.supabase_memory import SupabaseMemory, WorkingMemory, ConversationMemory, ResultCache"),
35
- ("Memory fix", "from memory.fix_supabase_tables import apply_monkey_patch, ensure_tables_exist"),
36
- ("Agent config", "from agent.config import get_memory_config"),
37
- ("Agent module", "from agent.agent import GAIAAgent"),
38
- ("Agent graph", "from agent.graph import create_agent_graph, run_agent_graph"),
39
- ("DuckDuckGo", "from tools.web_tools import duckduckgo_search"),
40
- ("Serper", "from tools.web_tools import serper_search")
41
- ]
42
-
43
- for name, import_statement in import_tests:
44
- logger.info(f"Testing import: {name}")
45
- try:
46
- exec(import_statement)
47
- logger.info(f"✓ Successfully imported {name}")
48
- except Exception as e:
49
- logger.error(f"✗ Failed to import {name}: {str(e)}")
50
- logger.debug(traceback.format_exc())
51
-
52
- def check_supabase_connection():
53
- """Test Supabase connection"""
54
- logger.info("Testing Supabase connection...")
55
- try:
56
- from supabase import create_client
57
- import os
58
-
59
- supabase_url = os.environ.get("SUPABASE_URL")
60
- supabase_key = os.environ.get("SUPABASE_KEY")
61
-
62
- if not supabase_url or not supabase_key:
63
- logger.error("Supabase credentials not found in environment variables")
64
- return
65
-
66
- client = create_client(supabase_url, supabase_key)
67
- response = client.table("agents").select("id").limit(1).execute()
68
-
69
- logger.info("Supabase connection successful")
70
- except Exception as e:
71
- logger.error(f"Supabase connection error: {str(e)}")
72
- logger.debug(traceback.format_exc())
73
-
74
- def check_openai_connection():
75
- """Test OpenAI connection"""
76
- logger.info("Testing OpenAI connection...")
77
- try:
78
- from openai import OpenAI
79
-
80
- if not os.environ.get("OPENAI_API_KEY"):
81
- logger.error("OpenAI API key not found in environment variables")
82
- return
83
-
84
- client = OpenAI()
85
- response = client.chat.completions.create(
86
- model="gpt-3.5-turbo",
87
- messages=[{"role": "user", "content": "Hello"}],
88
- max_tokens=5
89
- )
90
-
91
- logger.info("OpenAI connection successful")
92
- except Exception as e:
93
- logger.error(f"OpenAI connection error: {str(e)}")
94
- logger.debug(traceback.format_exc())
95
-
96
- def main():
97
- """Run all debug tests"""
98
- logger.info("=== STARTING DEBUG TESTS ===")
99
-
100
- # Check environment variables
101
- check_environment()
102
-
103
- # Test imports
104
- test_imports()
105
-
106
- # Test API connections
107
- check_supabase_connection()
108
- check_openai_connection()
109
-
110
- logger.info("=== DEBUG TESTS COMPLETED ===")
111
-
112
- if __name__ == "__main__":
113
- main()
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
dependency_conflict_fix_guide.md DELETED
@@ -1,84 +0,0 @@
1
- # Fixing Dependency Conflict in GAIA Agent Deployment
2
-
3
- ## The Specific Error
4
-
5
- The build is failing with a dependency conflict between `langchain-openai` and `openai`:
6
-
7
- ```
8
- ERROR: Cannot install -r /tmp/requirements.txt (line 5) and openai==1.3.7 because these package versions have conflicting dependencies.
9
-
10
- The conflict is caused by:
11
- The user requested openai==1.3.7
12
- langchain-openai 0.0.2.post1 depends on openai<2.0.0 and >=1.6.1
13
- ```
14
-
15
- ## Solution
16
-
17
- Update the `requirements.txt` file to use a compatible version of OpenAI that meets the requirements of `langchain-openai`.
18
-
19
- ## Steps to Fix the Dependency Conflict
20
-
21
- 1. Navigate to your Space on Hugging Face
22
- 2. Click on the "Files" tab
23
- 3. Open `requirements.txt`
24
- 4. Click "Edit" (pencil icon)
25
- 5. Replace the contents with the following fixed requirements:
26
-
27
- ```
28
- # Fixed requirements file with compatible versions
29
- gradio==3.50.2
30
- requests==2.31.0
31
- langchain==0.0.292
32
- # Fix for the dependency conflict - OpenAI version must be >=1.6.1 for langchain-openai
33
- openai==1.6.1
34
- langchain_openai==0.0.2.post1
35
- langgraph==0.0.19
36
- supabase==1.2.0
37
- duckduckgo-search==3.9.5
38
- beautifulsoup4==4.12.2
39
- pillow==10.0.1
40
- arxiv==1.4.8
41
- python-dotenv==1.0.0
42
-
43
- # Additional dependencies
44
- pydantic==1.10.13
45
- httpx==0.24.1
46
- numpy==1.24.4
47
- pandas==2.0.3
48
- ```
49
-
50
- 6. Click "Commit changes" at the bottom of the page
51
- 7. Enter a commit message like "Fix dependency conflict between langchain-openai and openai"
52
- 8. Click "Commit" to save the changes
53
- 9. Restart your Space (Settings tab → Factory reboot → Restart Space)
54
-
55
- ## Why This Works
56
-
57
- The error occurs because:
58
- - We were trying to install `openai==1.3.7`
59
- - But `langchain-openai 0.0.2.post1` requires `openai>=1.6.1`
60
-
61
- By updating to `openai==1.6.1`, we satisfy the version constraint imposed by langchain-openai while maintaining compatibility with other packages.
62
-
63
- ## Alternative Approaches
64
-
65
- If this solution doesn't work, you can try:
66
-
67
- 1. **Option 1**: Use a newer version of both packages
68
- ```
69
- openai==1.7.2
70
- langchain_openai==0.0.3
71
- ```
72
-
73
- 2. **Option 2**: Pin langchain-openai to an older version that works with openai 1.3.7
74
- (Note: This might not be available as langchain-openai seems to require >=1.6.1)
75
-
76
- 3. **Option 3**: Temporarily remove langchain-openai from requirements and install it in a separate step with compatible dependencies
77
-
78
- ## Testing After Fix
79
-
80
- After updating requirements.txt and restarting your Space:
81
-
82
- 1. Monitor the build logs to confirm successful installation without dependency errors
83
- 2. Verify the app loads correctly in the browser
84
- 3. Test the OpenAI integration to ensure it's working with the updated version
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
deploy_to_hf.py DELETED
@@ -1,392 +0,0 @@
1
- #!/usr/bin/env python
2
- """
3
- Automated deployment script for GAIA Agent to Hugging Face Spaces
4
-
5
- This script automates the deployment of the GAIA Agent to a Hugging Face Space:
6
- 1. Verifies the local project structure
7
- 2. Creates or updates the Hugging Face Space
8
- 3. Sets up environment variables
9
- 4. Adds the code to git and pushes it
10
-
11
- Usage:
12
- python deploy_to_hf.py --username YOUR_USERNAME --space YOUR_SPACE_NAME
13
- """
14
-
15
- import os
16
- import sys
17
- import argparse
18
- import subprocess
19
- import json
20
- import logging
21
- import shutil
22
- import tempfile
23
- from pathlib import Path
24
- from typing import Dict, Any, List, Optional, Union
25
-
26
- # Configure logging
27
- logging.basicConfig(
28
- level=logging.INFO,
29
- format="%(asctime)s - %(name)s - %(levelname)s - %(message)s"
30
- )
31
- logger = logging.getLogger("deploy_to_hf")
32
-
33
- def parse_arguments():
34
- """Parse command line arguments."""
35
- parser = argparse.ArgumentParser(description="Deploy GAIA Agent to Hugging Face Spaces")
36
-
37
- parser.add_argument("--username", required=True, help="Your Hugging Face username")
38
- parser.add_argument("--space", required=True, help="Name for your Hugging Face Space")
39
- parser.add_argument("--hardware", default="cpu-basic", choices=["cpu-basic", "cpu-upgrade", "t4-small", "t4-medium", "a10g-small", "a10g-large"], help="Space hardware tier")
40
- parser.add_argument("--private", action="store_true", help="Make the Space private")
41
- parser.add_argument("--debug", action="store_true", help="Enable debug logging")
42
- parser.add_argument("--sdk", default="gradio", choices=["gradio", "streamlit", "docker"], help="Space SDK")
43
- parser.add_argument("--skip-verification", action="store_true", help="Skip local verification")
44
-
45
- return parser.parse_args()
46
-
47
- def check_prerequisites():
48
- """Check if all prerequisites are met."""
49
- logger.info("Checking prerequisites...")
50
-
51
- # Check HF CLI
52
- try:
53
- result = subprocess.run(["huggingface-cli", "--version"], capture_output=True, text=True)
54
- logger.info(f"Hugging Face CLI found: {result.stdout.strip()}")
55
- except FileNotFoundError:
56
- logger.error("Hugging Face CLI not found. Please install with 'pip install huggingface_hub'")
57
- return False
58
-
59
- # Check Git
60
- try:
61
- result = subprocess.run(["git", "--version"], capture_output=True, text=True)
62
- logger.info(f"Git found: {result.stdout.strip()}")
63
- except FileNotFoundError:
64
- logger.error("Git not found. Please install Git and try again.")
65
- return False
66
-
67
- # Check HF Token
68
- hf_token = os.getenv("HF_TOKEN")
69
- if not hf_token:
70
- logger.warning("HF_TOKEN environment variable not found. You may need to log in manually.")
71
-
72
- return True
73
-
74
- def verify_local_project():
75
- """Verify the local project structure."""
76
- logger.info("Verifying local project structure...")
77
-
78
- # Required files
79
- required_files = [
80
- "app.py",
81
- "requirements.txt",
82
- "README.md",
83
- "agent/agent.py",
84
- "agent/config.py",
85
- "agent/graph.py"
86
- ]
87
-
88
- missing_files = []
89
- for file_path in required_files:
90
- if not os.path.isfile(file_path):
91
- missing_files.append(file_path)
92
- logger.error(f"Missing required file: {file_path}")
93
-
94
- if missing_files:
95
- logger.error(f"Missing {len(missing_files)} required files. Please fix before deployment.")
96
- return False
97
-
98
- # Check Gradio dependencies in requirements.txt
99
- with open("requirements.txt", "r") as f:
100
- requirements = f.read()
101
- if "gradio" not in requirements:
102
- logger.error("Gradio not found in requirements.txt")
103
- return False
104
-
105
- # Run basic code tests
106
- try:
107
- import importlib.util
108
-
109
- # Try to import the main modules to check for syntax errors
110
- modules_to_check = [
111
- ("agent.agent", "agent/agent.py"),
112
- ("agent.config", "agent/config.py"),
113
- ("agent.graph", "agent/graph.py"),
114
- ]
115
-
116
- for module_name, module_path in modules_to_check:
117
- spec = importlib.util.spec_from_file_location(module_name, module_path)
118
- module = importlib.util.module_from_spec(spec)
119
- spec.loader.exec_module(module)
120
- logger.info(f"Successfully imported {module_name}")
121
-
122
- except Exception as e:
123
- logger.error(f"Error importing project modules: {str(e)}")
124
- return False
125
-
126
- logger.info("Local project structure verification passed")
127
- return True
128
-
129
- def create_or_update_space(args):
130
- """Create or update the Hugging Face Space."""
131
- logger.info(f"Creating/updating Space: {args.username}/{args.space}")
132
-
133
- space_url = f"https://huggingface.co/spaces/{args.username}/{args.space}"
134
-
135
- # Check if Space exists
136
- try:
137
- result = subprocess.run(
138
- ["curl", "-s", "-o", "/dev/null", "-w", "%{http_code}", space_url],
139
- capture_output=True,
140
- text=True
141
- )
142
- space_exists = result.stdout.strip() == "200"
143
- except Exception:
144
- space_exists = False
145
-
146
- if space_exists:
147
- logger.info(f"Space already exists: {space_url}")
148
- else:
149
- logger.info(f"Creating new Space: {args.username}/{args.space}")
150
-
151
- # Create the Space
152
- cmd = [
153
- "huggingface-cli", "repo", "create",
154
- f"{args.username}/{args.space}",
155
- "--type", "space",
156
- "--sdk", args.sdk
157
- ]
158
-
159
- if args.private:
160
- cmd.append("--private")
161
- else:
162
- cmd.append("--public")
163
-
164
- try:
165
- subprocess.run(cmd, check=True)
166
- logger.info(f"Space created successfully: {space_url}")
167
- except subprocess.CalledProcessError as e:
168
- logger.error(f"Error creating Space: {str(e)}")
169
- return False
170
-
171
- # Update Space hardware if specified
172
- try:
173
- hardware_cmd = [
174
- "huggingface-cli", "repo", "update-space",
175
- f"{args.username}/{args.space}",
176
- "--hardware", args.hardware
177
- ]
178
- subprocess.run(hardware_cmd, check=True)
179
- logger.info(f"Space hardware set to: {args.hardware}")
180
- except subprocess.CalledProcessError as e:
181
- logger.error(f"Error updating Space hardware: {str(e)}")
182
-
183
- return True
184
-
185
- def setup_environment_variables(args):
186
- """Set up environment variables for the Hugging Face Space."""
187
- logger.info("Setting up environment variables...")
188
-
189
- # Get environment variables from current environment
190
- env_vars = {
191
- "OPENAI_API_KEY": os.getenv("OPENAI_API_KEY", ""),
192
- "SUPABASE_URL": os.getenv("SUPABASE_URL", ""),
193
- "SUPABASE_KEY": os.getenv("SUPABASE_KEY", ""),
194
- "SERPER_API_KEY": os.getenv("SERPER_API_KEY", ""),
195
- "HF_TOKEN": os.getenv("HF_TOKEN", ""),
196
- "SPACE_ID": f"{args.username}/{args.space}"
197
- }
198
-
199
- # Ask for missing variables
200
- for var_name, var_value in list(env_vars.items()):
201
- if not var_value:
202
- print(f"\n{var_name} not found in environment.")
203
-
204
- if var_name == "SPACE_ID":
205
- # This one can be auto-generated
206
- env_vars[var_name] = f"{args.username}/{args.space}"
207
- continue
208
-
209
- user_input = input(f"Enter {var_name} (leave blank to skip): ")
210
- if user_input:
211
- env_vars[var_name] = user_input
212
-
213
- # Generate commands to set variables in Hugging Face Space
214
- env_commands = []
215
- for var_name, var_value in env_vars.items():
216
- if var_value:
217
- # Mask the value for logging
218
- masked_value = var_value[:4] + "*" * (len(var_value) - 8) + var_value[-4:] if len(var_value) > 8 else "****"
219
- logger.info(f"Adding environment variable: {var_name}={masked_value}")
220
-
221
- cmd = [
222
- "huggingface-cli", "secret", "set",
223
- f"{args.username}/{args.space}",
224
- var_name, var_value
225
- ]
226
- env_commands.append(cmd)
227
-
228
- # Execute the commands
229
- success = True
230
- for cmd in env_commands:
231
- try:
232
- # Don't capture output to avoid showing the secrets in logs
233
- subprocess.run(cmd, check=True, capture_output=True)
234
- except subprocess.CalledProcessError as e:
235
- logger.error(f"Error setting environment variable: {str(e)}")
236
- success = False
237
-
238
- if success:
239
- logger.info("Environment variables set successfully")
240
-
241
- return success
242
-
243
- def deploy_code_to_space(args):
244
- """Deploy the code to the Hugging Face Space."""
245
- logger.info("Deploying code to Hugging Face Space...")
246
-
247
- # Create a temporary directory for deployment
248
- with tempfile.TemporaryDirectory() as temp_dir:
249
- # Clone the Space repository
250
- space_repo = f"https://huggingface.co/spaces/{args.username}/{args.space}"
251
- logger.info(f"Cloning Space repository: {space_repo}")
252
-
253
- try:
254
- subprocess.run(
255
- ["git", "clone", space_repo, temp_dir],
256
- check=True,
257
- capture_output=True,
258
- text=True
259
- )
260
- except subprocess.CalledProcessError as e:
261
- logger.error(f"Error cloning Space repository: {e.stderr}")
262
- return False
263
-
264
- # Copy project files to the cloned repository
265
- logger.info("Copying project files to the cloned repository")
266
-
267
- # Define files/directories to copy (excluding .git and other unnecessary files)
268
- items_to_copy = [
269
- "agent",
270
- "docs",
271
- "memory",
272
- "tests",
273
- "tools",
274
- "utils",
275
- "app.py",
276
- "requirements.txt",
277
- "README.md",
278
- "verify_gaia_assessment.py",
279
- "test_deployment_readiness.py"
280
- ]
281
-
282
- for item in items_to_copy:
283
- if os.path.exists(item):
284
- src_path = os.path.abspath(item)
285
- dst_path = os.path.join(temp_dir, os.path.basename(item))
286
-
287
- try:
288
- if os.path.isdir(src_path):
289
- if os.path.exists(dst_path):
290
- shutil.rmtree(dst_path)
291
- shutil.copytree(src_path, dst_path)
292
- else:
293
- shutil.copy2(src_path, dst_path)
294
- logger.info(f"Copied {item}")
295
- except Exception as e:
296
- logger.error(f"Error copying {item}: {str(e)}")
297
-
298
- # Create a .gitignore file if it doesn't exist
299
- gitignore_path = os.path.join(temp_dir, ".gitignore")
300
- if not os.path.exists(gitignore_path):
301
- with open(gitignore_path, "w") as f:
302
- f.write("__pycache__/\n*.py[cod]\n*$py.class\n.env\n.venv\nenv/\nvenv/\n*.log\n")
303
- logger.info("Created .gitignore file")
304
-
305
- # Commit and push the changes
306
- logger.info("Committing and pushing changes")
307
- try:
308
- # Configure git (necessary for CI/CD)
309
- subprocess.run(
310
- ["git", "config", "--global", "user.email", "huggingface@example.com"],
311
- cwd=temp_dir,
312
- check=True,
313
- capture_output=True
314
- )
315
- subprocess.run(
316
- ["git", "config", "--global", "user.name", "GAIA Deployment Script"],
317
- cwd=temp_dir,
318
- check=True,
319
- capture_output=True
320
- )
321
-
322
- # Add all files
323
- subprocess.run(
324
- ["git", "add", "."],
325
- cwd=temp_dir,
326
- check=True,
327
- capture_output=True
328
- )
329
-
330
- # Commit
331
- subprocess.run(
332
- ["git", "commit", "-m", "Deploy GAIA Agent"],
333
- cwd=temp_dir,
334
- check=True,
335
- capture_output=True
336
- )
337
-
338
- # Push
339
- subprocess.run(
340
- ["git", "push"],
341
- cwd=temp_dir,
342
- check=True,
343
- capture_output=True
344
- )
345
-
346
- logger.info("Code deployed successfully")
347
- return True
348
- except subprocess.CalledProcessError as e:
349
- logger.error(f"Error deploying code: {e.stderr if hasattr(e, 'stderr') else str(e)}")
350
- return False
351
-
352
- def main():
353
- """Main entry point for the deployment script."""
354
- args = parse_arguments()
355
-
356
- if args.debug:
357
- logger.setLevel(logging.DEBUG)
358
-
359
- logger.info("===== GAIA Agent Deployment to Hugging Face =====")
360
-
361
- if not check_prerequisites():
362
- logger.error("Prerequisites check failed. Please fix issues and try again.")
363
- return 1
364
-
365
- if not args.skip_verification and not verify_local_project():
366
- logger.error("Local project verification failed. Please fix issues and try again.")
367
- return 1
368
-
369
- if not create_or_update_space(args):
370
- logger.error("Failed to create or update Hugging Face Space.")
371
- return 1
372
-
373
- if not setup_environment_variables(args):
374
- logger.warning("Some environment variables could not be set. Check manually.")
375
-
376
- if not deploy_code_to_space(args):
377
- logger.error("Failed to deploy code to Hugging Face Space.")
378
- return 1
379
-
380
- space_url = f"https://huggingface.co/spaces/{args.username}/{args.space}"
381
- logger.info(f"\n🚀 Deployment successful! Your GAIA Agent is available at:\n{space_url}\n")
382
- logger.info("Note: It may take a few minutes for the Space to build and start.")
383
-
384
- logger.info("===== Post-Deployment Steps =====")
385
- logger.info("1. Check Space logs in the Factory tab if there are issues")
386
- logger.info("2. Test the agent with a simple query")
387
- logger.info("3. Monitor performance and memory usage")
388
-
389
- return 0
390
-
391
- if __name__ == "__main__":
392
- sys.exit(main())
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
deploy_to_huggingface.py DELETED
@@ -1,333 +0,0 @@
1
- #!/usr/bin/env python
2
- """
3
- Hugging Face Space Deployment Script
4
-
5
- This script deploys the GAIA project to Hugging Face Spaces using git.
6
- It handles authentication, remote setup, and deployment.
7
-
8
- Usage:
9
- python deploy_to_huggingface.py [--space-id SPACE_ID]
10
-
11
- Author: DevOps Team
12
- Date: April 30, 2025
13
- """
14
-
15
- import os
16
- import sys
17
- import argparse
18
- import subprocess
19
- import logging
20
- import json
21
- import time
22
- from pathlib import Path
23
-
24
- # Configure logging
25
- logging.basicConfig(
26
- level=logging.INFO,
27
- format="%(asctime)s - %(levelname)s - %(message)s"
28
- )
29
- logger = logging.getLogger("hf_deploy")
30
-
31
- def parse_arguments():
32
- """Parse command line arguments"""
33
- parser = argparse.ArgumentParser(description="Deploy GAIA to Hugging Face Space")
34
- parser.add_argument("--space-id",
35
- help="Hugging Face Space ID (username/space-name)",
36
- default="JoachimVC/GAIA-Assistant")
37
- parser.add_argument("--check-only", action="store_true",
38
- help="Only check prerequisites without deploying")
39
- return parser.parse_args()
40
-
41
- def run_command(command, shell=False):
42
- """Run a shell command and return the result"""
43
- try:
44
- logger.info(f"Running command: {command}")
45
- result = subprocess.run(
46
- command,
47
- shell=shell,
48
- check=True,
49
- text=True,
50
- capture_output=True
51
- )
52
- return result.stdout.strip()
53
- except subprocess.CalledProcessError as e:
54
- logger.error(f"Command failed with error: {e}")
55
- logger.error(f"Error output: {e.stderr}")
56
- return None
57
-
58
- def check_prerequisites():
59
- """Check if all prerequisites are met"""
60
- # Check if git is installed
61
- git_version = run_command(["git", "--version"])
62
- if not git_version:
63
- logger.error("Git is not installed or not in PATH")
64
- return False
65
- logger.info(f"Git version: {git_version}")
66
-
67
- # Check if huggingface_hub is installed
68
- try:
69
- import huggingface_hub
70
- logger.info(f"huggingface_hub version: {huggingface_hub.__version__}")
71
- except ImportError:
72
- logger.error("huggingface_hub is not installed")
73
- logger.info("Installing huggingface_hub...")
74
- if run_command(["pip", "install", "huggingface_hub"]) is None:
75
- return False
76
-
77
- # Check if user is logged in to Hugging Face
78
- hf_token_exists = os.path.exists(os.path.expanduser("~/.huggingface/token"))
79
- if not hf_token_exists:
80
- logger.error("Not logged in to Hugging Face")
81
- logger.info("Please run 'huggingface-cli login' to log in")
82
- return False
83
-
84
- # Check if the project has necessary files
85
- required_files = ["app.py", "requirements.txt", "space_config.json", "HF_README.md"]
86
- missing_files = [f for f in required_files if not os.path.exists(f)]
87
- if missing_files:
88
- logger.error(f"Missing required files: {', '.join(missing_files)}")
89
- return False
90
-
91
- logger.info("All prerequisites met")
92
- return True
93
-
94
- def setup_git_repo(space_id):
95
- """Set up git repository for deployment"""
96
- # Check if .git exists
97
- if not os.path.exists(".git"):
98
- logger.info("Initializing git repository...")
99
- if run_command(["git", "init"]) is None:
100
- return False
101
-
102
- # Check if the Hugging Face remote exists
103
- remotes = run_command(["git", "remote"])
104
- if remotes is None:
105
- return False
106
-
107
- if "huggingface" in remotes.split():
108
- logger.info("Updating Hugging Face remote...")
109
- if run_command(["git", "remote", "set-url", "huggingface", f"https://huggingface.co/spaces/{space_id}"]) is None:
110
- return False
111
- else:
112
- logger.info("Adding Hugging Face remote...")
113
- if run_command(["git", "remote", "add", "huggingface", f"https://huggingface.co/spaces/{space_id}"]) is None:
114
- return False
115
-
116
- logger.info("Git repository set up successfully")
117
- return True
118
-
119
- def validate_space_id(space_id):
120
- """Validate the Hugging Face Space ID format"""
121
- if "/" not in space_id:
122
- logger.error(f"Invalid Space ID format: {space_id}")
123
- logger.error("Space ID should be in format 'username/space-name'")
124
- return False
125
-
126
- username, space_name = space_id.split("/", 1)
127
- if not username or not space_name:
128
- logger.error(f"Invalid Space ID components: {space_id}")
129
- return False
130
-
131
- return True
132
-
133
- def rename_files_for_hf():
134
- """Rename files for Hugging Face Space"""
135
- if os.path.exists("HF_README.md"):
136
- if os.path.exists("README.md"):
137
- if not os.path.exists("README_original.md"):
138
- logger.info("Backing up README.md to README_original.md...")
139
- os.rename("README.md", "README_original.md")
140
- else:
141
- logger.info("README_original.md already exists, keeping it")
142
- # Remove README.md to avoid confusion
143
- if os.path.exists("README.md"):
144
- os.remove("README.md")
145
-
146
- logger.info("Renaming HF_README.md to README.md...")
147
- os.rename("HF_README.md", "README.md")
148
-
149
- return True
150
-
151
- def update_git_ignore():
152
- """Update .gitignore file for Hugging Face Space"""
153
- gitignore_entries = [
154
- ".env",
155
- "__pycache__/",
156
- "*.py[cod]",
157
- "*$py.class",
158
- "*.so",
159
- ".Python",
160
- "env/",
161
- "build/",
162
- "develop-eggs/",
163
- "dist/",
164
- "downloads/",
165
- "eggs/",
166
- ".eggs/",
167
- "lib/",
168
- "lib64/",
169
- "parts/",
170
- "sdist/",
171
- "var/",
172
- "*.egg-info/",
173
- ".installed.cfg",
174
- "*.egg",
175
- ".env",
176
- ".venv",
177
- "venv/",
178
- "ENV/",
179
- ".idea/",
180
- ".vscode/",
181
- "*.log",
182
- "deploy_to_huggingface.py",
183
- "test_components.py",
184
- "verify_production.py",
185
- "production_verification_report.md",
186
- "deployment_readiness_report.md",
187
- "final_verification_summary.md",
188
- "prepare_for_hf_deployment.py"
189
- ]
190
-
191
- # Read existing .gitignore if it exists
192
- existing_entries = []
193
- if os.path.exists(".gitignore"):
194
- with open(".gitignore", "r") as f:
195
- existing_entries = [line.strip() for line in f.readlines()]
196
-
197
- # Combine entries, removing duplicates
198
- combined_entries = list(set(existing_entries + gitignore_entries))
199
- combined_entries.sort()
200
-
201
- # Write updated .gitignore
202
- with open(".gitignore", "w") as f:
203
- f.write("\n".join(combined_entries) + "\n")
204
-
205
- logger.info("Updated .gitignore file")
206
- return True
207
-
208
- def commit_and_push(space_id):
209
- """Commit changes and push to Hugging Face Space"""
210
- # Stage all files
211
- if run_command(["git", "add", "."]) is None:
212
- return False
213
-
214
- # Commit changes
215
- commit_message = f"Deploy GAIA to Hugging Face Space {space_id}"
216
- if run_command(["git", "commit", "-m", commit_message]) is None:
217
- logger.warning("No changes to commit or commit failed")
218
- # Continue anyway, as there might be no changes
219
-
220
- # Push to Hugging Face
221
- logger.info(f"Pushing to Hugging Face Space {space_id}...")
222
- if run_command(["git", "push", "huggingface", "main"]) is None:
223
- # Try pushing to master branch if main fails
224
- logger.info("Pushing to master branch instead...")
225
- if run_command(["git", "push", "huggingface", "master"]) is None:
226
- # Try force pushing to main as last resort
227
- logger.info("Force pushing to main branch...")
228
- if run_command(["git", "push", "-f", "huggingface", "main"]) is None:
229
- return False
230
-
231
- logger.info(f"Successfully pushed to Hugging Face Space {space_id}")
232
- return True
233
-
234
- def create_space_if_needed(space_id):
235
- """Create Hugging Face Space if it doesn't exist"""
236
- try:
237
- from huggingface_hub import HfApi, SpaceHardware, SpaceSdk
238
-
239
- api = HfApi()
240
- username, space_name = space_id.split("/", 1)
241
-
242
- # Check if space exists
243
- try:
244
- api.get_space_info(space_id)
245
- logger.info(f"Space {space_id} already exists")
246
- return True
247
- except Exception:
248
- logger.info(f"Space {space_id} doesn't exist, creating it...")
249
-
250
- # Create space
251
- api.create_space(
252
- space_id=space_name,
253
- token=None, # Use token from ~/.huggingface/token
254
- sdk=SpaceSdk.GRADIO,
255
- hardware=SpaceHardware.CPU_BASIC,
256
- private=False
257
- )
258
-
259
- logger.info(f"Created Hugging Face Space {space_id}")
260
- return True
261
-
262
- except ImportError:
263
- logger.warning("huggingface_hub not installed properly, skipping space creation check")
264
- return True
265
- except Exception as e:
266
- logger.error(f"Error creating space: {e}")
267
- return False
268
-
269
- def wait_for_deployment(space_id):
270
- """Wait for deployment to complete and provide status link"""
271
- logger.info(f"Deployment initiated to {space_id}")
272
- logger.info(f"You can check the deployment status at: https://huggingface.co/spaces/{space_id}")
273
- logger.info(f"Once deployment is complete, your app will be available at: https://huggingface.co/spaces/{space_id}")
274
-
275
- logger.info("Waiting for 60 seconds to allow deployment to start...")
276
- for i in range(12):
277
- time.sleep(5)
278
- logger.info(f"Elapsed time: {(i+1)*5} seconds")
279
-
280
- return True
281
-
282
- def main():
283
- """Main function to deploy to Hugging Face Space"""
284
- args = parse_arguments()
285
- space_id = args.space_id
286
-
287
- logger.info(f"Starting deployment to Hugging Face Space: {space_id}")
288
-
289
- # Validate Space ID
290
- if not validate_space_id(space_id):
291
- return False
292
-
293
- # Check prerequisites
294
- if not check_prerequisites():
295
- return False
296
-
297
- if args.check_only:
298
- logger.info("Check-only mode: prerequisites satisfied, stopping before deployment")
299
- return True
300
-
301
- # Rename files for Hugging Face
302
- if not rename_files_for_hf():
303
- return False
304
-
305
- # Update .gitignore
306
- if not update_git_ignore():
307
- return False
308
-
309
- # Create Space if needed
310
- if not create_space_if_needed(space_id):
311
- logger.warning("Failed to create space, but continuing with deployment")
312
-
313
- # Set up git repository
314
- if not setup_git_repo(space_id):
315
- return False
316
-
317
- # Commit and push changes
318
- if not commit_and_push(space_id):
319
- return False
320
-
321
- # Wait for deployment and show status
322
- if not wait_for_deployment(space_id):
323
- return False
324
-
325
- logger.info("Deployment process completed successfully!")
326
- logger.info(f"Your GAIA application will be available at: https://huggingface.co/spaces/{space_id}")
327
- logger.info("Note: It may take a few minutes for the deployment to complete on Hugging Face")
328
-
329
- return True
330
-
331
- if __name__ == "__main__":
332
- success = main()
333
- sys.exit(0 if success else 1)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
deploy_to_huggingface_credential.py DELETED
@@ -1,320 +0,0 @@
1
- """
2
- Git Credential-based Deployment Script for Hugging Face Spaces
3
-
4
- This script handles the git operations required to deploy to a Hugging Face Space
5
- using credential helper for token-based authentication.
6
-
7
- Usage:
8
- python deploy_to_huggingface_credential.py [--space SPACE_ID]
9
-
10
- Environment variables (from .env file):
11
- HF_TOKEN - Your Hugging Face API token
12
- """
13
- import os
14
- import subprocess
15
- import sys
16
- import argparse
17
- from pathlib import Path
18
-
19
- def parse_args():
20
- parser = argparse.ArgumentParser(description="Deploy to Hugging Face Space using git credential helper")
21
- parser.add_argument("--space", help="Space ID (username/space-name)",
22
- default="JoachimVC/Final_Assignment_Template")
23
- return parser.parse_args()
24
-
25
- def load_dotenv():
26
- """Load environment variables from .env file"""
27
- env_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), '.env')
28
- if not os.path.exists(env_path):
29
- print(f"ERROR: .env file not found at {env_path}")
30
- return False
31
-
32
- print(f"Loading environment variables from {env_path}")
33
- with open(env_path, 'r') as file:
34
- for line in file:
35
- line = line.strip()
36
- if not line or line.startswith('#'):
37
- continue
38
-
39
- key, value = line.split('=', 1)
40
- os.environ[key] = value
41
-
42
- return True
43
-
44
- def run_command(cmd, silent=False):
45
- """Run a shell command and return output"""
46
- if not silent:
47
- print(f"Running: {cmd}")
48
- try:
49
- process = subprocess.Popen(
50
- cmd,
51
- shell=True,
52
- stdout=subprocess.PIPE,
53
- stderr=subprocess.PIPE,
54
- text=True
55
- )
56
- stdout, stderr = process.communicate()
57
-
58
- if process.returncode != 0:
59
- print(f"Command failed with error code {process.returncode}")
60
- print(f"Error: {stderr.strip()}")
61
- return False
62
-
63
- if not silent and stdout.strip():
64
- print(stdout.strip())
65
-
66
- return True
67
- except Exception as e:
68
- print(f"Failed to execute command: {str(e)}")
69
- return False
70
-
71
- def check_git_installed():
72
- """Check if git is installed"""
73
- print("Checking if git is installed...")
74
- return run_command("git --version", silent=True)
75
-
76
- def get_token_from_env():
77
- """Get Hugging Face token from environment variables"""
78
- # Try various environment variable names that might contain the token
79
- token_vars = ["HF_TOKEN", "HUGGINGFACE_TOKEN", "HUGGING_FACE_TOKEN",
80
- "HF_API_TOKEN", "HUGGINGFACE_API_TOKEN"]
81
-
82
- for var_name in token_vars:
83
- token = os.environ.get(var_name)
84
- if token:
85
- print(f"Found token in environment variable: {var_name}")
86
- return token
87
-
88
- return None
89
-
90
- def create_deploy_token_file(token):
91
- """Create a temporary token file for deployment"""
92
- token_dir = Path.home() / ".huggingface"
93
- token_dir.mkdir(exist_ok=True)
94
-
95
- token_file = token_dir / "token"
96
- with open(token_file, "w") as f:
97
- f.write(token)
98
-
99
- print(f"Created HF token file at {token_file}")
100
- return token_file
101
-
102
- def setup_git_credential_helper(token):
103
- """Set up git credential helper to use the token"""
104
- # Store the token in the git credential helper
105
- credential_input = f"protocol=https\nhost=huggingface.co\nusername=hf_token\npassword={token}\n"
106
-
107
- # Use git credential-store to save the token
108
- print("Setting up git credential helper...")
109
- process = subprocess.Popen(
110
- ["git", "credential-store", "store"],
111
- stdin=subprocess.PIPE,
112
- stdout=subprocess.PIPE,
113
- stderr=subprocess.PIPE,
114
- text=True
115
- )
116
-
117
- process.communicate(input=credential_input)
118
-
119
- # Configure git to use the credential helper
120
- return run_command("git config --global credential.helper store")
121
-
122
- def setup_git(space_id):
123
- """Initialize git if needed and configure remote"""
124
- # Check if git is already initialized
125
- if not os.path.exists(".git"):
126
- print("Initializing git repository...")
127
- if not run_command("git init"):
128
- return False
129
-
130
- # Configure git user if not set
131
- result = subprocess.run("git config user.name", shell=True, capture_output=True, text=True)
132
- if not result.stdout.strip():
133
- print("Setting up git user name...")
134
- run_command('git config user.name "GAIA Deployment"')
135
-
136
- result = subprocess.run("git config user.email", shell=True, capture_output=True, text=True)
137
- if not result.stdout.strip():
138
- print("Setting up git user email...")
139
- run_command('git config user.email "gaia-deployment@example.com"')
140
-
141
- # Set up the Hugging Face remote without token in URL
142
- print("Setting up git remote for Hugging Face...")
143
-
144
- # Check if remote exists
145
- result = subprocess.run("git remote -v", shell=True, capture_output=True, text=True)
146
- remote_output = result.stdout
147
-
148
- if "huggingface" in remote_output:
149
- run_command("git remote remove huggingface")
150
-
151
- # Add the HF remote without token (will use credential helper)
152
- repo_url = f"https://huggingface.co/spaces/{space_id}"
153
- return run_command(f"git remote add huggingface {repo_url}")
154
-
155
- def create_gitignore():
156
- """Create .gitignore file with common patterns"""
157
- patterns = [
158
- ".env",
159
- "__pycache__/",
160
- "*.py[cod]",
161
- "*$py.class",
162
- ".Python",
163
- "env/",
164
- "venv/",
165
- ".venv",
166
- ".coverage",
167
- "*.log",
168
- "deploy_with_token.py",
169
- "deploy_to_huggingface_env.py",
170
- "deploy_to_huggingface_dotenv.py",
171
- "deploy_to_huggingface_credential.py" # Don't include this file with token handling
172
- ]
173
-
174
- print("Creating .gitignore file...")
175
- with open(".gitignore", "w") as f:
176
- for pattern in patterns:
177
- f.write(f"{pattern}\n")
178
-
179
- return True
180
-
181
- def commit_changes():
182
- """Stage and commit all changes"""
183
- print("Staging files for commit...")
184
- if not run_command("git add -A"):
185
- return False
186
-
187
- print("Committing changes...")
188
- return run_command('git commit -m "Deploy GAIA to Hugging Face Space"')
189
-
190
- def push_to_huggingface():
191
- """Push to Hugging Face Space"""
192
- print("Pushing to Hugging Face Space...")
193
-
194
- # Try direct push with git credential store
195
- cmd = "git push huggingface main"
196
-
197
- # Execute the push command
198
- if run_command(cmd):
199
- return True
200
-
201
- # If regular push fails, try force push
202
- print("Regular push failed, attempting force push...")
203
- if run_command("git push huggingface main --force"):
204
- return True
205
-
206
- print("Force push failed. You may need to manually push:")
207
- print("1. Ensure your token has write permissions")
208
- print("2. Try: git push huggingface main --force")
209
- return False
210
-
211
- def setup_cli_auth_file(token):
212
- """Set up CLI authentication file for Hugging Face"""
213
- # Create .huggingface directory in home if it doesn't exist
214
- hf_dir = os.path.expanduser("~/.huggingface")
215
- os.makedirs(hf_dir, exist_ok=True)
216
-
217
- # Create or update token file
218
- token_file = os.path.join(hf_dir, "token")
219
- with open(token_file, "w") as f:
220
- f.write(token)
221
-
222
- print(f"Created HF token file at {token_file}")
223
- return True
224
-
225
- def setup_git_with_netrc(token):
226
- """Set up .netrc file for git authentication"""
227
- netrc_path = os.path.expanduser("~/.netrc")
228
-
229
- # Check if file exists and has huggingface.co entry
230
- netrc_content = ""
231
- if os.path.exists(netrc_path):
232
- with open(netrc_path, "r") as f:
233
- netrc_content = f.read()
234
-
235
- # Add or update huggingface.co entry
236
- if "machine huggingface.co" not in netrc_content:
237
- with open(netrc_path, "a") as f:
238
- f.write(f"\nmachine huggingface.co\nlogin hf_token\npassword {token}\n")
239
- else:
240
- # Replace existing entry
241
- import re
242
- pattern = r"machine huggingface\.co\s+login.*\s+password.*\s+"
243
- replacement = f"machine huggingface.co\nlogin hf_token\npassword {token}\n"
244
- netrc_content = re.sub(pattern, replacement, netrc_content)
245
- with open(netrc_path, "w") as f:
246
- f.write(netrc_content)
247
-
248
- # Set permissions for .netrc file (Unix only)
249
- if os.name != "nt": # not Windows
250
- os.chmod(netrc_path, 0o600)
251
-
252
- print(f"Updated .netrc file at {netrc_path}")
253
- return True
254
-
255
- def main():
256
- """Run deployment process"""
257
- args = parse_args()
258
- space_id = args.space
259
-
260
- # Load .env file
261
- if not load_dotenv():
262
- print("WARNING: Failed to load .env file, continuing with existing environment variables")
263
-
264
- # Get token from environment
265
- token = get_token_from_env()
266
- if not token:
267
- print("ERROR: No Hugging Face token found in environment variables.")
268
- print("Please ensure your .env file contains one of the following variables:")
269
- print(" - HF_TOKEN")
270
- print(" - HUGGINGFACE_TOKEN")
271
- print(" - HUGGING_FACE_TOKEN")
272
- print(" - HF_API_TOKEN")
273
- print(" - HUGGINGFACE_API_TOKEN")
274
- sys.exit(1)
275
-
276
- repo_url = f"https://huggingface.co/spaces/{space_id}"
277
-
278
- print("\n===== DEPLOYING TO HUGGING FACE SPACE =====")
279
- print(f"Target Space: {repo_url}")
280
-
281
- # Ensure Git is installed
282
- if not check_git_installed():
283
- print("Git is not installed. Please install Git and try again.")
284
- sys.exit(1)
285
-
286
- # Set up token file in ~/.huggingface/token
287
- if not setup_cli_auth_file(token):
288
- print("Failed to set up CLI authentication file")
289
- sys.exit(1)
290
-
291
- # Set up .netrc file with token
292
- if not setup_git_with_netrc(token):
293
- print("Failed to set up .netrc file")
294
- sys.exit(1)
295
-
296
- # Set up git credential helper
297
- if not setup_git_credential_helper(token):
298
- print("Failed to set up git credential helper")
299
- sys.exit(1)
300
-
301
- # Setup steps
302
- steps = [
303
- ("Setting up git", lambda: setup_git(space_id)),
304
- ("Creating .gitignore", create_gitignore),
305
- ("Committing changes", commit_changes),
306
- ("Pushing to Hugging Face", push_to_huggingface)
307
- ]
308
-
309
- for name, func in steps:
310
- print(f"\n>> {name}...")
311
- if not func():
312
- print(f"Step '{name}' failed. Deployment process halted.")
313
- sys.exit(1)
314
-
315
- print("\n===== DEPLOYMENT COMPLETED SUCCESSFULLY =====")
316
- print(f"Your project has been deployed to: {repo_url}")
317
- print("Note: It may take a few minutes for the changes to be visible.")
318
-
319
- if __name__ == "__main__":
320
- main()
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
deploy_to_huggingface_dotenv.py DELETED
@@ -1,227 +0,0 @@
1
- """
2
- Dotenv-based Git Deployment Script for Hugging Face Spaces
3
-
4
- This script handles the git operations required to deploy to a Hugging Face Space
5
- using token-based authentication, with the token loaded from a .env file.
6
-
7
- Usage:
8
- python deploy_to_huggingface_dotenv.py [--space SPACE_ID]
9
-
10
- Environment variables (from .env file):
11
- HF_TOKEN - Your Hugging Face API token
12
- """
13
- import os
14
- import subprocess
15
- import sys
16
- import argparse
17
-
18
- def parse_args():
19
- parser = argparse.ArgumentParser(description="Deploy to Hugging Face Space using .env file")
20
- parser.add_argument("--space", help="Space ID (username/space-name)",
21
- default="JoachimVC/Final_Assignment_Template")
22
- return parser.parse_args()
23
-
24
- def load_dotenv():
25
- """Load environment variables from .env file"""
26
- env_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), '.env')
27
- if not os.path.exists(env_path):
28
- print(f"ERROR: .env file not found at {env_path}")
29
- return False
30
-
31
- print(f"Loading environment variables from {env_path}")
32
- with open(env_path, 'r') as file:
33
- for line in file:
34
- line = line.strip()
35
- if not line or line.startswith('#'):
36
- continue
37
-
38
- key, value = line.split('=', 1)
39
- os.environ[key] = value
40
-
41
- return True
42
-
43
- def run_command(cmd, silent=False):
44
- """Run a shell command and return output"""
45
- if not silent:
46
- print(f"Running: {cmd}")
47
- try:
48
- process = subprocess.Popen(
49
- cmd,
50
- shell=True,
51
- stdout=subprocess.PIPE,
52
- stderr=subprocess.PIPE,
53
- text=True
54
- )
55
- stdout, stderr = process.communicate()
56
-
57
- if process.returncode != 0:
58
- print(f"Command failed with error code {process.returncode}")
59
- print(f"Error: {stderr.strip()}")
60
- return False
61
-
62
- if not silent and stdout.strip():
63
- print(stdout.strip())
64
-
65
- return True
66
- except Exception as e:
67
- print(f"Failed to execute command: {str(e)}")
68
- return False
69
-
70
- def check_git_installed():
71
- """Check if git is installed"""
72
- print("Checking if git is installed...")
73
- return run_command("git --version", silent=True)
74
-
75
- def get_token_from_env():
76
- """Get Hugging Face token from environment variables"""
77
- # Try various environment variable names that might contain the token
78
- token_vars = ["HF_TOKEN", "HUGGINGFACE_TOKEN", "HUGGING_FACE_TOKEN",
79
- "HF_API_TOKEN", "HUGGINGFACE_API_TOKEN"]
80
-
81
- for var_name in token_vars:
82
- token = os.environ.get(var_name)
83
- if token:
84
- print(f"Found token in environment variable: {var_name}")
85
- return token
86
-
87
- return None
88
-
89
- def setup_git(space_id, token):
90
- """Initialize git if needed and configure remote with token"""
91
- # Check if git is already initialized
92
- if not os.path.exists(".git"):
93
- print("Initializing git repository...")
94
- if not run_command("git init"):
95
- return False
96
-
97
- # Configure git user if not set
98
- result = subprocess.run("git config user.name", shell=True, capture_output=True, text=True)
99
- if not result.stdout.strip():
100
- print("Setting up git user name...")
101
- run_command('git config user.name "GAIA Deployment"')
102
-
103
- result = subprocess.run("git config user.email", shell=True, capture_output=True, text=True)
104
- if not result.stdout.strip():
105
- print("Setting up git user email...")
106
- run_command('git config user.email "gaia-deployment@example.com"')
107
-
108
- # Set up the Hugging Face remote with token
109
- print("Setting up git remote for Hugging Face...")
110
-
111
- # Check if remote exists
112
- result = subprocess.run("git remote -v", shell=True, capture_output=True, text=True)
113
- remote_output = result.stdout
114
-
115
- if "huggingface" in remote_output:
116
- run_command("git remote remove huggingface")
117
-
118
- # Add the HF remote with token authentication
119
- token_url = f"https://{token}@huggingface.co/spaces/{space_id}"
120
-
121
- # Don't print the token in the console output
122
- print(f"Adding remote: huggingface -> https://****@huggingface.co/spaces/{space_id}")
123
- return run_command(f"git remote add huggingface {token_url}")
124
-
125
- def create_gitignore():
126
- """Create .gitignore file with common patterns"""
127
- patterns = [
128
- ".env",
129
- "__pycache__/",
130
- "*.py[cod]",
131
- "*$py.class",
132
- ".Python",
133
- "env/",
134
- "venv/",
135
- ".venv",
136
- ".coverage",
137
- "*.log",
138
- "deploy_with_token.py",
139
- "deploy_to_huggingface_env.py",
140
- "deploy_to_huggingface_dotenv.py" # Don't include this file with token handling
141
- ]
142
-
143
- print("Creating .gitignore file...")
144
- with open(".gitignore", "w") as f:
145
- for pattern in patterns:
146
- f.write(f"{pattern}\n")
147
-
148
- return True
149
-
150
- def commit_changes():
151
- """Stage and commit all changes"""
152
- print("Staging files for commit...")
153
- if not run_command("git add -A"):
154
- return False
155
-
156
- print("Committing changes...")
157
- return run_command('git commit -m "Deploy GAIA to Hugging Face Space"')
158
-
159
- def push_to_huggingface():
160
- """Push to Hugging Face Space"""
161
- print("Pushing to Hugging Face Space...")
162
-
163
- # Try regular push first
164
- if run_command("git push huggingface main"):
165
- return True
166
-
167
- # If regular push fails, try force push
168
- print("Regular push failed, attempting force push...")
169
- if run_command("git push huggingface main --force"):
170
- return True
171
-
172
- print("Force push failed. You may need to manually push:")
173
- print("1. Ensure your token has write permissions")
174
- print("2. Try: git push huggingface main --force")
175
- return False
176
-
177
- def main():
178
- """Run deployment process"""
179
- args = parse_args()
180
- space_id = args.space
181
-
182
- # Load .env file
183
- if not load_dotenv():
184
- print("WARNING: Failed to load .env file, continuing with existing environment variables")
185
-
186
- # Get token from environment
187
- token = get_token_from_env()
188
- if not token:
189
- print("ERROR: No Hugging Face token found in environment variables.")
190
- print("Please ensure your .env file contains one of the following variables:")
191
- print(" - HF_TOKEN")
192
- print(" - HUGGINGFACE_TOKEN")
193
- print(" - HUGGING_FACE_TOKEN")
194
- print(" - HF_API_TOKEN")
195
- print(" - HUGGINGFACE_API_TOKEN")
196
- sys.exit(1)
197
-
198
- repo_url = f"https://huggingface.co/spaces/{space_id}"
199
-
200
- print("\n===== DEPLOYING TO HUGGING FACE SPACE =====")
201
- print(f"Target Space: {repo_url}")
202
-
203
- # Ensure Git is installed
204
- if not check_git_installed():
205
- print("Git is not installed. Please install Git and try again.")
206
- sys.exit(1)
207
-
208
- # Setup steps
209
- steps = [
210
- ("Setting up git", lambda: setup_git(space_id, token)),
211
- ("Creating .gitignore", create_gitignore),
212
- ("Committing changes", commit_changes),
213
- ("Pushing to Hugging Face", push_to_huggingface)
214
- ]
215
-
216
- for name, func in steps:
217
- print(f"\n>> {name}...")
218
- if not func():
219
- print(f"Step '{name}' failed. Deployment process halted.")
220
- sys.exit(1)
221
-
222
- print("\n===== DEPLOYMENT COMPLETED SUCCESSFULLY =====")
223
- print(f"Your project has been deployed to: {repo_url}")
224
- print("Note: It may take a few minutes for the changes to be visible.")
225
-
226
- if __name__ == "__main__":
227
- main()
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
deploy_to_huggingface_env.py DELETED
@@ -1,203 +0,0 @@
1
- """
2
- Environment-based Git Deployment Script for Hugging Face Spaces
3
-
4
- This script handles the git operations required to deploy to a Hugging Face Space
5
- using token-based authentication, with the token taken from environment variables.
6
-
7
- Usage:
8
- python deploy_to_huggingface_env.py [--space SPACE_ID]
9
-
10
- Environment variables:
11
- HUGGINGFACE_TOKEN - Your Hugging Face API token
12
- """
13
- import os
14
- import subprocess
15
- import sys
16
- import argparse
17
-
18
- def parse_args():
19
- parser = argparse.ArgumentParser(description="Deploy to Hugging Face Space using environment token")
20
- parser.add_argument("--space", help="Space ID (username/space-name)",
21
- default="JoachimVC/Final_Assignment_Template")
22
- return parser.parse_args()
23
-
24
- def run_command(cmd, silent=False):
25
- """Run a shell command and return output"""
26
- if not silent:
27
- print(f"Running: {cmd}")
28
- try:
29
- process = subprocess.Popen(
30
- cmd,
31
- shell=True,
32
- stdout=subprocess.PIPE,
33
- stderr=subprocess.PIPE,
34
- text=True
35
- )
36
- stdout, stderr = process.communicate()
37
-
38
- if process.returncode != 0:
39
- print(f"Command failed with error code {process.returncode}")
40
- print(f"Error: {stderr.strip()}")
41
- return False
42
-
43
- if not silent and stdout.strip():
44
- print(stdout.strip())
45
-
46
- return True
47
- except Exception as e:
48
- print(f"Failed to execute command: {str(e)}")
49
- return False
50
-
51
- def check_git_installed():
52
- """Check if git is installed"""
53
- print("Checking if git is installed...")
54
- return run_command("git --version", silent=True)
55
-
56
- def get_token_from_env():
57
- """Get Hugging Face token from environment variables"""
58
- # Try various environment variable names that might contain the token
59
- token_vars = ["HUGGINGFACE_TOKEN", "HF_TOKEN", "HUGGING_FACE_TOKEN",
60
- "HF_API_TOKEN", "HUGGINGFACE_API_TOKEN"]
61
-
62
- for var_name in token_vars:
63
- token = os.environ.get(var_name)
64
- if token:
65
- print(f"Found token in environment variable: {var_name}")
66
- return token
67
-
68
- return None
69
-
70
- def setup_git(space_id, token):
71
- """Initialize git if needed and configure remote with token"""
72
- # Check if git is already initialized
73
- if not os.path.exists(".git"):
74
- print("Initializing git repository...")
75
- if not run_command("git init"):
76
- return False
77
-
78
- # Configure git user if not set
79
- result = subprocess.run("git config user.name", shell=True, capture_output=True, text=True)
80
- if not result.stdout.strip():
81
- print("Setting up git user name...")
82
- run_command('git config user.name "GAIA Deployment"')
83
-
84
- result = subprocess.run("git config user.email", shell=True, capture_output=True, text=True)
85
- if not result.stdout.strip():
86
- print("Setting up git user email...")
87
- run_command('git config user.email "gaia-deployment@example.com"')
88
-
89
- # Set up the Hugging Face remote with token
90
- print("Setting up git remote for Hugging Face...")
91
-
92
- # Check if remote exists
93
- result = subprocess.run("git remote -v", shell=True, capture_output=True, text=True)
94
- remote_output = result.stdout
95
-
96
- if "huggingface" in remote_output:
97
- run_command("git remote remove huggingface")
98
-
99
- # Add the HF remote with token authentication
100
- token_url = f"https://{token}@huggingface.co/spaces/{space_id}"
101
-
102
- # Don't print the token in the console output
103
- print(f"Adding remote: huggingface -> https://****@huggingface.co/spaces/{space_id}")
104
- return run_command(f"git remote add huggingface {token_url}")
105
-
106
- def create_gitignore():
107
- """Create .gitignore file with common patterns"""
108
- patterns = [
109
- ".env",
110
- "__pycache__/",
111
- "*.py[cod]",
112
- "*$py.class",
113
- ".Python",
114
- "env/",
115
- "venv/",
116
- ".venv",
117
- ".coverage",
118
- "*.log",
119
- "deploy_with_token.py",
120
- "deploy_to_huggingface_env.py" # Don't include this file with token handling
121
- ]
122
-
123
- print("Creating .gitignore file...")
124
- with open(".gitignore", "w") as f:
125
- for pattern in patterns:
126
- f.write(f"{pattern}\n")
127
-
128
- return True
129
-
130
- def commit_changes():
131
- """Stage and commit all changes"""
132
- print("Staging files for commit...")
133
- if not run_command("git add -A"):
134
- return False
135
-
136
- print("Committing changes...")
137
- return run_command('git commit -m "Deploy GAIA to Hugging Face Space"')
138
-
139
- def push_to_huggingface():
140
- """Push to Hugging Face Space"""
141
- print("Pushing to Hugging Face Space...")
142
-
143
- # Try regular push first
144
- if run_command("git push huggingface main"):
145
- return True
146
-
147
- # If regular push fails, try force push
148
- print("Regular push failed, attempting force push...")
149
- if run_command("git push huggingface main --force"):
150
- return True
151
-
152
- print("Force push failed. You may need to manually push:")
153
- print("1. Ensure your token has write permissions")
154
- print("2. Try: git push huggingface main --force")
155
- return False
156
-
157
- def main():
158
- """Run deployment process"""
159
- args = parse_args()
160
- space_id = args.space
161
-
162
- # Get token from environment
163
- token = get_token_from_env()
164
- if not token:
165
- print("ERROR: No Hugging Face token found in environment variables.")
166
- print("Please set one of the following environment variables:")
167
- print(" - HUGGINGFACE_TOKEN")
168
- print(" - HF_TOKEN")
169
- print(" - HUGGING_FACE_TOKEN")
170
- print(" - HF_API_TOKEN")
171
- print(" - HUGGINGFACE_API_TOKEN")
172
- sys.exit(1)
173
-
174
- repo_url = f"https://huggingface.co/spaces/{space_id}"
175
-
176
- print("\n===== DEPLOYING TO HUGGING FACE SPACE =====")
177
- print(f"Target Space: {repo_url}")
178
-
179
- # Ensure Git is installed
180
- if not check_git_installed():
181
- print("Git is not installed. Please install Git and try again.")
182
- sys.exit(1)
183
-
184
- # Setup steps
185
- steps = [
186
- ("Setting up git", lambda: setup_git(space_id, token)),
187
- ("Creating .gitignore", create_gitignore),
188
- ("Committing changes", commit_changes),
189
- ("Pushing to Hugging Face", push_to_huggingface)
190
- ]
191
-
192
- for name, func in steps:
193
- print(f"\n>> {name}...")
194
- if not func():
195
- print(f"Step '{name}' failed. Deployment process halted.")
196
- sys.exit(1)
197
-
198
- print("\n===== DEPLOYMENT COMPLETED SUCCESSFULLY =====")
199
- print(f"Your project has been deployed to: {repo_url}")
200
- print("Note: It may take a few minutes for the changes to be visible.")
201
-
202
- if __name__ == "__main__":
203
- main()
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
deploy_with_token.py DELETED
@@ -1,176 +0,0 @@
1
- """
2
- Token-Based Git Deployment Script for Hugging Face Spaces
3
-
4
- This script handles the git operations required to deploy to a Hugging Face Space
5
- using token-based authentication.
6
-
7
- Usage:
8
- python deploy_with_token.py --token YOUR_HF_TOKEN
9
-
10
- """
11
- import os
12
- import subprocess
13
- import sys
14
- import argparse
15
-
16
- def parse_args():
17
- parser = argparse.ArgumentParser(description="Deploy to Hugging Face Space with token")
18
- parser.add_argument("--token", help="Hugging Face API token", required=True)
19
- parser.add_argument("--space", help="Space ID (username/space-name)",
20
- default="JoachimVC/Final_Assignment_Template")
21
- return parser.parse_args()
22
-
23
- def run_command(cmd, silent=False):
24
- """Run a shell command and return output"""
25
- if not silent:
26
- print(f"Running: {cmd}")
27
- try:
28
- process = subprocess.Popen(
29
- cmd,
30
- shell=True,
31
- stdout=subprocess.PIPE,
32
- stderr=subprocess.PIPE,
33
- text=True
34
- )
35
- stdout, stderr = process.communicate()
36
-
37
- if process.returncode != 0:
38
- print(f"Command failed with error code {process.returncode}")
39
- print(f"Error: {stderr.strip()}")
40
- return False
41
-
42
- if not silent and stdout.strip():
43
- print(stdout.strip())
44
-
45
- return True
46
- except Exception as e:
47
- print(f"Failed to execute command: {str(e)}")
48
- return False
49
-
50
- def check_git_installed():
51
- """Check if git is installed"""
52
- print("Checking if git is installed...")
53
- return run_command("git --version", silent=True)
54
-
55
- def setup_git(space_id, token):
56
- """Initialize git if needed and configure remote with token"""
57
- # Check if git is already initialized
58
- if not os.path.exists(".git"):
59
- print("Initializing git repository...")
60
- if not run_command("git init"):
61
- return False
62
-
63
- # Configure git user if not set
64
- result = subprocess.run("git config user.name", shell=True, capture_output=True, text=True)
65
- if not result.stdout.strip():
66
- print("Setting up git user name...")
67
- run_command('git config user.name "GAIA Deployment"')
68
-
69
- result = subprocess.run("git config user.email", shell=True, capture_output=True, text=True)
70
- if not result.stdout.strip():
71
- print("Setting up git user email...")
72
- run_command('git config user.email "gaia-deployment@example.com"')
73
-
74
- # Set up the Hugging Face remote with token
75
- print("Setting up git remote for Hugging Face...")
76
-
77
- # Check if remote exists
78
- result = subprocess.run("git remote -v", shell=True, capture_output=True, text=True)
79
- remote_output = result.stdout
80
-
81
- if "huggingface" in remote_output:
82
- run_command("git remote remove huggingface")
83
-
84
- # Add the HF remote with token authentication
85
- token_url = f"https://{token}@huggingface.co/spaces/{space_id}"
86
-
87
- # Don't print the token in the console output
88
- print(f"Adding remote: huggingface -> https://****@huggingface.co/spaces/{space_id}")
89
- return run_command(f"git remote add huggingface {token_url}")
90
-
91
- def create_gitignore():
92
- """Create .gitignore file with common patterns"""
93
- patterns = [
94
- ".env",
95
- "__pycache__/",
96
- "*.py[cod]",
97
- "*$py.class",
98
- ".Python",
99
- "env/",
100
- "venv/",
101
- ".venv",
102
- ".coverage",
103
- "*.log",
104
- "deploy_with_token.py" # Don't include this file with token handling
105
- ]
106
-
107
- print("Creating .gitignore file...")
108
- with open(".gitignore", "w") as f:
109
- for pattern in patterns:
110
- f.write(f"{pattern}\n")
111
-
112
- return True
113
-
114
- def commit_changes():
115
- """Stage and commit all changes"""
116
- print("Staging files for commit...")
117
- if not run_command("git add -A"):
118
- return False
119
-
120
- print("Committing changes...")
121
- return run_command('git commit -m "Deploy GAIA to Hugging Face Space"')
122
-
123
- def push_to_huggingface():
124
- """Push to Hugging Face Space"""
125
- print("Pushing to Hugging Face Space...")
126
-
127
- # Try regular push first
128
- if run_command("git push huggingface main"):
129
- return True
130
-
131
- # If regular push fails, try force push
132
- print("Regular push failed, attempting force push...")
133
- if run_command("git push huggingface main --force"):
134
- return True
135
-
136
- print("Force push failed. You may need to manually push:")
137
- print("1. Ensure your token has write permissions")
138
- print("2. Try: git push huggingface main --force")
139
- return False
140
-
141
- def main():
142
- """Run deployment process"""
143
- args = parse_args()
144
- token = args.token
145
- space_id = args.space
146
-
147
- repo_url = f"https://huggingface.co/spaces/{space_id}"
148
-
149
- print("\n===== DEPLOYING TO HUGGING FACE SPACE =====")
150
- print(f"Target Space: {repo_url}")
151
-
152
- # Ensure Git is installed
153
- if not check_git_installed():
154
- print("Git is not installed. Please install Git and try again.")
155
- sys.exit(1)
156
-
157
- # Setup steps
158
- steps = [
159
- ("Setting up git", lambda: setup_git(space_id, token)),
160
- ("Creating .gitignore", create_gitignore),
161
- ("Committing changes", commit_changes),
162
- ("Pushing to Hugging Face", push_to_huggingface)
163
- ]
164
-
165
- for name, func in steps:
166
- print(f"\n>> {name}...")
167
- if not func():
168
- print(f"Step '{name}' failed. Deployment process halted.")
169
- sys.exit(1)
170
-
171
- print("\n===== DEPLOYMENT COMPLETED SUCCESSFULLY =====")
172
- print(f"Your project has been deployed to: {repo_url}")
173
- print("Note: It may take a few minutes for the changes to be visible.")
174
-
175
- if __name__ == "__main__":
176
- main()
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
deployment_checklist.md DELETED
@@ -1,83 +0,0 @@
1
- # GAIA Agent Deployment Checklist for Hugging Face Space
2
-
3
- ## Pre-Deployment Verification
4
-
5
- - [ ] Verify all required files are present
6
- - [ ] app.py
7
- - [ ] requirements.txt
8
- - [ ] agent/agent.py, agent/config.py, agent/graph.py
9
- - [ ] memory/supabase_memory.py, memory/supabase_memory_fix.py
10
- - [ ] tools/ directory with all required tools
11
-
12
- - [ ] Verify requirements.txt contains all necessary dependencies
13
- - [ ] openai
14
- - [ ] gradio
15
- - [ ] langchain
16
- - [ ] supabase
17
- - [ ] requests
18
- - [ ] pandas
19
-
20
- - [ ] Verify app.py configuration
21
- - [ ] OAuth handling is configured
22
- - [ ] Gradio components are properly referenced
23
- - [ ] Error handling is in place
24
-
25
- ## Environment Variables Configuration
26
-
27
- - [ ] Set up required environment variables in Hugging Face Space:
28
- - [ ] `OPENAI_API_KEY` - Required for language model access
29
- - [ ] `SUPABASE_URL` - Required for memory functionality
30
- - [ ] `SUPABASE_KEY` - Required for memory functionality
31
-
32
- - [ ] Set up optional environment variables:
33
- - [ ] `SERPER_API_KEY` - For enhanced search capabilities
34
- - [ ] `SPACE_ID` - Usually auto-set by Hugging Face
35
- - [ ] `SPACE_HOST` - Usually auto-set by Hugging Face
36
-
37
- - [ ] Configure OAuth (for full functionality):
38
- - [ ] Set `hf_oauth: true` in Space repository secrets
39
-
40
- ## Deployment Process
41
-
42
- 1. Push code to Hugging Face Space repository
43
- 2. Monitor build logs for any errors
44
- 3. Check if the Space is successfully built and running
45
- 4. Verify the interface loads correctly
46
-
47
- ## Post-Deployment Verification
48
-
49
- - [ ] Verify the app loads without errors
50
- - [ ] Check that the correct mode (full or demo) is active based on OAuth configuration
51
- - [ ] Test the agent with a sample question
52
- - [ ] Verify results display correctly in the interface
53
- - [ ] Check integrated tools functionality:
54
- - [ ] OpenAI integration
55
- - [ ] Supabase memory integration
56
- - [ ] Search tools
57
-
58
- ## Troubleshooting Common Issues
59
-
60
- ### OAuth Configuration Issues
61
- - Symptom: "OAuth is required but OAUTH_CLIENT_ID environment variable is not set" error
62
- - Solution: Set `hf_oauth: true` in Space repository secrets
63
-
64
- ### OpenAI API Key Issues
65
- - Symptom: "Error: OpenAI API key is required" or similar errors
66
- - Solution: Verify OPENAI_API_KEY is correctly set in Space secrets
67
-
68
- ### Supabase Connection Issues
69
- - Symptom: "Error connecting to Supabase" or memory-related errors
70
- - Solution: Verify SUPABASE_URL and SUPABASE_KEY are correctly set in Space secrets
71
-
72
- ### Gradio Interface Issues
73
- - Symptom: Interface not loading properly or component errors
74
- - Solution: Check Space logs for specific component errors and update app.py accordingly
75
-
76
- ## Rollback Plan
77
-
78
- If critical issues are encountered after deployment:
79
-
80
- 1. Identify the specific component causing the issue from Space logs
81
- 2. If needed, revert to the previous working commit using Git
82
- 3. Fix issues locally and re-deploy with verified fixes
83
- 4. Consider temporarily enabling demo mode while fixing issues
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
deployment_readiness_report.json DELETED
@@ -1,38 +0,0 @@
1
- {
2
- "timestamp": "2025-05-01T09:14:06.477036",
3
- "status": "not_ready",
4
- "results": {
5
- "file_structure": {
6
- "passed": true,
7
- "issues": []
8
- },
9
- "environment_variables": {
10
- "passed": false,
11
- "issues": [
12
- "OPENAI_API_KEY",
13
- "SUPABASE_URL",
14
- "SUPABASE_KEY"
15
- ]
16
- },
17
- "gradio_interface": {
18
- "passed": false,
19
- "issues": [
20
- "Error checking app.py: 'charmap' codec can't decode byte 0x9d in position 8062: character maps to <undefined>"
21
- ]
22
- },
23
- "agent_configuration": {
24
- "passed": true,
25
- "issues": []
26
- },
27
- "requirements": {
28
- "passed": true,
29
- "issues": []
30
- }
31
- },
32
- "recommendations": [
33
- "Set the required environment variable: OPENAI_API_KEY",
34
- "Set the required environment variable: SUPABASE_URL",
35
- "Set the required environment variable: SUPABASE_KEY",
36
- "Fix Gradio interface issue: Error checking app.py: 'charmap' codec can't decode byte 0x9d in position 8062: character maps to <undefined>"
37
- ]
38
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
deployment_readiness_report.md DELETED
@@ -1,97 +0,0 @@
1
- # GAIA Deployment Readiness Report
2
-
3
- ## 1. Deployment Status: ✅ SUCCESSFUL
4
-
5
- The GAIA agent system has been successfully deployed to Hugging Face Space at:
6
- `https://huggingface.co/spaces/JoachimVC/Final_Assignment_Template`
7
-
8
- **Deployment Timestamp:** April 30, 2025
9
-
10
- ## 2. Environment Configuration
11
-
12
- | Category | Status | Details |
13
- |----------|--------|---------|
14
- | Code Files | ✅ Complete | All necessary code files have been uploaded |
15
- | Environment Variables | ✅ Configured | All required API keys and configuration variables set |
16
- | Secret Management | ✅ Secure | Using Hugging Face Secrets for sensitive credentials |
17
-
18
- ### Environment Variables Configured:
19
- - `OPENAI_API_KEY` ✅
20
- - `SERPER_API_KEY` ✅
21
- - `PERPLEXITY_API_KEY` ✅
22
- - `SUPABASE_URL` ✅
23
- - `SUPABASE_KEY` ✅
24
- - `HF_TOKEN` ✅
25
-
26
- ## 3. Component Integration Status
27
-
28
- | Component | Status | Verification |
29
- |-----------|--------|-------------|
30
- | OpenAI API | ✅ Working | Successfully connected to API and retrieved model list |
31
- | Serper Search | ✅ Working | Successfully performed search query with expected results |
32
- | Supabase Memory | ⚠️ Needs Verification | Environment variables set, but actual integration needs testing |
33
- | DuckDuckGo Search | ⚠️ Needs Verification | Functionality available but needs end-to-end testing |
34
- | Perplexity | ⚠️ Needs Verification | API key configured, but integration needs testing |
35
- | Agent Graph Workflow | ⚠️ Needs Verification | Core functionality needs verification in the deployed environment |
36
-
37
- ## 4. Cleanup Recommendations
38
-
39
- Several files and directories in the deployed space are not required for runtime operation and can be safely removed to optimize the environment. Refer to `huggingface_space_cleanup.md` for a detailed list of files that can be removed, including:
40
-
41
- - Test directories (`tests/`, `backup_tests/`)
42
- - Development configuration (`.roo/`, `.roomodes/`, etc.)
43
- - Deployment and cleaning scripts
44
- - Non-essential documentation
45
-
46
- ## 5. Verification Steps
47
-
48
- The following steps are recommended to complete verification of the deployed system:
49
-
50
- 1. **Run Production Verification Script**
51
- - Execute `verify_production.py` on the Hugging Face Space to test all components
52
- - This script will generate a detailed report on component status
53
-
54
- 2. **Test Conversation Memory**
55
- - Initiate a multi-turn conversation that relies on memory
56
- - Verify that the agent can recall information from previous interactions
57
-
58
- 3. **Test Search Tool Integration**
59
- - Ask a question requiring web search
60
- - Verify that search results are properly integrated into responses
61
-
62
- 4. **Performance Monitoring**
63
- - Monitor response times for different query types
64
- - Check for any memory leaks or increasing latency over time
65
-
66
- ## 6. Recommendations
67
-
68
- 1. **Memory Persistence**
69
- - Verify Supabase table structure matches the required schema
70
- - Check that memory persistence works across session restarts
71
-
72
- 2. **Error Handling**
73
- - Implement robust error handling for API failures
74
- - Add automatic fallback to alternative models if primary model is unavailable
75
-
76
- 3. **Monitoring**
77
- - Set up periodic health checks to verify all integrations remain functional
78
- - Implement usage tracking to monitor API consumption and costs
79
-
80
- 4. **Documentation**
81
- - Update user-facing documentation with examples of supported capabilities
82
- - Provide clear guidance on limitations and expected behaviors
83
-
84
- ## 7. Rollback Plan
85
-
86
- In case issues are discovered in the deployed system:
87
-
88
- 1. Identify the specific component causing the issue
89
- 2. If possible, fix directly in the Hugging Face interface
90
- 3. For more complex issues, fix locally and redeploy
91
- 4. If critical failure occurs, consider reverting to previous stable version
92
-
93
- ## Conclusion
94
-
95
- The GAIA system has been successfully deployed to Hugging Face Space with all required environment variables configured. Initial component testing shows successful integration with OpenAI and Serper APIs. Comprehensive testing of the Supabase memory integration and other components is recommended to ensure full functionality in the production environment.
96
-
97
- The deployment is currently in a good state for final verification and production use, with minor cleanup recommended to optimize the space.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
deployment_verification.py DELETED
@@ -1,327 +0,0 @@
1
- """
2
- Deployment verification script for GAIA assessment
3
- Checks key components without OAuth requirements
4
- """
5
- import os
6
- import sys
7
- import logging
8
- import time
9
- from typing import Dict, List, Any, Optional
10
- import traceback
11
-
12
- # Configure logging
13
- logging.basicConfig(
14
- level=logging.INFO,
15
- format="%(asctime)s - %(levelname)s - %(message)s"
16
- )
17
- logger = logging.getLogger("deployment_verification")
18
-
19
- class DeploymentVerifier:
20
- """Verifies deployment readiness of the GAIA system"""
21
-
22
- def __init__(self):
23
- self.results = {}
24
- self.all_tests_passed = True
25
-
26
- def verify_environment_variables(self):
27
- """Check required environment variables"""
28
- logger.info("Checking environment variables...")
29
- required_vars = ["OPENAI_API_KEY", "SUPABASE_URL", "SUPABASE_KEY"]
30
- optional_vars = ["SERPER_API_KEY", "SPACE_ID", "SPACE_HOST"]
31
-
32
- missing = []
33
- for var in required_vars:
34
- if not os.environ.get(var):
35
- missing.append(var)
36
- self.all_tests_passed = False
37
-
38
- if missing:
39
- logger.error(f"❌ Missing required environment variables: {', '.join(missing)}")
40
- self.results["environment_variables"] = False
41
- else:
42
- logger.info("✅ All required environment variables are set")
43
- self.results["environment_variables"] = True
44
-
45
- # Check optional variables
46
- for var in optional_vars:
47
- if os.environ.get(var):
48
- logger.info(f"✅ Optional variable {var} is set")
49
- else:
50
- logger.warning(f"⚠️ Optional variable {var} is not set")
51
-
52
- return self.results["environment_variables"]
53
-
54
- def verify_oauth_configuration(self):
55
- """Check OAuth configuration for Hugging Face Space"""
56
- logger.info("Checking OAuth configuration...")
57
-
58
- # For local testing, this will always be skipped/failed
59
- space_host = os.environ.get("SPACE_HOST")
60
- if not space_host:
61
- logger.warning("⚠️ Not running in Hugging Face Space environment. OAuth check skipped.")
62
- self.results["oauth_configuration"] = None
63
- return None
64
-
65
- # Check if OAUTH_CLIENT_ID is set when in a Space
66
- oauth_client_id = os.environ.get("OAUTH_CLIENT_ID")
67
- if oauth_client_id:
68
- logger.info("✅ OAUTH_CLIENT_ID is set")
69
- self.results["oauth_configuration"] = True
70
- else:
71
- logger.error("❌ OAUTH_CLIENT_ID is not set")
72
- logger.error("For deployment, ensure 'hf_oauth: true' is set in the Space metadata")
73
- self.results["oauth_configuration"] = False
74
- self.all_tests_passed = False
75
-
76
- return self.results["oauth_configuration"]
77
-
78
- def verify_agent_initialization(self):
79
- """Verify that the GAIA agent can be initialized"""
80
- logger.info("Verifying GAIA agent initialization...")
81
- try:
82
- from agent import GAIAAgent
83
-
84
- # Record start time
85
- start_time = time.time()
86
-
87
- # Initialize the agent
88
- agent = GAIAAgent()
89
-
90
- # Calculate initialization time
91
- init_time = time.time() - start_time
92
-
93
- logger.info(f"✅ GAIA Agent initialized successfully in {init_time:.2f} seconds")
94
- self.results["agent_initialization"] = True
95
- return True
96
- except Exception as e:
97
- logger.error(f"❌ Failed to initialize GAIA agent: {e}")
98
- logger.error(traceback.format_exc())
99
- self.results["agent_initialization"] = False
100
- self.all_tests_passed = False
101
- return False
102
-
103
- def verify_memory_system(self):
104
- """Verify that the memory system works correctly"""
105
- logger.info("Verifying memory system...")
106
- try:
107
- from memory.supabase_memory import SupabaseMemory
108
- from agent.config import get_memory_config
109
-
110
- # Get memory configuration
111
- memory_config = get_memory_config()
112
- memory_config["enabled"] = True
113
-
114
- # Initialize memory
115
- memory = SupabaseMemory(memory_config)
116
-
117
- # Test basic operations
118
- test_key = f"test_key_{int(time.time())}"
119
- test_value = {"message": "Test value", "timestamp": time.time()}
120
-
121
- # Store
122
- memory.store(test_key, test_value)
123
- logger.info(f"✅ Successfully stored test value with key: {test_key}")
124
-
125
- # Retrieve
126
- retrieved = memory.retrieve(test_key)
127
-
128
- if retrieved and retrieved.get("message") == "Test value":
129
- logger.info("✅ Successfully retrieved test value from memory")
130
- self.results["memory_system"] = True
131
- return True
132
- else:
133
- logger.error(f"❌ Retrieved value doesn't match expected. Got: {retrieved}")
134
- self.results["memory_system"] = False
135
- self.all_tests_passed = False
136
- return False
137
- except Exception as e:
138
- logger.error(f"❌ Failed to verify memory system: {e}")
139
- logger.error(traceback.format_exc())
140
- self.results["memory_system"] = False
141
- self.all_tests_passed = False
142
- return False
143
-
144
- def verify_tool_integrations(self):
145
- """Verify that required tools are available and working"""
146
- logger.info("Verifying tool integrations...")
147
- tool_results = {}
148
-
149
- # Check DuckDuckGo
150
- try:
151
- from tools.web_tools import duckduckgo_search
152
-
153
- # Test with a simple query
154
- test_query = "What is the capital of France?"
155
- results = duckduckgo_search(test_query, num_results=1)
156
-
157
- if results and len(results) > 0:
158
- logger.info("✅ DuckDuckGo search is working")
159
- tool_results["duckduckgo"] = True
160
- else:
161
- logger.warning("⚠️ DuckDuckGo search returned no results")
162
- tool_results["duckduckgo"] = False
163
- self.all_tests_passed = False
164
- except Exception as e:
165
- logger.error(f"❌ DuckDuckGo search integration failed: {e}")
166
- tool_results["duckduckgo"] = False
167
- self.all_tests_passed = False
168
-
169
- # Check Serper if API key is available
170
- serper_api_key = os.environ.get("SERPER_API_KEY")
171
- if serper_api_key:
172
- try:
173
- from tools.web_tools import serper_search
174
-
175
- # Test with a simple query
176
- test_query = "What is the capital of France?"
177
- results = serper_search(test_query, num_results=1)
178
-
179
- if results and len(results) > 0:
180
- logger.info("✅ Serper search is working")
181
- tool_results["serper"] = True
182
- else:
183
- logger.warning("⚠️ Serper search returned no results")
184
- tool_results["serper"] = False
185
- except Exception as e:
186
- logger.error(f"❌ Serper search integration failed: {e}")
187
- tool_results["serper"] = False
188
- # Don't fail the overall test for optional Serper
189
- else:
190
- logger.warning("⚠️ SERPER_API_KEY not set, skipping Serper verification")
191
- tool_results["serper"] = None
192
-
193
- self.results["tool_integrations"] = tool_results
194
- return any(result for result in tool_results.values() if result is True)
195
-
196
- def verify_agent_graph(self):
197
- """Verify that the agent graph can be created"""
198
- logger.info("Verifying agent graph...")
199
- try:
200
- from agent.graph import create_agent_graph
201
-
202
- # Create the agent graph
203
- graph = create_agent_graph()
204
-
205
- if graph:
206
- logger.info("✅ Agent graph created successfully")
207
- self.results["agent_graph"] = True
208
- return True
209
- else:
210
- logger.error("❌ Agent graph creation failed")
211
- self.results["agent_graph"] = False
212
- self.all_tests_passed = False
213
- return False
214
- except Exception as e:
215
- logger.error(f"❌ Failed to verify agent graph: {e}")
216
- logger.error(traceback.format_exc())
217
- self.results["agent_graph"] = False
218
- self.all_tests_passed = False
219
- return False
220
-
221
- def verify_gradio_app(self):
222
- """Verify Gradio app can be initialized (without OAuth)"""
223
- logger.info("Verifying Gradio app initialization...")
224
- try:
225
- import gradio as gr
226
-
227
- # Create a simple test app
228
- with gr.Blocks() as demo:
229
- gr.Markdown("# Test App")
230
- gr.Button("Test Button")
231
-
232
- logger.info("✅ Gradio app initialization successful")
233
- self.results["gradio_app"] = True
234
- return True
235
- except Exception as e:
236
- logger.error(f"❌ Failed to initialize Gradio app: {e}")
237
- logger.error(traceback.format_exc())
238
- self.results["gradio_app"] = False
239
- self.all_tests_passed = False
240
- return False
241
-
242
- def generate_deployment_recommendation(self):
243
- """Generate deployment recommendation based on test results"""
244
- logger.info("\n=== GENERATING DEPLOYMENT RECOMMENDATION ===")
245
-
246
- if self.all_tests_passed:
247
- logger.info("✅ All critical tests passed!")
248
- recommendation = """
249
- DEPLOYMENT RECOMMENDATION: READY FOR DEPLOYMENT
250
-
251
- Your system has passed all critical verification tests and is ready for deployment.
252
-
253
- For Hugging Face Space deployment, remember to:
254
- 1. Set 'hf_oauth: true' in the Space metadata
255
- 2. Ensure all environment variables are properly configured in the Space settings
256
- """
257
- else:
258
- logger.error("❌ Some tests failed!")
259
- failed_tests = [test for test, result in self.results.items()
260
- if result is False or (isinstance(result, dict) and not any(r for r in result.values()))]
261
-
262
- recommendation = f"""
263
- DEPLOYMENT RECOMMENDATION: NOT READY FOR DEPLOYMENT
264
-
265
- The following issues need to be addressed before deployment:
266
- {', '.join(failed_tests)}
267
-
268
- Please fix these issues and run the verification again.
269
- """
270
-
271
- logger.info(recommendation)
272
- return recommendation
273
-
274
- def run_all_tests(self):
275
- """Run all verification tests"""
276
- logger.info("=== STARTING DEPLOYMENT VERIFICATION ===")
277
-
278
- # 1. Verify environment variables
279
- self.verify_environment_variables()
280
-
281
- # 2. Verify OAuth configuration
282
- self.verify_oauth_configuration()
283
-
284
- # 3. Verify agent initialization
285
- self.verify_agent_initialization()
286
-
287
- # 4. Verify memory system
288
- self.verify_memory_system()
289
-
290
- # 5. Verify tool integrations
291
- self.verify_tool_integrations()
292
-
293
- # 6. Verify agent graph
294
- self.verify_agent_graph()
295
-
296
- # 7. Verify Gradio app
297
- self.verify_gradio_app()
298
-
299
- # Generate deployment recommendation
300
- recommendation = self.generate_deployment_recommendation()
301
-
302
- # Print summary
303
- logger.info("\n=== DEPLOYMENT VERIFICATION RESULTS ===")
304
- for component, result in self.results.items():
305
- if isinstance(result, dict):
306
- logger.info(f"{component}:")
307
- for subcomponent, subresult in result.items():
308
- status = "✅ PASS" if subresult else "❌ FAIL" if subresult is False else "⚠️ SKIPPED"
309
- logger.info(f" {subcomponent}: {status}")
310
- else:
311
- status = "✅ PASS" if result else "❌ FAIL" if result is False else "⚠️ SKIPPED"
312
- logger.info(f"{component}: {status}")
313
-
314
- logger.info("\n" + recommendation)
315
-
316
- return self.all_tests_passed
317
-
318
- if __name__ == "__main__":
319
- verifier = DeploymentVerifier()
320
- success = verifier.run_all_tests()
321
-
322
- if success:
323
- logger.info("\n✅ System is ready for deployment!")
324
- sys.exit(0)
325
- else:
326
- logger.error("\n❌ System is NOT ready for deployment!")
327
- sys.exit(1)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
docs/all_apis_integration_testing.md DELETED
@@ -1,170 +0,0 @@
1
- # All APIs Integration Testing
2
-
3
- This document provides information about the comprehensive integration test for all APIs in the GAIA agent.
4
-
5
- ## Overview
6
-
7
- The integration test verifies that all APIs work together seamlessly in the GAIA agent. It tests:
8
-
9
- 1. **OpenAI** - For language model capabilities
10
- 2. **Perplexity** - For enhanced search and reasoning
11
- 3. **Serper** - For Google search results
12
- 4. **DuckDuckGo** - For web search without API key requirements
13
- 5. **arXiv** - For academic and scientific paper search
14
-
15
- The test ensures that the agent can switch between different tools based on the task and maintain memory across different tool executions.
16
-
17
- ## Prerequisites
18
-
19
- Before running the integration test, ensure you have the following environment variables set:
20
-
21
- - `OPENAI_API_KEY` (required) - Your OpenAI API key
22
- - `SERPER_API_KEY` (optional) - Your Serper API key for Google search
23
- - `PERPLEXITY_API_KEY` (optional) - Your Perplexity API key
24
-
25
- If optional API keys are not provided, the corresponding tests will be skipped.
26
-
27
- ## Running the Test
28
-
29
- You can run the integration test using the provided runner script:
30
-
31
- ```bash
32
- python tests/run_all_apis_integration_test.py
33
- ```
34
-
35
- ### Command-line Options
36
-
37
- - `--verbose` or `-v`: Enable verbose logging for detailed output
38
- - `--report-file` or `-r`: Specify a custom output file for the test report (default: `gaia_all_apis_integration_report.json`)
39
-
40
- Example with options:
41
-
42
- ```bash
43
- python tests/run_all_apis_integration_test.py --verbose --report-file my_report.json
44
- ```
45
-
46
- ## Test Components
47
-
48
- The integration test consists of several components:
49
-
50
- ### 1. API Connection Tests
51
-
52
- Verifies that the agent can connect to each API service. For each API, it:
53
- - Checks if the required API key is available
54
- - Attempts to establish a connection
55
- - Verifies that the connection is successful
56
-
57
- ### 2. Individual Tool Execution Tests
58
-
59
- Tests each tool individually with simple queries to ensure they work correctly:
60
- - DuckDuckGo search
61
- - Serper search
62
- - Perplexity search
63
- - arXiv search
64
- - Robust search (fallback mechanism)
65
- - Merged search (combining results from multiple providers)
66
-
67
- ### 3. Tool Switching Tests
68
-
69
- Verifies that the agent can switch between different tools based on the task:
70
- - Scientific questions (should use arXiv and Perplexity)
71
- - Factual questions (should use DuckDuckGo and Serper)
72
- - Explanatory questions (should use Perplexity)
73
-
74
- ### 4. Memory Integration Tests
75
-
76
- Tests the agent's ability to maintain memory across different tool executions:
77
- - Processes a sequence of related questions
78
- - Verifies that the conversation history is maintained
79
- - Checks for cached results and intermediate results
80
-
81
- ## Test Report
82
-
83
- After running the test, a comprehensive report is generated in JSON format. The report includes:
84
-
85
- - API connection results
86
- - Tool execution results
87
- - Complex query results
88
- - Memory integration results
89
- - Summary statistics
90
-
91
- The report is displayed in the console and saved to the specified file.
92
-
93
- ## Interpreting Results
94
-
95
- The test results are displayed with the following indicators:
96
-
97
- - ✅ **PASS**: The test was successful
98
- - ❌ **FAIL**: The test failed
99
- - ⚠️ **SKIP**: The test was skipped (usually due to missing API keys)
100
-
101
- ### Example Output
102
-
103
- ```
104
- ================================================================================
105
- GAIA ALL APIs INTEGRATION TEST REPORT
106
- ================================================================================
107
-
108
- --------------------------------------------------------------------------------
109
- SUMMARY
110
- --------------------------------------------------------------------------------
111
- Timestamp: 2025-04-30 18:00:00
112
- Total APIs Tested: 5
113
- Successful Connections: 4
114
- Successful Tool Executions: 6
115
- Successful Complex Queries: 3
116
- Memory Integration Success: True
117
-
118
- --------------------------------------------------------------------------------
119
- API CONNECTIONS
120
- --------------------------------------------------------------------------------
121
- ✅ PASS | openai: Successfully connected to OpenAI API
122
- ✅ PASS | perplexity: Successfully connected to Perplexity API
123
- ⚠️ SKIP | serper: Serper API key not available
124
- ✅ PASS | duckduckgo: Successfully connected to DuckDuckGo
125
- ✅ PASS | arxiv: Successfully connected to arXiv
126
-
127
- ...
128
- ```
129
-
130
- ## Troubleshooting
131
-
132
- ### Missing API Keys
133
-
134
- If you see warnings about missing API keys, ensure that the environment variables are properly set:
135
-
136
- ```bash
137
- export OPENAI_API_KEY=your_openai_api_key
138
- export SERPER_API_KEY=your_serper_api_key
139
- export PERPLEXITY_API_KEY=your_perplexity_api_key
140
- ```
141
-
142
- ### Connection Issues
143
-
144
- If API connection tests fail:
145
- 1. Check your internet connection
146
- 2. Verify that your API keys are valid and not expired
147
- 3. Check if the API service is experiencing downtime
148
-
149
- ### Test Failures
150
-
151
- If specific tests fail:
152
- 1. Check the error message in the report
153
- 2. Look for detailed error logs if running with `--verbose`
154
- 3. Verify that the required dependencies are installed
155
- 4. Check if there are any rate limits or quotas on the APIs
156
-
157
- ## Extending the Test
158
-
159
- To add more test cases or test additional APIs:
160
-
161
- 1. Add new test methods to the `TestGAIAAllAPIsIntegration` class in `tests/test_gaia_all_apis_integration.py`
162
- 2. Update the report generation to include the new test results
163
- 3. Update this documentation to reflect the changes
164
-
165
- ## Related Documentation
166
-
167
- - [OpenAI Integration Testing](openai_integration_testing.md)
168
- - [Serper Integration](serper_integration.md)
169
- - [DuckDuckGo Integration](duckduckgo_integration.md)
170
- - [Multi-Search Integration](multi_search_integration.md)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
docs/configuration.md DELETED
@@ -1,357 +0,0 @@
1
- # GAIA Agent Configuration Guide
2
-
3
- This document provides detailed information about the configuration options available for the GAIA agent. Understanding these options will help you customize the agent's behavior to suit your specific needs.
4
-
5
- ## Configuration Overview
6
-
7
- The GAIA agent uses a hierarchical configuration structure with the following main sections:
8
-
9
- 1. **Model Configuration**: Settings related to the language models used by the agent
10
- 2. **Tools Configuration**: Settings for the various tools used by the agent
11
- 3. **Memory Configuration**: Settings for the agent's memory components
12
- 4. **Agent Configuration**: General settings for the agent's behavior
13
-
14
- ## Configuration Methods
15
-
16
- There are several ways to configure the GAIA agent:
17
-
18
- 1. **Environment Variables**: Set configuration values using environment variables
19
- 2. **Configuration Files**: Use a `.env` file or JSON configuration files
20
- 3. **Direct Configuration**: Pass a configuration dictionary when initializing the agent
21
-
22
- ## Model Configuration
23
-
24
- The model configuration controls which language models are used by the agent and how they behave.
25
-
26
- ```python
27
- model_config = {
28
- "reasoning_model": "gpt-4o", # Model used for reasoning tasks (updated from gpt-4-turbo)
29
- "tool_model": "gpt-3.5-turbo", # Model used for tool execution
30
- "vision_model": "gpt-4o", # Model used for image analysis (updated from gpt-4-vision-preview)
31
- "embedding_model": "text-embedding-3-large", # Model used for embeddings (updated from text-embedding-ada-002)
32
- "temperature": 0.1, # Controls randomness (0.0 to 1.0)
33
- "max_tokens": 4096, # Maximum number of tokens to generate
34
- "top_p": 0.95, # Controls diversity via nucleus sampling
35
- "frequency_penalty": 0.0, # Penalizes repeated tokens
36
- "presence_penalty": 0.0 # Penalizes repeated topics
37
- }
38
- ```
39
-
40
- ### Environment Variables
41
-
42
- ```
43
- OPENAI_API_KEY=your-api-key
44
- REASONING_MODEL=gpt-4o
45
- TOOL_MODEL=gpt-3.5-turbo
46
- VISION_MODEL=gpt-4o
47
- EMBEDDING_MODEL=text-embedding-3-large
48
- MODEL_TEMPERATURE=0.1
49
- MODEL_MAX_TOKENS=4096
50
- ```
51
-
52
- ## Tools Configuration
53
-
54
- The tools configuration controls which tools are available to the agent and how they behave.
55
-
56
- ```python
57
- tools_config = {
58
- "web_search": {
59
- "enabled": True,
60
- "providers": ["duckduckgo", "serper"],
61
- "result_count": 5,
62
- "timeout": 10
63
- },
64
- "web_content": {
65
- "enabled": True,
66
- "max_content_length": 10000,
67
- "timeout": 15
68
- },
69
- "reasoning": {
70
- "enabled": True,
71
- "step_by_step": True,
72
- "verification": True
73
- },
74
- "multimodal": {
75
- "enabled": True,
76
- "image_analysis": True,
77
- "chart_interpretation": True,
78
- "document_parsing": True
79
- },
80
- "arxiv": {
81
- "enabled": True,
82
- "max_results": 5,
83
- "timeout": 10
84
- },
85
- "perplexity": {
86
- "enabled": True,
87
- "model": "sonar-medium-online",
88
- "timeout": 15
89
- }
90
- }
91
- ```
92
-
93
- ### Environment Variables
94
-
95
- ```
96
- ENABLE_WEB_SEARCH=true
97
- SEARCH_PROVIDERS=duckduckgo,serper
98
- SEARCH_RESULT_COUNT=5
99
- SEARCH_TIMEOUT=10
100
- ENABLE_WEB_CONTENT=true
101
- MAX_CONTENT_LENGTH=10000
102
- ENABLE_REASONING=true
103
- ENABLE_MULTIMODAL=true
104
- ENABLE_ARXIV=true
105
- ARXIV_MAX_RESULTS=5
106
- ENABLE_PERPLEXITY=true
107
- PERPLEXITY_MODEL=sonar-medium-online
108
- ```
109
-
110
- ## Memory Configuration
111
-
112
- The memory configuration controls how the agent stores and retrieves information.
113
-
114
- ```python
115
- memory_config = {
116
- "supabase": {
117
- "url": "https://tjamxhvvtnypbadvrkjq.supabase.co",
118
- "key": "your-api-key",
119
- "project_id": "tjamxhvvtnypbadvrkjq",
120
- "project_name": "HF_GAIA_Assessment_AGT",
121
- "table": "gaia_memory"
122
- },
123
- "conversation_memory": {
124
- "enabled": True,
125
- "max_messages": 100,
126
- "table": "gaia_memory_conversations"
127
- },
128
- "result_cache": {
129
- "enabled": True,
130
- "ttl": 3600, # Time-to-live in seconds
131
- "table": "gaia_memory_cache"
132
- },
133
- "working_memory": {
134
- "enabled": True,
135
- "max_items": 50,
136
- "table": "gaia_memory_working"
137
- }
138
- }
139
- ```
140
-
141
- ### Environment Variables
142
-
143
- ```
144
- # Supabase Configuration
145
- SUPABASE_URL=https://tjamxhvvtnypbadvrkjq.supabase.co
146
- SUPABASE_KEY=your-api-key
147
- SUPABASE_PROJECT_ID=tjamxhvvtnypbadvrkjq
148
- SUPABASE_PROJECT_NAME=HF_GAIA_Assessment_AGT
149
- MEMORY_TABLE_NAME=gaia_memory
150
-
151
- # Memory Settings
152
- MEMORY_ENABLED=true
153
- MEMORY_TTL=86400
154
- MEMORY_CACHE_SIZE=100
155
- MEMORY_SIMILARITY_THRESHOLD=0.85
156
- MEMORY_MAX_ENTRIES=1000
157
- ```
158
-
159
- ## Agent Configuration
160
-
161
- The agent configuration controls the general behavior of the agent.
162
-
163
- ```python
164
- agent_config = {
165
- "max_retries": 3, # Maximum number of retries for failed operations
166
- "retry_delay": 1.0, # Delay between retries in seconds
167
- "timeout": 300, # Overall timeout for processing a question
168
- "max_iterations": 10, # Maximum number of iterations in the workflow
169
- "return_error_message": False, # Whether to return error messages or raise exceptions
170
- "verbose": False # Whether to enable verbose logging
171
- }
172
- ```
173
-
174
- ### Environment Variables
175
-
176
- ```
177
- AGENT_MAX_RETRIES=3
178
- AGENT_RETRY_DELAY=1.0
179
- AGENT_TIMEOUT=300
180
- AGENT_MAX_ITERATIONS=10
181
- RETURN_ERROR_MESSAGE=false
182
- VERBOSE=false
183
- ```
184
-
185
- ## Complete Configuration Example
186
-
187
- Here's a complete example of a configuration dictionary that you can pass when initializing the agent:
188
-
189
- ```python
190
- config = {
191
- "model": {
192
- "reasoning_model": "gpt-4o",
193
- "tool_model": "gpt-3.5-turbo",
194
- "vision_model": "gpt-4o",
195
- "embedding_model": "text-embedding-3-large",
196
- "temperature": 0.1,
197
- "max_tokens": 4096
198
- },
199
- "tools": {
200
- "web_search": {
201
- "enabled": True,
202
- "providers": ["duckduckgo", "serper"],
203
- "result_count": 5,
204
- "timeout": 10
205
- },
206
- "web_content": {
207
- "enabled": True,
208
- "max_content_length": 10000,
209
- "timeout": 15
210
- },
211
- "reasoning": {
212
- "enabled": True,
213
- "step_by_step": True,
214
- "verification": True
215
- },
216
- "multimodal": {
217
- "enabled": True,
218
- "image_analysis": True,
219
- "chart_interpretation": True,
220
- "document_parsing": True
221
- },
222
- "arxiv": {
223
- "enabled": True,
224
- "max_results": 5,
225
- "timeout": 10
226
- },
227
- "perplexity": {
228
- "enabled": True,
229
- "model": "sonar-medium-online",
230
- "timeout": 15
231
- }
232
- },
233
- "memory": {
234
- "supabase": {
235
- "url": "https://tjamxhvvtnypbadvrkjq.supabase.co",
236
- "key": "your-api-key",
237
- "project_id": "tjamxhvvtnypbadvrkjq",
238
- "project_name": "HF_GAIA_Assessment_AGT",
239
- "table": "gaia_memory"
240
- },
241
- "conversation_memory": {
242
- "enabled": True,
243
- "max_messages": 100,
244
- "table": "gaia_memory_conversations"
245
- },
246
- "result_cache": {
247
- "enabled": True,
248
- "ttl": 3600,
249
- "table": "gaia_memory_cache"
250
- },
251
- "working_memory": {
252
- "enabled": True,
253
- "max_items": 50,
254
- "table": "gaia_memory_working"
255
- }
256
- },
257
- "agent": {
258
- "max_retries": 3,
259
- "retry_delay": 1.0,
260
- "timeout": 300,
261
- "max_iterations": 10,
262
- "return_error_message": False,
263
- "verbose": False
264
- }
265
- }
266
-
267
- from agent import GAIAAgent
268
- agent = GAIAAgent(config)
269
- ```
270
-
271
- ## Environment Variables File (.env)
272
-
273
- You can also use a `.env` file to configure the agent. Here's an example:
274
-
275
- ```
276
- # API Keys and Endpoints
277
- OPENAI_API_KEY=your-openai-api-key
278
- SERPER_API_KEY=your-serper-api-key
279
- PERPLEXITY_API_KEY=your-perplexity-api-key
280
-
281
- # Supabase Configuration
282
- SUPABASE_URL=https://tjamxhvvtnypbadvrkjq.supabase.co
283
- SUPABASE_KEY=your-supabase-api-key
284
- SUPABASE_PROJECT_ID=tjamxhvvtnypbadvrkjq
285
- SUPABASE_PROJECT_NAME=HF_GAIA_Assessment_AGT
286
-
287
- # Hugging Face Configuration
288
- HF_TOKEN=your-huggingface-token
289
-
290
- # Model Configuration
291
- REASONING_MODEL=gpt-4o
292
- TOOL_MODEL=gpt-3.5-turbo
293
- VISION_MODEL=gpt-4o
294
- EMBEDDING_MODEL=text-embedding-3-large
295
- MODEL_TEMPERATURE=0.1
296
- MODEL_MAX_TOKENS=4096
297
-
298
- # Tools Configuration
299
- ENABLE_WEB_SEARCH=true
300
- SEARCH_PROVIDERS=duckduckgo,serper
301
- SEARCH_RESULT_COUNT=5
302
- ENABLE_WEB_CONTENT=true
303
- ENABLE_REASONING=true
304
- ENABLE_MULTIMODAL=true
305
- ENABLE_ARXIV=true
306
- ENABLE_PERPLEXITY=true
307
-
308
- # Memory Configuration
309
- ENABLE_CONVERSATION_MEMORY=true
310
- ENABLE_RESULT_CACHE=true
311
- ENABLE_WORKING_MEMORY=true
312
-
313
- # Agent Configuration
314
- AGENT_MAX_RETRIES=3
315
- AGENT_TIMEOUT=300
316
- VERBOSE=false
317
- ```
318
-
319
- ## Dependencies
320
-
321
- The GAIA agent relies on several key dependencies:
322
-
323
- 1. **langchain**: Core framework for building language model applications
324
- 2. **langchain_openai**: OpenAI integration for LangChain (separate package)
325
- 3. **langgraph**: Framework for building complex agent workflows
326
- 4. **supabase**: Python client for Supabase
327
- 5. **duckduckgo-search**: Client for DuckDuckGo search
328
- 6. **perplexity-python**: Client for Perplexity AI
329
- 7. **arxiv**: Client for arXiv search
330
-
331
- Make sure these dependencies are installed and up-to-date:
332
-
333
- ```bash
334
- pip install --upgrade langchain langchain_openai langgraph supabase duckduckgo-search perplexity-python arxiv
335
- ```
336
-
337
- > **Note**: The `langchain_openai` package is now separate from the core `langchain` package. If you encounter import errors, make sure you have the latest version installed.
338
-
339
- ## Configuration Precedence
340
-
341
- The agent uses the following precedence order when determining configuration values:
342
-
343
- 1. Direct configuration passed to the constructor
344
- 2. Environment variables
345
- 3. Default values defined in the code
346
-
347
- This means that values passed directly to the constructor will override environment variables, which in turn override the default values.
348
-
349
- ## Troubleshooting
350
-
351
- If you encounter configuration issues, check the [Troubleshooting Guide](troubleshooting.md) for common problems and solutions.
352
-
353
- ## Next Steps
354
-
355
- For usage examples, see [Usage Examples](examples.md).
356
-
357
- For deployment instructions, see [Deployment Guide](deployment.md).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
docs/connection_verification.md DELETED
@@ -1,114 +0,0 @@
1
- # Connection Verification Guide
2
-
3
- This guide explains how to verify connections to external services used by the GAIA agent.
4
-
5
- ## Overview
6
-
7
- The GAIA agent relies on several external services:
8
-
9
- 1. **OpenAI API** - For language model access
10
- 2. **Perplexity API** - For web search and reasoning
11
- 3. **Supabase** - For memory storage
12
- 4. **Serper API** - For web search
13
-
14
- Proper connection to these services is essential for the agent to function correctly.
15
-
16
- ## Verification Tools
17
-
18
- The project includes several tools to verify connections:
19
-
20
- 1. `tests/verify_connections.py` - A comprehensive test script that verifies all connections
21
- 2. `direct_config_check.py` - A simple script to verify configuration values
22
-
23
- ## Verifying All Connections
24
-
25
- To verify all connections at once, run:
26
-
27
- ```bash
28
- python -m tests.verify_connections
29
- ```
30
-
31
- This script will:
32
- - Check if all required API keys are set
33
- - Test connections to each service
34
- - Report success or failure for each connection
35
-
36
- ## Verifying Individual Connections
37
-
38
- ### OpenAI API
39
-
40
- To verify the OpenAI API connection:
41
-
42
- ```bash
43
- python -m tests.verify_connections --service openai
44
- ```
45
-
46
- ### Perplexity API
47
-
48
- To verify the Perplexity API connection:
49
-
50
- ```bash
51
- python -m tests.verify_connections --service perplexity
52
- ```
53
-
54
- You can also use the dedicated Perplexity test:
55
-
56
- ```bash
57
- python -m tests.test_perplexity_sonar
58
- ```
59
-
60
- ### Supabase
61
-
62
- To verify the Supabase connection:
63
-
64
- ```bash
65
- python -m tests.verify_connections --service supabase
66
- ```
67
-
68
- ### Serper API
69
-
70
- To verify the Serper API connection:
71
-
72
- ```bash
73
- python -m tests.verify_connections --service serper
74
- ```
75
-
76
- ## Troubleshooting Connection Issues
77
-
78
- ### API Key Issues
79
-
80
- If you encounter API key issues:
81
-
82
- 1. Check that the key is correctly set in your `.env` file
83
- 2. Verify the key format (no extra spaces or characters)
84
- 3. Ensure the key has not expired
85
- 4. Check if the service has usage limits that you've exceeded
86
-
87
- ### Network Issues
88
-
89
- If you encounter network issues:
90
-
91
- 1. Check your internet connection
92
- 2. Verify that your firewall or network settings allow connections to the service
93
- 3. Check if the service is experiencing downtime
94
-
95
- ### Configuration Issues
96
-
97
- If you encounter configuration issues:
98
-
99
- 1. Run `python direct_config_check.py` to verify configuration values
100
- 2. Check that the configuration in `agent/config.py` is correct
101
- 3. Ensure environment variables are properly loaded
102
-
103
- ## Setting Up API Keys
104
-
105
- All API keys should be stored in a `.env` file in the project root:
106
-
107
- ```
108
- OPENAI_API_KEY=your_openai_key
109
- PERPLEXITY_API_KEY=your_perplexity_key
110
- SERPER_API_KEY=your_serper_key
111
- SUPABASE_KEY=your_supabase_key
112
- ```
113
-
114
- Never commit this file to version control.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
docs/deployment.md DELETED
@@ -1,303 +0,0 @@
1
- # GAIA Agent Deployment Guide
2
-
3
- This document provides instructions for deploying the GAIA agent in various environments, from local development to production settings.
4
-
5
- ## Prerequisites
6
-
7
- Before deploying the GAIA agent, ensure you have:
8
-
9
- 1. Python 3.8 or higher installed
10
- 2. All required dependencies installed
11
- 3. API keys for any external services (OpenAI, Serper, Supabase, etc.)
12
- 4. Sufficient resources for your expected workload
13
-
14
- ## Local Deployment
15
-
16
- ### Installation
17
-
18
- 1. Clone the repository:
19
- ```bash
20
- git clone https://github.com/yourusername/gaia-agent.git
21
- cd gaia-agent
22
- ```
23
-
24
- 2. Create a virtual environment:
25
- ```bash
26
- python -m venv venv
27
- source venv/bin/activate # On Windows: venv\Scripts\activate
28
- ```
29
-
30
- 3. Install dependencies:
31
- ```bash
32
- pip install -r requirements.txt
33
- ```
34
-
35
- 4. Create a `.env` file with your configuration:
36
- ```
37
- OPENAI_API_KEY=your-openai-api-key
38
- SERPER_API_KEY=your-serper-api-key
39
- SUPABASE_URL=https://your-project-url.supabase.co
40
- SUPABASE_KEY=your-supabase-api-key
41
- ```
42
-
43
- 5. Run the application:
44
- ```bash
45
- python app.py
46
- ```
47
-
48
- ### Testing
49
-
50
- Before deploying to production, run the test suite to ensure everything is working correctly:
51
-
52
- ```bash
53
- # Run all tests
54
- python -m unittest discover tests
55
-
56
- # Run benchmark tests
57
- python -m tests.benchmark --output benchmark_results.json
58
- ```
59
-
60
- ## Deployment Options
61
-
62
- ### Hugging Face Spaces
63
-
64
- The GAIA agent is designed to be easily deployed to Hugging Face Spaces:
65
-
66
- 1. Create a new Space on Hugging Face:
67
- - Go to https://huggingface.co/spaces
68
- - Click "Create new Space"
69
- - Select "Gradio" as the SDK
70
- - Choose a name for your Space
71
-
72
- 2. Configure your Space:
73
- - Add your API keys as secrets in the Space settings
74
- - Set the Python version to 3.8 or higher
75
-
76
- 3. Push your code to the Space:
77
- ```bash
78
- git remote add space https://huggingface.co/spaces/yourusername/your-space-name
79
- git push space main
80
- ```
81
-
82
- 4. Your agent will be automatically deployed and available at `https://huggingface.co/spaces/yourusername/your-space-name`
83
-
84
- ### Docker Deployment
85
-
86
- For containerized deployment, you can use Docker:
87
-
88
- 1. Create a Dockerfile:
89
- ```dockerfile
90
- FROM python:3.9-slim
91
-
92
- WORKDIR /app
93
-
94
- COPY requirements.txt .
95
- RUN pip install --no-cache-dir -r requirements.txt
96
-
97
- COPY . .
98
-
99
- CMD ["python", "app.py"]
100
- ```
101
-
102
- 2. Build the Docker image:
103
- ```bash
104
- docker build -t gaia-agent .
105
- ```
106
-
107
- 3. Run the container:
108
- ```bash
109
- docker run -p 7860:7860 --env-file .env gaia-agent
110
- ```
111
-
112
- ### Cloud Deployment
113
-
114
- #### AWS Elastic Beanstalk
115
-
116
- 1. Install the EB CLI:
117
- ```bash
118
- pip install awsebcli
119
- ```
120
-
121
- 2. Initialize your EB application:
122
- ```bash
123
- eb init -p python-3.8 gaia-agent
124
- ```
125
-
126
- 3. Create an environment:
127
- ```bash
128
- eb create gaia-agent-env
129
- ```
130
-
131
- 4. Configure environment variables:
132
- ```bash
133
- eb setenv OPENAI_API_KEY=your-openai-api-key SERPER_API_KEY=your-serper-api-key
134
- ```
135
-
136
- 5. Deploy your application:
137
- ```bash
138
- eb deploy
139
- ```
140
-
141
- #### Google Cloud Run
142
-
143
- 1. Build and push your Docker image to Google Container Registry:
144
- ```bash
145
- gcloud builds submit --tag gcr.io/your-project-id/gaia-agent
146
- ```
147
-
148
- 2. Deploy to Cloud Run:
149
- ```bash
150
- gcloud run deploy gaia-agent \
151
- --image gcr.io/your-project-id/gaia-agent \
152
- --platform managed \
153
- --set-env-vars OPENAI_API_KEY=your-openai-api-key,SERPER_API_KEY=your-serper-api-key
154
- ```
155
-
156
- ## Scaling Considerations
157
-
158
- ### Memory Usage
159
-
160
- The GAIA agent can be memory-intensive, especially when processing complex questions or using large language models. Consider the following:
161
-
162
- - Monitor memory usage during benchmark testing
163
- - Adjust `max_tokens` and other model parameters to control memory usage
164
- - Consider using a memory cache for frequently asked questions
165
-
166
- ### API Rate Limits
167
-
168
- Be aware of rate limits for external APIs:
169
-
170
- - OpenAI has rate limits based on your subscription tier
171
- - Serper and other search APIs typically have daily or monthly quotas
172
- - Implement retry logic with exponential backoff for rate limit errors
173
-
174
- ### Cost Management
175
-
176
- Using external APIs can incur costs:
177
-
178
- - Monitor your API usage and costs
179
- - Consider caching results to reduce API calls
180
- - Use smaller or less expensive models for simpler tasks
181
-
182
- ## Production Best Practices
183
-
184
- ### Logging
185
-
186
- Implement comprehensive logging:
187
-
188
- ```python
189
- import logging
190
-
191
- logging.basicConfig(
192
- level=logging.INFO,
193
- format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
194
- filename='gaia_agent.log'
195
- )
196
- ```
197
-
198
- ### Monitoring
199
-
200
- Set up monitoring for:
201
-
202
- - API response times
203
- - Error rates
204
- - Memory and CPU usage
205
- - Cost metrics
206
-
207
- ### Security
208
-
209
- Protect sensitive information:
210
-
211
- - Never hardcode API keys in your code
212
- - Use environment variables or a secure secrets manager
213
- - Implement proper authentication for your API endpoints
214
- - Validate and sanitize all user inputs
215
-
216
- ### Backup and Recovery
217
-
218
- Implement backup and recovery procedures:
219
-
220
- - Regularly backup your Supabase database
221
- - Create snapshots of your deployment environment
222
- - Document recovery procedures
223
-
224
- ## Continuous Integration/Continuous Deployment (CI/CD)
225
-
226
- Set up CI/CD pipelines to automate testing and deployment:
227
-
228
- ### GitHub Actions Example
229
-
230
- Create a `.github/workflows/main.yml` file:
231
-
232
- ```yaml
233
- name: CI/CD Pipeline
234
-
235
- on:
236
- push:
237
- branches: [ main ]
238
- pull_request:
239
- branches: [ main ]
240
-
241
- jobs:
242
- test:
243
- runs-on: ubuntu-latest
244
- steps:
245
- - uses: actions/checkout@v2
246
- - name: Set up Python
247
- uses: actions/setup-python@v2
248
- with:
249
- python-version: '3.9'
250
- - name: Install dependencies
251
- run: |
252
- python -m pip install --upgrade pip
253
- pip install -r requirements.txt
254
- - name: Run tests
255
- run: |
256
- python -m unittest discover tests
257
-
258
- deploy:
259
- needs: test
260
- if: github.event_name == 'push' && github.ref == 'refs/heads/main'
261
- runs-on: ubuntu-latest
262
- steps:
263
- - uses: actions/checkout@v2
264
- - name: Deploy to Hugging Face Spaces
265
- uses: huggingface/huggingface-spaces-deploy-action@main
266
- with:
267
- space_id: ${{ secrets.SPACE_ID }}
268
- token: ${{ secrets.HF_TOKEN }}
269
- ```
270
-
271
- ## Troubleshooting
272
-
273
- ### Common Issues
274
-
275
- 1. **API Key Errors**:
276
- - Ensure all API keys are correctly set in your environment variables
277
- - Check for typos or trailing spaces in your API keys
278
-
279
- 2. **Memory Issues**:
280
- - Reduce the `max_tokens` parameter in your model configuration
281
- - Implement memory cleanup after processing each question
282
-
283
- 3. **Timeout Errors**:
284
- - Increase the timeout values in your configuration
285
- - Optimize your agent's workflow to reduce processing time
286
-
287
- 4. **Rate Limit Errors**:
288
- - Implement retry logic with exponential backoff
289
- - Consider upgrading your API subscription tier
290
-
291
- ### Getting Help
292
-
293
- If you encounter issues not covered in this guide:
294
-
295
- 1. Check the project's GitHub issues for similar problems
296
- 2. Consult the documentation for the specific components (LangChain, LangGraph, etc.)
297
- 3. Reach out to the community for assistance
298
-
299
- ## Next Steps
300
-
301
- For usage examples, see [Usage Examples](examples.md).
302
-
303
- For configuration options, see [Configuration Guide](configuration.md).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
docs/duckduckgo_integration.md DELETED
@@ -1,156 +0,0 @@
1
- # DuckDuckGo Integration for GAIA Agent
2
-
3
- This document describes how to set up and use the DuckDuckGo search integration with the GAIA agent.
4
-
5
- ## Overview
6
-
7
- The GAIA agent now supports DuckDuckGo as an alternative search engine for web searches. This integration allows the agent to perform web searches without requiring an API key, making it a good fallback option when other search providers are unavailable or rate-limited.
8
-
9
- ## Features
10
-
11
- - **No API Key Required**: DuckDuckGo search works without requiring an API key
12
- - **Configurable Result Count**: Control how many search results are returned
13
- - **Error Handling**: Robust error handling for various search scenarios
14
- - **Standardized Results**: Results are formatted consistently with other search providers
15
-
16
- ## Installation
17
-
18
- The DuckDuckGo integration requires the `duckduckgo-search` package:
19
-
20
- ```bash
21
- pip install duckduckgo-search
22
- ```
23
-
24
- ## Configuration
25
-
26
- The DuckDuckGo search tool can be configured in the GAIA agent's configuration:
27
-
28
- ```python
29
- # In agent/config.py
30
- class GaiaConfig:
31
- def __init__(self):
32
- # ...
33
- self.duckduckgo_enabled = True
34
- self.duckduckgo_result_count = 5
35
- # ...
36
- ```
37
-
38
- ### Configuration Options
39
-
40
- | Option | Description | Default |
41
- |--------|-------------|---------|
42
- | `duckduckgo_enabled` | Enable/disable DuckDuckGo search | `True` |
43
- | `duckduckgo_result_count` | Number of results to return | `5` |
44
- | `duckduckgo_timeout` | Timeout in seconds for search requests | `30` |
45
-
46
- ## Usage
47
-
48
- ### Basic Usage
49
-
50
- ```python
51
- from tools.web_tools import create_duckduckgo_search
52
-
53
- # Create a DuckDuckGo search tool
54
- search_tool = create_duckduckgo_search()
55
-
56
- # Perform a search
57
- results = search_tool.search("python programming")
58
-
59
- # Process the results
60
- for result in results:
61
- print(f"Title: {result['title']}")
62
- print(f"URL: {result['link']}")
63
- print(f"Snippet: {result['snippet']}")
64
- print("---")
65
- ```
66
-
67
- ### Integration with GAIA Agent
68
-
69
- The DuckDuckGo search tool is automatically integrated with the GAIA agent if enabled in the configuration:
70
-
71
- ```python
72
- from agent.agent import GaiaAgent
73
- from agent.config import GaiaConfig
74
-
75
- # Create a configuration with DuckDuckGo enabled
76
- config = GaiaConfig()
77
- config.duckduckgo_enabled = True
78
-
79
- # Create a GAIA agent with the configuration
80
- agent = GaiaAgent(config=config)
81
-
82
- # The agent will now use DuckDuckGo for web searches
83
- response = agent.process("What is the latest version of Python?")
84
- ```
85
-
86
- ## Result Format
87
-
88
- The DuckDuckGo search tool returns results in the following format:
89
-
90
- ```json
91
- [
92
- {
93
- "title": "Result title",
94
- "link": "https://example.com/result",
95
- "snippet": "A brief excerpt from the search result..."
96
- },
97
- // More results...
98
- ]
99
- ```
100
-
101
- ## Fallback Mechanism
102
-
103
- The GAIA agent can be configured to use DuckDuckGo as a fallback when other search providers fail:
104
-
105
- ```python
106
- # In agent/config.py
107
- class GaiaConfig:
108
- def __init__(self):
109
- # ...
110
- self.search_fallback_order = ["serper", "duckduckgo", "perplexity"]
111
- # ...
112
- ```
113
-
114
- With this configuration, the agent will try Serper first, then DuckDuckGo if Serper fails, and finally Perplexity if both fail.
115
-
116
- ## Testing
117
-
118
- You can test the DuckDuckGo integration using the provided test scripts:
119
-
120
- ```bash
121
- # Test just the DuckDuckGo search
122
- python tests/test_duckduckgo_search.py
123
-
124
- # Test all search providers together
125
- python tests/test_gaia_multi_search_providers.py
126
- ```
127
-
128
- ## Troubleshooting
129
-
130
- ### Common Issues
131
-
132
- 1. **No Results Returned**: DuckDuckGo may return no results for certain queries. Try reformulating the query.
133
-
134
- 2. **Connection Errors**: If you're experiencing connection errors, check your internet connection and ensure that DuckDuckGo is accessible from your network.
135
-
136
- 3. **Rate Limiting**: While DuckDuckGo doesn't require an API key, it may still rate-limit excessive requests. Consider adding delays between searches if you're making many requests.
137
-
138
- ### Debugging
139
-
140
- To enable debug logging for the DuckDuckGo search tool:
141
-
142
- ```python
143
- import logging
144
- logging.basicConfig(level=logging.DEBUG)
145
- logger = logging.getLogger("tools.web_tools")
146
- ```
147
-
148
- ## Limitations
149
-
150
- - DuckDuckGo search may not provide as many results as commercial search APIs
151
- - Some advanced search features (like filtering by date) may not be available
152
- - Results may be less comprehensive than those from paid search providers
153
-
154
- ## Conclusion
155
-
156
- The DuckDuckGo integration provides a reliable, no-cost alternative for web searches in the GAIA agent. It's particularly useful as a fallback option or for development and testing purposes.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
docs/enhanced_web_search.md DELETED
@@ -1,299 +0,0 @@
1
- # Enhanced Web Search
2
-
3
- The Enhanced Web Search system provides a comprehensive three-tier approach to web search, intelligently routing queries to the most appropriate search tools based on query type and available resources. This system is designed to handle a wide range of search needs, from simple factual queries to complex reasoning questions.
4
-
5
- ## Overview
6
-
7
- The Enhanced Web Search system consists of three specialized search tools:
8
-
9
- 1. **BrowserSearchTool** - For direct browser-based searches of any website
10
- 2. **LibrarySearchTool** - For searches using imported libraries like arXiv and DuckDuckGo
11
- 3. **ApiSearchTool** - For searches using API services like Perplexity and Serper
12
-
13
- These tools work together in a fallback chain, ensuring that if one tool fails or is unavailable, the system can fall back to another tool. This approach provides robust search capabilities that can adapt to different environments and query types.
14
-
15
- ## Three-Tier Search Approach
16
-
17
- ### 1. BrowserSearchTool
18
-
19
- The BrowserSearchTool enables direct browser-based searches of any website using the browser_action capability.
20
-
21
- **Purpose:**
22
- - Directly interact with websites through a browser interface
23
- - Access content that may be difficult to retrieve through APIs or libraries
24
- - Provide visual context for search results
25
-
26
- **Advantages:**
27
- - Can access any website, including those without APIs
28
- - Provides visual context and can interact with dynamic content
29
- - Works with a wide range of websites without requiring API keys
30
-
31
- **Use Cases:**
32
- - Searching websites that don't have public APIs
33
- - Accessing content behind login screens (with proper credentials)
34
- - Interacting with dynamic web content
35
- - Visual verification of search results
36
-
37
- **Example:**
38
-
39
- ```python
40
- from tools.web_tools import create_browser_search
41
-
42
- # Create the browser search tool
43
- browser_tool = create_browser_search()
44
-
45
- # Search Wikipedia
46
- results = browser_tool.search("quantum computing", source="wikipedia")
47
-
48
- # The results include instructions for using browser_action
49
- print(results[0]["instructions"])
50
- ```
51
-
52
- ### 2. LibrarySearchTool
53
-
54
- The LibrarySearchTool leverages imported Python libraries (DuckDuckGo and arXiv) for search operations, intelligently routing queries to the appropriate library based on query type.
55
-
56
- **Purpose:**
57
- - Use established Python libraries for search operations
58
- - Provide specialized academic search capabilities
59
- - Offer general web search without requiring API keys
60
-
61
- **Advantages:**
62
- - No API keys required
63
- - Specialized academic search through arXiv
64
- - General web search through DuckDuckGo
65
- - Works offline with locally installed libraries
66
-
67
- **Use Cases:**
68
- - Academic research and paper searches
69
- - General web searches when API keys aren't available
70
- - Environments with limited internet access but installed libraries
71
-
72
- **Example:**
73
-
74
- ```python
75
- from tools.web_tools import create_library_search
76
-
77
- # Create the library search tool
78
- library_tool = create_library_search()
79
-
80
- # For academic queries, it automatically uses arXiv
81
- academic_results = library_tool.search("recent papers on quantum computing")
82
-
83
- # For general queries, it automatically uses DuckDuckGo
84
- general_results = library_tool.search("weather in New York")
85
- ```
86
-
87
- ### 3. ApiSearchTool
88
-
89
- The ApiSearchTool uses external API services (Perplexity and Serper) for search operations, intelligently selecting between them based on the query type.
90
-
91
- **Purpose:**
92
- - Leverage powerful commercial search APIs
93
- - Provide high-quality results for complex queries
94
- - Offer fast and reliable search capabilities
95
-
96
- **Advantages:**
97
- - High-quality results for complex queries through Perplexity
98
- - Fast and reliable search through Serper's Google search API
99
- - Structured and consistent results
100
-
101
- **Use Cases:**
102
- - Complex, reasoning-based queries that benefit from Perplexity's capabilities
103
- - Factual and simple queries that can be handled by Serper
104
- - Production environments with API access
105
-
106
- **Example:**
107
-
108
- ```python
109
- from tools.web_tools import create_api_search
110
-
111
- # Create the API search tool
112
- api_tool = create_api_search()
113
-
114
- # For complex queries, it automatically uses Perplexity
115
- complex_results = api_tool.search("Explain quantum computing in detail and its potential impact on cryptography")
116
-
117
- # For simple queries, it automatically uses Serper
118
- simple_results = api_tool.search("weather in New York")
119
- ```
120
-
121
- ## Fallback Chain
122
-
123
- The three search tools work together in a fallback chain, ensuring robust search capabilities even when some tools are unavailable or fail. The EnhancedWebSearchTool manages this fallback chain.
124
-
125
- **Fallback Process:**
126
-
127
- 1. First, try the most appropriate tool based on the query type
128
- 2. If that tool fails or is unavailable, try the next most appropriate tool
129
- 3. Continue until a successful result is found or all tools have been tried
130
-
131
- **Example:**
132
-
133
- ```python
134
- from tools.web_tools import create_enhanced_web_search
135
-
136
- # Create the enhanced web search tool
137
- enhanced_tool = create_enhanced_web_search()
138
-
139
- # Search for information
140
- # This will automatically select the most appropriate tool
141
- # and fall back to others if needed
142
- results = enhanced_tool.search("Who nominated the only Featured Article on English Wikipedia about a dinosaur that was promoted in November 2016?")
143
- ```
144
-
145
- ## Query Type Detection
146
-
147
- Each tool includes specialized methods to detect specific types of queries:
148
-
149
- - **BrowserSearchTool**: Detects website-specific queries (e.g., "wikipedia article about...")
150
- - **LibrarySearchTool**: Detects academic queries for arXiv vs. general queries for DuckDuckGo
151
- - **ApiSearchTool**: Detects complex queries for Perplexity vs. simple queries for Serper
152
-
153
- These detection methods ensure that queries are routed to the most appropriate search tool.
154
-
155
- ## Implementation Details
156
-
157
- ### BrowserSearchTool
158
-
159
- The BrowserSearchTool provides search templates for common websites:
160
-
161
- ```python
162
- self.search_templates = {
163
- "wikipedia": "https://en.wikipedia.org/wiki/Special:Search?search={query}",
164
- "arxiv": "https://arxiv.org/search/?query={query}&searchtype=all",
165
- "google": "https://www.google.com/search?q={query}",
166
- "youtube": "https://www.youtube.com/results?search_query={query}",
167
- # Additional templates...
168
- }
169
- ```
170
-
171
- It can detect the most appropriate source based on the query content and provides specific instructions for using browser_action with each source.
172
-
173
- ### LibrarySearchTool
174
-
175
- The LibrarySearchTool includes methods to detect academic queries:
176
-
177
- ```python
178
- def _is_academic_query(self, query: str) -> bool:
179
- # Check for academic keywords, fields, and citation patterns
180
- # ...
181
- return has_academic_keyword or has_academic_field or has_citation_pattern
182
- ```
183
-
184
- It uses the DuckDuckGo library for general queries and the arXiv library for academic queries, formatting the results consistently.
185
-
186
- ### ApiSearchTool
187
-
188
- The ApiSearchTool includes methods to detect complex queries:
189
-
190
- ```python
191
- def _is_complex_query(self, query: str) -> bool:
192
- # Check for question words, complex indicators, and query length
193
- # ...
194
- return has_question_word or has_complex_indicator or is_long_query
195
- ```
196
-
197
- It uses the Perplexity API for complex queries and the Serper API for simple queries, formatting the results consistently.
198
-
199
- ## Configuration
200
-
201
- Each tool can be configured through environment variables and configuration files:
202
-
203
- ### ApiSearchTool
204
-
205
- - **PERPLEXITY_API_KEY**: API key for Perplexity
206
- - **SERPER_API_KEY**: API key for Serper
207
-
208
- ### LibrarySearchTool
209
-
210
- - No API keys required, but the DuckDuckGo and arXiv libraries must be installed
211
-
212
- ### BrowserSearchTool
213
-
214
- - No API keys required, but browser_action capability must be available
215
-
216
- ## Usage
217
-
218
- ### Basic Usage
219
-
220
- ```python
221
- from tools.web_tools import create_enhanced_web_search
222
-
223
- # Create the enhanced web search tool
224
- enhanced_tool = create_enhanced_web_search()
225
-
226
- # Search for information
227
- results = enhanced_tool.search("Who nominated the only Featured Article on English Wikipedia about a dinosaur that was promoted in November 2016?")
228
-
229
- # Process the results
230
- for result in results:
231
- print(f"Title: {result.get('title')}")
232
- print(f"Snippet: {result.get('snippet')}")
233
- print(f"Source: {result.get('source')}")
234
- print(f"Relevance: {result.get('relevance_score')}")
235
- print("---")
236
- ```
237
-
238
- ### Using with the Tool Registry
239
-
240
- ```python
241
- from agent.tool_registry import create_default_registry, unified_search
242
-
243
- # Create the registry
244
- registry = create_default_registry()
245
-
246
- # Search using unified_search
247
- results = unified_search(registry, "How many studio albums were published by Mercedes Sosa between 2000 and 2009?")
248
-
249
- # Process the results
250
- for result in results["results"]:
251
- print(f"Title: {result.get('title')}")
252
- print(f"Snippet: {result.get('snippet')}")
253
- print(f"Source: {result.get('source')}")
254
- print(f"Relevance: {result.get('relevance_score')}")
255
- print("---")
256
- ```
257
-
258
- ## Testing
259
-
260
- You can test the Enhanced Web Search system using the provided test scripts:
261
-
262
- ```bash
263
- # Run all tests
264
- python tests/run_gaia_web_search_tests.py
265
-
266
- # Run only direct tests (without registry)
267
- python tests/run_gaia_web_search_tests.py --direct-only
268
-
269
- # Run only registry tests
270
- python tests/run_gaia_web_search_tests.py --registry-only
271
-
272
- # Enable verbose logging
273
- python tests/run_gaia_web_search_tests.py --verbose
274
- ```
275
-
276
- ## Examples
277
-
278
- See the following example scripts for complete examples of how to use the Enhanced Web Search system:
279
-
280
- - `examples/enhanced_web_search_example.py` - Example of using the EnhancedWebSearchTool
281
- - `examples/multi_search_example.py` - Example of using multiple search tools together
282
-
283
- ## Requirements
284
-
285
- - Perplexity API key (set as `PERPLEXITY_API_KEY` environment variable) for ApiSearchTool
286
- - Serper API key (set as `SERPER_API_KEY` environment variable) for ApiSearchTool
287
- - DuckDuckGo and arXiv libraries installed for LibrarySearchTool
288
- - Browser_action capability for BrowserSearchTool
289
-
290
- ## Conclusion
291
-
292
- The Enhanced Web Search system provides a robust, flexible approach to web search through its three-tier architecture. By intelligently routing queries to the most appropriate search tools and implementing a fallback chain, it ensures reliable search capabilities across a wide range of environments and query types.
293
-
294
- This system significantly improves search performance by leveraging the strengths of each search approach:
295
- - Direct browser-based searches for website-specific content
296
- - Library-based searches for academic and general queries without API keys
297
- - API-based searches for high-quality results on complex queries
298
-
299
- Together, these tools provide a comprehensive solution for web search needs in various contexts and environments.