diff --git a/README_original.md b/README_original.md deleted file mode 100644 index 22d94bb5530f2035dbdc04c8e3e66a8067f2bc5a..0000000000000000000000000000000000000000 --- a/README_original.md +++ /dev/null @@ -1,294 +0,0 @@ -# GAIA Agent - Hugging Face Agents Course Final Project - -## Project Overview - -This project implements an intelligent agent capable of answering questions from the GAIA (Grounded AI Alignment) benchmark as part of the Hugging Face Agents Course final assignment. The agent leverages LangChain/LangGraph frameworks and various tools to solve complex, multi-step reasoning tasks. - -## What is GAIA? - -GAIA is a benchmark designed to evaluate AI assistants on real-world tasks that require a combination of core capabilities: -- Reasoning across multiple steps -- Multimodal understanding (text, images, etc.) -- Web browsing and information retrieval -- Proficient tool use - -The benchmark consists of 466 carefully curated questions that are conceptually simple for humans (~92% success rate) but challenging for current AI systems (GPT-4 with plugins: ~15% success rate). - -## Challenge Requirements - -- **Task**: Create an agent that answers questions from a subset of GAIA (20 questions from level 1) -- **Success Criteria**: Score at least 30% on the benchmark -- **Evaluation Method**: Exact match comparison with ground truth answers -- **Response Format**: Answers should be concise (number, few words, or comma-separated list) - -## Technical Architecture - -Our implementation uses: - -1. **LangChain/LangGraph**: For creating a flexible, modular agent architecture -2. **Supabase**: For memory management and persistence -3. **Web Tools**: Integration with search engines (DuckDuckGo, Serper), academic resources (arXiv), and AI assistants (Perplexity) -4. **Gradio**: For the user interface and submission process - -## Project Structure - -``` -├── README.md # Project documentation -├── app.py # Main application file with Gradio interface -├── requirements.txt # Project dependencies -├── agent/ # Agent implementation -│ ├── __init__.py # Package initialization -│ ├── agent.py # Main agent class -│ ├── config.py # Configuration settings -│ ├── graph.py # LangGraph workflow definition -│ └── tool_registry.py # Tool registration and management -├── tools/ # Tool implementations -│ ├── __init__.py # Package initialization -│ ├── web_tools.py # Web browsing and search tools -│ ├── reasoning_tools.py # Reasoning and analysis tools -│ ├── multimodal_tools.py # Tools for handling images and other media -│ ├── arxiv_tool.py # Tools for searching academic papers -│ └── perplexity_tool.py # Integration with Perplexity AI -├── memory/ # Memory management -│ ├── __init__.py # Package initialization -│ ├── supabase_memory.py # Supabase integration for memory -│ ├── supabase_memory_patch.py # Patches for Supabase memory -│ └── simplified_memory_fix.py # Simplified memory implementation -└── utils/ # Utility functions - ├── __init__.py # Package initialization - ├── parsing.py # Functions for parsing questions and answers - └── formatting.py # Formatting utilities -``` - -## Implementation Approach - -Our agent follows a multi-step process to answer GAIA questions: - -1. **Question Analysis**: Parse and understand the question, identifying required tools and information sources -2. **Planning**: Develop a step-by-step plan to answer the question -3. **Tool Execution**: Use appropriate tools in the correct sequence -4. **Answer Formulation**: Extract and format the precise answer from the results -5. **Verification**: Validate the answer format and content before submission - -## Key Features - -- **Modular Architecture**: Easily extensible with new tools and capabilities -- **Robust Error Handling**: Graceful recovery from tool failures or unexpected results -- **Efficient Tool Selection**: Smart selection of the most appropriate tools for each question -- **Answer Precision**: Focus on extracting exact answers that match the ground truth format -- **Memory Integration**: Persistent memory across sessions using Supabase - -## Getting Started - -### Prerequisites - -- Python 3.8 or higher -- OpenAI API key -- Supabase account and API key (for memory features) -- Optional: Serper API key (for enhanced search) -- Optional: Perplexity API key (for enhanced reasoning) - -### Installation - -1. Clone this repository -2. Install dependencies: - ```bash - pip install -r requirements.txt - ``` - - > **Note**: If you encounter issues with the `langchain_openai` dependency, make sure you have the latest version installed: - > ```bash - > pip install --upgrade langchain_openai - > ``` - -3. Set up environment variables (create a `.env` file in the project root): - ``` - OPENAI_API_KEY=your_openai_api_key - SUPABASE_KEY=your_supabase_api_key - SERPER_API_KEY=your_serper_api_key (optional) - PERPLEXITY_API_KEY=your_perplexity_api_key (optional) - ``` - -4. Run the application: - ```bash - python app.py - ``` - -5. Log in with your Hugging Face account -6. Click "Run Evaluation & Submit All Answers" to test the agent - -## Testing - -The project includes comprehensive unit tests for all components: - -- **Agent Tests**: Test the GAIAAgent class and its workflow -- **Tools Tests**: Test the web, reasoning, and multimodal tools -- **Memory Tests**: Test the memory components and Supabase integration -- **Utils Tests**: Test utility functions for parsing and formatting -- **Integration Tests**: Test integration with external APIs like OpenAI and Perplexity - -To run all tests: -```bash -python -m unittest discover tests -``` - -To run specific test files: -```bash -python -m unittest tests.test_agent -python -m unittest tests.test_tools -python -m unittest tests.test_memory -python -m unittest tests.test_utils -``` - -### OpenAI Integration Testing - -The project includes specialized tests for OpenAI integration with newer models: - -```bash -# Test OpenAI integration with newer models -python tests/test_openai_integration.py - -# Test OpenAI integration with GAIA agent -python tests/test_openai_integration.py --test-agent - -# Test GAIA agent with multiple AI providers (OpenAI and Perplexity) -python tests/test_gaia_multi_provider.py -``` - -For convenience, you can run all OpenAI integration tests at once: - -```bash -# Run all OpenAI integration tests -python tests/run_openai_tests.py - -# Run all tests with verbose logging -python tests/run_openai_tests.py --verbose - -# Run all tests and save results to a custom directory -python tests/run_openai_tests.py --output my_test_results - -# Stop running tests after the first failure -python tests/run_openai_tests.py --stop-on-failure -``` - -These tests verify that: -- The OpenAI API connection works correctly -- Newer models like GPT-4o can be used successfully -- The GAIA agent can use both OpenAI and Perplexity tools together - -### API Integration Testing - -To test all API integrations together: - -```bash -# Run all API integration tests -python tests/run_all_apis_integration_test.py - -# Run with verbose logging -python tests/run_all_apis_integration_test.py --verbose - -# Specify a custom output file -python tests/run_all_apis_integration_test.py --report-file my_report.json -``` - -### Supabase Integration Testing - -To test the Supabase memory integration: - -```bash -# Run all Supabase integration tests -python tests/run_supabase_integration_tests.py - -# Run with verbose logging -python tests/run_supabase_integration_tests.py --verbose - -# Specify a custom output directory -python tests/run_supabase_integration_tests.py --output-dir custom_docs -``` - -## Evaluation - -The agent is evaluated on its ability to provide exact matches to the ground truth answers for the GAIA benchmark questions. The submission process includes: - -- Fetching questions from the course API -- Running the agent on each question -- Submitting answers for scoring -- Viewing results on the leaderboard - -## Benchmark Testing - -The project includes a benchmark testing framework to evaluate the agent's performance: - -```bash -# Run benchmark tests with default settings -python -m tests.benchmark --output benchmark_results.json - -# Run benchmark tests with custom questions -python -m tests.benchmark --questions custom_questions.json --output benchmark_results.json - -# Run benchmark tests with a specific number of questions -python -m tests.benchmark --max-questions 5 --output benchmark_results.json -``` - -The benchmark testing framework: -- Tests the agent against sample questions -- Measures performance metrics (accuracy, latency, resource usage) -- Generates detailed reports of benchmark results -- Allows comparison of different agent configurations - -## Troubleshooting - -### Common Issues - -#### 1. langchain_openai Dependency Issues - -If you encounter errors related to the `langchain_openai` package, such as: -- `ImportError: No module named 'langchain_openai'` -- `AttributeError: module 'langchain_openai' has no attribute 'ChatOpenAI'` - -Try the following solutions: -- Ensure you have the latest version installed: `pip install --upgrade langchain_openai` -- Make sure you're using the correct import: `from langchain_openai import ChatOpenAI` -- Check for conflicts with older langchain versions: `pip uninstall langchain-openai` (note the hyphen) - -#### 2. API Connection Issues - -If you're having trouble connecting to external APIs: -- Verify that your API keys are correctly set in the environment variables -- Check that the API service is not experiencing downtime -- Ensure you have a stable internet connection -- Check for rate limiting or quota issues - -#### 3. Supabase Connection Issues - -If you're having trouble with Supabase: -- Verify that your Supabase URL and API key are correctly set -- Ensure that the required tables exist in your Supabase project -- Check that your API key has the necessary permissions -- Verify that your Supabase project is active - -#### 4. Memory Issues - -If you're experiencing issues with the agent's memory: -- Check that memory is enabled in the configuration -- Verify that the table names are correctly set -- Ensure that the TTL and cache size settings are appropriate -- Check for errors in the Supabase logs - -## Documentation - -Comprehensive documentation is available in the `docs/` directory: - -- [Usage Examples](docs/examples.md): Examples of how to use the GAIA agent in different scenarios -- [Configuration Guide](docs/configuration.md): Detailed information about configuration options -- [Deployment Guide](docs/deployment.md): Instructions for deploying the agent in various environments -- [OpenAI Integration Testing](docs/openai_integration_testing.md): Guide for testing OpenAI integration with newer models -- [Connection Verification](docs/connection_verification.md): Guide for verifying connections to external services -- [Supabase Memory Integration](docs/supabase_memory_integration.md): Guide for setting up and using Supabase memory -- [All APIs Integration Testing](docs/all_apis_integration_testing.md): Guide for testing all API integrations together -- [Multi-Search Integration](docs/multi_search_integration.md): Guide for using multiple search providers -- [GAIA Memory Integration](docs/gaia_memory_integration.md): Guide for using memory in the GAIA agent - -## License - -This project is part of the Hugging Face Agents Course and follows its licensing terms. \ No newline at end of file diff --git a/SUPABASE_TEST_README.md b/SUPABASE_TEST_README.md deleted file mode 100644 index 0aa93f674711a9ca12dd66ed9e5670b9674b6be0..0000000000000000000000000000000000000000 --- a/SUPABASE_TEST_README.md +++ /dev/null @@ -1,104 +0,0 @@ -# Standalone Supabase Connection Test - -This directory contains scripts for testing the Supabase connection without relying on any project modules. - -## Main Script: `supabase_direct_test.py` - -This script tests the connection to Supabase, verifies access to all required tables, and performs basic CRUD operations on the test_connection table. - -### Prerequisites - -- Python 3.6+ -- Required packages: - - supabase - - python-dotenv - -You can install the required packages with: - -```bash -pip install supabase python-dotenv -``` - -### Usage - -You can run the script in several ways: - -1. Using environment variables: - -```bash -# Set environment variables -export SUPABASE_URL=https://your-project-id.supabase.co -export SUPABASE_KEY=your-supabase-api-key - -# Run the test -python supabase_direct_test.py -``` - -2. Using command-line arguments: - -```bash -python supabase_direct_test.py --key your-supabase-api-key -``` - -3. Using a .env file: - -Create a `.env` file in the same directory with the following content: - -``` -SUPABASE_URL=https://your-project-id.supabase.co -SUPABASE_KEY=your-supabase-api-key -``` - -Then run the script: - -```bash -python supabase_direct_test.py -``` - -### Options - -- `--verbose`: Enable verbose logging -- `--url`: Specify the Supabase URL (overrides environment variable) -- `--key`: Specify the Supabase API key (overrides environment variable) - -Example with verbose logging: - -```bash -python supabase_direct_test.py --verbose --key your-supabase-api-key -``` - -## Helper Scripts - -The repository also includes several helper scripts for troubleshooting: - -- `check_supabase_env.py`: Checks if the required Supabase environment variables are set -- `supabase_import_test.py`: Tests importing the Supabase library -- `supabase_init_test.py`: Tests initializing the Supabase client -- `supabase_minimal_test.py`: Minimal test for Supabase connection -- `create_env_and_test.py`: Creates a .env file and tests the connection - -## Tables Tested - -The script verifies connection to the following tables: - -1. `gaia_memory`: Main memory table -2. `gaia_memory_conversations`: Conversation history table -3. `gaia_memory_cache`: Cache table -4. `gaia_memory_working`: Working memory table -5. `test_connection`: Test table for connection verification - -If any of these tables don't exist, you can create them by running the SQL statements in `supabase_tables.sql`. - -## Troubleshooting - -If you encounter issues: - -1. Check that your Supabase API key is correct -2. Verify that the Supabase URL is correct -3. Ensure that the required tables exist in your Supabase project -4. Check network connectivity to Supabase -5. Run with the `--verbose` flag for more detailed error information - -## Security Note - -Never hardcode your Supabase API key in your scripts. Always use environment variables, .env files, or command-line arguments to provide the key securely. \ No newline at end of file diff --git a/agent/__init__.py b/agent/__init__.py deleted file mode 100644 index b1bbbdf4c5f28fcbf3a98484360db16feb97c673..0000000000000000000000000000000000000000 --- a/agent/__init__.py +++ /dev/null @@ -1,22 +0,0 @@ -""" -Agent module for GAIA implementation. -Contains the core agent architecture and configuration. -""" - -from agent.agent import GAIAAgent -from agent.config import ( - get_logging_config, - get_model_config, - get_tool_config, - get_memory_config, - get_agent_config -) - -__all__ = [ - 'GAIAAgent', - 'get_logging_config', - 'get_model_config', - 'get_tool_config', - 'get_memory_config', - 'get_agent_config' -] \ No newline at end of file diff --git a/agent/__pycache__/__init__.cpython-312.pyc b/agent/__pycache__/__init__.cpython-312.pyc deleted file mode 100644 index 3efa0538cd811d1abfebe1f0ef83dc541ce6d890..0000000000000000000000000000000000000000 Binary files a/agent/__pycache__/__init__.cpython-312.pyc and /dev/null differ diff --git a/agent/__pycache__/__init__.cpython-313.pyc b/agent/__pycache__/__init__.cpython-313.pyc deleted file mode 100644 index 889631e210f0972fe8ae56944d8a7a7ed63577ce..0000000000000000000000000000000000000000 Binary files a/agent/__pycache__/__init__.cpython-313.pyc and /dev/null differ diff --git a/agent/__pycache__/agent.cpython-312.pyc b/agent/__pycache__/agent.cpython-312.pyc deleted file mode 100644 index 860d92beaa2db988dc310d03de249e29ae3aa471..0000000000000000000000000000000000000000 Binary files a/agent/__pycache__/agent.cpython-312.pyc and /dev/null differ diff --git a/agent/__pycache__/agent.cpython-313.pyc b/agent/__pycache__/agent.cpython-313.pyc deleted file mode 100644 index 2662d643a752bae28e019f762ee8fba4e2ae392c..0000000000000000000000000000000000000000 Binary files a/agent/__pycache__/agent.cpython-313.pyc and /dev/null differ diff --git a/agent/__pycache__/assessment_analyzer.cpython-313.pyc b/agent/__pycache__/assessment_analyzer.cpython-313.pyc deleted file mode 100644 index 9eae94278d60ff5cee5b1f2af9e55bb378595882..0000000000000000000000000000000000000000 Binary files a/agent/__pycache__/assessment_analyzer.cpython-313.pyc and /dev/null differ diff --git a/agent/__pycache__/config.cpython-312.pyc b/agent/__pycache__/config.cpython-312.pyc deleted file mode 100644 index 91411752eeb530ead18ee753d0857d01de20b5be..0000000000000000000000000000000000000000 Binary files a/agent/__pycache__/config.cpython-312.pyc and /dev/null differ diff --git a/agent/__pycache__/config.cpython-313.pyc b/agent/__pycache__/config.cpython-313.pyc deleted file mode 100644 index 8fef98574e815f1ece056e9cf638493a891dd9c7..0000000000000000000000000000000000000000 Binary files a/agent/__pycache__/config.cpython-313.pyc and /dev/null differ diff --git a/agent/__pycache__/graph.cpython-312.pyc b/agent/__pycache__/graph.cpython-312.pyc deleted file mode 100644 index 608ccaec854ce78e2c89090d0222f43b582389da..0000000000000000000000000000000000000000 Binary files a/agent/__pycache__/graph.cpython-312.pyc and /dev/null differ diff --git a/agent/__pycache__/graph.cpython-313.pyc b/agent/__pycache__/graph.cpython-313.pyc deleted file mode 100644 index 09a2ef131f1269bf1b2896a4651c60342a23c1f4..0000000000000000000000000000000000000000 Binary files a/agent/__pycache__/graph.cpython-313.pyc and /dev/null differ diff --git a/agent/__pycache__/tool_registry.cpython-313.pyc b/agent/__pycache__/tool_registry.cpython-313.pyc deleted file mode 100644 index 9c70314159e82b2ff05bdc428ed99829c1d7bcc4..0000000000000000000000000000000000000000 Binary files a/agent/__pycache__/tool_registry.cpython-313.pyc and /dev/null differ diff --git a/agent/agent.py b/agent/agent.py deleted file mode 100644 index cb1f65798d38495e9a7dcdaf29d7ac457884c10c..0000000000000000000000000000000000000000 --- a/agent/agent.py +++ /dev/null @@ -1,291 +0,0 @@ -""" -Main agent implementation for the GAIA benchmark. - -This module contains the GAIAAgent class which is responsible for: -- Processing questions from the GAIA benchmark -- Selecting and executing appropriate tools -- Formulating precise answers -- Handling errors and logging - -The agent uses LangGraph for workflow management and Supabase for memory, -with configuration from the config module. It can be extended with -additional capabilities as needed. -""" - -import logging -import time -import traceback -import uuid -from typing import Dict, Any, Optional, List, Union, Tuple - -from agent.config import ( - get_logging_config, - get_model_config, - get_tool_config, - get_memory_config, - get_agent_config, - VERBOSE -) - -# Import LangGraph workflow -from agent.graph import run_agent_graph - -from memory import SupabaseMemory, ConversationMemory, ResultCache, WorkingMemory - -logging_config = get_logging_config() -logging.basicConfig( - level=logging_config["level"], - format=logging_config["format"], - filename=logging_config["filename"] -) -logger = logging.getLogger("gaia_agent") - -class GAIAAgent: - """ - Agent for answering questions from the GAIA benchmark. - - This agent processes questions, selects appropriate tools, - executes a reasoning process, and formulates precise answers. - """ - - def __init__(self, config: Optional[Dict[str, Any]] = None): - """ - Initialize the GAIA agent with configuration. - - Args: - config (Optional[Dict[str, Any]]): Custom configuration dictionary. - If None, the default configuration from config.py is used. - """ - logger.info("Initializing GAIAAgent") - - self.config = config or { - "model": get_model_config(), - "tools": get_tool_config(), - "memory": get_memory_config(), - "agent": get_agent_config() - } - - if VERBOSE: - logger.setLevel(logging.DEBUG) - if not any(isinstance(h, logging.StreamHandler) for h in logger.handlers): - console_handler = logging.StreamHandler() - console_handler.setLevel(logging.DEBUG) - console_handler.setFormatter(logging.Formatter(logging_config["format"])) - logger.addHandler(console_handler) - - self._initialize_memory() - - self.reset_state() - - logger.info("GAIAAgent initialized successfully") - - def _initialize_memory(self) -> None: - """Initialize memory components.""" - - try: - self.memory = SupabaseMemory(self.config["memory"]) - - self.session_id = str(uuid.uuid4()) - - self.conversation_memory = ConversationMemory(self.memory, self.session_id) - self.result_cache = ResultCache(self.memory) - self.working_memory = WorkingMemory(self.memory, self.session_id) - - except Exception as e: - logger.error(f"Error initializing memory components: {str(e)}") - logger.warning("Continuing without persistent memory") - - self.memory = None - self.conversation_memory = None - self.result_cache = None - self.working_memory = None - - def reset_state(self) -> None: - """Reset the agent's internal state.""" - self.state = { - "current_question": None, - "tools_used": [], - "intermediate_results": [], - "start_time": None, - "end_time": None, - "error": None - } - - def __call__(self, question: str) -> str: - """ - Process a question and return an answer. - - This is the main entry point for the agent. It takes a question, - processes it, and returns a concise answer. - - Args: - question (str): The question to answer - - Returns: - str: The answer to the question - - Raises: - Exception: If an error occurs during processing - """ - self.reset_state() - self.state["current_question"] = question - self.state["start_time"] = time.time() - - logger.info(f"Processing question: {question[:100]}...") - - try: - max_retries = self.config["agent"]["max_retries"] - retry_delay = self.config["agent"]["retry_delay"] - - for attempt in range(max_retries): - try: - answer = self._process_question(question) - - self.state["end_time"] = time.time() - processing_time = self.state["end_time"] - self.state["start_time"] - logger.info(f"Question processed in {processing_time:.2f} seconds. Answer: {answer[:100]}...") - - return answer - - except Exception as e: - if attempt < max_retries - 1: - logger.warning(f"Attempt {attempt + 1}/{max_retries} failed: {str(e)}. Retrying in {retry_delay} seconds...") - time.sleep(retry_delay) - else: - raise - - except Exception as e: - self.state["error"] = str(e) - self.state["end_time"] = time.time() - processing_time = self.state["end_time"] - self.state["start_time"] - - logger.error(f"Error processing question after {processing_time:.2f} seconds: {str(e)}") - logger.error(traceback.format_exc()) - - if self.config["agent"].get("return_error_message", False): - return f"Error: {str(e)}" - else: - raise - - def _process_question(self, question: str) -> str: - """ - Internal method to process a question and generate an answer. - - This method implements the core logic of the agent using LangGraph: - 1. Analyze the question - 2. Plan the approach - 3. Execute tools - 4. Formulate the answer - - Args: - question (str): The question to process - - Returns: - str: The answer to the question - """ - - if self.result_cache: - import hashlib - cache_key = f"answer:{hashlib.md5(question.encode()).hexdigest()}" - - cached_answer = self.result_cache.get_result(cache_key) - if cached_answer: - logger.info(f"Using cached answer for question: {question[:30]}...") - return cached_answer - - graph_config = { - "recursion_limit": self.config["agent"].get("max_iterations", 10), - "timeout": self.config["agent"].get("timeout", 300) - } - - try: - result = run_agent_graph(question, graph_config) - - if result.get("answer"): - answer = result["answer"] - elif result.get("error"): - raise Exception(result["error"]) - else: - answer = f"Unable to answer: {question[:30]}..." - - if self.result_cache: - self.result_cache.cache_result(cache_key, answer) - - if self.working_memory: - if result.get("analysis"): - self.working_memory.store_intermediate_result( - "analysis", result["analysis"] - ) - - if result.get("plan"): - self.working_memory.store_intermediate_result( - "plan", result["plan"] - ) - - if result.get("tool_results"): - self.working_memory.store_intermediate_result( - "tool_results", result["tool_results"] - ) - - if self.conversation_memory: - self.conversation_memory.add_message("user", question) - self.conversation_memory.add_message("assistant", answer) - - return answer - except Exception as e: - logger.error(f"Error running LangGraph workflow: {str(e)}") - raise - - def get_memory_snapshot(self) -> Dict[str, Any]: - """ - Get a snapshot of the agent's memory. - - This method returns a dictionary containing the current state of the - agent's memory, including conversation history, cached results, and - intermediate results. - - Returns: - Dict[str, Any]: Memory snapshot - """ - snapshot = { - "conversation": [], - "cached_results": {}, - "intermediate_results": {} - } - - if self.conversation_memory: - snapshot["conversation"] = self.conversation_memory.get_messages() - - if self.result_cache and self.memory: - cache_keys = self.memory.list_keys("cache:") - for key in cache_keys: - original_key = key[6:] # Remove "cache:" prefix - - result = self.memory.retrieve(key) - - snapshot["cached_results"][original_key] = result - - if self.working_memory: - snapshot["intermediate_results"] = self.working_memory.list_intermediate_results() - - return snapshot - - def clear_memory(self) -> None: - """ - Clear the agent's memory. - - This method clears all memory components, including conversation history, - cached results, and intermediate results. - """ - logger.info("Clearing agent memory") - - if self.conversation_memory: - self.conversation_memory.clear() - - if self.result_cache: - self.result_cache.clear() - - if self.working_memory: - self.working_memory.clear() - - logger.info("Agent memory cleared") \ No newline at end of file diff --git a/agent/assessment_analyzer.py b/agent/assessment_analyzer.py deleted file mode 100644 index ec19dcfead5dbdd042a047a391bc2d82c9cbb2e5..0000000000000000000000000000000000000000 --- a/agent/assessment_analyzer.py +++ /dev/null @@ -1,306 +0,0 @@ -""" -Assessment analyzer for the GAIA agent. - -This module provides functionality to analyze assessment results, -categorize questions, calculate success rates, identify error patterns, -and generate recommendations for further improvements. -""" - -import logging -from typing import Dict, Any, List, Optional, Set, Tuple -import re -from collections import Counter, defaultdict - -logger = logging.getLogger("gaia_agent.assessment_analyzer") - -class AnalysisResult: - """Class to store assessment analysis results.""" - - def __init__( - self, - category: str, - success_rate: float, - successful_questions: Dict[str, Dict[str, Any]], - failed_questions: Dict[str, Dict[str, Any]], - error_patterns: Dict[str, int], - recommendations: List[str] - ): - """ - Initialize the analysis result. - - Args: - category: The category of questions analyzed - success_rate: The success rate (0.0 to 1.0) - successful_questions: Dictionary of successfully answered questions - failed_questions: Dictionary of failed questions - error_patterns: Dictionary of error patterns and their frequencies - recommendations: List of recommendations for improvement - """ - self.category = category - self.success_rate = success_rate - self.successful_questions = successful_questions - self.failed_questions = failed_questions - self.error_patterns = error_patterns - self.recommendations = recommendations - - def __str__(self) -> str: - """Return a string representation of the analysis result.""" - return ( - f"Analysis Result for {self.category} questions:\n" - f"Success Rate: {self.success_rate:.2%}\n" - f"Successful Questions: {len(self.successful_questions)}\n" - f"Failed Questions: {len(self.failed_questions)}\n" - f"Top Error Patterns: {dict(sorted(self.error_patterns.items(), key=lambda x: x[1], reverse=True)[:3])}\n" - f"Recommendations: {self.recommendations}" - ) - -def categorize_question(question_id: str, question_data: Dict[str, Any]) -> str: - """ - Categorize a question based on its content and ID. - - Args: - question_id: The ID of the question - question_data: The question data - - Returns: - The category of the question - """ - question_text = question_data.get("question", "").lower() - - # Web search questions - web_search_keywords = [ - "mercedes sosa", "wikipedia", "featured article", "malko competition", - "yankees", "olympics", "universe today", "polish tv actor" - ] - - # Multimodal questions - multimodal_keywords = [ - "video", "youtube", "watch?v=", "audio", "listen", "chess position", "image" - ] - - # File processing questions - file_processing_keywords = [ - "excel", "spreadsheet", "python code", "execute", "run this code" - ] - - # Academic paper questions - academic_paper_keywords = [ - "arxiv", "paper", "scientific", "research", "publication", "journal" - ] - - # Mathematical reasoning questions - math_keywords = [ - "math", "calculate", "commutativity", "table", "operation", "subset" - ] - - # Check for web search questions - for keyword in web_search_keywords: - if keyword in question_text: - return "web_search" - - # Check for multimodal questions - for keyword in multimodal_keywords: - if keyword in question_text: - return "multimodal" - - # Check for file processing questions - for keyword in file_processing_keywords: - if keyword in question_text: - return "file_processing" - - # Check for academic paper questions - for keyword in academic_paper_keywords: - if keyword in question_text: - return "academic_paper" - - # Check for mathematical reasoning questions - for keyword in math_keywords: - if keyword in question_text: - return "mathematical_reasoning" - - # Default to "other" if no category matches - return "other" - -def identify_error_patterns(failed_questions: Dict[str, Dict[str, Any]]) -> Dict[str, int]: - """ - Identify common error patterns in failed questions. - - Args: - failed_questions: Dictionary of failed questions - - Returns: - Dictionary of error patterns and their frequencies - """ - error_patterns = Counter() - - for question_id, question_data in failed_questions.items(): - error = question_data.get("error", "") - answer = question_data.get("answer", "") - expected_answer = question_data.get("expected_answer", "") - - # Check for specific error types - if error: - # Check for timeout-related errors - if any(term in error.lower() for term in ["timeout", "timed out", "time limit"]): - error_patterns["timeout"] += 1 - # Check for API-related errors - elif any(term in error.lower() for term in ["api", "rate limit", "quota"]): - error_patterns["api_error"] += 1 - # Check for unhashable type errors - elif "unhashable" in error.lower(): - error_patterns["unhashable_type"] += 1 - else: - error_patterns["other_error"] += 1 - # Check for incorrect answers - elif answer and expected_answer and answer != expected_answer: - # Check for partial matches - if any(word in answer.lower() for word in expected_answer.lower().split()): - error_patterns["partial_match"] += 1 - else: - error_patterns["completely_wrong_answer"] += 1 - else: - error_patterns["unknown_failure"] += 1 - - return error_patterns - -def generate_recommendations( - category: str, - success_rate: float, - error_patterns: Dict[str, int], - failed_questions: Dict[str, Dict[str, Any]] -) -> List[str]: - """ - Generate recommendations for improvement based on analysis. - - Args: - category: The category of questions - success_rate: The success rate - error_patterns: Dictionary of error patterns and their frequencies - failed_questions: Dictionary of failed questions - - Returns: - List of recommendations - """ - recommendations = [] - - # General recommendations based on success rate - if success_rate < 0.3: - recommendations.append(f"Major improvements needed for {category} questions") - elif success_rate < 0.7: - recommendations.append(f"Moderate improvements needed for {category} questions") - elif success_rate < 1.0: - recommendations.append(f"Minor improvements needed for {category} questions") - else: - recommendations.append(f"All {category} questions are answered correctly") - - # Specific recommendations based on error patterns - if error_patterns.get("timeout", 0) > 0: - recommendations.append("Optimize search performance to prevent timeouts") - - if error_patterns.get("api_error", 0) > 0: - recommendations.append("Improve error handling for API failures") - - if error_patterns.get("unhashable_type", 0) > 0: - recommendations.append("Fix unhashable type errors in the agent implementation") - - if error_patterns.get("partial_match", 0) > 0: - recommendations.append("Improve answer extraction precision") - - if error_patterns.get("completely_wrong_answer", 0) > 0: - recommendations.append("Enhance search result relevance filtering") - - # Category-specific recommendations - if category == "web_search": - if success_rate < 1.0: - recommendations.append("Implement specialized search for specific domains (Wikipedia, news, etc.)") - recommendations.append("Improve result ranking based on query relevance") - recommendations.append("Add better error handling for failed searches") - - elif category == "multimodal": - if success_rate < 1.0: - recommendations.append("Enhance video content extraction capabilities") - recommendations.append("Improve audio transcription accuracy") - recommendations.append("Add specialized image analysis for specific domains") - - elif category == "file_processing": - if success_rate < 1.0: - recommendations.append("Improve Excel file parsing and data extraction") - recommendations.append("Enhance code execution sandbox for better isolation") - - elif category == "academic_paper": - if success_rate < 1.0: - recommendations.append("Improve arXiv paper content extraction") - recommendations.append("Add support for more academic paper sources") - - elif category == "mathematical_reasoning": - if success_rate < 1.0: - recommendations.append("Enhance mathematical operation analysis") - recommendations.append("Improve structured data parsing for tables and matrices") - - return recommendations - -def analyze_assessment_results( - assessment_results: Dict[str, Dict[str, Any]], - category: Optional[str] = None -) -> AnalysisResult: - """ - Analyze assessment results and generate insights. - - Args: - assessment_results: Dictionary of assessment results - category: Optional category to filter questions by - - Returns: - AnalysisResult object with analysis results - """ - # Categorize questions - categorized_questions = defaultdict(dict) - - for question_id, question_data in assessment_results.items(): - question_category = categorize_question(question_id, question_data) - categorized_questions[question_category][question_id] = question_data - - # If a specific category is requested, filter questions - if category: - questions_to_analyze = categorized_questions.get(category, {}) - category_name = category - else: - questions_to_analyze = assessment_results - category_name = "all" - - # Separate successful and failed questions - successful_questions = {} - failed_questions = {} - - for question_id, question_data in questions_to_analyze.items(): - if question_data.get("correct", False): - successful_questions[question_id] = question_data - else: - failed_questions[question_id] = question_data - - # Calculate success rate - total_questions = len(questions_to_analyze) - successful_count = len(successful_questions) - - success_rate = successful_count / total_questions if total_questions > 0 else 0.0 - - # Identify error patterns - error_patterns = identify_error_patterns(failed_questions) - - # Generate recommendations - recommendations = generate_recommendations( - category_name, - success_rate, - error_patterns, - failed_questions - ) - - # Create and return analysis result - return AnalysisResult( - category=category_name, - success_rate=success_rate, - successful_questions=successful_questions, - failed_questions=failed_questions, - error_patterns=error_patterns, - recommendations=recommendations - ) \ No newline at end of file diff --git a/agent/config.py b/agent/config.py deleted file mode 100644 index 79e21b017f8b0e2c5dd2fbb48bc189073c1efcea..0000000000000000000000000000000000000000 --- a/agent/config.py +++ /dev/null @@ -1,187 +0,0 @@ -""" -Configuration module for the GAIA agent. - -This module centralizes all configuration settings for the agent, including: -- API endpoints -- Model selection -- Tool parameters -- Memory settings -- Logging configuration -- Web browsing settings -- Multimodal processing parameters - -It uses python-dotenv to load environment variables for sensitive credentials. -""" - -import os -import logging -from typing import Dict, Any, Optional, List, Union, Tuple - -try: - from dotenv import load_dotenv - load_dotenv(verbose=False) -except ImportError: - def load_dotenv(*args, **kwargs): - pass # No-op implementation when dotenv is not available - -OPENAI_API_KEY = os.getenv("OPENAI_API_KEY", "") -SERPER_API_KEY = os.getenv("SERPER_API_KEY", "") -PERPLEXITY_API_KEY = os.getenv("PERPLEXITY_API_KEY", "") -SUPABASE_KEY = os.getenv("SUPABASE_KEY", "") -HF_TOKEN = os.getenv("HF_TOKEN", "") - -# Supabase Configuration (partially hardcoded) -SUPABASE_URL = "https://tjamxhvvtnypbadvrkjq.supabase.co" -SUPABASE_PROJECT_ID = "tjamxhvvtnypbadvrkjq" -SUPABASE_PROJECT_NAME = "HF_GAIA_Assessment_AGT" - -OPENAI_API_BASE = "https://api.openai.com/v1" -SERPER_API_URL = "https://google.serper.dev/search" -PERPLEXITY_API_URL = "https://api.perplexity.ai" -HUGGINGFACE_API_URL = "https://api-inference.huggingface.co/models" - -# Model Configuration (hardcoded with updated models) -DEFAULT_MODEL = "gpt-4o" # Updated from gpt-4-turbo -EMBEDDING_MODEL = "text-embedding-3-large" # Updated from text-embedding-ada-002 -FALLBACK_MODEL = "gpt-3.5-turbo" -VISION_MODEL = "gpt-4o" # Updated from gpt-4-vision-preview -REASONING_MODEL = "gpt-4o" # Updated from gpt-4-turbo -MODEL_TEMPERATURE = 0.1 -MODEL_MAX_TOKENS = 4096 - -# Tool Parameters (hardcoded) -WEB_SEARCH_RESULT_COUNT = 5 -WEB_SEARCH_TIMEOUT = 10 -ARXIV_MAX_RESULTS = 3 -DUCKDUCKGO_TIMEOUT = 10 -DUCKDUCKGO_MAX_RESULTS = 5 -PERPLEXITY_TIMEOUT = 30 -PERPLEXITY_MODEL = "sonar-reasoning" -WEB_SCRAPING_TIMEOUT = 15 -WEB_SCRAPING_MAX_LINKS = 3 -WEB_SCRAPING_MAX_CONTENT_LENGTH = 10000 - -# Memory Settings (hardcoded) -MEMORY_ENABLED = True -MEMORY_TABLE_NAME = "gaia_memory" -MEMORY_TTL = 86400 # 24 hours in seconds -MEMORY_CACHE_SIZE = 100 -MEMORY_SIMILARITY_THRESHOLD = 0.85 -MEMORY_MAX_ENTRIES = 1000 - -# Agent Settings (hardcoded) -MAX_ITERATIONS = 10 -VERBOSE = False -TIMEOUT = 300 # 5 minutes in seconds -MAX_RETRIES = 3 -RETRY_DELAY = 2 # seconds -PARALLEL_TOOL_EXECUTION = False - -# Logging Configuration (hardcoded) -LOG_LEVEL = "INFO" -LOG_FILE = "gaia_agent.log" -LOG_FORMAT = "%(asctime)s - %(name)s - %(levelname)s - %(message)s" - -# Multimodal Settings (hardcoded) -IMAGE_MAX_SIZE = 1024 # pixels -IMAGE_FORMAT = "JPEG" -VISION_DETAIL_LEVEL = "high" # low, medium, high - -# Web Browsing Settings (hardcoded) -USER_AGENT = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36" -BROWSER_TIMEOUT = 30 -BROWSER_HEADLESS = True -BROWSER_IMAGES_ENABLED = True - -def get_logging_config() -> Dict[str, Any]: - """ - Returns a dictionary with logging configuration. - - Returns: - Dict[str, Any]: Logging configuration dictionary - """ - return { - "level": getattr(logging, LOG_LEVEL.upper(), logging.INFO), - "format": LOG_FORMAT, - "filename": LOG_FILE if LOG_FILE else None, - } - -def get_model_config() -> Dict[str, Any]: - """ - Returns a dictionary with model configuration. - - Returns: - Dict[str, Any]: Model configuration dictionary - """ - return { - "default_model": DEFAULT_MODEL, - "embedding_model": EMBEDDING_MODEL, - "fallback_model": FALLBACK_MODEL, - "vision_model": VISION_MODEL, - "reasoning_model": REASONING_MODEL, - "temperature": MODEL_TEMPERATURE, - "max_tokens": MODEL_MAX_TOKENS, - } - -def get_tool_config() -> Dict[str, Dict[str, Any]]: - """ - Returns a dictionary with tool configurations. - - Returns: - Dict[str, Dict[str, Any]]: Tool configuration dictionary - """ - return { - "web_search": { - "result_count": WEB_SEARCH_RESULT_COUNT, - "timeout": WEB_SEARCH_TIMEOUT, - }, - "arxiv": { - "max_results": ARXIV_MAX_RESULTS, - }, - "duckduckgo": { - "timeout": DUCKDUCKGO_TIMEOUT, - "max_results": DUCKDUCKGO_MAX_RESULTS, - }, - "perplexity": { - "timeout": PERPLEXITY_TIMEOUT, - "model": PERPLEXITY_MODEL, - "api_url": PERPLEXITY_API_URL, - }, - "web_scraping": { - "timeout": WEB_SCRAPING_TIMEOUT, - "max_links": WEB_SCRAPING_MAX_LINKS, - "max_content_length": WEB_SCRAPING_MAX_CONTENT_LENGTH, - }, - } - -def get_memory_config() -> Dict[str, Any]: - """ - Returns a dictionary with memory configuration. - - Returns: - Dict[str, Any]: Memory configuration dictionary - """ - return { - "enabled": MEMORY_ENABLED, - "table_name": MEMORY_TABLE_NAME, - "ttl": MEMORY_TTL, - "cache_size": MEMORY_CACHE_SIZE, - "similarity_threshold": MEMORY_SIMILARITY_THRESHOLD, - "max_entries": MEMORY_MAX_ENTRIES, - } - -def get_agent_config() -> Dict[str, Any]: - """ - Returns a dictionary with agent configuration. - - Returns: - Dict[str, Any]: Agent configuration dictionary - """ - return { - "max_iterations": MAX_ITERATIONS, - "verbose": VERBOSE, - "timeout": TIMEOUT, - "max_retries": MAX_RETRIES, - "retry_delay": RETRY_DELAY, - "parallel_tool_execution": PARALLEL_TOOL_EXECUTION, - } \ No newline at end of file diff --git a/agent/graph.py b/agent/graph.py deleted file mode 100644 index ac2331bf22ee27ad88549f0a4f7f115f5cfd8abc..0000000000000000000000000000000000000000 --- a/agent/graph.py +++ /dev/null @@ -1,703 +0,0 @@ -""" -LangGraph workflow definition for the GAIA agent. - -This module defines the workflow graph for the GAIA agent using LangGraph. -It implements nodes for question analysis, planning, tool selection, -tool execution, and answer formulation, as well as the edges between them. - -The graph is designed to be integrated with the GAIAAgent class and can be -extended with additional nodes and edges as needed. -""" - -import logging -import json -import re -import traceback -import openai -from typing import Dict, Any, List, Tuple, Optional, Annotated, TypedDict, Literal, Union, cast - -from langchain.prompts import PromptTemplate -from langchain_core.messages import HumanMessage, AIMessage -from langchain_core.runnables import RunnableConfig -from langchain_core.output_parsers import StrOutputParser - -from langgraph.graph import StateGraph, END - -from agent.config import get_model_config, get_agent_config, VERBOSE - -logger = logging.getLogger("gaia_agent.graph") - -class QuestionAnalysis(TypedDict): - """Type for question analysis results.""" - question_type: str - required_tools: List[str] - information_sources: List[str] - complexity: str - -class PlanStep(TypedDict): - """Type for a single step in the plan.""" - step_number: int - description: str - tool: Optional[str] - tool_input: Optional[Dict[str, Any]] - -class ToolResult(TypedDict): - """Type for tool execution results.""" - tool_name: str - success: bool - result: Any - error: Optional[str] - -class AgentState(TypedDict): - """Type for the agent state.""" - question: str - analysis: Optional[QuestionAnalysis] - plan: Optional[List[PlanStep]] - current_step: int - tools_to_use: Optional[List[str]] - tool_results: List[ToolResult] - intermediate_thoughts: List[str] - answer: Optional[str] - error: Optional[str] - check_count: int # Counter to prevent infinite loops - -def extract_json_from_text(text: str) -> str: - """ - Extract JSON from text that might contain additional content. - - Args: - text: Text that might contain JSON - - Returns: - Extracted JSON string or original text if no JSON found - """ - # Try to find JSON-like content using regex - json_pattern = r'(\{[\s\S]*\})' - json_matches = re.findall(json_pattern, text) - - if json_matches: - # Return the first match that is valid JSON - for match in json_matches: - try: - json.loads(match) - return match - except: - continue - - # If no valid JSON found in matches, return the original text - return text - -def call_openai_api(prompt: str, model_config: Dict[str, Any]) -> str: - """ - Call the OpenAI API directly instead of using langchain_openai.ChatOpenAI. - - Args: - prompt: The prompt to send to the API - model_config: Configuration for the model - - Returns: - The response from the API - """ - try: - response = openai.ChatCompletion.create( - model=model_config["reasoning_model"], - messages=[{"role": "user", "content": prompt}], - temperature=model_config["temperature"], - max_tokens=model_config["max_tokens"] - ) - return response.choices[0].message.content - except Exception as e: - logger.error(f"Error calling OpenAI API: {str(e)}") - raise - -def analyze_question(state: AgentState) -> AgentState: - """ - Analyze the question to determine its requirements. - - Args: - state: The current agent state - - Returns: - Updated agent state with question analysis - """ - - model_config = get_model_config() - - analysis_prompt = PromptTemplate.from_template( - """You are an expert question analyzer for the GAIA benchmark. - - Your task is to analyze the following question and determine: - 1. The question type (factual, mathematical, reasoning, etc.) - 2. The tools required to answer it (web_search, calculator, reasoning, etc.) - 3. The information sources needed (web, knowledge base, computation, etc.) - 4. The complexity level (simple, medium, complex) - - Question: {question} - - Provide your analysis in the following JSON format: - {{ - "question_type": "type of question", - "required_tools": ["tool1", "tool2", ...], - "information_sources": ["source1", "source2", ...], - "complexity": "complexity level" - }} - - IMPORTANT: Your response must be valid JSON. Do not include any text before or after the JSON. - - Analysis:""" - ) - - try: - # Use the prompt template to format the prompt - formatted_prompt = analysis_prompt.format(question=state["question"]) - - # Call OpenAI API directly - analysis_str = call_openai_api(formatted_prompt, model_config) - - # Try to extract JSON from the response - json_str = extract_json_from_text(analysis_str) - - # If we still can't parse the JSON, provide a default analysis - try: - analysis = json.loads(json_str) - except json.JSONDecodeError: - # Fallback to a default analysis if JSON parsing fails - analysis = { - "question_type": "unknown", - "required_tools": ["web_search"], - "information_sources": ["web"], - "complexity": "medium" - } - logger.warning(f"Failed to parse analysis JSON, using default analysis. Response was: {analysis_str}") - - new_state = state.copy() - new_state["analysis"] = analysis - new_state["check_count"] = 0 # Initialize check_count - - return new_state - except Exception as e: - logger.error(f"Error analyzing question: {str(e)}") - - # Create a default analysis instead of setting an error - default_analysis = { - "question_type": "unknown", - "required_tools": ["web_search"], - "information_sources": ["web"], - "complexity": "medium" - } - - new_state = state.copy() - new_state["analysis"] = default_analysis - new_state["check_count"] = 0 # Initialize check_count - - # Add the error as a thought rather than stopping the workflow - thought = f"Error during question analysis: {str(e)}. Using default analysis." - new_state["intermediate_thoughts"] = state.get("intermediate_thoughts", []) + [thought] - - return new_state - -def create_plan(state: AgentState) -> AgentState: - """ - Create a plan to answer the question based on the analysis. - - Args: - state: The current agent state - - Returns: - Updated agent state with plan - """ - - if state.get("error"): - return state - - model_config = get_model_config() - - planning_prompt = PromptTemplate.from_template( - """You are an expert planner for the GAIA benchmark. - - Your task is to create a step-by-step plan to answer the following question: - - Question: {question} - - Question Analysis: - {analysis} - - Create a detailed plan with the following steps: - 1. Break down the question into logical steps - 2. For each step, specify which tool to use (if any) - 3. Specify the inputs for each tool - 4. Describe how to combine the results to formulate the final answer - - Provide your plan in the following JSON format: - [ - {{ - "step_number": 1, - "description": "description of step", - "tool": "tool_name or null if no tool needed", - "tool_input": {{ "param1": "value1", ... }} or null if no tool needed - }}, - ... - ] - - IMPORTANT: Your response must be valid JSON. Do not include any text before or after the JSON. - - Plan:""" - ) - - try: - # Use the prompt template to format the prompt - formatted_prompt = planning_prompt.format( - question=state["question"], - analysis=state["analysis"] - ) - - # Call OpenAI API directly - plan_str = call_openai_api(formatted_prompt, model_config) - - # Try to extract JSON from the response - json_str = extract_json_from_text(plan_str) - - # If we still can't parse the JSON, provide a default plan - try: - plan = json.loads(json_str) - except json.JSONDecodeError: - # Fallback to a default plan if JSON parsing fails - plan = [ - { - "step_number": 1, - "description": "Search for information related to the question", - "tool": "web_search", - "tool_input": {"query": state["question"]} - }, - { - "step_number": 2, - "description": "Formulate an answer based on search results", - "tool": None, - "tool_input": None - } - ] - logger.warning(f"Failed to parse plan JSON, using default plan. Response was: {plan_str}") - - new_state = state.copy() - new_state["plan"] = plan - new_state["current_step"] = 0 - - tools_to_use = [] - for step in plan: - if step.get("tool"): - tools_to_use.append(step["tool"]) - - new_state["tools_to_use"] = list(set(tools_to_use)) # Remove duplicates - - return new_state - except Exception as e: - logger.error(f"Error creating plan: {str(e)}") - - # Create a default plan instead of setting an error - default_plan = [ - { - "step_number": 1, - "description": "Search for information related to the question", - "tool": "web_search", - "tool_input": {"query": state["question"]} - }, - { - "step_number": 2, - "description": "Formulate an answer based on search results", - "tool": None, - "tool_input": None - } - ] - - new_state = state.copy() - new_state["plan"] = default_plan - new_state["current_step"] = 0 - new_state["tools_to_use"] = ["web_search"] - - # Add the error as a thought rather than stopping the workflow - thought = f"Error during plan creation: {str(e)}. Using default plan." - new_state["intermediate_thoughts"] = state.get("intermediate_thoughts", []) + [thought] - - return new_state - -def check_next_step(state: AgentState) -> AgentState: - """ - Check the current state and update it to prepare for the next step. - - Args: - state: The current agent state - - Returns: - Updated agent state - """ - if state.get("error"): - return state - - if state.get("answer"): - return state - - # Increment check count to prevent infinite loops - new_state = state.copy() - new_state["check_count"] = state.get("check_count", 0) + 1 - - # If we've checked too many times, force move to formulate_answer - if new_state["check_count"] > 3: - thought = "Too many checks without progress, moving to formulate answer." - new_state["intermediate_thoughts"] = state.get("intermediate_thoughts", []) + [thought] - return new_state - - if state["current_step"] >= len(state["plan"]): - # All steps completed, move to formulate answer - return new_state - - current_step = state["plan"][state["current_step"]] - - if not current_step.get("tool"): - # No tool needed for this step, increment step counter and continue - new_state["current_step"] += 1 - - thought = f"Step {current_step['step_number']}: {current_step['description']} - No tool needed, moving to next step." - new_state["intermediate_thoughts"] = state.get("intermediate_thoughts", []) + [thought] - - return new_state - -def route_next_step(state: AgentState) -> str: - """ - Determine the next step in the workflow based on the current state. - - Args: - state: The current agent state - - Returns: - String indicating the next node - """ - if state.get("error"): - return "end" - - if state.get("answer"): - return "end" - - # If we've checked too many times, force move to formulate_answer - if state.get("check_count", 0) > 3: - return "formulate_answer" - - if state["current_step"] >= len(state["plan"]): - return "formulate_answer" - - current_step = state["plan"][state["current_step"]] - - if current_step.get("tool"): - return "execute_tool" - else: - return "check_next_step" - -def execute_tool(state: AgentState) -> AgentState: - """ - Execute the selected tool based on the current plan step. - - Args: - state: The current agent state - - Returns: - Updated agent state with tool execution results - """ - if state.get("error"): - return state - - current_step = state["plan"][state["current_step"]] - tool_name = current_step.get("tool") - tool_input = current_step.get("tool_input", {}) - - try: - # Import the tool registry here to avoid circular imports - from agent.tool_registry import create_default_registry, search, robust_search, merged_search - - # Create the tool registry - registry = create_default_registry() - - # Execute the appropriate tool based on the tool name - if tool_name == "web_search": - query = tool_input.get("query") - if not query: - raise ValueError("Query is required for web search") - - try: - # Get working memory if available - working_memory = None - if hasattr(state, "get") and callable(state.get): - from memory import WorkingMemory - working_memory_instance = state.get("working_memory") - if isinstance(working_memory_instance, WorkingMemory): - working_memory = working_memory_instance - - # Use the unified search wrapper with working memory - result = search(registry, query, format_type="unified", working_memory=working_memory) - logger.info(f"Web search successful using unified search") - except Exception as search_error: - logger.error(f"Unified search failed: {str(search_error)}") - # If unified search fails, try robust search as a fallback - try: - result = robust_search(registry, query) - logger.info(f"Web search successful using fallback robust search") - except Exception as robust_error: - logger.error(f"Robust search failed: {str(robust_error)}") - raise ValueError(f"All search methods failed: {str(search_error)}") - - elif tool_name in ["duckduckgo_search", "serper_search", "perplexity_search", "arxiv_search"]: - query = tool_input.get("query") - if not query: - raise ValueError("Query is required for search tools") - - tool = registry.get_tool(tool_name) - if not tool: - raise ValueError(f"Tool not found: {tool_name}") - - result = registry.execute_tool(tool_name, query=query) - - elif tool_name == "arxiv_get_paper": - paper_id = tool_input.get("paper_id") - if not paper_id: - raise ValueError("Paper ID is required for arXiv paper retrieval") - - tool = registry.get_tool("arxiv_search") - if not tool: - raise ValueError("arXiv tool not found") - - result = registry.execute_tool("arxiv_get_paper", paper_id=paper_id) - - elif tool_name == "extract_web_content": - url = tool_input.get("url") - if not url: - raise ValueError("URL is required for web content extraction") - - # Import the web content extractor - from tools.web_tools import create_web_content_extractor - extractor = create_web_content_extractor() - result = extractor.extract_content(url) - - elif tool_name == "youtube_video": - video_id_or_url = tool_input.get("video_id_or_url") - language = tool_input.get("language") - if not video_id_or_url: - raise ValueError("Video ID or URL is required for YouTube video analysis") - - tool = registry.get_tool("youtube_video") - if not tool: - raise ValueError("YouTube tool not found") - - result = registry.execute_tool("youtube_video", video_id_or_url=video_id_or_url, language=language) - - else: - # Fallback to simulated result for unknown tools - result = f"Simulated result for {tool_name} with input {tool_input}" - logger.warning(f"Using simulated result for unknown tool: {tool_name}") - - tool_result = { - "tool_name": tool_name, - "success": True, - "result": result, - "error": None - } - - new_state = state.copy() - new_state["tool_results"] = state.get("tool_results", []) + [tool_result] - new_state["current_step"] += 1 - new_state["check_count"] = 0 # Reset check count after successful tool execution - - thought = f"Step {current_step['step_number']}: {current_step['description']} - Executed {tool_name} and got result: {str(result)[:200]}..." - new_state["intermediate_thoughts"] = state.get("intermediate_thoughts", []) + [thought] - - return new_state - except Exception as e: - logger.error(f"Error executing tool {tool_name}: {str(e)}") - logger.error(f"Stack trace: {traceback.format_exc()}") - - tool_result = { - "tool_name": tool_name, - "success": False, - "result": None, - "error": str(e) - } - - new_state = state.copy() - new_state["tool_results"] = state.get("tool_results", []) + [tool_result] - new_state["current_step"] += 1 - new_state["check_count"] = 0 # Reset check count after tool execution (even if failed) - - thought = f"Step {current_step['step_number']}: {current_step['description']} - Failed to execute {tool_name}: {str(e)}" - new_state["intermediate_thoughts"] = state.get("intermediate_thoughts", []) + [thought] - - return new_state - -def formulate_answer(state: AgentState) -> AgentState: - """ - Formulate the final answer based on tool results. - - Args: - state: The current agent state - - Returns: - Updated agent state with answer - """ - - if state.get("error"): - return state - - model_config = get_model_config() - - answer_prompt = PromptTemplate.from_template( - """You are an expert answer formulator for the GAIA benchmark. - - Your task is to formulate a precise, concise answer to the following question based on the tool results: - - Question: {question} - - Question Analysis: - {analysis} - - Plan: - {plan} - - Tool Results: - {tool_results} - - Intermediate Thoughts: - {thoughts} - - Remember, GAIA benchmark answers should be very concise - either a number, a few words, or a comma-separated list. - Do not provide explanations or additional context in your answer. - - Concise Answer:""" - ) - - try: - # Use the prompt template to format the prompt - formatted_prompt = answer_prompt.format( - question=state["question"], - analysis=state["analysis"], - plan=state["plan"], - tool_results=state["tool_results"], - thoughts=state.get("intermediate_thoughts", []) - ) - - # Call OpenAI API directly - answer = call_openai_api(formatted_prompt, model_config) - - new_state = state.copy() - new_state["answer"] = answer.strip() - - return new_state - except Exception as e: - logger.error(f"Error formulating answer: {str(e)}") - - # Provide a default answer instead of setting an error - default_answer = "Unable to provide a specific answer due to technical limitations." - - new_state = state.copy() - new_state["answer"] = default_answer - - # Add the error as a thought rather than stopping the workflow - thought = f"Error during answer formulation: {str(e)}. Using default answer." - new_state["intermediate_thoughts"] = state.get("intermediate_thoughts", []) + [thought] - - return new_state - -def create_agent_graph() -> Tuple[StateGraph, Dict[str, Any]]: - """ - Create the agent workflow graph. - - Returns: - Tuple containing the StateGraph and the initial state - """ - workflow = StateGraph(AgentState) - - # Add nodes - workflow.add_node("analyze_question", analyze_question) - workflow.add_node("create_plan", create_plan) - workflow.add_node("check_next_step", check_next_step) - workflow.add_node("execute_tool", execute_tool) - workflow.add_node("formulate_answer", formulate_answer) - - # Set entry point - workflow.set_entry_point("analyze_question") - - # Add standard edges - workflow.add_edge("analyze_question", "create_plan") - workflow.add_edge("formulate_answer", END) - - # Add conditional edges - workflow.add_conditional_edges( - "create_plan", - route_next_step, - { - "check_next_step": "check_next_step", - "execute_tool": "execute_tool", - "formulate_answer": "formulate_answer", - "end": END - } - ) - - workflow.add_conditional_edges( - "check_next_step", - route_next_step, - { - "check_next_step": "check_next_step", - "execute_tool": "execute_tool", - "formulate_answer": "formulate_answer", - "end": END - } - ) - - workflow.add_conditional_edges( - "execute_tool", - route_next_step, - { - "check_next_step": "check_next_step", - "execute_tool": "execute_tool", - "formulate_answer": "formulate_answer", - "end": END - } - ) - - initial_state = { - "question": "", - "analysis": None, - "plan": None, - "current_step": 0, - "tools_to_use": None, - "tool_results": [], - "intermediate_thoughts": [], - "answer": None, - "error": None, - "check_count": 0 # Initialize check_count - } - - return workflow, initial_state - -def run_agent_graph(question: str, config: Optional[Dict[str, Any]] = None) -> Dict[str, Any]: - """ - Run the agent workflow graph with a question. - - Args: - question: The question to answer - config: Optional configuration for the graph - - Returns: - The final state of the graph - """ - workflow, initial_state = create_agent_graph() - - app = workflow.compile() - - state = initial_state.copy() - state["question"] = question - - # Set a higher recursion limit to prevent errors - config_with_recursion = (config or {}).copy() - config_with_recursion["recursion_limit"] = 25 - - runnable_config = RunnableConfig(config_with_recursion) - - result = app.invoke(state, config=runnable_config) - - return result \ No newline at end of file diff --git a/agent/tool_registry.py b/agent/tool_registry.py deleted file mode 100644 index ef147405327d3564be4e63e8b2f3b88b5fedb236..0000000000000000000000000000000000000000 --- a/agent/tool_registry.py +++ /dev/null @@ -1,732 +0,0 @@ -""" -Tool registry for the GAIA agent. - -This module provides a registry for tools that can be used by the GAIA agent. -It includes factory functions for creating tool instances and a registry class -for managing tool instances. - -The registry is designed to be used with the LangGraph workflow in agent/graph.py. -""" - -import logging -import os -import re -import time -from typing import Dict, Any, Optional, List, Callable - -from tools.web_tools import ( - DuckDuckGoSearchTool, - SerperSearchTool, - EnhancedWebSearchTool, - LibrarySearchTool, - ApiSearchTool, - create_duckduckgo_search, - create_serper_search, - create_wikipedia_search, - create_enhanced_web_search, - create_library_search, - create_api_search -) -from tools.perplexity_tool import PerplexityTool, create_perplexity_tool -from tools.arxiv_tool import ArxivSearchTool, create_arxiv_search -from tools.multimodal_tools import YouTubeVideoTool, create_youtube_video_tool, BrowserSearchTool, create_browser_search_tool - -logger = logging.getLogger("gaia_agent.tool_registry") - -class ToolRegistry: - """Registry for tools used by the GAIA agent.""" - - def __init__(self): - """Initialize an empty tool registry.""" - self.tools = {} - - def register_tool(self, name: str, tool: Any) -> None: - """ - Register a tool in the registry. - - Args: - name: The name of the tool - tool: The tool instance - """ - self.tools[name] = tool - - def get_tool(self, name: str) -> Optional[Any]: - """ - Get a tool from the registry. - - Args: - name: The name of the tool - - Returns: - The tool instance, or None if not found - """ - tool = self.tools.get(name) - if not tool: - logger.warning(f"Tool not found in registry: {name}") - return tool - - def list_tools(self) -> List[str]: - """ - List all tools in the registry. - - Returns: - List of tool names - """ - return list(self.tools.keys()) - - def execute_tool(self, name: str, **kwargs) -> Any: - """ - Execute a tool from the registry. - - Args: - name: The name of the tool - **kwargs: Arguments to pass to the tool - - Returns: - The result of the tool execution - - Raises: - Exception: If the tool is not found or execution fails - """ - tool = self.get_tool(name) - if not tool: - raise Exception(f"Tool not found in registry: {name}") - - - try: - if name in ["duckduckgo_search", "serper_search", "wikipedia_search", "enhanced_web_search", - "library_search", "api_search"]: - query = kwargs.get("query") - if not query: - raise ValueError("Query is required for search tools") - return tool.search(query) - - elif name == "browser_search": - query = kwargs.get("query") - source = kwargs.get("source") - if not query: - raise ValueError("Query is required for browser search") - return tool.search(query, source) - - elif name == "perplexity_search": - query = kwargs.get("query") - if not query: - raise ValueError("Query is required for Perplexity search") - return tool.search(query) - - elif name == "arxiv_search": - query = kwargs.get("query") - max_results = kwargs.get("max_results") - if not query: - raise ValueError("Query is required for arXiv search") - return tool.search(query, max_results) - - elif name == "arxiv_get_paper": - paper_id = kwargs.get("paper_id") - if not paper_id: - raise ValueError("Paper ID is required for arXiv paper retrieval") - return tool.get_paper_by_id(paper_id) - - elif name == "arxiv_search_category": - category = kwargs.get("category") - max_results = kwargs.get("max_results") - if not category: - raise ValueError("Category is required for arXiv category search") - return tool.search_by_category(category, max_results) - - elif name == "wikipedia_extract_page": - url = kwargs.get("url") - if not url: - raise ValueError("URL is required for Wikipedia page extraction") - return tool.extract_page_content(url) - - elif name == "wikipedia_featured_articles": - topic = kwargs.get("topic") - return tool.find_featured_articles(topic) - - elif name == "youtube_video": - video_id_or_url = kwargs.get("video_id_or_url") - language = kwargs.get("language") - if not video_id_or_url: - raise ValueError("Video ID or URL is required for YouTube video analysis") - return tool.extract_transcript(video_id_or_url, language) - - else: - return tool.run(**kwargs) - - except Exception as e: - logger.error(f"Error executing tool {name}: {str(e)}") - raise - -def create_default_registry() -> ToolRegistry: - """ - Create a default tool registry with all available tools. - - Returns: - ToolRegistry: A registry with all available tools - """ - registry = ToolRegistry() - - # Register Enhanced Web Search tool (handles GAIA assessment questions) - try: - enhanced_web_tool = create_enhanced_web_search() - registry.register_tool("enhanced_web_search", enhanced_web_tool) - logger.info("Registered Enhanced Web Search tool") - except Exception as e: - logger.warning(f"Failed to create Enhanced Web Search tool: {str(e)}") - - # Register DuckDuckGo search tool (no API key required) - try: - duckduckgo_tool = create_duckduckgo_search() - registry.register_tool("duckduckgo_search", duckduckgo_tool) - except Exception as e: - logger.warning(f"Failed to create DuckDuckGo search tool: {str(e)}") - - # Register Serper search tool if API key is available - serper_api_key = os.environ.get("SERPER_API_KEY") - if serper_api_key: - try: - serper_tool = create_serper_search() - registry.register_tool("serper_search", serper_tool) - except Exception as e: - logger.warning(f"Failed to create Serper search tool: {str(e)}") - else: - logger.warning("Serper API key not available, skipping Serper search tool") - - # Register Perplexity tool if API key is available - perplexity_api_key = os.environ.get("PERPLEXITY_API_KEY") - if perplexity_api_key: - try: - perplexity_tool = create_perplexity_tool() - registry.register_tool("perplexity_search", perplexity_tool) - except Exception as e: - logger.warning(f"Failed to create Perplexity tool: {str(e)}") - else: - logger.warning("Perplexity API key not available, skipping Perplexity tool") - - try: - arxiv_tool = create_arxiv_search() - registry.register_tool("arxiv_search", arxiv_tool) - registry.register_tool("arxiv_get_paper", arxiv_tool) - registry.register_tool("arxiv_search_category", arxiv_tool) - except Exception as e: - logger.warning(f"Failed to create arXiv search tool: {str(e)}") - - # Register Wikipedia search tool - try: - wikipedia_tool = create_wikipedia_search() - registry.register_tool("wikipedia_search", wikipedia_tool) - except Exception as e: - logger.warning(f"Failed to create Wikipedia search tool: {str(e)}") - - # Register YouTube video tool - try: - youtube_tool = create_youtube_video_tool() - registry.register_tool("youtube_video", youtube_tool) - except Exception as e: - logger.warning(f"Failed to create YouTube video tool: {str(e)}") - - # Register Browser Search tool - try: - browser_search_tool = create_browser_search_tool() - registry.register_tool("browser_search", browser_search_tool) - logger.info("Registered Browser Search tool") - except Exception as e: - logger.warning(f"Failed to create Browser Search tool: {str(e)}") - - # Register Library Search tool - try: - library_search_tool = create_library_search() - registry.register_tool("library_search", library_search_tool) - logger.info("Registered Library Search tool") - except Exception as e: - logger.warning(f"Failed to create Library Search tool: {str(e)}") - - # Register API Search tool if API keys are available - if os.environ.get("PERPLEXITY_API_KEY") or os.environ.get("SERPER_API_KEY"): - try: - api_search_tool = create_api_search() - registry.register_tool("api_search", api_search_tool) - logger.info("Registered API Search tool") - except Exception as e: - logger.warning(f"Failed to create API Search tool: {str(e)}") - else: - logger.warning("Neither Perplexity nor Serper API keys available, skipping API Search tool") - - logger.info(f"Created default tool registry with {len(registry.list_tools())} tools") - return registry - -def analyze_query(query: str) -> Dict[str, Any]: - """ - Analyze a query to determine the best search strategy. - - This function examines the query to identify: - - Source-specific keywords (Wikipedia, YouTube, arXiv) - - Question type (factual, research, multimedia) - - Information depth needed - - Args: - query: The search query - - Returns: - Dict with analysis results - """ - analysis = { - "source_specific": False, - "preferred_sources": [], - "question_type": "factual", # Default to factual - "depth_needed": "medium", # Default to medium depth - "is_multimedia": False - } - - # Check for source-specific keywords - query_lower = query.lower() - - # Wikipedia specific - if "wikipedia" in query_lower or "featured article" in query_lower: - analysis["source_specific"] = True - analysis["preferred_sources"].append("wikipedia") - - # YouTube specific - if "youtube" in query_lower or "video" in query_lower: - analysis["source_specific"] = True - analysis["preferred_sources"].append("youtube") - analysis["is_multimedia"] = True - - # arXiv specific - if "arxiv" in query_lower or "paper" in query_lower or "research paper" in query_lower: - analysis["source_specific"] = True - analysis["preferred_sources"].append("arxiv") - analysis["question_type"] = "research" - analysis["depth_needed"] = "high" - - # Determine question type if not already set - if "how" in query_lower or "why" in query_lower or "explain" in query_lower: - analysis["question_type"] = "explanatory" - analysis["depth_needed"] = "high" - elif "when" in query_lower or "where" in query_lower or "who" in query_lower: - analysis["question_type"] = "factual" - elif "compare" in query_lower or "difference" in query_lower: - analysis["question_type"] = "comparative" - analysis["depth_needed"] = "high" - - # Check for indicators of needed depth - if "detailed" in query_lower or "comprehensive" in query_lower or "in depth" in query_lower: - analysis["depth_needed"] = "high" - elif "brief" in query_lower or "summary" in query_lower or "overview" in query_lower: - analysis["depth_needed"] = "low" - - return analysis - -def unified_search(registry: ToolRegistry, query: str, working_memory=None) -> Dict[str, Any]: - """ - Perform a unified search using an intelligent routing approach. - - This function: - 1. Analyzes the query to determine the best search strategy - 2. Routes to appropriate tools based on the analysis - 3. Executes tools in parallel when appropriate - 4. Stores intermediate results in working_memory - 5. Combines and ranks results - - Args: - registry: The tool registry - query: The search query - working_memory: Optional working memory instance for storing results - - Returns: - Dict with search results and metadata - """ - from tools.web_tools import calculate_query_relevance - import concurrent.futures - - # Analyze the query - analysis = analyze_query(query) - - # Store the analysis in working memory if available - if working_memory: - working_memory.store_intermediate_result("query_analysis", analysis) - - # Initialize results container - all_results = [] - metadata = { - "providers_used": [], - "analysis": analysis, - "execution_times": {} - } - - # Check if we should use the enhanced web search tool - # This tool is especially useful for GAIA assessment questions - if registry.get_tool("enhanced_web_search"): - # First try the enhanced web search tool directly (not in parallel) - # This is more efficient for GAIA assessment questions - try: - logger.info(f"Using enhanced web search for query: {query}") - start_time = time.time() - enhanced_results = registry.execute_tool("enhanced_web_search", query=query) - end_time = time.time() - - metadata["execution_times"]["enhanced_web_search"] = end_time - start_time - metadata["providers_used"].append("enhanced_web_search") - - # If we got good results from the enhanced tool, return them directly - if enhanced_results and len(enhanced_results) > 0: - # Check if we have a high-quality result (like from Perplexity) - has_high_quality = False - for result in enhanced_results: - if result.get("source") == "perplexity" or result.get("relevance_score", 0) > 8.0: - has_high_quality = True - break - - if has_high_quality: - logger.info("Enhanced web search returned high-quality results, skipping other tools") - - # Store results in working memory - if working_memory: - working_memory.store_intermediate_result("enhanced_search_results", enhanced_results) - - return { - "results": enhanced_results, - "metadata": metadata - } - except Exception as e: - logger.warning(f"Enhanced web search failed: {str(e)}") - - # Determine which tools to use based on analysis - tools_to_use = [] - - # If source-specific, prioritize those sources - if analysis["source_specific"]: - for source in analysis["preferred_sources"]: - if source == "wikipedia" and registry.get_tool("wikipedia_search"): - tools_to_use.append("wikipedia_search") - elif source == "youtube" and registry.get_tool("youtube_video"): - tools_to_use.append("youtube_video") - elif source == "arxiv" and registry.get_tool("arxiv_search"): - tools_to_use.append("arxiv_search") - - # For high depth questions, always include Perplexity if available - if analysis["depth_needed"] == "high" and registry.get_tool("perplexity_search"): - if "perplexity_search" not in tools_to_use: - tools_to_use.append("perplexity_search") - - # Check for API search tool (combines Perplexity and Serper) - if registry.get_tool("api_search") and "api_search" not in tools_to_use: - tools_to_use.append("api_search") - - # Check for library search tool (combines DuckDuckGo and arXiv) - if registry.get_tool("library_search") and "library_search" not in tools_to_use: - tools_to_use.append("library_search") - - # Always include general search tools as fallbacks - if registry.get_tool("duckduckgo_search") and "duckduckgo_search" not in tools_to_use: - tools_to_use.append("duckduckgo_search") - - if registry.get_tool("serper_search") and "serper_search" not in tools_to_use: - tools_to_use.append("serper_search") - - # For visual or interactive queries, include browser search tool - if analysis["is_multimedia"] and registry.get_tool("browser_search") and "browser_search" not in tools_to_use: - tools_to_use.append("browser_search") - - # Execute tools in parallel - results_dict = {} - with concurrent.futures.ThreadPoolExecutor() as executor: - future_to_tool = {} - - for tool_name in tools_to_use: - if tool_name == "youtube_video": - # YouTube tool requires different handling - continue - - future = executor.submit(registry.execute_tool, tool_name, query=query) - future_to_tool[future] = tool_name - - for future in concurrent.futures.as_completed(future_to_tool): - tool_name = future_to_tool[future] - try: - start_time = time.time() - result = future.result() - end_time = time.time() - - metadata["execution_times"][tool_name] = end_time - start_time - metadata["providers_used"].append(tool_name) - - results_dict[tool_name] = result - - # Store intermediate results in working memory - if working_memory: - working_memory.store_intermediate_result(f"search_result_{tool_name}", result) - - except Exception as e: - logger.warning(f"{tool_name} search failed: {str(e)}") - metadata["execution_times"][tool_name] = -1 # Indicate failure - - # Process and merge results - seen_urls = set() - - # Process source-specific results first - for source in analysis["preferred_sources"]: - tool_name = None - if source == "wikipedia": - tool_name = "wikipedia_search" - elif source == "arxiv": - tool_name = "arxiv_search" - - if tool_name and tool_name in results_dict: - results = results_dict[tool_name] - - # Format results if needed - formatted_results = [] - if tool_name == "arxiv_search": - for result in results: - if "url" in result and result["url"] not in seen_urls: - title = result.get("title", "") - summary = result.get("summary", "") - - # Calculate relevance - title_relevance = calculate_query_relevance(title, query) - summary_relevance = calculate_query_relevance(summary, query) - relevance_score = (title_relevance * 2 + summary_relevance) / 3 - - formatted_result = { - "title": title, - "link": result.get("url", ""), - "snippet": summary[:200] + "..." if summary else "", - "relevance_score": relevance_score * 1.2, # Boost source-specific results - "source": "arxiv" - } - - formatted_results.append(formatted_result) - seen_urls.add(result["url"]) - else: - # For other tools, just add source and boost relevance - for result in results: - if result["link"] not in seen_urls: - if "relevance_score" not in result: - title_relevance = calculate_query_relevance(result.get("title", ""), query) - snippet_relevance = calculate_query_relevance(result.get("snippet", ""), query) - result["relevance_score"] = (title_relevance * 2 + snippet_relevance) / 3 - - # Boost source-specific results - result["relevance_score"] = result["relevance_score"] * 1.2 - result["source"] = source - - formatted_results.append(result) - seen_urls.add(result["link"]) - - all_results.extend(formatted_results) - - # Process general search results - for tool_name in ["duckduckgo_search", "serper_search", "library_search", "api_search"]: - if tool_name in results_dict: - for result in results_dict[tool_name]: - if "link" in result and result["link"] not in seen_urls: - if "relevance_score" not in result: - title_relevance = calculate_query_relevance(result.get("title", ""), query) - snippet_relevance = calculate_query_relevance(result.get("snippet", ""), query) - result["relevance_score"] = (title_relevance * 2 + snippet_relevance) / 3 - - # If source is not already set, derive it from the tool name - if "source" not in result: - result["source"] = tool_name.replace("_search", "") - - all_results.append(result) - seen_urls.add(result["link"]) - - # Process Perplexity results - if "perplexity_search" in results_dict: - perplexity_result = results_dict["perplexity_search"] - perplexity_content = None - - if isinstance(perplexity_result, dict) and "content" in perplexity_result: - perplexity_content = perplexity_result["content"] - - # Add perplexity as a result if it's not empty - if perplexity_content and perplexity_content.strip(): - relevance_score = calculate_query_relevance(perplexity_content, query) - - # For high depth questions, boost Perplexity even more - if analysis["depth_needed"] == "high": - relevance_score = relevance_score * 1.5 - - formatted_result = { - "title": "Perplexity AI Search Result", - "link": "https://perplexity.ai/", - "snippet": perplexity_content[:200] + "..." if len(perplexity_content) > 200 else perplexity_content, - "relevance_score": relevance_score, - "source": "perplexity" - } - all_results.append(formatted_result) - - # Store the full perplexity content in metadata - metadata["perplexity_content"] = perplexity_content - - # Process Browser Search results - if "browser_search" in results_dict: - browser_results = results_dict["browser_search"] - if browser_results and isinstance(browser_results, list): - for result in browser_results: - if "link" in result and result["link"] not in seen_urls: - # Browser search results already have high relevance scores - if "relevance_score" not in result: - result["relevance_score"] = 9.0 # High default score for browser results - - all_results.append(result) - seen_urls.add(result["link"]) - - # Sort all results by relevance score - all_results.sort(key=lambda x: x.get("relevance_score", 0), reverse=True) - - # Store final merged results in working memory - if working_memory: - working_memory.store_intermediate_result("merged_search_results", all_results) - working_memory.store_intermediate_result("search_metadata", metadata) - - return { - "results": all_results[:10], # Return top 10 results - "metadata": metadata - } - -def search(registry: ToolRegistry, query: str, format_type: str = "unified", working_memory=None) -> Dict[str, Any]: - """ - Unified wrapper function for all search types. - - This function serves as a single entry point for all search operations, - eliminating redundancy while maintaining backward compatibility with - different output formats. - - Args: - registry: The tool registry - query: The search query - format_type: The desired output format ("unified", "robust", or "merged") - working_memory: Optional working memory instance for storing results - - Returns: - Dict with search results formatted according to format_type - """ - # Execute the unified search - search_result = unified_search(registry, query, working_memory) - - # Return results in the requested format - if format_type == "robust": - # Format as robust_search result - providers = [result.get("source", "unknown") for result in search_result["results"]] - unique_providers = list(set(providers)) - - return { - "provider": ",".join(unique_providers), - "results": search_result["results"] - } - - elif format_type == "merged": - # Format as merged_search result - perplexity_content = search_result["metadata"].get("perplexity_content") - - # Extract arxiv results if available - arxiv_results = [] - browser_results = [] - library_results = [] - api_results = [] - - for result in search_result["results"]: - source = result.get("source", "") - - if source == "arxiv": - # Try to reconstruct original arxiv result format - arxiv_result = { - "title": result.get("title", ""), - "url": result.get("link", ""), - "summary": result.get("snippet", "") - } - arxiv_results.append(arxiv_result) - - elif source == "browser": - browser_results.append({ - "title": result.get("title", ""), - "url": result.get("link", ""), - "snippet": result.get("snippet", "") - }) - - elif source == "library": - library_results.append({ - "title": result.get("title", ""), - "url": result.get("link", ""), - "snippet": result.get("snippet", "") - }) - - elif source == "api": - api_results.append({ - "title": result.get("title", ""), - "url": result.get("link", ""), - "snippet": result.get("snippet", "") - }) - - return { - "merged_results": search_result["results"], - "perplexity_context": perplexity_content, - "arxiv_context": arxiv_results, - "browser_context": browser_results, - "library_context": library_results, - "api_context": api_results - } - - else: # "unified" or any other value - # Return the unified search result directly - return search_result - -def robust_search(registry: ToolRegistry, query: str) -> Dict[str, Any]: - """ - Legacy robust search function - now uses the unified search wrapper. - - This function is maintained for backward compatibility. - New code should use the 'search' function with format_type="robust". - - Args: - registry: The tool registry - query: The search query - - Returns: - Dict with provider name and search results - """ - return search(registry, query, format_type="robust") - -def merged_search(registry: ToolRegistry, query: str, working_memory=None) -> Dict[str, Any]: - """ - Legacy merged search function - now uses the unified search wrapper. - - This function is maintained for backward compatibility. - New code should use the 'search' function with format_type="merged". - - Args: - registry: The tool registry - query: The search query - working_memory: Optional working memory instance - - Returns: - Dict with merged results and context - """ - return search(registry, query, format_type="merged", working_memory=working_memory) - -if __name__ == "__main__": - logging.basicConfig(level=logging.INFO) - - registry = create_default_registry() - - try: - query = "latest Python version" - result = robust_search(registry, query) - print(f"Robust search found {len(result.get('results', []))} results") - except Exception as e: - print(f"Robust search failed: {str(e)}") - - try: - query = "latest Python version" - result = merged_search(registry, query) - if result["perplexity_context"]: - print("Perplexity context available") - if result.get("arxiv_context"): - print(f"arXiv context available with {len(result.get('arxiv_context', []))} results") - print(f"Merged search found {len(result.get('merged_results', []))} results") - except Exception as e: - print(f"Merged search failed: {str(e)}") diff --git a/app.py b/app.py index e58e8147634b21cc35281efc0b176f3304deb131..2b4a25377b1cd082884ee4d75f40334aa227dd27 100644 --- a/app.py +++ b/app.py @@ -1,347 +1,347 @@ -import os -import gradio as gr -import requests -import inspect -import pandas as pd -import logging -import time -from typing import Dict, Any, List, Optional, Tuple, Union - -DEFAULT_API_URL = "https://agents-course-unit4-scoring.hf.space" - -from agent import GAIAAgent -from agent.config import VERBOSE - -logger = logging.getLogger("gaia_app") -if VERBOSE: - logging.basicConfig(level=logging.DEBUG, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s') -else: - logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s') - -def run_and_submit_all(username: str = "anonymous", progress=gr.Progress()) -> Tuple[str, Optional[pd.DataFrame]]: - """ - Fetches all questions, runs the GAIA Agent on them, submits all answers, - and displays the results with progress tracking. - - Args: - username: Username for submission (default: anonymous) - progress: Gradio progress tracker - - Returns: - Tuple containing status message and results dataframe - """ - space_id = os.getenv("SPACE_ID") # Get the SPACE_ID for sending link to the code - logger.info(f"Running with username: {username}") - - api_url = DEFAULT_API_URL - questions_url = f"{api_url}/questions" - submit_url = f"{api_url}/submit" - - progress(0, desc="Initializing GAIA Agent...") - try: - start_time = time.time() - agent = GAIAAgent() - init_time = time.time() - start_time - logger.info(f"GAIA Agent initialized successfully in {init_time:.2f} seconds.") - progress(0.1, desc=f"Agent initialized in {init_time:.2f}s") - except Exception as e: - error_msg = f"Error initializing GAIA agent: {e}" - logger.error(error_msg) - return error_msg, None - - agent_code = f"https://huggingface.co/spaces/{space_id}/tree/main" - logger.info(f"Agent code URL: {agent_code}") - - progress(0.15, desc="Fetching questions...") - logger.info(f"Fetching questions from: {questions_url}") - try: - response = requests.get(questions_url, timeout=15) - response.raise_for_status() - questions_data = response.json() - if not questions_data: - error_msg = "Fetched questions list is empty or invalid format." - logger.warning(error_msg) - return error_msg, None - - question_count = len(questions_data) - logger.info(f"Fetched {question_count} questions.") - progress(0.2, desc=f"Fetched {question_count} questions") - except requests.exceptions.RequestException as e: - error_msg = f"Error fetching questions: {e}" - logger.error(error_msg) - return error_msg, None - except requests.exceptions.JSONDecodeError as e: - error_msg = f"Error decoding JSON response from questions endpoint: {e}" - logger.error(f"{error_msg}\nResponse text: {response.text[:500]}") - return error_msg, None - except Exception as e: - error_msg = f"An unexpected error occurred fetching questions: {e}" - logger.error(error_msg) - return error_msg, None - - results_log = [] - answers_payload = [] - total_questions = len(questions_data) - - logger.info(f"Running agent on {total_questions} questions...") - - question_progress_increment = 0.6 / total_questions # 60% of progress bar for processing questions - - for i, item in enumerate(questions_data): - task_id = item.get("task_id") - question_text = item.get("question") - - if not task_id or question_text is None: - logger.warning(f"Skipping item with missing task_id or question: {item}") - continue - - progress_value = 0.2 + (i * question_progress_increment) - progress(progress_value, desc=f"Processing question {i+1}/{total_questions}...") - - logger.info(f"Processing question {i+1}/{total_questions}: {question_text[:50]}...") - question_start_time = time.time() - - try: - submitted_answer = agent(question_text) - processing_time = time.time() - question_start_time - - answers_payload.append({"task_id": task_id, "submitted_answer": submitted_answer}) - results_log.append({ - "Task ID": task_id, - "Question": question_text, - "Submitted Answer": submitted_answer, - "Processing Time": f"{processing_time:.2f}s", - "Status": "Success" - }) - - logger.info(f"Question {i+1} processed in {processing_time:.2f}s. Answer: {submitted_answer[:50]}...") - - except Exception as e: - error_msg = f"Error running agent on task {task_id}: {e}" - logger.error(error_msg) - - results_log.append({ - "Task ID": task_id, - "Question": question_text, - "Submitted Answer": f"AGENT ERROR: {e}", - "Processing Time": f"{time.time() - question_start_time:.2f}s", - "Status": "Error" - }) - - if not answers_payload: - error_msg = "Agent did not produce any answers to submit." - logger.warning(error_msg) - return error_msg, pd.DataFrame(results_log) - - # Only proceed with submission if not in demo mode - progress(0.8, desc="Preparing submission...") - submission_data = {"username": username.strip(), "agent_code": agent_code, "answers": answers_payload} - status_update = f"Agent finished. Submitting {len(answers_payload)} answers for user '{username}'..." - logger.info(status_update) - - progress(0.9, desc=f"Submitting {len(answers_payload)} answers...") - logger.info(f"Submitting {len(answers_payload)} answers to: {submit_url}") - try: - response = requests.post(submit_url, json=submission_data, timeout=60) - response.raise_for_status() - result_data = response.json() - - correct_count = result_data.get('correct_count', 0) - total_attempted = result_data.get('total_attempted', 0) - score = result_data.get('score', 'N/A') - - final_status = ( - f"[SUCCESS] Submission Successful!\n" - f"User: {result_data.get('username')}\n" - f"Overall Score: {score}% " - f"({correct_count}/{total_attempted} correct)\n" - f"Message: {result_data.get('message', 'No message received.')}" - ) - logger.info("Submission successful.") - - results_df = pd.DataFrame(results_log) - - if not results_df.empty: - success_count = sum(1 for status in results_df.get("Status", []) if status == "Success") - error_count = sum(1 for status in results_df.get("Status", []) if status == "Error") - - try: - avg_time = sum(float(time.strip('s')) for time in results_df["Processing Time"]) / len(results_df) - avg_time_str = f"{avg_time:.2f}s" - except: - avg_time_str = "N/A" - - logger.info(f"Processed {len(results_df)} questions: {success_count} successes, {error_count} errors, avg time: {avg_time_str}") - - progress(1.0, desc="Evaluation complete!") - return final_status, results_df - except requests.exceptions.HTTPError as e: - progress(1.0, desc="Submission failed!") - error_detail = f"Server responded with status {e.response.status_code}." - try: - error_json = e.response.json() - error_detail += f" Detail: {error_json.get('detail', e.response.text)}" - except requests.exceptions.JSONDecodeError: - error_detail += f" Response: {e.response.text[:500]}" - status_message = f"[ERROR] Submission Failed: {error_detail}" - logger.error(status_message) - results_df = pd.DataFrame(results_log) - return status_message, results_df - except requests.exceptions.Timeout: - progress(1.0, desc="Submission timed out!") - status_message = "[ERROR] Submission Failed: The request timed out." - logger.error(status_message) - results_df = pd.DataFrame(results_log) - return status_message, results_df - except requests.exceptions.RequestException as e: - progress(1.0, desc="Network error!") - status_message = f"[ERROR] Submission Failed: Network error - {e}" - logger.error(status_message) - results_df = pd.DataFrame(results_log) - return status_message, results_df - except Exception as e: - progress(1.0, desc="Error during submission!") - status_message = f"[ERROR] An unexpected error occurred during submission: {e}" - logger.error(status_message) - results_df = pd.DataFrame(results_log) - return status_message, results_df - - -# Check if we're running in a HF Space environment -space_host = os.getenv("SPACE_HOST") - -# Create Gradio interface -with gr.Blocks(theme=gr.themes.Soft()) as demo: - gr.Markdown("# GAIA Agent Evaluation Runner") - with gr.Tabs(): - with gr.TabItem("Run Evaluation"): - gr.Markdown( - """ - **Instructions:** - - 1. Enter your username (optional) for submission tracking. - 2. Click 'Run Evaluation & Submit All Answers' to fetch questions, run the GAIA agent, submit answers, and see the score. - 3. The progress bar will show you the current status of the evaluation. - 4. Results will appear in the table below once processing is complete. - - **Note:** Running the evaluation may take some time as the agent processes all questions. Please be patient. - """ - ) - - with gr.Row(): - with gr.Column(scale=1): - username_input = gr.Textbox( - label="Username (optional)", - placeholder="Enter username for submission", - value="anonymous" - ) - with gr.Column(scale=2): - run_button = gr.Button("Run Evaluation & Submit All Answers", - variant="primary", size="lg") - - status_output = gr.Textbox( - label="Run Status / Submission Result", - lines=6, - interactive=False, - elem_id="status_output" - ) - - results_accordion = gr.Accordion("Results", open=False, elem_id="results_accordion") - with results_accordion: - results_table = gr.DataFrame( - label="Questions and Agent Answers", - wrap=True, - elem_id="results_table" - ) - - with gr.Row(): - with gr.Column(): - gr.Markdown("### Result Statistics") - stats_output = gr.JSON(label="Statistics", visible=True) - - with gr.TabItem("About GAIA Agent"): - gr.Markdown( - """ - ## About GAIA Agent - - This agent is designed to answer questions from the GAIA (Grounded AI Alignment) benchmark. - - ### Key Features: - - - **Architecture**: Built with LangChain/LangGraph for workflow management - - **Tools**: Integrates multiple tools for web search, reasoning, and multimodal understanding - - **Memory**: Implements memory components for context retention - - **Structured Approach**: Uses a systematic process for question analysis and answer formulation - - ### Components: - - 1. **Web Tools**: Search engines, content extraction, and web navigation - 2. **Reasoning Tools**: Step-by-step reasoning, mathematical calculations, fact verification - 3. **Multimodal Tools**: Image analysis, chart interpretation, document parsing - 4. **Memory Components**: Conversation history, working memory, result caching - - ### Performance Metrics: - - The agent's performance is evaluated based on: - - Accuracy of answers - - Reasoning quality - - Ability to use appropriate tools - - Handling of edge cases - """ - ) - - with gr.TabItem("Help & Troubleshooting"): - gr.Markdown( - """ - ## Help & Troubleshooting - - ### Common Issues: - - 1. **Timeout Errors**: If the agent takes too long to process a question, try running the evaluation again. - 2. **API Errors**: If you encounter API errors, check your internet connection and try again. - - ### Error Messages: - - - **"Error initializing GAIA agent"**: There was a problem setting up the agent. Check the logs for details. - - **"Error fetching questions"**: The system couldn't retrieve questions from the API. - - **"Submission Failed"**: There was a problem submitting your answers. Check the error details. - - ### Contact: - - If you continue to experience issues, please open an issue on the GitHub repository or contact the course instructors. - """ - ) - - run_button.click( - fn=run_and_submit_all, - inputs=[username_input], - outputs=[status_output, results_table] - ).then( - fn=lambda df: { - "Total Questions": len(df) if df is not None else 0, - "Successful": sum(1 for status in df.get("Status", []) if status == "Success") if df is not None else 0, - "Errors": sum(1 for status in df.get("Status", []) if status == "Error") if df is not None else 0, - "Average Processing Time": f"{sum(float(time.strip('s')) for time in df['Processing Time']) / len(df):.2f}s" if df is not None and len(df) > 0 else "N/A" - } if df is not None else {}, - inputs=[results_table], - outputs=[stats_output] - ).then( - fn=lambda: gr.update(open=True, visible=True), - outputs=[results_accordion] - ) - -if __name__ == "__main__": - space_host_startup = os.getenv("SPACE_HOST") - space_id_startup = os.getenv("SPACE_ID") # Get SPACE_ID at startup - - # Handle Space host configurations - if space_host_startup: - logger.info(f"Running on Hugging Face Space host: {space_host_startup}") - - # Print repository URLs if SPACE_ID is available - if space_id_startup: - repo_url = f"https://huggingface.co/spaces/{space_id_startup}" - logger.info(f"Space repository URL: {repo_url}") - - # Launch the Gradio interface +import os +import gradio as gr +import requests +import inspect +import pandas as pd +import logging +import time +from typing import Dict, Any, List, Optional, Tuple, Union + +DEFAULT_API_URL = "https://agents-course-unit4-scoring.hf.space" + +from agent import GAIAAgent +from agent.config import VERBOSE + +logger = logging.getLogger("gaia_app") +if VERBOSE: + logging.basicConfig(level=logging.DEBUG, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s') +else: + logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s') + +def run_and_submit_all(username: str = "anonymous", progress=gr.Progress()) -> Tuple[str, Optional[pd.DataFrame]]: + """ + Fetches all questions, runs the GAIA Agent on them, submits all answers, + and displays the results with progress tracking. + + Args: + username: Username for submission (default: anonymous) + progress: Gradio progress tracker + + Returns: + Tuple containing status message and results dataframe + """ + space_id = os.getenv("SPACE_ID") # Get the SPACE_ID for sending link to the code + logger.info(f"Running with username: {username}") + + api_url = DEFAULT_API_URL + questions_url = f"{api_url}/questions" + submit_url = f"{api_url}/submit" + + progress(0, desc="Initializing GAIA Agent...") + try: + start_time = time.time() + agent = GAIAAgent() + init_time = time.time() - start_time + logger.info(f"GAIA Agent initialized successfully in {init_time:.2f} seconds.") + progress(0.1, desc=f"Agent initialized in {init_time:.2f}s") + except Exception as e: + error_msg = f"Error initializing GAIA agent: {e}" + logger.error(error_msg) + return error_msg, None + + agent_code = f"https://huggingface.co/spaces/{space_id}/tree/main" + logger.info(f"Agent code URL: {agent_code}") + + progress(0.15, desc="Fetching questions...") + logger.info(f"Fetching questions from: {questions_url}") + try: + response = requests.get(questions_url, timeout=15) + response.raise_for_status() + questions_data = response.json() + if not questions_data: + error_msg = "Fetched questions list is empty or invalid format." + logger.warning(error_msg) + return error_msg, None + + question_count = len(questions_data) + logger.info(f"Fetched {question_count} questions.") + progress(0.2, desc=f"Fetched {question_count} questions") + except requests.exceptions.RequestException as e: + error_msg = f"Error fetching questions: {e}" + logger.error(error_msg) + return error_msg, None + except requests.exceptions.JSONDecodeError as e: + error_msg = f"Error decoding JSON response from questions endpoint: {e}" + logger.error(f"{error_msg}\nResponse text: {response.text[:500]}") + return error_msg, None + except Exception as e: + error_msg = f"An unexpected error occurred fetching questions: {e}" + logger.error(error_msg) + return error_msg, None + + results_log = [] + answers_payload = [] + total_questions = len(questions_data) + + logger.info(f"Running agent on {total_questions} questions...") + + question_progress_increment = 0.6 / total_questions # 60% of progress bar for processing questions + + for i, item in enumerate(questions_data): + task_id = item.get("task_id") + question_text = item.get("question") + + if not task_id or question_text is None: + logger.warning(f"Skipping item with missing task_id or question: {item}") + continue + + progress_value = 0.2 + (i * question_progress_increment) + progress(progress_value, desc=f"Processing question {i+1}/{total_questions}...") + + logger.info(f"Processing question {i+1}/{total_questions}: {question_text[:50]}...") + question_start_time = time.time() + + try: + submitted_answer = agent(question_text) + processing_time = time.time() - question_start_time + + answers_payload.append({"task_id": task_id, "submitted_answer": submitted_answer}) + results_log.append({ + "Task ID": task_id, + "Question": question_text, + "Submitted Answer": submitted_answer, + "Processing Time": f"{processing_time:.2f}s", + "Status": "Success" + }) + + logger.info(f"Question {i+1} processed in {processing_time:.2f}s. Answer: {submitted_answer[:50]}...") + + except Exception as e: + error_msg = f"Error running agent on task {task_id}: {e}" + logger.error(error_msg) + + results_log.append({ + "Task ID": task_id, + "Question": question_text, + "Submitted Answer": f"AGENT ERROR: {e}", + "Processing Time": f"{time.time() - question_start_time:.2f}s", + "Status": "Error" + }) + + if not answers_payload: + error_msg = "Agent did not produce any answers to submit." + logger.warning(error_msg) + return error_msg, pd.DataFrame(results_log) + + # Only proceed with submission if not in demo mode + progress(0.8, desc="Preparing submission...") + submission_data = {"username": username.strip(), "agent_code": agent_code, "answers": answers_payload} + status_update = f"Agent finished. Submitting {len(answers_payload)} answers for user '{username}'..." + logger.info(status_update) + + progress(0.9, desc=f"Submitting {len(answers_payload)} answers...") + logger.info(f"Submitting {len(answers_payload)} answers to: {submit_url}") + try: + response = requests.post(submit_url, json=submission_data, timeout=60) + response.raise_for_status() + result_data = response.json() + + correct_count = result_data.get('correct_count', 0) + total_attempted = result_data.get('total_attempted', 0) + score = result_data.get('score', 'N/A') + + final_status = ( + f"[SUCCESS] Submission Successful!\n" + f"User: {result_data.get('username')}\n" + f"Overall Score: {score}% " + f"({correct_count}/{total_attempted} correct)\n" + f"Message: {result_data.get('message', 'No message received.')}" + ) + logger.info("Submission successful.") + + results_df = pd.DataFrame(results_log) + + if not results_df.empty: + success_count = sum(1 for status in results_df.get("Status", []) if status == "Success") + error_count = sum(1 for status in results_df.get("Status", []) if status == "Error") + + try: + avg_time = sum(float(time.strip('s')) for time in results_df["Processing Time"]) / len(results_df) + avg_time_str = f"{avg_time:.2f}s" + except: + avg_time_str = "N/A" + + logger.info(f"Processed {len(results_df)} questions: {success_count} successes, {error_count} errors, avg time: {avg_time_str}") + + progress(1.0, desc="Evaluation complete!") + return final_status, results_df + except requests.exceptions.HTTPError as e: + progress(1.0, desc="Submission failed!") + error_detail = f"Server responded with status {e.response.status_code}." + try: + error_json = e.response.json() + error_detail += f" Detail: {error_json.get('detail', e.response.text)}" + except requests.exceptions.JSONDecodeError: + error_detail += f" Response: {e.response.text[:500]}" + status_message = f"[ERROR] Submission Failed: {error_detail}" + logger.error(status_message) + results_df = pd.DataFrame(results_log) + return status_message, results_df + except requests.exceptions.Timeout: + progress(1.0, desc="Submission timed out!") + status_message = "[ERROR] Submission Failed: The request timed out." + logger.error(status_message) + results_df = pd.DataFrame(results_log) + return status_message, results_df + except requests.exceptions.RequestException as e: + progress(1.0, desc="Network error!") + status_message = f"[ERROR] Submission Failed: Network error - {e}" + logger.error(status_message) + results_df = pd.DataFrame(results_log) + return status_message, results_df + except Exception as e: + progress(1.0, desc="Error during submission!") + status_message = f"[ERROR] An unexpected error occurred during submission: {e}" + logger.error(status_message) + results_df = pd.DataFrame(results_log) + return status_message, results_df + + +# Check if we're running in a HF Space environment +space_host = os.getenv("SPACE_HOST") + +# Create Gradio interface +with gr.Blocks(theme=gr.themes.Soft()) as demo: + gr.Markdown("# GAIA Agent Evaluation Runner") + with gr.Tabs(): + with gr.TabItem("Run Evaluation"): + gr.Markdown( + """ + **Instructions:** + + 1. Enter your username (optional) for submission tracking. + 2. Click 'Run Evaluation & Submit All Answers' to fetch questions, run the GAIA agent, submit answers, and see the score. + 3. The progress bar will show you the current status of the evaluation. + 4. Results will appear in the table below once processing is complete. + + **Note:** Running the evaluation may take some time as the agent processes all questions. Please be patient. + """ + ) + + with gr.Row(): + with gr.Column(scale=1): + username_input = gr.Textbox( + label="Username (optional)", + placeholder="Enter username for submission", + value="anonymous" + ) + with gr.Column(scale=2): + run_button = gr.Button("Run Evaluation & Submit All Answers", + variant="primary", size="lg") + + status_output = gr.Textbox( + label="Run Status / Submission Result", + lines=6, + interactive=False, + elem_id="status_output" + ) + + results_accordion = gr.Accordion("Results", open=False, elem_id="results_accordion") + with results_accordion: + results_table = gr.DataFrame( + label="Questions and Agent Answers", + wrap=True, + elem_id="results_table" + ) + + with gr.Row(): + with gr.Column(): + gr.Markdown("### Result Statistics") + stats_output = gr.JSON(label="Statistics", visible=True) + + with gr.TabItem("About GAIA Agent"): + gr.Markdown( + """ + ## About GAIA Agent + + This agent is designed to answer questions from the GAIA (Grounded AI Alignment) benchmark. + + ### Key Features: + + - **Architecture**: Built with LangChain/LangGraph for workflow management + - **Tools**: Integrates multiple tools for web search, reasoning, and multimodal understanding + - **Memory**: Implements memory components for context retention + - **Structured Approach**: Uses a systematic process for question analysis and answer formulation + + ### Components: + + 1. **Web Tools**: Search engines, content extraction, and web navigation + 2. **Reasoning Tools**: Step-by-step reasoning, mathematical calculations, fact verification + 3. **Multimodal Tools**: Image analysis, chart interpretation, document parsing + 4. **Memory Components**: Conversation history, working memory, result caching + + ### Performance Metrics: + + The agent's performance is evaluated based on: + - Accuracy of answers + - Reasoning quality + - Ability to use appropriate tools + - Handling of edge cases + """ + ) + + with gr.TabItem("Help & Troubleshooting"): + gr.Markdown( + """ + ## Help & Troubleshooting + + ### Common Issues: + + 1. **Timeout Errors**: If the agent takes too long to process a question, try running the evaluation again. + 2. **API Errors**: If you encounter API errors, check your internet connection and try again. + + ### Error Messages: + + - **"Error initializing GAIA agent"**: There was a problem setting up the agent. Check the logs for details. + - **"Error fetching questions"**: The system couldn't retrieve questions from the API. + - **"Submission Failed"**: There was a problem submitting your answers. Check the error details. + + ### Contact: + + If you continue to experience issues, please open an issue on the GitHub repository or contact the course instructors. + """ + ) + + run_button.click( + fn=run_and_submit_all, + inputs=[username_input], + outputs=[status_output, results_table] + ).then( + fn=lambda df: { + "Total Questions": len(df) if df is not None else 0, + "Successful": sum(1 for status in df.get("Status", []) if status == "Success") if df is not None else 0, + "Errors": sum(1 for status in df.get("Status", []) if status == "Error") if df is not None else 0, + "Average Processing Time": f"{sum(float(time.strip('s')) for time in df['Processing Time']) / len(df):.2f}s" if df is not None and len(df) > 0 else "N/A" + } if df is not None else {}, + inputs=[results_table], + outputs=[stats_output] + ).then( + fn=lambda: gr.update(open=True, visible=True), + outputs=[results_accordion] + ) + +if __name__ == "__main__": + space_host_startup = os.getenv("SPACE_HOST") + space_id_startup = os.getenv("SPACE_ID") # Get SPACE_ID at startup + + # Handle Space host configurations + if space_host_startup: + logger.info(f"Running on Hugging Face Space host: {space_host_startup}") + + # Print repository URLs if SPACE_ID is available + if space_id_startup: + repo_url = f"https://huggingface.co/spaces/{space_id_startup}" + logger.info(f"Space repository URL: {repo_url}") + + # Launch the Gradio interface demo.launch(debug=True, share=False) \ No newline at end of file diff --git a/app_demo_mode_patch.py b/app_demo_mode_patch.py deleted file mode 100644 index f13511626d79900f322745b40c10a411069604de..0000000000000000000000000000000000000000 --- a/app_demo_mode_patch.py +++ /dev/null @@ -1,104 +0,0 @@ -""" -Patch for app.py to add demo mode support -This allows the app to run without OAuth for testing/development -""" -import os -import sys - -def apply_demo_mode_patch(): - """ - Applies a patch to app.py to add demo mode support - Returns True if successful, False otherwise - """ - # Check if app.py exists - if not os.path.exists('app.py'): - print("Error: app.py not found in current directory") - return False - - # Read the original file - with open('app.py', 'r') as f: - content = f.read() - - # Check if already patched - if 'DEMO_MODE' in content: - print("app.py already has demo mode support") - return True - - # Make the modifications - # 1. Add demo mode imports and configuration after the existing imports - import_section_end = "from agent.config import VERBOSE" - demo_mode_code = """from agent.config import VERBOSE - -# Set to True to enable demo mode (skip OAuth requirement) -DEMO_MODE = os.getenv("DEMO_MODE", "").lower() in ("true", "1", "yes") -""" - - # Replace the import section - content = content.replace(import_section_end, demo_mode_code) - - # 2. Modify the OAuth check in run_and_submit_all function - oauth_check = """ # OAuth is REQUIRED for assessment submission - if not profile: - logger.warning("User not logged in.") - return "Please Login to Hugging Face with the button. OAuth login is required for assessment submission.", None""" - - demo_mode_oauth_check = """ # OAuth is REQUIRED for assessment submission (unless in demo mode) - if not profile and not DEMO_MODE: - logger.warning("User not logged in.") - return "Please Login to Hugging Face with the button. OAuth login is required for assessment submission.", None - - # Use demo username in demo mode - if DEMO_MODE and not profile: - logger.warning("Running in DEMO MODE - using demo username") - username = "demo_user" - else: - username = f"{profile.username}\"""" - - # Replace the OAuth check - content = content.replace(oauth_check, demo_mode_oauth_check) - - # 3. Modify the blocks initialization to add a demo mode indicator - blocks_init = "with gr.Blocks(theme=gr.themes.Soft()) as demo:" - demo_mode_blocks_init = """# Add demo mode indicator if enabled -if DEMO_MODE: - logger.warning("Running in DEMO MODE - OAuth requirement disabled") - -with gr.Blocks(theme=gr.themes.Soft()) as demo: - if DEMO_MODE: - gr.Markdown("# GAIA Agent Evaluation Runner (DEMO MODE)") - else: - gr.Markdown("# GAIA Agent Evaluation Runner")""" - - # Replace blocks initialization - content = content.replace(blocks_init, demo_mode_blocks_init) - - # 4. Add demo mode warning at the end - demo_warning = """ -# Add demo mode warning -if DEMO_MODE: - gr.Markdown( - \"\"\" - ## 🚨 DEMO MODE ACTIVE 🚨 - - Running in demo mode - OAuth login is disabled. - Submissions will not be recorded to the leaderboard. - - For actual assessment submission, set DEMO_MODE=false and enable OAuth in Space settings. - \"\"\" - ) -""" - - # Add the warning before the if __name__ section - name_main_section = "if __name__ == \"__main__\":" - content = content.replace(name_main_section, demo_warning + "\n" + name_main_section) - - # Write the modified content back to app.py - with open('app.py', 'w') as f: - f.write(content) - - print("Successfully applied demo mode patch to app.py") - return True - -if __name__ == "__main__": - success = apply_demo_mode_patch() - sys.exit(0 if success else 1) \ No newline at end of file diff --git a/app_fixed.py b/app_fixed.py deleted file mode 100644 index 80e9d87e26b98ef27477b0b2d87cf3f625b709d8..0000000000000000000000000000000000000000 --- a/app_fixed.py +++ /dev/null @@ -1,367 +0,0 @@ -import os -import gradio as gr -import requests -import inspect -import pandas as pd -import logging -import time -from typing import Dict, Any, List, Optional, Tuple, Union - -DEFAULT_API_URL = "https://agents-course-unit4-scoring.hf.space" - -from agent import GAIAAgent -from agent.config import VERBOSE - -logger = logging.getLogger("gaia_app") -if VERBOSE: - logging.basicConfig(level=logging.DEBUG, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s') -else: - logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s') - -def run_and_submit_all(profile: Optional[gr.OAuthProfile] = None, progress=gr.Progress()) -> Tuple[str, Optional[pd.DataFrame]]: - """ - Fetches all questions, runs the GAIA Agent on them, submits all answers, - and displays the results with progress tracking. - - Args: - profile: Hugging Face user profile for authentication - progress: Gradio progress tracker - - Returns: - Tuple containing status message and results dataframe - """ - space_id = os.getenv("SPACE_ID") # Get the SPACE_ID for sending link to the code - # OAuth is REQUIRED for assessment submission - if not profile: - logger.warning("User not logged in.") - return "Please Login to Hugging Face with the button. OAuth login is required for assessment submission.", None - - username = f"{profile.username}" - logger.info(f"User logged in: {username}") - - api_url = DEFAULT_API_URL - questions_url = f"{api_url}/questions" - submit_url = f"{api_url}/submit" - - progress(0, desc="Initializing GAIA Agent...") - try: - start_time = time.time() - agent = GAIAAgent() - init_time = time.time() - start_time - logger.info(f"GAIA Agent initialized successfully in {init_time:.2f} seconds.") - progress(0.1, desc=f"Agent initialized in {init_time:.2f}s") - except Exception as e: - error_msg = f"Error initializing GAIA agent: {e}" - logger.error(error_msg) - return error_msg, None - - agent_code = f"https://huggingface.co/spaces/{space_id}/tree/main" - logger.info(f"Agent code URL: {agent_code}") - - progress(0.15, desc="Fetching questions...") - logger.info(f"Fetching questions from: {questions_url}") - try: - response = requests.get(questions_url, timeout=15) - response.raise_for_status() - questions_data = response.json() - if not questions_data: - error_msg = "Fetched questions list is empty or invalid format." - logger.warning(error_msg) - return error_msg, None - - question_count = len(questions_data) - logger.info(f"Fetched {question_count} questions.") - progress(0.2, desc=f"Fetched {question_count} questions") - except requests.exceptions.RequestException as e: - error_msg = f"Error fetching questions: {e}" - logger.error(error_msg) - return error_msg, None - except requests.exceptions.JSONDecodeError as e: - error_msg = f"Error decoding JSON response from questions endpoint: {e}" - logger.error(f"{error_msg}\nResponse text: {response.text[:500]}") - return error_msg, None - except Exception as e: - error_msg = f"An unexpected error occurred fetching questions: {e}" - logger.error(error_msg) - return error_msg, None - - results_log = [] - answers_payload = [] - total_questions = len(questions_data) - - logger.info(f"Running agent on {total_questions} questions...") - - question_progress_increment = 0.6 / total_questions # 60% of progress bar for processing questions - - for i, item in enumerate(questions_data): - task_id = item.get("task_id") - question_text = item.get("question") - - if not task_id or question_text is None: - logger.warning(f"Skipping item with missing task_id or question: {item}") - continue - - progress_value = 0.2 + (i * question_progress_increment) - progress(progress_value, desc=f"Processing question {i+1}/{total_questions}...") - - logger.info(f"Processing question {i+1}/{total_questions}: {question_text[:50]}...") - question_start_time = time.time() - - try: - submitted_answer = agent(question_text) - processing_time = time.time() - question_start_time - - answers_payload.append({"task_id": task_id, "submitted_answer": submitted_answer}) - results_log.append({ - "Task ID": task_id, - "Question": question_text, - "Submitted Answer": submitted_answer, - "Processing Time": f"{processing_time:.2f}s", - "Status": "Success" - }) - - logger.info(f"Question {i+1} processed in {processing_time:.2f}s. Answer: {submitted_answer[:50]}...") - - except Exception as e: - error_msg = f"Error running agent on task {task_id}: {e}" - logger.error(error_msg) - - results_log.append({ - "Task ID": task_id, - "Question": question_text, - "Submitted Answer": f"AGENT ERROR: {e}", - "Processing Time": f"{time.time() - question_start_time:.2f}s", - "Status": "Error" - }) - - if not answers_payload: - error_msg = "Agent did not produce any answers to submit." - logger.warning(error_msg) - return error_msg, pd.DataFrame(results_log) - - # Only proceed with submission if not in demo mode - progress(0.8, desc="Preparing submission...") - submission_data = {"username": username.strip(), "agent_code": agent_code, "answers": answers_payload} - status_update = f"Agent finished. Submitting {len(answers_payload)} answers for user '{username}'..." - logger.info(status_update) - - progress(0.9, desc=f"Submitting {len(answers_payload)} answers...") - logger.info(f"Submitting {len(answers_payload)} answers to: {submit_url}") - try: - response = requests.post(submit_url, json=submission_data, timeout=60) - response.raise_for_status() - result_data = response.json() - - correct_count = result_data.get('correct_count', 0) - total_attempted = result_data.get('total_attempted', 0) - score = result_data.get('score', 'N/A') - - final_status = ( - f"[SUCCESS] Submission Successful!\n" - f"User: {result_data.get('username')}\n" - f"Overall Score: {score}% " - f"({correct_count}/{total_attempted} correct)\n" - f"Message: {result_data.get('message', 'No message received.')}" - ) - logger.info("Submission successful.") - - results_df = pd.DataFrame(results_log) - - if not results_df.empty: - success_count = sum(1 for status in results_df.get("Status", []) if status == "Success") - error_count = sum(1 for status in results_df.get("Status", []) if status == "Error") - - try: - avg_time = sum(float(time.strip('s')) for time in results_df["Processing Time"]) / len(results_df) - avg_time_str = f"{avg_time:.2f}s" - except: - avg_time_str = "N/A" - - logger.info(f"Processed {len(results_df)} questions: {success_count} successes, {error_count} errors, avg time: {avg_time_str}") - - progress(1.0, desc="Evaluation complete!") - return final_status, results_df - except requests.exceptions.HTTPError as e: - progress(1.0, desc="Submission failed!") - error_detail = f"Server responded with status {e.response.status_code}." - try: - error_json = e.response.json() - error_detail += f" Detail: {error_json.get('detail', e.response.text)}" - except requests.exceptions.JSONDecodeError: - error_detail += f" Response: {e.response.text[:500]}" - status_message = f"[ERROR] Submission Failed: {error_detail}" - logger.error(status_message) - results_df = pd.DataFrame(results_log) - return status_message, results_df - except requests.exceptions.Timeout: - progress(1.0, desc="Submission timed out!") - status_message = "[ERROR] Submission Failed: The request timed out." - logger.error(status_message) - results_df = pd.DataFrame(results_log) - return status_message, results_df - except requests.exceptions.RequestException as e: - progress(1.0, desc="Network error!") - status_message = f"[ERROR] Submission Failed: Network error - {e}" - logger.error(status_message) - results_df = pd.DataFrame(results_log) - return status_message, results_df - except Exception as e: - progress(1.0, desc="Error during submission!") - status_message = f"[ERROR] An unexpected error occurred during submission: {e}" - logger.error(status_message) - results_df = pd.DataFrame(results_log) - return status_message, results_df - - -# Check if we're running in a HF Space environment -space_host = os.getenv("SPACE_HOST") - -# Import HfApi for OAuth functionality -try: - from huggingface_hub import HfApi -except ImportError: - logger.warning("huggingface_hub not available - please install it for OAuth functionality") -except Exception as e: - logger.warning(f"Error importing huggingface_hub: {e}") - -with gr.Blocks(theme=gr.themes.Soft()) as demo: - gr.Markdown("# GAIA Agent Evaluation Runner") - with gr.Tabs(): - with gr.TabItem("Run Evaluation"): - gr.Markdown( - """ - **Instructions:** - - 1. Log in to your Hugging Face account using the button below. This uses your HF username for submission. - 2. Click 'Run Evaluation & Submit All Answers' to fetch questions, run the GAIA agent, submit answers, and see the score. - 3. The progress bar will show you the current status of the evaluation. - 4. Results will appear in the table below once processing is complete. - - **Note:** Running the evaluation may take some time as the agent processes all questions. Please be patient. - """ - ) - - with gr.Row(): - with gr.Column(scale=1): - gr.LoginButton() - with gr.Column(scale=2): - run_button = gr.Button("Run Evaluation & Submit All Answers", - variant="primary", size="lg") - - status_output = gr.Textbox( - label="Run Status / Submission Result", - lines=6, - interactive=False, - elem_id="status_output" - ) - - results_accordion = gr.Accordion("Results", open=False, elem_id="results_accordion") - with results_accordion: - results_table = gr.DataFrame( - label="Questions and Agent Answers", - wrap=True, - elem_id="results_table" - ) - - with gr.Row(): - with gr.Column(): - gr.Markdown("### Result Statistics") - stats_output = gr.JSON(label="Statistics", visible=True) - - with gr.TabItem("About GAIA Agent"): - gr.Markdown( - """ - ## About GAIA Agent - - This agent is designed to answer questions from the GAIA (Grounded AI Alignment) benchmark. - - ### Key Features: - - - **Architecture**: Built with LangChain/LangGraph for workflow management - - **Tools**: Integrates multiple tools for web search, reasoning, and multimodal understanding - - **Memory**: Implements memory components for context retention - - **Structured Approach**: Uses a systematic process for question analysis and answer formulation - - ### Components: - - 1. **Web Tools**: Search engines, content extraction, and web navigation - 2. **Reasoning Tools**: Step-by-step reasoning, mathematical calculations, fact verification - 3. **Multimodal Tools**: Image analysis, chart interpretation, document parsing - 4. **Memory Components**: Conversation history, working memory, result caching - - ### Performance Metrics: - - The agent's performance is evaluated based on: - - Accuracy of answers - - Reasoning quality - - Ability to use appropriate tools - - Handling of edge cases - """ - ) - - with gr.TabItem("Help & Troubleshooting"): - gr.Markdown( - """ - ## Help & Troubleshooting - - ### Common Issues: - - 1. **Login Problems**: Make sure you're logged in to Hugging Face before running the evaluation. - 2. **Timeout Errors**: If the agent takes too long to process a question, try running the evaluation again. - 3. **API Errors**: If you encounter API errors, check your internet connection and try again. - - ### Error Messages: - - - **"Please Login to Hugging Face"**: You need to log in using the login button. - - **"Error initializing GAIA agent"**: There was a problem setting up the agent. Check the logs for details. - - **"Error fetching questions"**: The system couldn't retrieve questions from the API. - - **"Submission Failed"**: There was a problem submitting your answers. Check the error details. - - ### Contact: - - If you continue to experience issues, please open an issue on the GitHub repository or contact the course instructors. - """ - ) - - run_button.click( - fn=run_and_submit_all, - outputs=[status_output, results_table] - ).then( - fn=lambda df: { - "Total Questions": len(df) if df is not None else 0, - "Successful": sum(1 for status in df.get("Status", []) if status == "Success") if df is not None else 0, - "Errors": sum(1 for status in df.get("Status", []) if status == "Error") if df is not None else 0, - "Average Processing Time": f"{sum(float(time.strip('s')) for time in df['Processing Time']) / len(df):.2f}s" if df is not None and len(df) > 0 else "N/A" - } if df is not None else {}, - inputs=[results_table], - outputs=[stats_output] - ).then( - fn=lambda: gr.update(open=True, visible=True), - outputs=[results_accordion] - ) - -# Add note that OAuth is required -gr.Markdown( - """ - ## [WARNING] Important Note - - This GAIA Assessment requires OAuth login for submission to the leaderboard. - Please make sure to log in with your Hugging Face account before running the evaluation. - """ -) - -if __name__ == "__main__": - space_host_startup = os.getenv("SPACE_HOST") - space_id_startup = os.getenv("SPACE_ID") # Get SPACE_ID at startup - - # Handle Space host configurations - if space_host_startup: - logger.info(f"Running on Hugging Face Space host: {space_host_startup}") - - # Print repository URLs if SPACE_ID is available - if space_id_startup: - repo_url = f"https://huggingface.co/spaces/{space_id_startup}" - logger.info(f"Space repository URL: {repo_url}") - - # Launch the Gradio interface - demo.launch(debug=True, share=False) \ No newline at end of file diff --git a/app_no_oauth.py b/app_no_oauth.py deleted file mode 100644 index b947fc3a0885ba7e2110a3dec4ebd553e5577fa8..0000000000000000000000000000000000000000 --- a/app_no_oauth.py +++ /dev/null @@ -1,226 +0,0 @@ -import os -import gradio as gr -import requests -import inspect -import pandas as pd -import logging -import time -from typing import Dict, Any, List, Optional, Tuple, Union - -DEFAULT_API_URL = "https://agents-course-unit4-scoring.hf.space" - -from agent import GAIAAgent -from agent.config import VERBOSE - -logger = logging.getLogger("gaia_app") -if VERBOSE: - logging.basicConfig(level=logging.DEBUG, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s') -else: - logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s') - -def run_and_submit_all(username: str = "anonymous", progress=gr.Progress()) -> Tuple[str, Optional[pd.DataFrame]]: - """ - Fetches all questions, runs the GAIA Agent on them, submits all answers, - and displays the results with progress tracking. - - Args: - username: Username for submission (default: anonymous) - progress: Gradio progress tracker - - Returns: - Tuple of (status message, results dataframe) - """ - api_url = os.environ.get("GAIA_API_URL", DEFAULT_API_URL) - - try: - # Step 1: Fetch questions - progress(0, desc="Fetching questions...") - questions_response = requests.get(f"{api_url}/questions") - questions_response.raise_for_status() - questions = questions_response.json() - - if not questions: - return "Error: No questions received from the API.", None - - # Step 2: Initialize agent and prepare for processing - progress(0.1, desc="Initializing GAIA Agent...") - agent = GAIAAgent() - - # Step 3: Process each question - results = [] - total_questions = len(questions) - - for i, question in enumerate(questions): - question_id = question.get("id", f"q{i+1}") - question_text = question.get("question", "") - - progress_value = 0.1 + (0.7 * (i / total_questions)) - progress(progress_value, desc=f"Processing question {i+1}/{total_questions}: {question_text[:30]}...") - - # Run the agent on the question - try: - start_time = time.time() - answer = agent.run(question_text) - end_time = time.time() - processing_time = end_time - start_time - - results.append({ - "question_id": question_id, - "question": question_text, - "answer": answer, - "processing_time": f"{processing_time:.2f}s" - }) - - logger.info(f"Processed question {question_id} in {processing_time:.2f}s") - except Exception as e: - error_msg = f"Error processing question {question_id}: {str(e)}" - logger.error(error_msg) - results.append({ - "question_id": question_id, - "question": question_text, - "answer": f"ERROR: {str(e)}", - "processing_time": "N/A" - }) - - # Step 4: Submit all answers - progress(0.8, desc="Submitting answers...") - - answers_payload = { - "username": username, - "answers": {item["question_id"]: item["answer"] for item in results} - } - - submission_response = requests.post( - f"{api_url}/submit", - json=answers_payload - ) - - # Step 5: Process submission response - progress(0.9, desc="Processing submission results...") - - if submission_response.status_code == 200: - submission_data = submission_response.json() - score = submission_data.get("score", "N/A") - feedback = submission_data.get("feedback", {}) - - # Add score and feedback to results - for item in results: - question_id = item["question_id"] - item["score"] = feedback.get(question_id, {}).get("score", "N/A") - item["feedback"] = feedback.get(question_id, {}).get("feedback", "") - - # Create DataFrame for display - results_df = pd.DataFrame(results) - - status_message = f"Submission successful! Overall score: {score}" - logger.info(status_message) - - progress(1.0, desc="Complete!") - return status_message, results_df - else: - error_message = f"Submission failed with status code {submission_response.status_code}: {submission_response.text}" - logger.error(error_message) - progress(1.0, desc="Failed!") - return error_message, pd.DataFrame(results) - - except Exception as e: - error_message = f"Error during evaluation process: {str(e)}" - logger.error(error_message) - progress(1.0, desc="Failed!") - return error_message, None - -# Create Gradio interface without OAuth -with gr.Blocks(theme=gr.themes.Soft()) as demo: - gr.Markdown("# GAIA Agent Evaluation Runner (No OAuth)") - - with gr.Tabs(): - with gr.TabItem("Run Evaluation"): - gr.Markdown( - """ - **Instructions:** - - 1. Enter your username (optional) for submission tracking. - 2. Click 'Run Evaluation & Submit All Answers' to fetch questions, run the GAIA agent, submit answers, and see the score. - 3. The progress bar will show you the current status of the evaluation. - 4. Results will appear in the table below once processing is complete. - - **Note:** Running the evaluation may take some time as the agent processes all questions. Please be patient. - """ - ) - - with gr.Row(): - with gr.Column(scale=1): - username_input = gr.Textbox( - label="Username (optional)", - placeholder="Enter username for submission", - value="anonymous" - ) - with gr.Column(scale=2): - run_button = gr.Button("Run Evaluation & Submit All Answers", - variant="primary", size="lg") - - status_output = gr.Textbox( - label="Run Status / Submission Result", - lines=6, - interactive=False, - elem_id="status_output" - ) - - results_accordion = gr.Accordion("Results", open=False, elem_id="results_accordion") - with results_accordion: - results_table = gr.DataFrame( - label="Questions and Agent Answers", - wrap=True, - elem_id="results_table" - ) - - with gr.Row(): - with gr.Column(): - score_chart = gr.Plot( - label="Score Distribution", - elem_id="score_chart" - ) - with gr.Column(): - time_chart = gr.Plot( - label="Processing Time", - elem_id="time_chart" - ) - - with gr.TabItem("About"): - gr.Markdown( - """ - ## About GAIA Agent Evaluation - - This application runs the GAIA (Generative AI Assistant) agent on a set of evaluation questions - and submits the answers for scoring. The agent uses various tools and APIs to answer questions - about factual information. - - ### Features - - - Fetches evaluation questions from the scoring API - - Runs the GAIA agent on each question - - Submits answers for scoring - - Displays results with feedback - - ### Technical Details - - The GAIA agent is built using: - - - LangChain for agent orchestration - - Various search tools and APIs - - Supabase for memory storage - - For more information, see the project documentation. - """ - ) - - # Set up event handlers - run_button.click( - fn=run_and_submit_all, - inputs=[username_input], - outputs=[status_output, results_table] - ) - -# Launch the app -if __name__ == "__main__": - demo.launch() \ No newline at end of file diff --git a/backup_tests/minimal_supabase_test.py b/backup_tests/minimal_supabase_test.py deleted file mode 100644 index f67a10cad18e826f163fefcfe0c7484df5dc68b2..0000000000000000000000000000000000000000 --- a/backup_tests/minimal_supabase_test.py +++ /dev/null @@ -1,75 +0,0 @@ -""" -Minimal Supabase Connection Test - -This script performs a minimal test of the Supabase connection. -""" - -import os -import sys -import logging -from dotenv import load_dotenv - -logging.basicConfig(level=logging.DEBUG) -logger = logging.getLogger("minimal_supabase_test") - -dotenv_path = os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), '.env') -logger.info(f"Loading environment variables from {dotenv_path}") -load_dotenv(dotenv_path) - -supabase_url = os.environ.get("SUPABASE_URL") -supabase_key = os.environ.get("SUPABASE_KEY") - -logger.info(f"SUPABASE_URL: {supabase_url}") -logger.info(f"SUPABASE_KEY: {'***' if supabase_key else None}") - -if not supabase_url or not supabase_key: - logger.error("Supabase URL or key not provided") - logger.error("Please set SUPABASE_URL and SUPABASE_KEY environment variables") - sys.exit(1) - -try: - from supabase import create_client, Client - logger.info("Supabase package imported successfully") -except ImportError as e: - logger.error(f"Error importing supabase: {str(e)}") - logger.error("Please install the supabase package with: pip install supabase") - sys.exit(1) - -try: - logger.info(f"Connecting to Supabase at {supabase_url}") - client = create_client(supabase_url, supabase_key) - logger.info("Supabase client created successfully") - - import time - from datetime import datetime - - test_id = int(time.time()) - test_name = f"Test Record {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}" - - logger.info(f"Inserting test record with ID {test_id} and name '{test_name}'...") - insert_result = client.table("test_connection").insert({ - "id": test_id, - "name": test_name - }).execute() - - if insert_result.data: - logger.info("Test record inserted successfully") - else: - logger.warning("Failed to insert test record") - - logger.info("Querying test_connection table...") - result = client.table("test_connection").select("*").limit(5).execute() - - if result.data is not None: - logger.info(f"Query successful, found {len(result.data)} records") - for record in result.data: - else: - logger.warning("Query successful but no data returned") - - logger.info("Connection to Supabase successful") -except Exception as e: - logger.error(f"Error connecting to Supabase: {str(e)}") - import traceback - logger.error(f"Traceback: {traceback.format_exc()}") - sys.exit(1) - diff --git a/backup_tests/test_supabase_tables_final.py b/backup_tests/test_supabase_tables_final.py deleted file mode 100644 index 1b090aa33f1d1db5867117c25cd56370f9d441e6..0000000000000000000000000000000000000000 --- a/backup_tests/test_supabase_tables_final.py +++ /dev/null @@ -1,167 +0,0 @@ -""" -Final test script for Supabase tables in GAIA Agent. - -This script applies the fix to ensure all Supabase tables are properly utilized, -then runs a simple test to verify the fix works correctly. - -Usage: - python -m tests.test_supabase_tables_final -""" - -import os -import sys -import json -import logging -import time -import uuid -import datetime -from typing import Dict, Any, List, Optional - -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(name)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("test_supabase_tables_final") - -sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) - -try: - from memory.supabase_memory import ( - SupabaseMemory, - WorkingMemory - ) - from agent.config import get_memory_config - - from memory.fix_supabase_tables import apply_monkey_patch, ensure_tables_exist -except ImportError as e: - logger.error(f"Error importing required components: {str(e)}") - sys.exit(1) - -def apply_fix_and_verify(): - """Apply the fix and verify it works correctly.""" - logger.info("Applying fix for Supabase tables...") - - if not ensure_tables_exist(): - logger.error("Failed to ensure tables exist") - return False - - if not apply_monkey_patch(): - logger.error("Failed to apply monkey patch") - return False - - logger.info("Fix applied successfully") - return True - -def test_working_memory(): - """Test if working memory is using the correct table.""" - logger.info("Testing working memory...") - - memory_config = get_memory_config() - memory_config["enabled"] = True - - base_memory = SupabaseMemory(config=memory_config) - session_id = f"test-session-{uuid.uuid4()}" - working_memory = WorkingMemory(base_memory, session_id) - - logger.info(f"Base memory table: {base_memory.table_name}") - logger.info(f"Working memory table: {working_memory.table_name}") - - test_key = f"test_working_{int(time.time())}" - test_value = { - "test_data": "This is test data for working memory", - "timestamp": time.time(), - "random_id": str(uuid.uuid4()) - } - - logger.info(f"Storing data in working memory: {test_key}") - store_result = working_memory.store_intermediate_result(test_key, test_value) - - if not store_result: - logger.error("Failed to store test value") - return False - - logger.info(f"Retrieving data from working memory: {test_key}") - retrieved_value = working_memory.get_intermediate_result(test_key) - - if retrieved_value != test_value: - logger.error(f"Retrieved value does not match test value") - logger.error(f"Expected: {test_value}") - logger.error(f"Got: {retrieved_value}") - return False - - logger.info("✅ Store and retrieve operations successful") - - logger.info("Verifying data was stored in the working memory table...") - - if base_memory.client and base_memory.initialized: - try: - working_key = f"working:{session_id}:{test_key}" - response = base_memory.client.table(working_memory.table_name).select("*").eq("key", working_key).execute() - - if not response.data or len(response.data) == 0: - logger.error("No data found in working memory table") - - response = base_memory.client.table(base_memory.table_name).select("*").eq("key", working_key).execute() - - if response.data and len(response.data) > 0: - logger.error("❌ Data was stored in base memory table instead of working memory table") - return False - else: - logger.error("❌ Data was not stored in any table") - return False - - logger.info(f"✅ Data found in working memory table: {len(response.data)} entries") - - entry_data = response.data[0] - stored_value = entry_data.get("value") - - if isinstance(stored_value, str): - try: - stored_value = json.loads(stored_value) - except json.JSONDecodeError: - pass - - if stored_value == test_value or stored_value == json.dumps(test_value): - logger.info("✅ Stored data matches expected value") - else: - logger.error("❌ Stored data does not match expected value") - logger.error(f"Expected: {test_value}") - logger.error(f"Got: {stored_value}") - return False - - logger.info("Cleaning up test data...") - working_memory.clear() - - return True - except Exception as e: - logger.error(f"Error verifying data storage: {str(e)}") - return False - - return False - -def main(): - """Main entry point for the test script.""" - - if not apply_fix_and_verify(): - sys.exit(1) - - if test_working_memory(): - - timestamp = datetime.datetime.now().strftime("%Y%m%d_%H%M%S") - filename = f"gaia_memory_test_results_{timestamp}.json" - - results = { - "timestamp": timestamp, - "success": True, - "message": "Working memory table is now being used correctly" - } - - with open(filename, "w") as f: - json.dump(results, f, indent=2) - - sys.exit(0) - else: - sys.exit(1) - -if __name__ == "__main__": - main() \ No newline at end of file diff --git a/backup_tests/test_supabase_tables_fix.py b/backup_tests/test_supabase_tables_fix.py deleted file mode 100644 index ea6c031f00afd7b410a29a692abbd31d25b7c567..0000000000000000000000000000000000000000 --- a/backup_tests/test_supabase_tables_fix.py +++ /dev/null @@ -1,276 +0,0 @@ -""" -Test script for verifying the Supabase tables fix. - -This script tests if the working memory table (gaia_memory_working) is properly -utilized by the GAIA agent. It verifies that data is correctly stored in and -retrieved from the working memory table. - -Usage: - python -m tests.test_supabase_tables_fix -""" - -import os -import sys -import json -import logging -import time -import uuid -from typing import Dict, Any, List, Optional - -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(name)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("test_supabase_tables") - -sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) - -try: - from memory.supabase_memory import ( - SupabaseMemory, - WorkingMemory - ) - from agent.config import get_memory_config - - from memory.simplified_memory_fix import apply_fix, create_working_memory_table -except ImportError as e: - logger.error(f"Error importing required components: {str(e)}") - sys.exit(1) - -class SupabaseTablesTest: - """Test class for verifying Supabase tables.""" - - def __init__(self): - """Initialize the test class.""" - self.memory_config = get_memory_config() - self.memory_config["enabled"] = True - - self.base_memory = None - self.working_memory = None - self.session_id = str(uuid.uuid4()) - - self.table_names = { - "base": "gaia_memory", - "working": "gaia_memory_working" - } - - self.results = { - "initialization": False, - "table_existence": {}, - "working_memory": {} - } - - def initialize_memory(self) -> bool: - """Initialize memory components.""" - logger.info("Initializing memory components...") - - try: - self.base_memory = SupabaseMemory(self.memory_config) - - self.working_memory = WorkingMemory(self.base_memory, self.session_id) - - if self.base_memory and self.base_memory.initialized: - logger.info("✅ Memory components initialized successfully") - self.results["initialization"] = True - return True - else: - logger.error("❌ Memory components initialization failed") - self.results["initialization"] = False - return False - - except Exception as e: - logger.error(f"Error initializing memory components: {str(e)}") - self.results["initialization"] = False - return False - - def check_table_existence(self) -> Dict[str, bool]: - """Check if the required tables exist.""" - logger.info("Checking table existence...") - - results = {} - - if not self.base_memory or not self.base_memory.initialized: - logger.error("Memory components not initialized") - return {"error": "Memory components not initialized"} - - for table_type, table_name in self.table_names.items(): - try: - response = self.base_memory.client.table(table_name).select("*").limit(1).execute() - - if hasattr(response, "data"): - logger.info(f"✅ Table {table_name} exists") - results[table_name] = True - else: - logger.warning(f"❌ Table {table_name} query failed") - results[table_name] = False - - except Exception as e: - logger.error(f"Error checking table {table_name}: {str(e)}") - results[table_name] = False - - self.results["table_existence"] = results - return results - - def test_working_memory(self) -> Dict[str, bool]: - """Test working memory operations.""" - logger.info("Testing working memory operations...") - - results = {} - - if not self.working_memory: - logger.error("Working memory not initialized") - return {"error": "Working memory not initialized"} - - try: - test_key = f"test_working_{int(time.time())}" - test_value = { - "test_data": "This is test data for working memory", - "timestamp": time.time(), - "random_id": str(uuid.uuid4()) - } - - logger.info(f"Storing data in working memory: {test_key}") - store_result = self.working_memory.store_intermediate_result(test_key, test_value) - - if store_result: - logger.info("✅ Store operation successful") - results["store"] = True - else: - logger.warning("❌ Store operation failed") - results["store"] = False - - logger.info(f"Retrieving data from working memory: {test_key}") - retrieved_value = self.working_memory.get_intermediate_result(test_key) - - if retrieved_value and retrieved_value == test_value: - logger.info("✅ Retrieve operation successful") - results["retrieve"] = True - else: - logger.warning(f"❌ Retrieve operation failed or data mismatch") - logger.warning(f"Expected: {test_value}") - logger.warning(f"Got: {retrieved_value}") - results["retrieve"] = False - - logger.info("Verifying data was stored in the correct table...") - - if self.base_memory.client and self.base_memory.initialized: - try: - working_key = f"working:{self.session_id}:{test_key}" - response = self.base_memory.client.table(self.table_names["working"]).select("*").eq("key", working_key).execute() - - if response.data and len(response.data) > 0: - logger.info(f"✅ Data found in working memory table: {len(response.data)} entries") - results["correct_table"] = True - - entry_data = response.data[0] - stored_value = entry_data.get("value") - - if isinstance(stored_value, str): - try: - stored_value = json.loads(stored_value) - except json.JSONDecodeError: - pass - - if stored_value == test_value or stored_value == json.dumps(test_value): - logger.info("✅ Stored data matches expected value") - results["data_match"] = True - else: - logger.warning("❌ Stored data does not match expected value") - logger.warning(f"Expected: {test_value}") - logger.warning(f"Got: {stored_value}") - results["data_match"] = False - else: - logger.warning("❌ No data found in working memory table") - results["correct_table"] = False - - response = self.base_memory.client.table(self.table_names["base"]).select("*").eq("key", working_key).execute() - - if response.data and len(response.data) > 0: - logger.warning("⚠️ Data was stored in base memory table instead of working memory table") - results["stored_in_base"] = True - else: - logger.warning("❌ Data was not stored in any table") - results["stored_in_base"] = False - - except Exception as e: - logger.error(f"Error verifying data storage: {str(e)}") - results["correct_table"] = False - - logger.info("Cleaning up test data...") - self.working_memory.clear() - - except Exception as e: - logger.error(f"Error testing working memory: {str(e)}") - results["error"] = str(e) - - self.results["working_memory"] = results - return results - - def run_all_tests(self) -> Dict[str, Any]: - """Run all tests.""" - logger.info("Running all tests...") - - if not self.initialize_memory(): - logger.error("Memory initialization failed, aborting tests") - return self.results - - self.check_table_existence() - - if self.results["table_existence"].get(self.table_names["working"], False): - logger.info("Working memory table exists, applying fix...") - apply_fix() - else: - logger.info("Working memory table does not exist, creating it...") - create_working_memory_table() - - self.check_table_existence() - - self.test_working_memory() - - return self.results - - def print_test_summary(self) -> None: - """Print a summary of the test results.""" - - if self.results["initialization"]: - else: - - for table_name, exists in self.results["table_existence"].items(): - if exists: - else: - - working_results = self.results["working_memory"] - - if working_results.get("store", False): - else: - - if working_results.get("retrieve", False): - else: - - if working_results.get("correct_table", False): - else: - - if working_results.get("stored_in_base", False): - - if working_results.get("data_match", False): - else: - - if (self.results["initialization"] and - all(self.results["table_existence"].values()) and - working_results.get("store", False) and - working_results.get("retrieve", False) and - working_results.get("correct_table", False) and - working_results.get("data_match", False)): - else: - - -def main(): - """Main entry point for the test script.""" - - test = SupabaseTablesTest() - test.run_all_tests() - - test.print_test_summary() - -if __name__ == "__main__": - main() \ No newline at end of file diff --git a/backup_tests/test_supabase_tables_production.py b/backup_tests/test_supabase_tables_production.py deleted file mode 100644 index fd2f4f9dfc4b45d0d31893f9281e15e9f5bbc5fb..0000000000000000000000000000000000000000 --- a/backup_tests/test_supabase_tables_production.py +++ /dev/null @@ -1,481 +0,0 @@ -""" -Comprehensive Unit Tests for Supabase Tables - -This script tests all Supabase tables used by the GAIA agent: -- gaia_memory: Main memory table -- gaia_memory_working: Working memory table -- gaia_memory_conversations: Conversation history table -- gaia_memory_cache: Result cache table - -The tests verify: -1. Each table is created correctly -2. Each memory class uses its dedicated table -3. Data is properly isolated between tables -4. Performance metrics are captured -5. All test data is cleaned up after tests - -Usage: - python tests/test_supabase_tables_production.py -""" - -import os -import sys -import json -import time -import uuid -import logging -import unittest -from datetime import datetime, timedelta -from typing import Dict, Any, List, Optional, Union, Tuple - -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(name)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("test_supabase_tables_production") - -parent_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) -sys.path.insert(0, parent_dir) - -try: - from supabase import create_client, Client -except ImportError as e: - logger.error(f"Error importing supabase: {str(e)}") - sys.exit(1) - -try: - from agent.config import ( - SUPABASE_URL, - SUPABASE_KEY, - SUPABASE_PROJECT_ID, - SUPABASE_PROJECT_NAME - ) -except ImportError as e: - logger.error(f"Error importing configuration: {str(e)}") - sys.exit(1) - -def get_table_schema(client: Client, table_name: str) -> List[str]: - """Get the schema of a table.""" - try: - result = client.table(table_name).select("*").limit(1).execute() - - if result.data and len(result.data) > 0: - return list(result.data[0].keys()) - else: - schema_result = client.from_("information_schema.columns").select("column_name").eq("table_name", table_name).execute() - if schema_result.data: - return [col["column_name"] for col in schema_result.data] - else: - return [] - except Exception as e: - logger.error(f"Error getting schema for table {table_name}: {str(e)}") - return [] - -def ensure_table_exists(client: Client, table_name: str) -> bool: - """Ensure a table exists in Supabase.""" - try: - client.table(table_name).select("*").limit(1).execute() - logger.info(f"Table {table_name} exists") - return True - except Exception as e: - logger.warning(f"Error checking table {table_name}: {str(e)}") - logger.info(f"Creating table {table_name}") - - try: - sql = f""" - CREATE TABLE IF NOT EXISTS {table_name} ( - id UUID PRIMARY KEY DEFAULT uuid_generate_v4(), - key TEXT NOT NULL UNIQUE, - value TEXT NOT NULL, - metadata JSONB, - created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), - expires_at TIMESTAMPTZ - ); - - CREATE INDEX IF NOT EXISTS idx_{table_name}_key ON {table_name}(key); - CREATE INDEX IF NOT EXISTS idx_{table_name}_expires_at ON {table_name}(expires_at); - """ - - client.query(sql).execute() - - client.table(table_name).select("*").limit(1).execute() - - logger.info(f"Table {table_name} created successfully") - return True - except Exception as e: - logger.error(f"Error creating table {table_name}: {str(e)}") - return False - -def store_value(client: Client, table_name: str, key: str, value: Any, metadata: Optional[Dict[str, Any]] = None, ttl: Optional[int] = None, session_id: Optional[str] = None) -> bool: - """Store a value in a table.""" - try: - entry = { - "key": key, - "value": json.dumps(value) if not isinstance(value, str) else value, - "metadata": metadata or {} - } - - if ttl: - entry["expires_at"] = (datetime.now() + timedelta(seconds=ttl)).isoformat() - - if session_id and table_name.endswith("_working"): - entry["session_id"] = session_id - - client.table(table_name).upsert(entry).execute() - - logger.info(f"Stored value in table {table_name}: {key}") - return True - except Exception as e: - logger.error(f"Error storing value in table {table_name}: {key} - {str(e)}") - return False - -def retrieve_value(client: Client, table_name: str, key: str, default: Any = None) -> Any: - """Retrieve a value from a table.""" - try: - result = client.table(table_name).select("*").eq("key", key).execute() - - if result.data and len(result.data) > 0: - entry = result.data[0] - - value = entry.get("value") - if isinstance(value, str): - try: - value = json.loads(value) - except json.JSONDecodeError: - pass - - expires_at = entry.get("expires_at") - if expires_at: - try: - expiry = datetime.fromisoformat(expires_at) - if expiry < datetime.now(): - delete_value(client, table_name, key) - return default - except (ValueError, TypeError): - pass - - logger.info(f"Retrieved value from table {table_name}: {key}") - return value - except Exception as e: - logger.error(f"Error retrieving value from table {table_name}: {key} - {str(e)}") - - return default - -def delete_value(client: Client, table_name: str, key: str) -> bool: - """Delete a value from a table.""" - try: - client.table(table_name).delete().eq("key", key).execute() - - logger.info(f"Deleted value from table {table_name}: {key}") - return True - except Exception as e: - logger.error(f"Error deleting value from table {table_name}: {key} - {str(e)}") - return False - -def list_keys(client: Client, table_name: str, prefix: Optional[str] = None) -> List[str]: - """List keys in a table.""" - keys = [] - - try: - query = client.table(table_name).select("key") - - if prefix: - query = query.like("key", f"{prefix}%") - - result = query.execute() - - if result.data: - for entry in result.data: - key = entry.get("key") - if key: - keys.append(key) - - logger.info(f"Listed keys from table {table_name}: {len(keys)} keys") - except Exception as e: - logger.error(f"Error listing keys from table {table_name}: {str(e)}") - - return keys - -def clear_table(client: Client, table_name: str, prefix: Optional[str] = None) -> bool: - """Clear a table.""" - try: - if prefix: - keys = list_keys(client, table_name, prefix) - - for key in keys: - delete_value(client, table_name, key) - else: - client.table(table_name).delete().execute() - - logger.info(f"Cleared table {table_name}") - return True - except Exception as e: - logger.error(f"Error clearing table {table_name}: {str(e)}") - return False - -def run_tests(): - """Run the tests.""" - logger.info("Running Supabase tables tests...") - - if not SUPABASE_URL or not SUPABASE_KEY: - logger.error("Supabase URL or key not provided") - logger.error("Please set SUPABASE_URL and SUPABASE_KEY environment variables") - return False - - # Initialize Supabase client - client = create_client(SUPABASE_URL, SUPABASE_KEY) - - test_run_id = str(uuid.uuid4()) - session_id = f"test-session-{test_run_id}" - conversation_id = f"test-conversation-{test_run_id}" - - base_table = "gaia_memory" - tables = { - "base": base_table, - "working": f"{base_table}_working", - "conversations": f"{base_table}_conversations", - "cache": f"{base_table}_cache" - } - - for table_type, table_name in tables.items(): - if not ensure_table_exists(client, table_name): - logger.error(f"Failed to ensure table {table_name} exists") - return False - - schema = get_table_schema(client, table_name) - logger.info(f"Table {table_name} schema: {schema}") - - if table_type == "conversations": - if "conversation_id" in schema and "message_id" in schema and "content" in schema: - logger.info(f"Table {table_name} has a different schema for conversations") - - logger.info("Testing base table operations...") - - test_key = f"test-{test_run_id}-base" - test_value = { - "test_data": f"This is test data for {test_key}", - "timestamp": datetime.now().isoformat(), - "random_id": str(uuid.uuid4()) - } - - if not store_value(client, tables["base"], test_key, test_value): - logger.error("Failed to store test value in base table") - return False - - retrieved_value = retrieve_value(client, tables["base"], test_key) - if retrieved_value != test_value: - logger.error("Retrieved value does not match test value") - logger.error(f"Expected: {test_value}") - logger.error(f"Got: {retrieved_value}") - return False - - logger.info("[PASS] Base table operations successful") - - logger.info("Testing working memory table operations...") - - working_key = f"working:{session_id}:test-{test_run_id}-working" - working_value = { - "test_data": f"This is test data for {working_key}", - "timestamp": datetime.now().isoformat(), - "random_id": str(uuid.uuid4()) - } - - if not store_value(client, tables["working"], working_key, working_value, session_id=session_id): - logger.error("Failed to store test value in working table") - return False - - retrieved_value = retrieve_value(client, tables["working"], working_key) - if retrieved_value != working_value: - logger.error("Retrieved value does not match test value") - logger.error(f"Expected: {working_value}") - logger.error(f"Got: {retrieved_value}") - return False - - base_value = retrieve_value(client, tables["base"], working_key) - if base_value is not None: - logger.error("Working memory data was incorrectly stored in base table") - return False - - logger.info("[PASS] Working memory table operations successful") - - logger.info("Testing conversation memory table operations...") - - schema = get_table_schema(client, tables["conversations"]) - - if "key" in schema and "value" in schema: - conversation_key = f"conversation:{conversation_id}:message:{datetime.now().isoformat()}" - conversation_value = { - "role": "user", - "content": "Hello, how are you?", - "timestamp": datetime.now().isoformat() - } - - if not store_value(client, tables["conversations"], conversation_key, conversation_value): - logger.error("Failed to store test value in conversations table") - return False - - retrieved_value = retrieve_value(client, tables["conversations"], conversation_key) - if retrieved_value != conversation_value: - logger.error("Retrieved value does not match test value") - logger.error(f"Expected: {conversation_value}") - logger.error(f"Got: {retrieved_value}") - return False - - base_value = retrieve_value(client, tables["base"], conversation_key) - if base_value is not None: - logger.error("Conversation memory data was incorrectly stored in base table") - return False - else: - logger.info("Using different schema for conversations table") - - try: - message_id = f"message-{datetime.now().isoformat()}" - entry = { - "conversation_id": conversation_id, - "message_id": message_id, - "role": "user", - "content": "Hello, how are you?", - "created_at": datetime.now().isoformat() - } - - client.table(tables["conversations"]).insert(entry).execute() - logger.info(f"Inserted message in conversations table: {message_id}") - - result = client.table(tables["conversations"]).select("*").eq("conversation_id", conversation_id).execute() - - if not result.data or len(result.data) == 0: - logger.error("No messages found in conversations table") - return False - - logger.info(f"Found {len(result.data)} messages in conversations table") - except Exception as e: - logger.error(f"Error testing conversations table: {str(e)}") - return False - - logger.info("[PASS] Conversation memory table operations successful") - - logger.info("Testing result cache table operations...") - - cache_key = f"cache:test-{test_run_id}-cache" - cache_value = { - "test_data": f"This is test data for {cache_key}", - "timestamp": datetime.now().isoformat(), - "random_id": str(uuid.uuid4()) - } - - if not store_value(client, tables["cache"], cache_key, cache_value, ttl=3600): - logger.error("Failed to store test value in cache table") - return False - - retrieved_value = retrieve_value(client, tables["cache"], cache_key) - if retrieved_value != cache_value: - logger.error("Retrieved value does not match test value") - logger.error(f"Expected: {cache_value}") - logger.error(f"Got: {retrieved_value}") - return False - - base_value = retrieve_value(client, tables["base"], cache_key) - if base_value is not None: - logger.error("Result cache data was incorrectly stored in base table") - return False - - logger.info("[PASS] Result cache table operations successful") - - logger.info("Testing data isolation between tables...") - - base_keys = list_keys(client, tables["base"]) - working_keys = list_keys(client, tables["working"]) - cache_keys = list_keys(client, tables["cache"]) - - schema = get_table_schema(client, tables["conversations"]) - if "key" in schema: - conversation_keys = list_keys(client, tables["conversations"]) - - if conversation_key not in conversation_keys: - logger.error(f"Conversation key {conversation_key} not found in conversations table") - return False - else: - try: - result = client.table(tables["conversations"]).select("*").eq("conversation_id", conversation_id).execute() - if not result.data or len(result.data) == 0: - logger.error(f"No messages found for conversation ID {conversation_id}") - return False - logger.info(f"Found {len(result.data)} messages for conversation ID {conversation_id}") - except Exception as e: - logger.error(f"Error checking conversations table: {str(e)}") - return False - - if test_key not in base_keys: - logger.error(f"Test key {test_key} not found in base table") - return False - - if working_key not in working_keys: - logger.error(f"Working key {working_key} not found in working table") - return False - - if cache_key not in cache_keys: - logger.error(f"Cache key {cache_key} not found in cache table") - return False - - logger.info("[PASS] Data isolation between tables verified") - - logger.info("Testing performance metrics...") - - num_operations = 10 - - base_store_times = [] - base_retrieve_times = [] - - for i in range(num_operations): - key = f"test-{test_run_id}-perf-base-{i}" - value = {"index": i, "data": f"Performance test data {i}"} - - start_time = time.time() - store_value(client, tables["base"], key, value) - base_store_times.append(time.time() - start_time) - - start_time = time.time() - retrieve_value(client, tables["base"], key) - base_retrieve_times.append(time.time() - start_time) - - working_store_times = [] - working_retrieve_times = [] - - for i in range(num_operations): - key = f"working:{session_id}:test-{test_run_id}-perf-working-{i}" - value = {"index": i, "data": f"Performance test data {i}"} - - start_time = time.time() - store_value(client, tables["working"], key, value, session_id=session_id) - working_store_times.append(time.time() - start_time) - - start_time = time.time() - retrieve_value(client, tables["working"], key) - working_retrieve_times.append(time.time() - start_time) - - logger.info(f"Base table: avg store {sum(base_store_times)/num_operations:.4f}s, avg retrieve {sum(base_retrieve_times)/num_operations:.4f}s") - logger.info(f"Working table: avg store {sum(working_store_times)/num_operations:.4f}s, avg retrieve {sum(working_retrieve_times)/num_operations:.4f}s") - - logger.info("[PASS] Performance metrics collected") - - logger.info("Cleaning up test data...") - - clear_table(client, tables["base"], f"test-{test_run_id}") - clear_table(client, tables["working"], f"working:{session_id}") - clear_table(client, tables["conversations"], f"conversation:{conversation_id}") - clear_table(client, tables["cache"], f"cache:test-{test_run_id}") - - logger.info("[PASS] Test data cleaned up") - - return True - -def main(): - """Main entry point for the test script.""" - - if run_tests(): - sys.exit(0) - else: - sys.exit(1) - -if __name__ == "__main__": - main() \ No newline at end of file diff --git a/build_error_fix_guide.md b/build_error_fix_guide.md deleted file mode 100644 index ebbb48d1cc163b95bbe789be719c4aa3a2ade63e..0000000000000000000000000000000000000000 --- a/build_error_fix_guide.md +++ /dev/null @@ -1,78 +0,0 @@ -# Fixing the GAIA Agent Build Error on Hugging Face Space - -## Problem - -The build process is failing with an "Unexpected job error" on Hugging Face Space. This is likely due to: - -1. Incompatible package versions -2. Dependency conflicts -3. Version resolution issues during installation - -## Solution - -Update the `requirements.txt` file with specific version pins to ensure all packages are compatible with each other and with the Hugging Face Space environment. - -## Steps to Fix the Build Error - -1. Navigate to your Space on Hugging Face -2. Click on the "Files" tab -3. Open `requirements.txt` -4. Click "Edit" (pencil icon) -5. Replace the contents with the following fixed requirements: - -``` -# Fixed requirements file with specific version pins -# Main dependencies -gradio==3.50.2 # Downgraded to ensure compatibility with Hugging Face Spaces -requests==2.31.0 -langchain==0.0.292 -langchain_openai==0.0.2.post1 -langgraph==0.0.19 -supabase==1.2.0 -duckduckgo-search==3.9.5 -beautifulsoup4==4.12.2 -pillow==10.0.1 -arxiv==1.4.8 -python-dotenv==1.0.0 - -# Benchmark testing dependencies (optional for production) -psutil==5.9.5 -matplotlib==3.7.3 -tabulate==0.9.0 - -# Additional dependencies to ensure compatibility -openai==1.3.7 -pydantic==1.10.13 # Using older version for compatibility -httpx==0.24.1 -numpy==1.24.4 -pandas==2.0.3 -``` - -6. Click "Commit changes" at the bottom of the page -7. Enter an optional commit message like "Fix requirements with specific version pins" -8. Click "Commit" to save the changes -9. Restart your Space (Settings tab → Factory reboot → Restart Space) - -## Why This Works - -1. **Specific Version Pinning**: Instead of allowing ranges or latest versions, specific versions ensure consistent builds -2. **Downgraded Gradio**: Using Gradio 3.50.2 instead of 5.x ensures better compatibility with Hugging Face Spaces -3. **Compatible Dependencies**: The specified versions are known to work together without conflicts -4. **Explicit Secondary Dependencies**: Including dependencies like openai and pydantic directly prevents version resolution issues - -## Additional Troubleshooting - -If you continue to encounter build errors: - -1. Check the Space logs for specific error messages -2. Consider removing optional testing dependencies (psutil, matplotlib, tabulate) -3. Try a minimal requirements set first, then add additional packages as needed -4. Check for any custom package installations in your code that might need to be added to requirements.txt - -## Testing After Fix - -After updating requirements.txt and restarting your Space: - -1. Monitor the build logs to confirm successful installation -2. Verify the app loads correctly in the browser -3. Test core functionality to ensure all components work properly \ No newline at end of file diff --git a/check_app.py b/check_app.py deleted file mode 100644 index ddadced3505d3902c639127e48cd6d99e8e34025..0000000000000000000000000000000000000000 --- a/check_app.py +++ /dev/null @@ -1,42 +0,0 @@ -""" -Simple script to check app.py file length and syntax -""" -import os -import sys -import ast - -def check_file(file_path): - try: - with open(file_path, 'r', encoding='utf-8') as f: - content = f.read() - lines = content.splitlines() - print(f"Total lines in {file_path}: {len(lines)}") - - # Try to parse the content to check for syntax errors - try: - ast.parse(content) - print(f"✓ No syntax errors found in {file_path}") - except SyntaxError as e: - print(f"✗ Syntax error in {file_path}:") - print(f" Line {e.lineno}, Column {e.offset}") - print(f" {e.text.strip()}") - print(f" {e}") - - # Print context around the error - start_line = max(0, e.lineno - 5) - end_line = min(len(lines), e.lineno + 5) - print("\nContext around error:") - for i in range(start_line, end_line): - line_marker = ">" if i + 1 == e.lineno else " " - print(f"{line_marker} {i+1:4d} | {lines[i]}") - - except Exception as e: - print(f"Error reading or parsing {file_path}: {e}") - -if __name__ == "__main__": - file_path = "app.py" - if not os.path.exists(file_path): - print(f"File not found: {file_path}") - sys.exit(1) - - check_file(file_path) \ No newline at end of file diff --git a/cleanup_results.txt b/cleanup_results.txt deleted file mode 100644 index 580cc00cdf9b81702cb848325ef2686b49c24ed9..0000000000000000000000000000000000000000 --- a/cleanup_results.txt +++ /dev/null @@ -1,340 +0,0 @@ -2025-04-30 22:18:05,446 - INFO - Finding temporary files to remove... -2025-04-30 22:18:05,563 - INFO - Found 39 temporary files to remove -2025-04-30 22:18:05,603 - INFO - Removed: gaia_assessment_results_20250430_205848.json -2025-04-30 22:18:05,615 - INFO - Removed: gaia_assessment_results_20250430_210047.json -2025-04-30 22:18:05,616 - INFO - Removed: gaia_assessment_results_20250430_210252.json -2025-04-30 22:18:05,619 - INFO - Removed: gaia_assessment_results_20250430_210521.json -2025-04-30 22:18:05,621 - INFO - Removed: gaia_assessment_results_20250430_211034.json -2025-04-30 22:18:05,622 - INFO - Removed: gaia_assessment_results_20250430_211508.json -2025-04-30 22:18:05,624 - INFO - Removed: gaia_assessment_results_20250430_213743.json -2025-04-30 22:18:05,627 - INFO - Removed: gaia_assessment_results_20250430_214034.json -2025-04-30 22:18:05,629 - INFO - Removed: gaia_assessment_results_20250430_214159.json -2025-04-30 22:18:05,631 - INFO - Removed: gaia_assessment_results_20250430_214711.json -2025-04-30 22:18:05,634 - INFO - Removed: gaia_assessment_results_20250430_220729.json -2025-04-30 22:18:05,636 - INFO - Removed: gaia_memory_test_results_20250430_185256.json -2025-04-30 22:18:05,639 - INFO - Removed: gaia_memory_test_results_20250430_185420.json -2025-04-30 22:18:05,641 - INFO - Removed: gaia_memory_test_results_20250430_185453.json -2025-04-30 22:18:05,644 - INFO - Removed: gaia_memory_test_results_20250430_185550.json -2025-04-30 22:18:05,648 - INFO - Removed: gaia_memory_test_results_20250430_185641.json -2025-04-30 22:18:05,671 - INFO - Removed: cleanup_script.bat -2025-04-30 22:18:05,672 - INFO - Removed: check_env_vars.py -2025-04-30 22:18:05,675 - INFO - Removed: check_imports.py -2025-04-30 22:18:05,678 - INFO - Removed: check_serper_config.py -2025-04-30 22:18:05,679 - INFO - Removed: check_supabase_env.py -2025-04-30 22:18:05,681 - INFO - Removed: direct_config_check.py -2025-04-30 22:18:05,683 - INFO - Removed: direct_serper_test.py -2025-04-30 22:18:05,685 - INFO - Removed: minimal_serper_test.py -2025-04-30 22:18:05,686 - INFO - Removed: simple_serper_test.py -2025-04-30 22:18:05,689 - INFO - Removed: simple_serper_test_final.py -2025-04-30 22:18:05,693 - INFO - Removed: supabase_direct_test.py -2025-04-30 22:18:05,695 - INFO - Removed: supabase_dotenv_test.py -2025-04-30 22:18:05,697 - INFO - Removed: supabase_import_test.py -2025-04-30 22:18:05,698 - INFO - Removed: supabase_init_test.py -2025-04-30 22:18:05,700 - INFO - Removed: supabase_minimal_test.py -2025-04-30 22:18:05,702 - INFO - Removed: test_serper_direct.py -2025-04-30 22:18:05,704 - INFO - Removed: verify_python.py -2025-04-30 22:18:05,706 - INFO - Removed: run_gaia_assessment.py -2025-04-30 22:18:05,707 - INFO - Removed: run_gaia_assessment_simple.py -2025-04-30 22:18:05,709 - INFO - Removed: run_gaia_fixed.py -2025-04-30 22:18:05,713 - INFO - Removed: run_gaia_minimal.py -2025-04-30 22:18:05,714 - INFO - Removed: run_gaia_with_fix.py -2025-04-30 22:18:05,717 - INFO - Removed: setup_serper_env.py -2025-04-30 22:18:05,717 - INFO - Finding duplicate test files... -2025-04-30 22:18:05,720 - INFO - Found 7 duplicate test files -2025-04-30 22:18:05,720 - INFO - Duplicate tests for test_config.py: -2025-04-30 22:18:05,720 - INFO - - tests\test_config.py -2025-04-30 22:18:05,720 - INFO - - tests\test_config_simple.py -2025-04-30 22:18:05,721 - INFO - Keeping: tests\test_config.py -2025-04-30 22:18:05,721 - INFO - Removing: tests\test_config_simple.py -2025-04-30 22:18:05,722 - INFO - Removed: tests\test_config_simple.py -2025-04-30 22:18:05,722 - INFO - Duplicate tests for test_serper_api.py: -2025-04-30 22:18:05,723 - INFO - - tests\test_serper_api.py -2025-04-30 22:18:05,723 - INFO - - tests\test_serper_api_final.py -2025-04-30 22:18:05,723 - INFO - Keeping: tests\test_serper_api_final.py -2025-04-30 22:18:05,723 - INFO - Removing: tests\test_serper_api.py -2025-04-30 22:18:05,724 - INFO - Removed: tests\test_serper_api.py -2025-04-30 22:18:05,726 - INFO - Duplicate tests for test_supabase_connection.py: -2025-04-30 22:18:05,726 - INFO - - tests\test_supabase_connection.py -2025-04-30 22:18:05,726 - INFO - - tests\test_supabase_connection_simple.py -2025-04-30 22:18:05,727 - INFO - Keeping: tests\test_supabase_connection.py -2025-04-30 22:18:05,727 - INFO - Removing: tests\test_supabase_connection_simple.py -2025-04-30 22:18:05,728 - INFO - Removed: tests\test_supabase_connection_simple.py -2025-04-30 22:18:05,729 - INFO - Duplicate tests for test_supabase_tables.py: -2025-04-30 22:18:05,729 - INFO - - tests\test_supabase_tables.py -2025-04-30 22:18:05,729 - INFO - - tests\test_supabase_tables_comprehensive.py -2025-04-30 22:18:05,729 - INFO - - tests\test_supabase_tables_final.py -2025-04-30 22:18:05,729 - INFO - - tests\test_supabase_tables_production.py -2025-04-30 22:18:05,729 - INFO - - tests\test_supabase_tables_simple.py -2025-04-30 22:18:05,730 - INFO - Keeping: tests\test_supabase_tables_comprehensive.py -2025-04-30 22:18:05,730 - INFO - Removing: tests\test_supabase_tables.py, tests\test_supabase_tables_simple.py -2025-04-30 22:18:05,731 - INFO - Removed: tests\test_supabase_tables.py -2025-04-30 22:18:05,733 - INFO - Removed: tests\test_supabase_tables_simple.py -2025-04-30 22:18:05,733 - INFO - Checking for debug statements and commented-out code... -2025-04-30 22:18:05,803 - INFO - Found 248 instances of debug statements or commented-out code -2025-04-30 22:18:05,803 - INFO - Issues in agent\agent.py: -2025-04-30 22:18:05,803 - INFO - Line 72 (Commented code): # Set up verbose logging if enabled -2025-04-30 22:18:05,803 - INFO - Line 75 (Commented code): # Add console handler if not already present -2025-04-30 22:18:05,803 - INFO - Line 92 (Debug statement): logger.debug("Initializing memory components") -2025-04-30 22:18:05,803 - INFO - Line 98 (Commented code): # Create a unique session ID for this agent instance -2025-04-30 22:18:05,803 - INFO - Line 106 (Debug statement): logger.debug("Memory components initialized successfully") -2025-04-30 22:18:05,804 - INFO - Line 161 (Commented code): # Record end time and return the answer -2025-04-30 22:18:05,804 - INFO - Line 207 (Debug statement): logger.debug(f"Processing question: {question}") -2025-04-30 22:18:05,804 - INFO - Line 209 (Commented code): # Check if we have a cached answer -2025-04-30 22:18:05,804 - INFO - Line 215 (Commented code): # Check if we have a cached answer -2025-04-30 22:18:05,804 - INFO - Line 221 (Commented code): # Create a config for the graph -2025-04-30 22:18:05,804 - INFO - Line 237 (Commented code): # Fallback answer if no answer or error is provided -2025-04-30 22:18:05,804 - INFO - Line 240 (Commented code): # Store the answer in the result cache if available -2025-04-30 22:18:05,804 - INFO - Line 244 (Commented code): # Store the intermediate results in working memory if available -2025-04-30 22:18:05,804 - INFO - Line 264 (Commented code): # Add the question and answer to conversation memory if available -2025-04-30 22:18:05,804 - INFO - Line 291 (Commented code): # Get conversation history if available -2025-04-30 22:18:05,804 - INFO - Line 295 (Commented code): # Get cached results if available -2025-04-30 22:18:05,805 - INFO - Line 308 (Commented code): # Get intermediate results if available -2025-04-30 22:18:05,805 - INFO - Line 323 (Commented code): # Clear conversation memory if available -2025-04-30 22:18:05,805 - INFO - Line 327 (Commented code): # Clear result cache if available -2025-04-30 22:18:05,805 - INFO - Line 331 (Commented code): # Clear working memory if available -2025-04-30 22:18:05,805 - INFO - Issues in agent\config.py: -2025-04-30 22:18:05,805 - INFO - Line 20 (Commented code): # Try to import dotenv, but don't fail if it's not available -2025-04-30 22:18:05,805 - INFO - Line 23 (Commented code): # Load environment variables from .env file if it exists -2025-04-30 22:18:05,805 - INFO - Line 28 (Debug statement): print("Warning: python-dotenv not installed. Environment variables will not be loaded from .env file.") -2025-04-30 22:18:05,806 - INFO - Line 30 (Commented code): # API Keys and Credentials (kept as environment variables) -2025-04-30 22:18:05,806 - INFO - Line 37 (Commented code): # Supabase Configuration (partially hardcoded) -2025-04-30 22:18:05,806 - INFO - Line 42 (Commented code): # API Endpoints (hardcoded) -2025-04-30 22:18:05,806 - INFO - Line 48 (Commented code): # Model Configuration (hardcoded with updated models) -2025-04-30 22:18:05,806 - INFO - Line 57 (Commented code): # Tool Parameters (hardcoded) -2025-04-30 22:18:05,806 - INFO - Line 69 (Commented code): # Memory Settings (hardcoded) -2025-04-30 22:18:05,806 - INFO - Line 77 (Commented code): # Agent Settings (hardcoded) -2025-04-30 22:18:05,806 - INFO - Line 85 (Commented code): # Logging Configuration (hardcoded) -2025-04-30 22:18:05,807 - INFO - Line 90 (Commented code): # Multimodal Settings (hardcoded) -2025-04-30 22:18:05,807 - INFO - Line 95 (Commented code): # Web Browsing Settings (hardcoded) -2025-04-30 22:18:05,807 - INFO - Line 101 (Commented code): # Helper functions for configuration -2025-04-30 22:18:05,807 - INFO - Issues in agent\graph.py: -2025-04-30 22:18:05,807 - INFO - Line 30 (Commented code): # Type definitions for state -2025-04-30 22:18:05,807 - INFO - Line 75 (Debug statement): logger.debug(f"Analyzing question: {state['question']}") -2025-04-30 22:18:05,807 - INFO - Line 117 (Commented code): # Parse the analysis (in a real implementation, use a proper JSON parser) -2025-04-30 22:18:05,807 - INFO - Line 118 (Commented code): # This is a simplified version for demonstration -2025-04-30 22:18:05,807 - INFO - Line 126 (Debug statement): logger.debug(f"Question analysis completed: {analysis}") -2025-04-30 22:18:05,807 - INFO - Line 148 (Debug statement): logger.debug("Creating plan based on question analysis") -2025-04-30 22:18:05,808 - INFO - Line 150 (Commented code): # If there's an error, return the state unchanged -2025-04-30 22:18:05,808 - INFO - Line 205 (Commented code): # Parse the plan (in a real implementation, use a proper JSON parser) -2025-04-30 22:18:05,808 - INFO - Line 222 (Debug statement): logger.debug(f"Plan created with {len(plan)} steps") -2025-04-30 22:18:05,809 - INFO - Line 244 (Debug statement): logger.debug(f"Selecting tool for step {state['current_step'] + 1}") -2025-04-30 22:18:05,809 - INFO - Line 246 (Commented code): # If there's an error, return END -2025-04-30 22:18:05,809 - INFO - Line 281 (Commented code): # If there's an error, return the state unchanged -2025-04-30 22:18:05,809 - INFO - Line 290 (Debug statement): logger.debug(f"Executing tool {tool_name} for step {current_step['step_number']}") -2025-04-30 22:18:05,809 - INFO - Line 315 (Debug statement): logger.debug(f"Tool {tool_name} executed successfully") -2025-04-30 22:18:05,810 - INFO - Line 350 (Debug statement): logger.debug("Formulating answer based on tool results") -2025-04-30 22:18:05,810 - INFO - Line 352 (Commented code): # If there's an error, return the state unchanged -2025-04-30 22:18:05,810 - INFO - Line 409 (Debug statement): logger.debug(f"Answer formulated: {answer.strip()}") -2025-04-30 22:18:05,810 - INFO - Line 431 (Commented code): # If there's an error, check if we should retry -2025-04-30 22:18:05,810 - INFO - Issues in agent\tool_registry.py: -2025-04-30 22:18:05,810 - INFO - Line 34 (Debug statement): logger.debug("Initialized empty tool registry") -2025-04-30 22:18:05,811 - INFO - Line 45 (Debug statement): logger.debug(f"Registered tool: {name}") -2025-04-30 22:18:05,811 - INFO - Line 89 (Debug statement): logger.debug(f"Executing tool: {name} with args: {kwargs}") -2025-04-30 22:18:05,811 - INFO - Line 147 (Commented code): # Register DuckDuckGo search tool (no API key required) -2025-04-30 22:18:05,811 - INFO - Line 154 (Commented code): # Register Serper search tool if API key is available -2025-04-30 22:18:05,811 - INFO - Line 165 (Commented code): # Register Perplexity tool if API key is available -2025-04-30 22:18:05,811 - INFO - Line 176 (Commented code): # Register arXiv tool (no API key required) -2025-04-30 22:18:05,812 - INFO - Line 180 (Commented code): # Register the same tool instance for different operations -2025-04-30 22:18:05,812 - INFO - Line 207 (Commented code): # Try DuckDuckGo first (no API key required) -2025-04-30 22:18:05,812 - INFO - Line 334 (Debug statement): print(f"Available tools: {registry.list_tools()}") -2025-04-30 22:18:05,812 - INFO - Line 340 (Debug statement): print(f"Robust search using {result['provider']} found {len(result['results'])} results") -2025-04-30 22:18:05,812 - INFO - Line 342 (Debug statement): print(f"Robust search failed: {str(e)}") -2025-04-30 22:18:05,812 - INFO - Line 348 (Debug statement): print(f"Merged search found {len(result['merged_results'])} results") -2025-04-30 22:18:05,813 - INFO - Line 350 (Debug statement): print(f"Perplexity context: {result['perplexity_context'][:100]}...") -2025-04-30 22:18:05,813 - INFO - Line 352 (Debug statement): print(f"arXiv papers: {len(result['arxiv_context'])}") -2025-04-30 22:18:05,813 - INFO - Line 354 (Debug statement): print(f"Merged search failed: {str(e)}") -2025-04-30 22:18:05,813 - INFO - Issues in memory\fix_supabase_tables.py: -2025-04-30 22:18:05,813 - INFO - Line 55 (Commented code): # Check if the client is initialized -2025-04-30 22:18:05,813 - INFO - Line 71 (Commented code): # Check if the table exists -2025-04-30 22:18:05,813 - INFO - Line 79 (Commented code): # SQL for creating the table -2025-04-30 22:18:05,814 - INFO - Line 130 (Commented code): # Trim cache if it's too large -2025-04-30 22:18:05,814 - INFO - Line 141 (Commented code): # Store in Supabase if initialized -2025-04-30 22:18:05,814 - INFO - Line 150 (Debug statement): logger.debug(f"Stored entry in table {current_table}: {key}") -2025-04-30 22:18:05,814 - INFO - Line 170 (Commented code): # Check if the entry has expired -2025-04-30 22:18:05,814 - INFO - Line 187 (Commented code): # Check if any entries were found -2025-04-30 22:18:05,814 - INFO - Line 196 (Commented code): # Check if the entry has expired -2025-04-30 22:18:05,814 - INFO - Line 208 (Debug statement): logger.debug(f"Retrieved entry from table {current_table}: {key}") -2025-04-30 22:18:05,814 - INFO - Line 227 (Commented code): # Remove from Supabase if initialized -2025-04-30 22:18:05,814 - INFO - Line 233 (Debug statement): logger.debug(f"Deleted entry from table {current_table}: {key}") -2025-04-30 22:18:05,814 - INFO - Line 256 (Commented code): # Get keys from Supabase if initialized -2025-04-30 22:18:05,815 - INFO - Line 262 (Commented code): # Filter by prefix if provided -2025-04-30 22:18:05,815 - INFO - Line 278 (Debug statement): logger.debug(f"Listed keys from table {current_table}: {len(keys)} keys") -2025-04-30 22:18:05,815 - INFO - Line 350 (Commented code): # Create a key for the message -2025-04-30 22:18:05,815 - INFO - Line 360 (Commented code): # Add metadata if provided -2025-04-30 22:18:05,816 - INFO - Line 384 (Commented code): # Filter by before/after if provided -2025-04-30 22:18:05,816 - INFO - Line 391 (Commented code): # Limit the number of messages if provided -2025-04-30 22:18:05,816 - INFO - Line 526 (Commented code): # Check if any entries were found -2025-04-30 22:18:05,816 - INFO - Line 546 (Debug statement): print("\nFixing Supabase Tables for GAIA Agent") -2025-04-30 22:18:05,816 - INFO - Line 547 (Debug statement): print("====================================\n") -2025-04-30 22:18:05,837 - INFO - Line 562 (Debug statement): print("All Supabase tables are now being used correctly.") -2025-04-30 22:18:05,839 - INFO - Line 566 (Debug statement): print("Please check the logs for more information") -2025-04-30 22:18:05,839 - INFO - Issues in memory\simplified_memory_fix.py: -2025-04-30 22:18:05,839 - INFO - Line 50 (Commented code): # Check if the client is initialized -2025-04-30 22:18:05,840 - INFO - Line 59 (Commented code): # Check if the table exists -2025-04-30 22:18:05,840 - INFO - Line 68 (Commented code): # SQL for creating the table -2025-04-30 22:18:05,840 - INFO - Line 118 (Commented code): # Trim cache if it's too large -2025-04-30 22:18:05,840 - INFO - Line 129 (Commented code): # Store in Supabase if initialized -2025-04-30 22:18:05,840 - INFO - Line 138 (Debug statement): logger.debug(f"Stored entry in table {current_table}: {key}") -2025-04-30 22:18:05,840 - INFO - Line 158 (Commented code): # Check if the entry has expired -2025-04-30 22:18:05,840 - INFO - Line 175 (Commented code): # Check if any entries were found -2025-04-30 22:18:05,840 - INFO - Line 184 (Commented code): # Check if the entry has expired -2025-04-30 22:18:05,841 - INFO - Line 196 (Debug statement): logger.debug(f"Retrieved entry from table {current_table}: {key}") -2025-04-30 22:18:05,841 - INFO - Line 215 (Commented code): # Remove from Supabase if initialized -2025-04-30 22:18:05,841 - INFO - Line 221 (Debug statement): logger.debug(f"Deleted entry from table {current_table}: {key}") -2025-04-30 22:18:05,841 - INFO - Line 244 (Commented code): # Get keys from Supabase if initialized -2025-04-30 22:18:05,841 - INFO - Line 250 (Commented code): # Filter by prefix if provided -2025-04-30 22:18:05,841 - INFO - Line 266 (Debug statement): logger.debug(f"Listed keys from table {current_table}: {len(keys)} keys") -2025-04-30 22:18:05,842 - INFO - Line 388 (Commented code): # Check if any entries were found -2025-04-30 22:18:05,842 - INFO - Line 408 (Debug statement): print("\nSimplified Fix for Supabase Tables in GAIA Agent") -2025-04-30 22:18:05,842 - INFO - Line 409 (Debug statement): print("==============================================\n") -2025-04-30 22:18:05,843 - INFO - Line 411 (Commented code): # Create the working memory table if it doesn't exist -2025-04-30 22:18:05,845 - INFO - Line 424 (Debug statement): print("The working memory table is now being used correctly.") -2025-04-30 22:18:05,846 - INFO - Line 425 (Debug statement): print("This should be sufficient for the GAIA benchmark test.") -2025-04-30 22:18:05,848 - INFO - Line 429 (Debug statement): print("Please check the logs for more information") -2025-04-30 22:18:05,848 - INFO - Issues in memory\supabase_memory.py: -2025-04-30 22:18:05,849 - INFO - Line 15 (Commented code): # Try to import supabase, but don't fail if it's not available -2025-04-30 22:18:05,849 - INFO - Line 19 (Commented code): # Define dummy types for type checking -2025-04-30 22:18:05,849 - INFO - Line 60 (Commented code): # Set expiration time if TTL is provided -2025-04-30 22:18:05,849 - INFO - Line 147 (Commented code): # Check if the main table exists -2025-04-30 22:18:05,849 - INFO - Line 181 (Commented code): # Check if the table exists by querying it -2025-04-30 22:18:05,850 - INFO - Line 183 (Debug statement): logger.debug(f"Table {actual_table} exists") -2025-04-30 22:18:05,850 - INFO - Line 249 (Commented code): # Trim cache if it's too large -2025-04-30 22:18:05,850 - INFO - Line 260 (Commented code): # Store in Supabase if initialized -2025-04-30 22:18:05,850 - INFO - Line 300 (Commented code): # Check if the entry has expired -2025-04-30 22:18:05,850 - INFO - Line 316 (Commented code): # Check if any entries were found -2025-04-30 22:18:05,850 - INFO - Line 324 (Commented code): # Check if the entry has expired -2025-04-30 22:18:05,851 - INFO - Line 359 (Commented code): # Remove from Supabase if initialized -2025-04-30 22:18:05,851 - INFO - Line 388 (Commented code): # Clear Supabase if initialized -2025-04-30 22:18:05,851 - INFO - Line 422 (Commented code): # Get keys from Supabase if initialized -2025-04-30 22:18:05,851 - INFO - Line 428 (Commented code): # Filter by prefix if provided -2025-04-30 22:18:05,851 - INFO - Line 484 (Commented code): # Create a key for the message -2025-04-30 22:18:05,851 - INFO - Line 494 (Commented code): # Add metadata if provided -2025-04-30 22:18:05,851 - INFO - Line 531 (Commented code): # Filter by before/after if provided -2025-04-30 22:18:05,851 - INFO - Line 538 (Commented code): # Limit the number of messages if provided -2025-04-30 22:18:05,852 - INFO - Issues in memory\supabase_memory_patch.py: -2025-04-30 22:18:05,852 - INFO - Line 60 (Commented code): # Check if the client is initialized -2025-04-30 22:18:05,852 - INFO - Line 73 (Commented code): # Check if the tables exist -2025-04-30 22:18:05,852 - INFO - Line 90 (Commented code): # SQL for creating the table -2025-04-30 22:18:05,853 - INFO - Line 151 (Debug statement): print("All tables are now properly utilized:") -2025-04-30 22:18:05,854 - INFO - Line 152 (Debug statement): print("- gaia_memory: Main memory table") -2025-04-30 22:18:05,854 - INFO - Line 153 (Debug statement): print("- gaia_memory_conversations: Conversation history table") -2025-04-30 22:18:05,854 - INFO - Line 154 (Debug statement): print("- gaia_memory_cache: Result cache table") -2025-04-30 22:18:05,854 - INFO - Line 155 (Debug statement): print("- gaia_memory_working: Working memory table") -2025-04-30 22:18:05,856 - INFO - Line 159 (Debug statement): print("Please check the logs for more information") -2025-04-30 22:18:05,856 - INFO - Issues in tools\arxiv_tool.py: -2025-04-30 22:18:05,856 - INFO - Line 60 (Debug statement): logger.debug(f"Searching arXiv for: {query}") -2025-04-30 22:18:05,856 - INFO - Line 82 (Debug statement): logger.debug(f"Found {len(results)} results from arXiv") -2025-04-30 22:18:05,856 - INFO - Line 134 (Debug statement): logger.debug(f"Getting arXiv paper with ID: {paper_id}") -2025-04-30 22:18:05,856 - INFO - Line 143 (Commented code): # Create a search for the specific paper -2025-04-30 22:18:05,856 - INFO - Line 201 (Debug statement): logger.debug(f"Searching arXiv for papers in category: {category}") -2025-04-30 22:18:05,856 - INFO - Line 203 (Commented code): # Create a query for the category -2025-04-30 22:18:05,856 - INFO - Line 222 (Commented code): # Search for papers -2025-04-30 22:18:05,857 - INFO - Line 225 (Debug statement): print(f"Found {len(results)} papers") -2025-04-30 22:18:05,857 - INFO - Line 227 (Debug statement): print(f"\nPaper {i+1}:") -2025-04-30 22:18:05,857 - INFO - Line 228 (Debug statement): print(f"Title: {paper['title']}") -2025-04-30 22:18:05,857 - INFO - Line 229 (Debug statement): print(f"Authors: {paper['authors']}") -2025-04-30 22:18:05,857 - INFO - Line 230 (Debug statement): print(f"Published: {paper['published']}") -2025-04-30 22:18:05,857 - INFO - Line 231 (Debug statement): print(f"URL: {paper['url']}") -2025-04-30 22:18:05,857 - INFO - Line 232 (Debug statement): print(f"Summary: {paper['summary'][:200]}...") -2025-04-30 22:18:05,857 - INFO - Line 234 (Debug statement): print(f"Error: {str(e)}") -2025-04-30 22:18:05,857 - INFO - Issues in tools\multimodal_tools.py: -2025-04-30 22:18:05,858 - INFO - Line 81 (Debug statement): logger.debug(f"Analyzing image: {image_path}") -2025-04-30 22:18:05,858 - INFO - Line 126 (Debug statement): logger.debug("Image analysis completed successfully") -2025-04-30 22:18:05,858 - INFO - Line 129 (Commented code): # If the result is not valid JSON, return it as plain text -2025-04-30 22:18:05,858 - INFO - Line 157 (Debug statement): logger.debug(f"Detecting objects in image: {image_path}") -2025-04-30 22:18:05,858 - INFO - Line 206 (Debug statement): logger.debug(f"Object detection completed, found {parsed_result.get('object_count', 0)} objects") -2025-04-30 22:18:05,859 - INFO - Line 255 (Debug statement): logger.debug(f"Interpreting chart: {chart_path}") -2025-04-30 22:18:05,859 - INFO - Line 306 (Debug statement): logger.debug(f"Chart interpretation completed: {parsed_result.get('chart_type', 'unknown')} chart") -2025-04-30 22:18:05,859 - INFO - Line 338 (Debug statement): logger.debug(f"Extracting data from chart: {chart_path}") -2025-04-30 22:18:05,859 - INFO - Line 386 (Debug statement): logger.debug(f"Chart data extraction completed with {len(parsed_result.get('data', []))} data points") -2025-04-30 22:18:05,859 - INFO - Line 422 (Commented code): # Check for required dependencies -2025-04-30 22:18:05,860 - INFO - Line 443 (Debug statement): logger.debug(f"Parsing document: {document_path}") -2025-04-30 22:18:05,860 - INFO - Line 468 (Commented code): # Truncate if too long -2025-04-30 22:18:05,860 - INFO - Line 504 (Debug statement): logger.debug(f"Extracting structured data from document: {document_path}") -2025-04-30 22:18:05,860 - INFO - Line 536 (Debug statement): logger.debug("Structured data extraction completed successfully") -2025-04-30 22:18:05,860 - INFO - Issues in tools\perplexity_tool.py: -2025-04-30 22:18:05,860 - INFO - Line 20 (Commented code): # Set up basic logging if not already configured -2025-04-30 22:18:05,860 - INFO - Line 40 (Commented code): # Get API key from config or environment (only configuration from environment) -2025-04-30 22:18:05,860 - INFO - Line 45 (Commented code): # Hardcoded values for API URL and model -2025-04-30 22:18:05,860 - INFO - Line 50 (Commented code): # Endpoint for chat completions -2025-04-30 22:18:05,861 - INFO - Line 53 (Debug statement): logger.debug(f"Initialized Perplexity tool with model {self.model}") -2025-04-30 22:18:05,861 - INFO - Line 73 (Debug statement): logger.debug(f"Querying Perplexity API: {question[:50]}...") -2025-04-30 22:18:05,861 - INFO - Line 82 (Commented code): # Add system prompt if provided -2025-04-30 22:18:05,861 - INFO - Line 134 (Debug statement): logger.debug(f"Perplexity API query successful, received {len(content) if content else 0} characters") -2025-04-30 22:18:05,862 - INFO - Line 169 (Debug statement): logger.debug(f"Searching with Perplexity API: {query[:50]}...") -2025-04-30 22:18:05,862 - INFO - Issues in tools\reasoning_tools.py: -2025-04-30 22:18:05,862 - INFO - Line 65 (Debug statement): logger.debug(f"Performing step-by-step reasoning on: {question}") -2025-04-30 22:18:05,862 - INFO - Line 93 (Commented code): # Add context if provided -2025-04-30 22:18:05,862 - INFO - Line 111 (Debug statement): logger.debug(f"Reasoning completed with {len(parsed_result.get('steps', []))} steps") -2025-04-30 22:18:05,862 - INFO - Line 134 (Debug statement): logger.debug(f"Performing chain-of-thought reasoning on: {question}") -2025-04-30 22:18:05,862 - INFO - Line 148 (Commented code): # Add context if provided -2025-04-30 22:18:05,862 - INFO - Line 163 (Debug statement): logger.debug("Chain-of-thought reasoning completed") -2025-04-30 22:18:05,862 - INFO - Line 203 (Debug statement): logger.debug(f"Calculating expression: {expression}") -2025-04-30 22:18:05,863 - INFO - Line 209 (Commented code): # Evaluate using sympy for symbolic math -2025-04-30 22:18:05,863 - INFO - Line 241 (Debug statement): logger.debug(f"Solving equation: {equation} for {variable}") -2025-04-30 22:18:05,863 - INFO - Line 262 (Commented code): # Convert to float or int if possible -2025-04-30 22:18:05,863 - INFO - Line 317 (Commented code): # Only replace if it's a whole word (not part of another word) -2025-04-30 22:18:05,863 - INFO - Line 354 (Debug statement): logger.debug(f"Verifying claim: {claim}") -2025-04-30 22:18:05,863 - INFO - Line 395 (Debug statement): logger.debug(f"Verification completed: {parsed_result.get('verification_result', 'unknown')}") -2025-04-30 22:18:05,863 - INFO - Line 418 (Debug statement): logger.debug(f"Comparing claims: '{claim1}' and '{claim2}'") -2025-04-30 22:18:05,863 - INFO - Line 455 (Debug statement): logger.debug(f"Comparison completed: {parsed_result.get('relationship', 'unknown')}") -2025-04-30 22:18:05,863 - INFO - Issues in tools\web_tools.py: -2025-04-30 22:18:05,864 - INFO - Line 124 (Debug statement): logger.debug(f"Searching DuckDuckGo for: {query}") -2025-04-30 22:18:05,864 - INFO - Line 137 (Debug statement): logger.debug(f"Found {len(results)} results from DuckDuckGo") -2025-04-30 22:18:05,864 - INFO - Line 187 (Debug statement): logger.debug(f"Searching Serper for: {query}") -2025-04-30 22:18:05,864 - INFO - Line 217 (Debug statement): logger.debug(f"Found {len(organic_results)} results from Serper") -2025-04-30 22:18:05,864 - INFO - Line 270 (Debug statement): logger.debug(f"Extracting content from: {url}") -2025-04-30 22:18:05,864 - INFO - Line 306 (Commented code): # Truncate if too long -2025-04-30 22:18:05,864 - INFO - Line 392 (Debug statement): logger.debug(f"Following links from: {url}") -2025-04-30 22:18:05,864 - INFO - Line 401 (Commented code): # Filter links by pattern if provided -2025-04-30 22:18:05,864 - INFO - Issues in tools\__init__.py: -2025-04-30 22:18:05,865 - INFO - Line 129 (Debug statement): logger.debug(f"Registered tool: {metadata.name} in category {metadata.category}") -2025-04-30 22:18:05,865 - INFO - Line 189 (Commented code): # Check for direct tool name match -2025-04-30 22:18:05,865 - INFO - Line 194 (Commented code): # Check for keyword matches -2025-04-30 22:18:05,865 - INFO - Line 659 (Commented code): # Convenience functions for accessing tools -2025-04-30 22:18:05,865 - INFO - Issues in utils\formatting.py: -2025-04-30 22:18:05,865 - INFO - Line 38 (Debug statement): logger.debug(f"Extracting answer from response (expected format: {expected_format})") -2025-04-30 22:18:05,865 - INFO - Line 40 (Commented code): # If the response is already concise, return it directly -2025-04-30 22:18:05,865 - INFO - Line 44 (Commented code): # Look for explicit answer indicators -2025-04-30 22:18:05,865 - INFO - Line 75 (Commented code): # Default: return the last sentence as the answer -2025-04-30 22:18:05,866 - INFO - Line 92 (Commented code): # Look for numbers with optional decimal points and commas -2025-04-30 22:18:05,866 - INFO - Line 108 (Commented code): # If no answer-indicating phrase, find all numbers and return the last one -2025-04-30 22:18:05,866 - INFO - Line 147 (Commented code): # If no answer-indicating phrase, find all dates and return the first one -2025-04-30 22:18:05,866 - INFO - Line 167 (Commented code): # Check for yes/no indicators at the beginning of the text or after answer indicators -2025-04-30 22:18:05,866 - INFO - Line 198 (Commented code): # If no clear yes/no, look for sentiment -2025-04-30 22:18:05,866 - INFO - Line 213 (Commented code): # Default to "Unknown" if can't determine -2025-04-30 22:18:05,866 - INFO - Line 226 (Commented code): # Look for explicit list indicators -2025-04-30 22:18:05,866 - INFO - Line 244 (Commented code): # Look for bullet points or numbered lists -2025-04-30 22:18:05,866 - INFO - Line 250 (Commented code): # Look for any comma-separated sequence -2025-04-30 22:18:05,866 - INFO - Line 268 (Commented code): # Look for entity indicators -2025-04-30 22:18:05,866 - INFO - Line 281 (Commented code): # Look for capitalized proper nouns -2025-04-30 22:18:05,867 - INFO - Line 306 (Commented code): # Look for JSON blocks -2025-04-30 22:18:05,867 - INFO - Line 318 (Commented code): # Look for JSON without code blocks -2025-04-30 22:18:05,867 - INFO - Line 341 (Debug statement): logger.debug(f"Formatting answer to {format_type} format") -2025-04-30 22:18:05,867 - INFO - Line 401 (Commented code): # Default: return as is -2025-04-30 22:18:05,867 - INFO - Line 415 (Debug statement): logger.debug(f"Validating answer format (expected: {expected_format})") -2025-04-30 22:18:05,867 - INFO - Line 435 (Commented code): # Validate date format (simple check) -2025-04-30 22:18:05,867 - INFO - Line 469 (Commented code): # Validate entity format (simple check for proper noun) -2025-04-30 22:18:05,867 - INFO - Line 502 (Debug statement): logger.debug(f"Processing answer (expected format: {expected_format})") -2025-04-30 22:18:05,867 - INFO - Line 504 (Commented code): # Default to text format if none specified -2025-04-30 22:18:05,867 - INFO - Issues in utils\parsing.py: -2025-04-30 22:18:05,868 - INFO - Line 34 (Debug statement): logger.debug(f"Extracting key information from question: {question}") -2025-04-30 22:18:05,868 - INFO - Line 47 (Commented code): # Extract entities (proper nouns, potentially) -2025-04-30 22:18:05,868 - INFO - Line 52 (Commented code): # Extract keywords (important nouns, verbs, adjectives) -2025-04-30 22:18:05,868 - INFO - Line 143 (Debug statement): logger.debug(f"Identifying required tools for question: {question}") -2025-04-30 22:18:05,868 - INFO - Line 197 (Debug statement): logger.debug(f"Determining complexity for question: {question}") -2025-04-30 22:18:05,868 - INFO - Line 293 (Debug statement): logger.debug(f"Parsing question: {question}") -2025-04-30 22:18:05,868 - INFO - Issues in utils\__init__.py: -2025-04-30 22:18:05,868 - INFO - Line 96 (Commented code): # Check for number format -2025-04-30 22:18:05,868 - INFO - Line 100 (Commented code): # Check for date format -2025-04-30 22:18:05,868 - INFO - Line 104 (Commented code): # Check for boolean format -2025-04-30 22:18:05,869 - INFO - Line 109 (Commented code): # Check for list format -2025-04-30 22:18:05,869 - INFO - Line 113 (Commented code): # Check for entity format -2025-04-30 22:18:05,869 - INFO - Checking for potential hardcoded credentials... -2025-04-30 22:18:05,990 - INFO - No potential hardcoded credentials found -2025-04-30 22:18:05,990 - INFO - -Cleanup Summary: -2025-04-30 22:18:05,991 - INFO - - Temporary files to remove: 39 -2025-04-30 22:18:05,991 - INFO - - Duplicate test files to remove: 7 -2025-04-30 22:18:05,991 - INFO - - Files with debug statements or commented code: 17 -2025-04-30 22:18:05,992 - INFO - - Files with potential hardcoded credentials: 0 diff --git a/complete_deployment_guide.md b/complete_deployment_guide.md deleted file mode 100644 index 3befedd39f1b44f00fef36b09808f6ed87f9fa9d..0000000000000000000000000000000000000000 --- a/complete_deployment_guide.md +++ /dev/null @@ -1,129 +0,0 @@ -# Complete GAIA Agent Deployment Guide for Hugging Face Space - -This guide addresses all issues encountered when deploying the GAIA agent to Hugging Face Space, including OAuth configuration and dependency conflicts. - -## Step 1: Fix OAuth Configuration - -The OAuth error occurs because Hugging Face Spaces requires OAuth for user identification. - -1. Navigate to your Space on Hugging Face -2. Click on the "Files" tab -3. Open **README.md** -4. Edit the file to include this YAML frontmatter at the top: - -```yaml ---- -title: GAIA Agent Assessment -emoji: 🤖 -colorFrom: blue -colorTo: green -sdk: gradio -sdk_version: 3.50.2 -app_file: app.py -pinned: false -license: mit -hf_oauth: true # This enables OAuth! ---- -``` - -5. Click "Commit changes" - -## Step 2: Fix Dependency Conflicts - -The build error is caused by conflicts between package versions, particularly around httpx: - -- supabase 1.2.0 requires httpx<0.25.0 and >=0.24.0 -- openai 1.6.1 requires httpx<1 and >=0.23.0 -- duckduckgo-search 3.9.5 requires httpx>=0.25.1 - -To resolve this: - -1. Navigate to your Space's "Files" tab -2. Open **requirements.txt** -3. Replace with this compatible configuration: - -``` -# Fixed requirements with compatible versions -gradio==3.50.2 -requests==2.31.0 -langchain==0.0.292 - -# OpenAI and related dependencies -openai==1.6.1 -langchain_openai==0.0.2.post1 -langgraph==0.0.19 - -# HTTP Libraries - key conflict area -httpx==0.24.1 # Compatible with supabase and openai - -# Database and storage -supabase==1.2.0 - -# Search tools - downgraded to avoid httpx conflict -duckduckgo-search==3.8.3 - -# Parsing and content handling -beautifulsoup4==4.12.2 -pillow==10.0.1 -arxiv==1.4.8 -python-dotenv==1.0.0 - -# Additional dependencies -pydantic==1.10.13 -numpy==1.24.4 -pandas==2.0.3 -``` - -4. Click "Commit changes" - -## Step 3: Configure Environment Variables - -1. Go to the "Settings" tab of your Space -2. Scroll to "Repository secrets" -3. Add the following environment variables: - - `OPENAI_API_KEY`: Your OpenAI API key - - `SUPABASE_URL`: Your Supabase project URL - - `SUPABASE_KEY`: Your Supabase API key - - `SERPER_API_KEY`: (Optional) Your Serper API key - -## Step 4: Restart Your Space - -1. Still in "Settings", scroll to "Factory reboot" -2. Click "Restart Space" to apply all changes - -## Verification Steps - -After restarting: -1. Check the build logs to confirm successful installation -2. Verify the app loads correctly in the browser -3. Test the OAuth login functionality -4. Run a test evaluation to ensure all components work - -## Troubleshooting Common Issues - -### If OAuth Still Doesn't Work -- Verify the `hf_oauth: true` line is correctly added to the README.md frontmatter -- Make sure you've restarted the Space after changes - -### If Dependency Conflicts Persist -Try these alternative approaches: - -1. **Further simplify dependencies** - - Remove optional packages temporarily - - Add them back one by one to identify conflicts - -2. **Use separate requirements files** - - Create a base requirements.txt with core dependencies - - Use post-installation scripts to install conflicting packages separately - -3. **Check for custom code dependencies** - - Review the codebase for any imports that might require additional packages - - Look for conditional imports that might need to be modified - -## Post-Deployment Monitoring - -After successful deployment: -1. Monitor the application logs for any runtime errors -2. Test all major functionality paths -3. Verify Supabase connections and memory functionality -4. Test search tool integrations \ No newline at end of file diff --git a/complete_deployment_solution.md b/complete_deployment_solution.md deleted file mode 100644 index 2e5cb6683bc103d29ef5ba8ffa1dce2b40ae32ff..0000000000000000000000000000000000000000 --- a/complete_deployment_solution.md +++ /dev/null @@ -1,183 +0,0 @@ -# GAIA Agent - Complete HuggingFace Space Deployment Solution - -This comprehensive solution addresses all deployment issues encountered with the GAIA Agent on Hugging Face Space: - -## Issues Identified & Fixed - -1. **OAuth Configuration Error** -2. **Dependency Conflicts** (OpenAI/langchain_openai incompatibility) -3. **Runtime Import Error** with missing module - -## Step 1: Fix OAuth Configuration - -1. Navigate to your Space's "Files" tab -2. Open **README.md** -3. Add this YAML frontmatter at the top: - -```yaml ---- -title: GAIA Agent Assessment -emoji: 🤖 -colorFrom: blue -colorTo: green -sdk: gradio -sdk_version: 3.50.2 -app_file: app.py -pinned: false -license: mit -hf_oauth: true # This enables OAuth! ---- -``` - -## Step 2: Fix Dependency Conflicts - -1. Navigate to your Space's "Files" tab -2. Open **requirements.txt** -3. Replace with these compatible versions: - -``` -# Final fixed requirements -gradio==3.50.2 -requests==2.31.0 - -# Langchain ecosystem with compatible versions -langchain==0.1.0 -langchain-core==0.1.17 -openai==1.10.0 # Specific version required by langchain_openai -langchain_openai==0.0.5 -langgraph==0.0.24 - -# HTTP Libraries -httpx==0.24.1 # Compatible with supabase - -# Database and storage -supabase==1.2.0 - -# Search tools - downgraded to avoid httpx conflict -duckduckgo-search==3.8.3 - -# Parsing and content handling -beautifulsoup4==4.12.2 -pillow==10.0.1 -arxiv==1.4.8 -python-dotenv==1.0.0 - -# Additional dependencies -pydantic==1.10.13 -numpy==1.24.4 -pandas==2.0.3 -``` - -## Step 3: Fix Runtime Import Error - -You need to modify **agent/graph.py**: - -1. Navigate to your Space's "Files" tab -2. Open **agent/graph.py** -3. Find line 22: `from langgraph.prebuilt import ToolNode` -4. Replace with: `# Removed problematic import: from langgraph.prebuilt import ToolNode` - -Alternatively, upload and run the **fix_graph_imports.py** script: - -```python -#!/usr/bin/env python -""" -Fix for the ModuleNotFoundError: No module named 'langchain.tools.render' error. -""" -import os -import re -import sys -import logging - -logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s') -logger = logging.getLogger(__name__) - -def patch_graph_file(): - """Patch the agent/graph.py file to remove the problematic import.""" - graph_file = "agent/graph.py" - - if not os.path.exists(graph_file): - logger.error(f"Could not find {graph_file}") - return False - - try: - with open(graph_file, "r") as f: - content = f.read() - - # Create a backup - backup_file = f"{graph_file}.bak" - with open(backup_file, "w") as f: - f.write(content) - logger.info(f"Created backup at {backup_file}") - - # Remove the problematic import - pattern = r"from langgraph\.prebuilt import ToolNode\n" - updated_content = re.sub(pattern, "# Removed problematic import: from langgraph.prebuilt import ToolNode\n", content) - - with open(graph_file, "w") as f: - f.write(updated_content) - - logger.info(f"Successfully patched {graph_file}") - return True - - except Exception as e: - logger.error(f"Error patching file: {str(e)}") - return False - -if __name__ == "__main__": - logger.info("Starting graph.py import patch...") - - if patch_graph_file(): - logger.info("Patch applied successfully!") - else: - logger.error("Failed to apply patch") - sys.exit(1) -``` - -## Step 4: Configure Environment Variables - -1. Go to the "Settings" tab of your Space -2. Scroll to "Repository secrets" -3. Add the following environment variables: - - `OPENAI_API_KEY`: Your OpenAI API key - - `SUPABASE_URL`: Your Supabase project URL - - `SUPABASE_KEY`: Your Supabase API key - - `SERPER_API_KEY`: (Optional) Your Serper API key - -## Step 5: Restart Your Space - -1. Still in "Settings", scroll to "Factory reboot" -2. Click "Restart Space" to apply all changes - -## Detailed Error Analysis - -### 1. OAuth Configuration Issue -Required for Hugging Face Spaces to properly identify users during the GAIA assessment. - -### 2. Dependency Conflict -The error was: -``` -Cannot install langchain-openai 0.0.5 and openai==1.6.1 because these package versions have conflicting dependencies. -The conflict is caused by: - The user requested openai==1.6.1 - langchain-openai 0.0.5 depends on openai<2.0.0 and >=1.10.0 -``` - -Our solution updates openai to 1.10.0 to satisfy langchain_openai's requirements. - -### 3. Runtime Import Error -The error was: -``` -ModuleNotFoundError: No module named 'langchain.tools.render' -``` -Caused by import statement at line 22 in agent/graph.py that tried to use a module from older versions of langchain. Since this import isn't used in the code, removing it solves the issue. - -## Verification - -After deploying: -1. Check build logs - verify dependencies installed correctly -2. Confirm the app loads in browser -3. Test OAuth login -4. Verify agent functionality works correctly - -All these fixes together should resolve the deployment issues and get your GAIA agent running successfully on Hugging Face Space. \ No newline at end of file diff --git a/create_env_and_test.py b/create_env_and_test.py deleted file mode 100644 index a13d6faaeb575388cf6578ef7a65322c525b3fcf..0000000000000000000000000000000000000000 --- a/create_env_and_test.py +++ /dev/null @@ -1,77 +0,0 @@ -""" -Create .env File and Test Supabase Connection - -This script creates a .env file with hardcoded values and then tests the connection to Supabase. -""" - -import os -import sys -import time - -def create_env_file(api_key): - """ - Create a .env file with the provided API key. - - Args: - api_key: The Supabase API key - """ - - env_content = f""" -# Supabase Configuration -SUPABASE_URL=https://tjamxhvvtnypbadvrkjq.supabase.co -SUPABASE_KEY={api_key} -""" - - with open(".env", "w") as f: - f.write(env_content) - - -def test_supabase(): - """Test the connection to Supabase.""" - - try: - from supabase import create_client - from dotenv import load_dotenv - - load_dotenv(override=True) - - # Get Supabase URL and key - supabase_url = os.getenv("SUPABASE_URL") - supabase_key = os.getenv("SUPABASE_KEY") - - if supabase_key: - masked_key = supabase_key[:4] + "*" * (len(supabase_key) - 8) + supabase_key[-4:] if len(supabase_key) > 8 else "****" - else: - return False - - client = create_client(supabase_url, supabase_key) - - result = client.table("test_connection").select("*").limit(1).execute() - - return True - except Exception as e: - import traceback - traceback.print_exc() - return False - -def main(): - """Main function.""" - - api_key = input("> ").strip() - - if not api_key: - return - - create_env_file(api_key) - - time.sleep(1) - - # Test Supabase connection - success = test_supabase() - - if success: - else: - - -if __name__ == "__main__": - main() \ No newline at end of file diff --git a/debug_test.py b/debug_test.py deleted file mode 100644 index 2138102b0c7c322128691c0e6dc82352b152d4b1..0000000000000000000000000000000000000000 --- a/debug_test.py +++ /dev/null @@ -1,113 +0,0 @@ -""" -Simple debug script to identify component issues -""" -import os -import sys -import logging -import traceback - -# Set up logging -logging.basicConfig( - level=logging.DEBUG, - format="%(asctime)s - %(name)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("debug_test") - -def check_environment(): - """Check environment variables""" - logger.info("Checking environment variables...") - vars_to_check = ["OPENAI_API_KEY", "SUPABASE_URL", "SUPABASE_KEY", "SERPER_API_KEY"] - for var in vars_to_check: - value = os.environ.get(var) - if value: - logger.info(f"{var}: ✓ (Set)") - else: - logger.warning(f"{var}: ✗ (Not set)") - -def test_imports(): - """Test importing all required modules""" - logger.info("Testing imports...") - - import_tests = [ - ("OpenAI", "from openai import OpenAI; client = OpenAI()"), - ("Supabase", "from supabase import create_client"), - ("Memory modules", "from memory.supabase_memory import SupabaseMemory, WorkingMemory, ConversationMemory, ResultCache"), - ("Memory fix", "from memory.fix_supabase_tables import apply_monkey_patch, ensure_tables_exist"), - ("Agent config", "from agent.config import get_memory_config"), - ("Agent module", "from agent.agent import GAIAAgent"), - ("Agent graph", "from agent.graph import create_agent_graph, run_agent_graph"), - ("DuckDuckGo", "from tools.web_tools import duckduckgo_search"), - ("Serper", "from tools.web_tools import serper_search") - ] - - for name, import_statement in import_tests: - logger.info(f"Testing import: {name}") - try: - exec(import_statement) - logger.info(f"✓ Successfully imported {name}") - except Exception as e: - logger.error(f"✗ Failed to import {name}: {str(e)}") - logger.debug(traceback.format_exc()) - -def check_supabase_connection(): - """Test Supabase connection""" - logger.info("Testing Supabase connection...") - try: - from supabase import create_client - import os - - supabase_url = os.environ.get("SUPABASE_URL") - supabase_key = os.environ.get("SUPABASE_KEY") - - if not supabase_url or not supabase_key: - logger.error("Supabase credentials not found in environment variables") - return - - client = create_client(supabase_url, supabase_key) - response = client.table("agents").select("id").limit(1).execute() - - logger.info("Supabase connection successful") - except Exception as e: - logger.error(f"Supabase connection error: {str(e)}") - logger.debug(traceback.format_exc()) - -def check_openai_connection(): - """Test OpenAI connection""" - logger.info("Testing OpenAI connection...") - try: - from openai import OpenAI - - if not os.environ.get("OPENAI_API_KEY"): - logger.error("OpenAI API key not found in environment variables") - return - - client = OpenAI() - response = client.chat.completions.create( - model="gpt-3.5-turbo", - messages=[{"role": "user", "content": "Hello"}], - max_tokens=5 - ) - - logger.info("OpenAI connection successful") - except Exception as e: - logger.error(f"OpenAI connection error: {str(e)}") - logger.debug(traceback.format_exc()) - -def main(): - """Run all debug tests""" - logger.info("=== STARTING DEBUG TESTS ===") - - # Check environment variables - check_environment() - - # Test imports - test_imports() - - # Test API connections - check_supabase_connection() - check_openai_connection() - - logger.info("=== DEBUG TESTS COMPLETED ===") - -if __name__ == "__main__": - main() \ No newline at end of file diff --git a/dependency_conflict_fix_guide.md b/dependency_conflict_fix_guide.md deleted file mode 100644 index 37a65df676219d9da4d31d1c5d0ac2eaea808fad..0000000000000000000000000000000000000000 --- a/dependency_conflict_fix_guide.md +++ /dev/null @@ -1,84 +0,0 @@ -# Fixing Dependency Conflict in GAIA Agent Deployment - -## The Specific Error - -The build is failing with a dependency conflict between `langchain-openai` and `openai`: - -``` -ERROR: Cannot install -r /tmp/requirements.txt (line 5) and openai==1.3.7 because these package versions have conflicting dependencies. - -The conflict is caused by: - The user requested openai==1.3.7 - langchain-openai 0.0.2.post1 depends on openai<2.0.0 and >=1.6.1 -``` - -## Solution - -Update the `requirements.txt` file to use a compatible version of OpenAI that meets the requirements of `langchain-openai`. - -## Steps to Fix the Dependency Conflict - -1. Navigate to your Space on Hugging Face -2. Click on the "Files" tab -3. Open `requirements.txt` -4. Click "Edit" (pencil icon) -5. Replace the contents with the following fixed requirements: - -``` -# Fixed requirements file with compatible versions -gradio==3.50.2 -requests==2.31.0 -langchain==0.0.292 -# Fix for the dependency conflict - OpenAI version must be >=1.6.1 for langchain-openai -openai==1.6.1 -langchain_openai==0.0.2.post1 -langgraph==0.0.19 -supabase==1.2.0 -duckduckgo-search==3.9.5 -beautifulsoup4==4.12.2 -pillow==10.0.1 -arxiv==1.4.8 -python-dotenv==1.0.0 - -# Additional dependencies -pydantic==1.10.13 -httpx==0.24.1 -numpy==1.24.4 -pandas==2.0.3 -``` - -6. Click "Commit changes" at the bottom of the page -7. Enter a commit message like "Fix dependency conflict between langchain-openai and openai" -8. Click "Commit" to save the changes -9. Restart your Space (Settings tab → Factory reboot → Restart Space) - -## Why This Works - -The error occurs because: -- We were trying to install `openai==1.3.7` -- But `langchain-openai 0.0.2.post1` requires `openai>=1.6.1` - -By updating to `openai==1.6.1`, we satisfy the version constraint imposed by langchain-openai while maintaining compatibility with other packages. - -## Alternative Approaches - -If this solution doesn't work, you can try: - -1. **Option 1**: Use a newer version of both packages - ``` - openai==1.7.2 - langchain_openai==0.0.3 - ``` - -2. **Option 2**: Pin langchain-openai to an older version that works with openai 1.3.7 - (Note: This might not be available as langchain-openai seems to require >=1.6.1) - -3. **Option 3**: Temporarily remove langchain-openai from requirements and install it in a separate step with compatible dependencies - -## Testing After Fix - -After updating requirements.txt and restarting your Space: - -1. Monitor the build logs to confirm successful installation without dependency errors -2. Verify the app loads correctly in the browser -3. Test the OpenAI integration to ensure it's working with the updated version \ No newline at end of file diff --git a/deploy_to_hf.py b/deploy_to_hf.py deleted file mode 100644 index 6cb94b1b94ba28bc70d717a84ed23a464e725f9a..0000000000000000000000000000000000000000 --- a/deploy_to_hf.py +++ /dev/null @@ -1,392 +0,0 @@ -#!/usr/bin/env python -""" -Automated deployment script for GAIA Agent to Hugging Face Spaces - -This script automates the deployment of the GAIA Agent to a Hugging Face Space: -1. Verifies the local project structure -2. Creates or updates the Hugging Face Space -3. Sets up environment variables -4. Adds the code to git and pushes it - -Usage: - python deploy_to_hf.py --username YOUR_USERNAME --space YOUR_SPACE_NAME -""" - -import os -import sys -import argparse -import subprocess -import json -import logging -import shutil -import tempfile -from pathlib import Path -from typing import Dict, Any, List, Optional, Union - -# Configure logging -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(name)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("deploy_to_hf") - -def parse_arguments(): - """Parse command line arguments.""" - parser = argparse.ArgumentParser(description="Deploy GAIA Agent to Hugging Face Spaces") - - parser.add_argument("--username", required=True, help="Your Hugging Face username") - parser.add_argument("--space", required=True, help="Name for your Hugging Face Space") - parser.add_argument("--hardware", default="cpu-basic", choices=["cpu-basic", "cpu-upgrade", "t4-small", "t4-medium", "a10g-small", "a10g-large"], help="Space hardware tier") - parser.add_argument("--private", action="store_true", help="Make the Space private") - parser.add_argument("--debug", action="store_true", help="Enable debug logging") - parser.add_argument("--sdk", default="gradio", choices=["gradio", "streamlit", "docker"], help="Space SDK") - parser.add_argument("--skip-verification", action="store_true", help="Skip local verification") - - return parser.parse_args() - -def check_prerequisites(): - """Check if all prerequisites are met.""" - logger.info("Checking prerequisites...") - - # Check HF CLI - try: - result = subprocess.run(["huggingface-cli", "--version"], capture_output=True, text=True) - logger.info(f"Hugging Face CLI found: {result.stdout.strip()}") - except FileNotFoundError: - logger.error("Hugging Face CLI not found. Please install with 'pip install huggingface_hub'") - return False - - # Check Git - try: - result = subprocess.run(["git", "--version"], capture_output=True, text=True) - logger.info(f"Git found: {result.stdout.strip()}") - except FileNotFoundError: - logger.error("Git not found. Please install Git and try again.") - return False - - # Check HF Token - hf_token = os.getenv("HF_TOKEN") - if not hf_token: - logger.warning("HF_TOKEN environment variable not found. You may need to log in manually.") - - return True - -def verify_local_project(): - """Verify the local project structure.""" - logger.info("Verifying local project structure...") - - # Required files - required_files = [ - "app.py", - "requirements.txt", - "README.md", - "agent/agent.py", - "agent/config.py", - "agent/graph.py" - ] - - missing_files = [] - for file_path in required_files: - if not os.path.isfile(file_path): - missing_files.append(file_path) - logger.error(f"Missing required file: {file_path}") - - if missing_files: - logger.error(f"Missing {len(missing_files)} required files. Please fix before deployment.") - return False - - # Check Gradio dependencies in requirements.txt - with open("requirements.txt", "r") as f: - requirements = f.read() - if "gradio" not in requirements: - logger.error("Gradio not found in requirements.txt") - return False - - # Run basic code tests - try: - import importlib.util - - # Try to import the main modules to check for syntax errors - modules_to_check = [ - ("agent.agent", "agent/agent.py"), - ("agent.config", "agent/config.py"), - ("agent.graph", "agent/graph.py"), - ] - - for module_name, module_path in modules_to_check: - spec = importlib.util.spec_from_file_location(module_name, module_path) - module = importlib.util.module_from_spec(spec) - spec.loader.exec_module(module) - logger.info(f"Successfully imported {module_name}") - - except Exception as e: - logger.error(f"Error importing project modules: {str(e)}") - return False - - logger.info("Local project structure verification passed") - return True - -def create_or_update_space(args): - """Create or update the Hugging Face Space.""" - logger.info(f"Creating/updating Space: {args.username}/{args.space}") - - space_url = f"https://huggingface.co/spaces/{args.username}/{args.space}" - - # Check if Space exists - try: - result = subprocess.run( - ["curl", "-s", "-o", "/dev/null", "-w", "%{http_code}", space_url], - capture_output=True, - text=True - ) - space_exists = result.stdout.strip() == "200" - except Exception: - space_exists = False - - if space_exists: - logger.info(f"Space already exists: {space_url}") - else: - logger.info(f"Creating new Space: {args.username}/{args.space}") - - # Create the Space - cmd = [ - "huggingface-cli", "repo", "create", - f"{args.username}/{args.space}", - "--type", "space", - "--sdk", args.sdk - ] - - if args.private: - cmd.append("--private") - else: - cmd.append("--public") - - try: - subprocess.run(cmd, check=True) - logger.info(f"Space created successfully: {space_url}") - except subprocess.CalledProcessError as e: - logger.error(f"Error creating Space: {str(e)}") - return False - - # Update Space hardware if specified - try: - hardware_cmd = [ - "huggingface-cli", "repo", "update-space", - f"{args.username}/{args.space}", - "--hardware", args.hardware - ] - subprocess.run(hardware_cmd, check=True) - logger.info(f"Space hardware set to: {args.hardware}") - except subprocess.CalledProcessError as e: - logger.error(f"Error updating Space hardware: {str(e)}") - - return True - -def setup_environment_variables(args): - """Set up environment variables for the Hugging Face Space.""" - logger.info("Setting up environment variables...") - - # Get environment variables from current environment - env_vars = { - "OPENAI_API_KEY": os.getenv("OPENAI_API_KEY", ""), - "SUPABASE_URL": os.getenv("SUPABASE_URL", ""), - "SUPABASE_KEY": os.getenv("SUPABASE_KEY", ""), - "SERPER_API_KEY": os.getenv("SERPER_API_KEY", ""), - "HF_TOKEN": os.getenv("HF_TOKEN", ""), - "SPACE_ID": f"{args.username}/{args.space}" - } - - # Ask for missing variables - for var_name, var_value in list(env_vars.items()): - if not var_value: - print(f"\n{var_name} not found in environment.") - - if var_name == "SPACE_ID": - # This one can be auto-generated - env_vars[var_name] = f"{args.username}/{args.space}" - continue - - user_input = input(f"Enter {var_name} (leave blank to skip): ") - if user_input: - env_vars[var_name] = user_input - - # Generate commands to set variables in Hugging Face Space - env_commands = [] - for var_name, var_value in env_vars.items(): - if var_value: - # Mask the value for logging - masked_value = var_value[:4] + "*" * (len(var_value) - 8) + var_value[-4:] if len(var_value) > 8 else "****" - logger.info(f"Adding environment variable: {var_name}={masked_value}") - - cmd = [ - "huggingface-cli", "secret", "set", - f"{args.username}/{args.space}", - var_name, var_value - ] - env_commands.append(cmd) - - # Execute the commands - success = True - for cmd in env_commands: - try: - # Don't capture output to avoid showing the secrets in logs - subprocess.run(cmd, check=True, capture_output=True) - except subprocess.CalledProcessError as e: - logger.error(f"Error setting environment variable: {str(e)}") - success = False - - if success: - logger.info("Environment variables set successfully") - - return success - -def deploy_code_to_space(args): - """Deploy the code to the Hugging Face Space.""" - logger.info("Deploying code to Hugging Face Space...") - - # Create a temporary directory for deployment - with tempfile.TemporaryDirectory() as temp_dir: - # Clone the Space repository - space_repo = f"https://huggingface.co/spaces/{args.username}/{args.space}" - logger.info(f"Cloning Space repository: {space_repo}") - - try: - subprocess.run( - ["git", "clone", space_repo, temp_dir], - check=True, - capture_output=True, - text=True - ) - except subprocess.CalledProcessError as e: - logger.error(f"Error cloning Space repository: {e.stderr}") - return False - - # Copy project files to the cloned repository - logger.info("Copying project files to the cloned repository") - - # Define files/directories to copy (excluding .git and other unnecessary files) - items_to_copy = [ - "agent", - "docs", - "memory", - "tests", - "tools", - "utils", - "app.py", - "requirements.txt", - "README.md", - "verify_gaia_assessment.py", - "test_deployment_readiness.py" - ] - - for item in items_to_copy: - if os.path.exists(item): - src_path = os.path.abspath(item) - dst_path = os.path.join(temp_dir, os.path.basename(item)) - - try: - if os.path.isdir(src_path): - if os.path.exists(dst_path): - shutil.rmtree(dst_path) - shutil.copytree(src_path, dst_path) - else: - shutil.copy2(src_path, dst_path) - logger.info(f"Copied {item}") - except Exception as e: - logger.error(f"Error copying {item}: {str(e)}") - - # Create a .gitignore file if it doesn't exist - gitignore_path = os.path.join(temp_dir, ".gitignore") - if not os.path.exists(gitignore_path): - with open(gitignore_path, "w") as f: - f.write("__pycache__/\n*.py[cod]\n*$py.class\n.env\n.venv\nenv/\nvenv/\n*.log\n") - logger.info("Created .gitignore file") - - # Commit and push the changes - logger.info("Committing and pushing changes") - try: - # Configure git (necessary for CI/CD) - subprocess.run( - ["git", "config", "--global", "user.email", "huggingface@example.com"], - cwd=temp_dir, - check=True, - capture_output=True - ) - subprocess.run( - ["git", "config", "--global", "user.name", "GAIA Deployment Script"], - cwd=temp_dir, - check=True, - capture_output=True - ) - - # Add all files - subprocess.run( - ["git", "add", "."], - cwd=temp_dir, - check=True, - capture_output=True - ) - - # Commit - subprocess.run( - ["git", "commit", "-m", "Deploy GAIA Agent"], - cwd=temp_dir, - check=True, - capture_output=True - ) - - # Push - subprocess.run( - ["git", "push"], - cwd=temp_dir, - check=True, - capture_output=True - ) - - logger.info("Code deployed successfully") - return True - except subprocess.CalledProcessError as e: - logger.error(f"Error deploying code: {e.stderr if hasattr(e, 'stderr') else str(e)}") - return False - -def main(): - """Main entry point for the deployment script.""" - args = parse_arguments() - - if args.debug: - logger.setLevel(logging.DEBUG) - - logger.info("===== GAIA Agent Deployment to Hugging Face =====") - - if not check_prerequisites(): - logger.error("Prerequisites check failed. Please fix issues and try again.") - return 1 - - if not args.skip_verification and not verify_local_project(): - logger.error("Local project verification failed. Please fix issues and try again.") - return 1 - - if not create_or_update_space(args): - logger.error("Failed to create or update Hugging Face Space.") - return 1 - - if not setup_environment_variables(args): - logger.warning("Some environment variables could not be set. Check manually.") - - if not deploy_code_to_space(args): - logger.error("Failed to deploy code to Hugging Face Space.") - return 1 - - space_url = f"https://huggingface.co/spaces/{args.username}/{args.space}" - logger.info(f"\n🚀 Deployment successful! Your GAIA Agent is available at:\n{space_url}\n") - logger.info("Note: It may take a few minutes for the Space to build and start.") - - logger.info("===== Post-Deployment Steps =====") - logger.info("1. Check Space logs in the Factory tab if there are issues") - logger.info("2. Test the agent with a simple query") - logger.info("3. Monitor performance and memory usage") - - return 0 - -if __name__ == "__main__": - sys.exit(main()) \ No newline at end of file diff --git a/deploy_to_huggingface.py b/deploy_to_huggingface.py deleted file mode 100644 index 8c1562a8873d77517610e9aeb6e4ad7bd3df9936..0000000000000000000000000000000000000000 --- a/deploy_to_huggingface.py +++ /dev/null @@ -1,333 +0,0 @@ -#!/usr/bin/env python -""" -Hugging Face Space Deployment Script - -This script deploys the GAIA project to Hugging Face Spaces using git. -It handles authentication, remote setup, and deployment. - -Usage: - python deploy_to_huggingface.py [--space-id SPACE_ID] - -Author: DevOps Team -Date: April 30, 2025 -""" - -import os -import sys -import argparse -import subprocess -import logging -import json -import time -from pathlib import Path - -# Configure logging -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("hf_deploy") - -def parse_arguments(): - """Parse command line arguments""" - parser = argparse.ArgumentParser(description="Deploy GAIA to Hugging Face Space") - parser.add_argument("--space-id", - help="Hugging Face Space ID (username/space-name)", - default="JoachimVC/GAIA-Assistant") - parser.add_argument("--check-only", action="store_true", - help="Only check prerequisites without deploying") - return parser.parse_args() - -def run_command(command, shell=False): - """Run a shell command and return the result""" - try: - logger.info(f"Running command: {command}") - result = subprocess.run( - command, - shell=shell, - check=True, - text=True, - capture_output=True - ) - return result.stdout.strip() - except subprocess.CalledProcessError as e: - logger.error(f"Command failed with error: {e}") - logger.error(f"Error output: {e.stderr}") - return None - -def check_prerequisites(): - """Check if all prerequisites are met""" - # Check if git is installed - git_version = run_command(["git", "--version"]) - if not git_version: - logger.error("Git is not installed or not in PATH") - return False - logger.info(f"Git version: {git_version}") - - # Check if huggingface_hub is installed - try: - import huggingface_hub - logger.info(f"huggingface_hub version: {huggingface_hub.__version__}") - except ImportError: - logger.error("huggingface_hub is not installed") - logger.info("Installing huggingface_hub...") - if run_command(["pip", "install", "huggingface_hub"]) is None: - return False - - # Check if user is logged in to Hugging Face - hf_token_exists = os.path.exists(os.path.expanduser("~/.huggingface/token")) - if not hf_token_exists: - logger.error("Not logged in to Hugging Face") - logger.info("Please run 'huggingface-cli login' to log in") - return False - - # Check if the project has necessary files - required_files = ["app.py", "requirements.txt", "space_config.json", "HF_README.md"] - missing_files = [f for f in required_files if not os.path.exists(f)] - if missing_files: - logger.error(f"Missing required files: {', '.join(missing_files)}") - return False - - logger.info("All prerequisites met") - return True - -def setup_git_repo(space_id): - """Set up git repository for deployment""" - # Check if .git exists - if not os.path.exists(".git"): - logger.info("Initializing git repository...") - if run_command(["git", "init"]) is None: - return False - - # Check if the Hugging Face remote exists - remotes = run_command(["git", "remote"]) - if remotes is None: - return False - - if "huggingface" in remotes.split(): - logger.info("Updating Hugging Face remote...") - if run_command(["git", "remote", "set-url", "huggingface", f"https://huggingface.co/spaces/{space_id}"]) is None: - return False - else: - logger.info("Adding Hugging Face remote...") - if run_command(["git", "remote", "add", "huggingface", f"https://huggingface.co/spaces/{space_id}"]) is None: - return False - - logger.info("Git repository set up successfully") - return True - -def validate_space_id(space_id): - """Validate the Hugging Face Space ID format""" - if "/" not in space_id: - logger.error(f"Invalid Space ID format: {space_id}") - logger.error("Space ID should be in format 'username/space-name'") - return False - - username, space_name = space_id.split("/", 1) - if not username or not space_name: - logger.error(f"Invalid Space ID components: {space_id}") - return False - - return True - -def rename_files_for_hf(): - """Rename files for Hugging Face Space""" - if os.path.exists("HF_README.md"): - if os.path.exists("README.md"): - if not os.path.exists("README_original.md"): - logger.info("Backing up README.md to README_original.md...") - os.rename("README.md", "README_original.md") - else: - logger.info("README_original.md already exists, keeping it") - # Remove README.md to avoid confusion - if os.path.exists("README.md"): - os.remove("README.md") - - logger.info("Renaming HF_README.md to README.md...") - os.rename("HF_README.md", "README.md") - - return True - -def update_git_ignore(): - """Update .gitignore file for Hugging Face Space""" - gitignore_entries = [ - ".env", - "__pycache__/", - "*.py[cod]", - "*$py.class", - "*.so", - ".Python", - "env/", - "build/", - "develop-eggs/", - "dist/", - "downloads/", - "eggs/", - ".eggs/", - "lib/", - "lib64/", - "parts/", - "sdist/", - "var/", - "*.egg-info/", - ".installed.cfg", - "*.egg", - ".env", - ".venv", - "venv/", - "ENV/", - ".idea/", - ".vscode/", - "*.log", - "deploy_to_huggingface.py", - "test_components.py", - "verify_production.py", - "production_verification_report.md", - "deployment_readiness_report.md", - "final_verification_summary.md", - "prepare_for_hf_deployment.py" - ] - - # Read existing .gitignore if it exists - existing_entries = [] - if os.path.exists(".gitignore"): - with open(".gitignore", "r") as f: - existing_entries = [line.strip() for line in f.readlines()] - - # Combine entries, removing duplicates - combined_entries = list(set(existing_entries + gitignore_entries)) - combined_entries.sort() - - # Write updated .gitignore - with open(".gitignore", "w") as f: - f.write("\n".join(combined_entries) + "\n") - - logger.info("Updated .gitignore file") - return True - -def commit_and_push(space_id): - """Commit changes and push to Hugging Face Space""" - # Stage all files - if run_command(["git", "add", "."]) is None: - return False - - # Commit changes - commit_message = f"Deploy GAIA to Hugging Face Space {space_id}" - if run_command(["git", "commit", "-m", commit_message]) is None: - logger.warning("No changes to commit or commit failed") - # Continue anyway, as there might be no changes - - # Push to Hugging Face - logger.info(f"Pushing to Hugging Face Space {space_id}...") - if run_command(["git", "push", "huggingface", "main"]) is None: - # Try pushing to master branch if main fails - logger.info("Pushing to master branch instead...") - if run_command(["git", "push", "huggingface", "master"]) is None: - # Try force pushing to main as last resort - logger.info("Force pushing to main branch...") - if run_command(["git", "push", "-f", "huggingface", "main"]) is None: - return False - - logger.info(f"Successfully pushed to Hugging Face Space {space_id}") - return True - -def create_space_if_needed(space_id): - """Create Hugging Face Space if it doesn't exist""" - try: - from huggingface_hub import HfApi, SpaceHardware, SpaceSdk - - api = HfApi() - username, space_name = space_id.split("/", 1) - - # Check if space exists - try: - api.get_space_info(space_id) - logger.info(f"Space {space_id} already exists") - return True - except Exception: - logger.info(f"Space {space_id} doesn't exist, creating it...") - - # Create space - api.create_space( - space_id=space_name, - token=None, # Use token from ~/.huggingface/token - sdk=SpaceSdk.GRADIO, - hardware=SpaceHardware.CPU_BASIC, - private=False - ) - - logger.info(f"Created Hugging Face Space {space_id}") - return True - - except ImportError: - logger.warning("huggingface_hub not installed properly, skipping space creation check") - return True - except Exception as e: - logger.error(f"Error creating space: {e}") - return False - -def wait_for_deployment(space_id): - """Wait for deployment to complete and provide status link""" - logger.info(f"Deployment initiated to {space_id}") - logger.info(f"You can check the deployment status at: https://huggingface.co/spaces/{space_id}") - logger.info(f"Once deployment is complete, your app will be available at: https://huggingface.co/spaces/{space_id}") - - logger.info("Waiting for 60 seconds to allow deployment to start...") - for i in range(12): - time.sleep(5) - logger.info(f"Elapsed time: {(i+1)*5} seconds") - - return True - -def main(): - """Main function to deploy to Hugging Face Space""" - args = parse_arguments() - space_id = args.space_id - - logger.info(f"Starting deployment to Hugging Face Space: {space_id}") - - # Validate Space ID - if not validate_space_id(space_id): - return False - - # Check prerequisites - if not check_prerequisites(): - return False - - if args.check_only: - logger.info("Check-only mode: prerequisites satisfied, stopping before deployment") - return True - - # Rename files for Hugging Face - if not rename_files_for_hf(): - return False - - # Update .gitignore - if not update_git_ignore(): - return False - - # Create Space if needed - if not create_space_if_needed(space_id): - logger.warning("Failed to create space, but continuing with deployment") - - # Set up git repository - if not setup_git_repo(space_id): - return False - - # Commit and push changes - if not commit_and_push(space_id): - return False - - # Wait for deployment and show status - if not wait_for_deployment(space_id): - return False - - logger.info("Deployment process completed successfully!") - logger.info(f"Your GAIA application will be available at: https://huggingface.co/spaces/{space_id}") - logger.info("Note: It may take a few minutes for the deployment to complete on Hugging Face") - - return True - -if __name__ == "__main__": - success = main() - sys.exit(0 if success else 1) \ No newline at end of file diff --git a/deploy_to_huggingface_credential.py b/deploy_to_huggingface_credential.py deleted file mode 100644 index 612345d23d12bb47fac2e67af5fabade59f7e3fb..0000000000000000000000000000000000000000 --- a/deploy_to_huggingface_credential.py +++ /dev/null @@ -1,320 +0,0 @@ -""" -Git Credential-based Deployment Script for Hugging Face Spaces - -This script handles the git operations required to deploy to a Hugging Face Space -using credential helper for token-based authentication. - -Usage: - python deploy_to_huggingface_credential.py [--space SPACE_ID] - -Environment variables (from .env file): - HF_TOKEN - Your Hugging Face API token -""" -import os -import subprocess -import sys -import argparse -from pathlib import Path - -def parse_args(): - parser = argparse.ArgumentParser(description="Deploy to Hugging Face Space using git credential helper") - parser.add_argument("--space", help="Space ID (username/space-name)", - default="JoachimVC/Final_Assignment_Template") - return parser.parse_args() - -def load_dotenv(): - """Load environment variables from .env file""" - env_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), '.env') - if not os.path.exists(env_path): - print(f"ERROR: .env file not found at {env_path}") - return False - - print(f"Loading environment variables from {env_path}") - with open(env_path, 'r') as file: - for line in file: - line = line.strip() - if not line or line.startswith('#'): - continue - - key, value = line.split('=', 1) - os.environ[key] = value - - return True - -def run_command(cmd, silent=False): - """Run a shell command and return output""" - if not silent: - print(f"Running: {cmd}") - try: - process = subprocess.Popen( - cmd, - shell=True, - stdout=subprocess.PIPE, - stderr=subprocess.PIPE, - text=True - ) - stdout, stderr = process.communicate() - - if process.returncode != 0: - print(f"Command failed with error code {process.returncode}") - print(f"Error: {stderr.strip()}") - return False - - if not silent and stdout.strip(): - print(stdout.strip()) - - return True - except Exception as e: - print(f"Failed to execute command: {str(e)}") - return False - -def check_git_installed(): - """Check if git is installed""" - print("Checking if git is installed...") - return run_command("git --version", silent=True) - -def get_token_from_env(): - """Get Hugging Face token from environment variables""" - # Try various environment variable names that might contain the token - token_vars = ["HF_TOKEN", "HUGGINGFACE_TOKEN", "HUGGING_FACE_TOKEN", - "HF_API_TOKEN", "HUGGINGFACE_API_TOKEN"] - - for var_name in token_vars: - token = os.environ.get(var_name) - if token: - print(f"Found token in environment variable: {var_name}") - return token - - return None - -def create_deploy_token_file(token): - """Create a temporary token file for deployment""" - token_dir = Path.home() / ".huggingface" - token_dir.mkdir(exist_ok=True) - - token_file = token_dir / "token" - with open(token_file, "w") as f: - f.write(token) - - print(f"Created HF token file at {token_file}") - return token_file - -def setup_git_credential_helper(token): - """Set up git credential helper to use the token""" - # Store the token in the git credential helper - credential_input = f"protocol=https\nhost=huggingface.co\nusername=hf_token\npassword={token}\n" - - # Use git credential-store to save the token - print("Setting up git credential helper...") - process = subprocess.Popen( - ["git", "credential-store", "store"], - stdin=subprocess.PIPE, - stdout=subprocess.PIPE, - stderr=subprocess.PIPE, - text=True - ) - - process.communicate(input=credential_input) - - # Configure git to use the credential helper - return run_command("git config --global credential.helper store") - -def setup_git(space_id): - """Initialize git if needed and configure remote""" - # Check if git is already initialized - if not os.path.exists(".git"): - print("Initializing git repository...") - if not run_command("git init"): - return False - - # Configure git user if not set - result = subprocess.run("git config user.name", shell=True, capture_output=True, text=True) - if not result.stdout.strip(): - print("Setting up git user name...") - run_command('git config user.name "GAIA Deployment"') - - result = subprocess.run("git config user.email", shell=True, capture_output=True, text=True) - if not result.stdout.strip(): - print("Setting up git user email...") - run_command('git config user.email "gaia-deployment@example.com"') - - # Set up the Hugging Face remote without token in URL - print("Setting up git remote for Hugging Face...") - - # Check if remote exists - result = subprocess.run("git remote -v", shell=True, capture_output=True, text=True) - remote_output = result.stdout - - if "huggingface" in remote_output: - run_command("git remote remove huggingface") - - # Add the HF remote without token (will use credential helper) - repo_url = f"https://huggingface.co/spaces/{space_id}" - return run_command(f"git remote add huggingface {repo_url}") - -def create_gitignore(): - """Create .gitignore file with common patterns""" - patterns = [ - ".env", - "__pycache__/", - "*.py[cod]", - "*$py.class", - ".Python", - "env/", - "venv/", - ".venv", - ".coverage", - "*.log", - "deploy_with_token.py", - "deploy_to_huggingface_env.py", - "deploy_to_huggingface_dotenv.py", - "deploy_to_huggingface_credential.py" # Don't include this file with token handling - ] - - print("Creating .gitignore file...") - with open(".gitignore", "w") as f: - for pattern in patterns: - f.write(f"{pattern}\n") - - return True - -def commit_changes(): - """Stage and commit all changes""" - print("Staging files for commit...") - if not run_command("git add -A"): - return False - - print("Committing changes...") - return run_command('git commit -m "Deploy GAIA to Hugging Face Space"') - -def push_to_huggingface(): - """Push to Hugging Face Space""" - print("Pushing to Hugging Face Space...") - - # Try direct push with git credential store - cmd = "git push huggingface main" - - # Execute the push command - if run_command(cmd): - return True - - # If regular push fails, try force push - print("Regular push failed, attempting force push...") - if run_command("git push huggingface main --force"): - return True - - print("Force push failed. You may need to manually push:") - print("1. Ensure your token has write permissions") - print("2. Try: git push huggingface main --force") - return False - -def setup_cli_auth_file(token): - """Set up CLI authentication file for Hugging Face""" - # Create .huggingface directory in home if it doesn't exist - hf_dir = os.path.expanduser("~/.huggingface") - os.makedirs(hf_dir, exist_ok=True) - - # Create or update token file - token_file = os.path.join(hf_dir, "token") - with open(token_file, "w") as f: - f.write(token) - - print(f"Created HF token file at {token_file}") - return True - -def setup_git_with_netrc(token): - """Set up .netrc file for git authentication""" - netrc_path = os.path.expanduser("~/.netrc") - - # Check if file exists and has huggingface.co entry - netrc_content = "" - if os.path.exists(netrc_path): - with open(netrc_path, "r") as f: - netrc_content = f.read() - - # Add or update huggingface.co entry - if "machine huggingface.co" not in netrc_content: - with open(netrc_path, "a") as f: - f.write(f"\nmachine huggingface.co\nlogin hf_token\npassword {token}\n") - else: - # Replace existing entry - import re - pattern = r"machine huggingface\.co\s+login.*\s+password.*\s+" - replacement = f"machine huggingface.co\nlogin hf_token\npassword {token}\n" - netrc_content = re.sub(pattern, replacement, netrc_content) - with open(netrc_path, "w") as f: - f.write(netrc_content) - - # Set permissions for .netrc file (Unix only) - if os.name != "nt": # not Windows - os.chmod(netrc_path, 0o600) - - print(f"Updated .netrc file at {netrc_path}") - return True - -def main(): - """Run deployment process""" - args = parse_args() - space_id = args.space - - # Load .env file - if not load_dotenv(): - print("WARNING: Failed to load .env file, continuing with existing environment variables") - - # Get token from environment - token = get_token_from_env() - if not token: - print("ERROR: No Hugging Face token found in environment variables.") - print("Please ensure your .env file contains one of the following variables:") - print(" - HF_TOKEN") - print(" - HUGGINGFACE_TOKEN") - print(" - HUGGING_FACE_TOKEN") - print(" - HF_API_TOKEN") - print(" - HUGGINGFACE_API_TOKEN") - sys.exit(1) - - repo_url = f"https://huggingface.co/spaces/{space_id}" - - print("\n===== DEPLOYING TO HUGGING FACE SPACE =====") - print(f"Target Space: {repo_url}") - - # Ensure Git is installed - if not check_git_installed(): - print("Git is not installed. Please install Git and try again.") - sys.exit(1) - - # Set up token file in ~/.huggingface/token - if not setup_cli_auth_file(token): - print("Failed to set up CLI authentication file") - sys.exit(1) - - # Set up .netrc file with token - if not setup_git_with_netrc(token): - print("Failed to set up .netrc file") - sys.exit(1) - - # Set up git credential helper - if not setup_git_credential_helper(token): - print("Failed to set up git credential helper") - sys.exit(1) - - # Setup steps - steps = [ - ("Setting up git", lambda: setup_git(space_id)), - ("Creating .gitignore", create_gitignore), - ("Committing changes", commit_changes), - ("Pushing to Hugging Face", push_to_huggingface) - ] - - for name, func in steps: - print(f"\n>> {name}...") - if not func(): - print(f"Step '{name}' failed. Deployment process halted.") - sys.exit(1) - - print("\n===== DEPLOYMENT COMPLETED SUCCESSFULLY =====") - print(f"Your project has been deployed to: {repo_url}") - print("Note: It may take a few minutes for the changes to be visible.") - -if __name__ == "__main__": - main() \ No newline at end of file diff --git a/deploy_to_huggingface_dotenv.py b/deploy_to_huggingface_dotenv.py deleted file mode 100644 index 8471989525028785ef3639243fb80381ad39036a..0000000000000000000000000000000000000000 --- a/deploy_to_huggingface_dotenv.py +++ /dev/null @@ -1,227 +0,0 @@ -""" -Dotenv-based Git Deployment Script for Hugging Face Spaces - -This script handles the git operations required to deploy to a Hugging Face Space -using token-based authentication, with the token loaded from a .env file. - -Usage: - python deploy_to_huggingface_dotenv.py [--space SPACE_ID] - -Environment variables (from .env file): - HF_TOKEN - Your Hugging Face API token -""" -import os -import subprocess -import sys -import argparse - -def parse_args(): - parser = argparse.ArgumentParser(description="Deploy to Hugging Face Space using .env file") - parser.add_argument("--space", help="Space ID (username/space-name)", - default="JoachimVC/Final_Assignment_Template") - return parser.parse_args() - -def load_dotenv(): - """Load environment variables from .env file""" - env_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), '.env') - if not os.path.exists(env_path): - print(f"ERROR: .env file not found at {env_path}") - return False - - print(f"Loading environment variables from {env_path}") - with open(env_path, 'r') as file: - for line in file: - line = line.strip() - if not line or line.startswith('#'): - continue - - key, value = line.split('=', 1) - os.environ[key] = value - - return True - -def run_command(cmd, silent=False): - """Run a shell command and return output""" - if not silent: - print(f"Running: {cmd}") - try: - process = subprocess.Popen( - cmd, - shell=True, - stdout=subprocess.PIPE, - stderr=subprocess.PIPE, - text=True - ) - stdout, stderr = process.communicate() - - if process.returncode != 0: - print(f"Command failed with error code {process.returncode}") - print(f"Error: {stderr.strip()}") - return False - - if not silent and stdout.strip(): - print(stdout.strip()) - - return True - except Exception as e: - print(f"Failed to execute command: {str(e)}") - return False - -def check_git_installed(): - """Check if git is installed""" - print("Checking if git is installed...") - return run_command("git --version", silent=True) - -def get_token_from_env(): - """Get Hugging Face token from environment variables""" - # Try various environment variable names that might contain the token - token_vars = ["HF_TOKEN", "HUGGINGFACE_TOKEN", "HUGGING_FACE_TOKEN", - "HF_API_TOKEN", "HUGGINGFACE_API_TOKEN"] - - for var_name in token_vars: - token = os.environ.get(var_name) - if token: - print(f"Found token in environment variable: {var_name}") - return token - - return None - -def setup_git(space_id, token): - """Initialize git if needed and configure remote with token""" - # Check if git is already initialized - if not os.path.exists(".git"): - print("Initializing git repository...") - if not run_command("git init"): - return False - - # Configure git user if not set - result = subprocess.run("git config user.name", shell=True, capture_output=True, text=True) - if not result.stdout.strip(): - print("Setting up git user name...") - run_command('git config user.name "GAIA Deployment"') - - result = subprocess.run("git config user.email", shell=True, capture_output=True, text=True) - if not result.stdout.strip(): - print("Setting up git user email...") - run_command('git config user.email "gaia-deployment@example.com"') - - # Set up the Hugging Face remote with token - print("Setting up git remote for Hugging Face...") - - # Check if remote exists - result = subprocess.run("git remote -v", shell=True, capture_output=True, text=True) - remote_output = result.stdout - - if "huggingface" in remote_output: - run_command("git remote remove huggingface") - - # Add the HF remote with token authentication - token_url = f"https://{token}@huggingface.co/spaces/{space_id}" - - # Don't print the token in the console output - print(f"Adding remote: huggingface -> https://****@huggingface.co/spaces/{space_id}") - return run_command(f"git remote add huggingface {token_url}") - -def create_gitignore(): - """Create .gitignore file with common patterns""" - patterns = [ - ".env", - "__pycache__/", - "*.py[cod]", - "*$py.class", - ".Python", - "env/", - "venv/", - ".venv", - ".coverage", - "*.log", - "deploy_with_token.py", - "deploy_to_huggingface_env.py", - "deploy_to_huggingface_dotenv.py" # Don't include this file with token handling - ] - - print("Creating .gitignore file...") - with open(".gitignore", "w") as f: - for pattern in patterns: - f.write(f"{pattern}\n") - - return True - -def commit_changes(): - """Stage and commit all changes""" - print("Staging files for commit...") - if not run_command("git add -A"): - return False - - print("Committing changes...") - return run_command('git commit -m "Deploy GAIA to Hugging Face Space"') - -def push_to_huggingface(): - """Push to Hugging Face Space""" - print("Pushing to Hugging Face Space...") - - # Try regular push first - if run_command("git push huggingface main"): - return True - - # If regular push fails, try force push - print("Regular push failed, attempting force push...") - if run_command("git push huggingface main --force"): - return True - - print("Force push failed. You may need to manually push:") - print("1. Ensure your token has write permissions") - print("2. Try: git push huggingface main --force") - return False - -def main(): - """Run deployment process""" - args = parse_args() - space_id = args.space - - # Load .env file - if not load_dotenv(): - print("WARNING: Failed to load .env file, continuing with existing environment variables") - - # Get token from environment - token = get_token_from_env() - if not token: - print("ERROR: No Hugging Face token found in environment variables.") - print("Please ensure your .env file contains one of the following variables:") - print(" - HF_TOKEN") - print(" - HUGGINGFACE_TOKEN") - print(" - HUGGING_FACE_TOKEN") - print(" - HF_API_TOKEN") - print(" - HUGGINGFACE_API_TOKEN") - sys.exit(1) - - repo_url = f"https://huggingface.co/spaces/{space_id}" - - print("\n===== DEPLOYING TO HUGGING FACE SPACE =====") - print(f"Target Space: {repo_url}") - - # Ensure Git is installed - if not check_git_installed(): - print("Git is not installed. Please install Git and try again.") - sys.exit(1) - - # Setup steps - steps = [ - ("Setting up git", lambda: setup_git(space_id, token)), - ("Creating .gitignore", create_gitignore), - ("Committing changes", commit_changes), - ("Pushing to Hugging Face", push_to_huggingface) - ] - - for name, func in steps: - print(f"\n>> {name}...") - if not func(): - print(f"Step '{name}' failed. Deployment process halted.") - sys.exit(1) - - print("\n===== DEPLOYMENT COMPLETED SUCCESSFULLY =====") - print(f"Your project has been deployed to: {repo_url}") - print("Note: It may take a few minutes for the changes to be visible.") - -if __name__ == "__main__": - main() \ No newline at end of file diff --git a/deploy_to_huggingface_env.py b/deploy_to_huggingface_env.py deleted file mode 100644 index edcd4194cc4ead7bd61b0660eada30ea34303d2d..0000000000000000000000000000000000000000 --- a/deploy_to_huggingface_env.py +++ /dev/null @@ -1,203 +0,0 @@ -""" -Environment-based Git Deployment Script for Hugging Face Spaces - -This script handles the git operations required to deploy to a Hugging Face Space -using token-based authentication, with the token taken from environment variables. - -Usage: - python deploy_to_huggingface_env.py [--space SPACE_ID] - -Environment variables: - HUGGINGFACE_TOKEN - Your Hugging Face API token -""" -import os -import subprocess -import sys -import argparse - -def parse_args(): - parser = argparse.ArgumentParser(description="Deploy to Hugging Face Space using environment token") - parser.add_argument("--space", help="Space ID (username/space-name)", - default="JoachimVC/Final_Assignment_Template") - return parser.parse_args() - -def run_command(cmd, silent=False): - """Run a shell command and return output""" - if not silent: - print(f"Running: {cmd}") - try: - process = subprocess.Popen( - cmd, - shell=True, - stdout=subprocess.PIPE, - stderr=subprocess.PIPE, - text=True - ) - stdout, stderr = process.communicate() - - if process.returncode != 0: - print(f"Command failed with error code {process.returncode}") - print(f"Error: {stderr.strip()}") - return False - - if not silent and stdout.strip(): - print(stdout.strip()) - - return True - except Exception as e: - print(f"Failed to execute command: {str(e)}") - return False - -def check_git_installed(): - """Check if git is installed""" - print("Checking if git is installed...") - return run_command("git --version", silent=True) - -def get_token_from_env(): - """Get Hugging Face token from environment variables""" - # Try various environment variable names that might contain the token - token_vars = ["HUGGINGFACE_TOKEN", "HF_TOKEN", "HUGGING_FACE_TOKEN", - "HF_API_TOKEN", "HUGGINGFACE_API_TOKEN"] - - for var_name in token_vars: - token = os.environ.get(var_name) - if token: - print(f"Found token in environment variable: {var_name}") - return token - - return None - -def setup_git(space_id, token): - """Initialize git if needed and configure remote with token""" - # Check if git is already initialized - if not os.path.exists(".git"): - print("Initializing git repository...") - if not run_command("git init"): - return False - - # Configure git user if not set - result = subprocess.run("git config user.name", shell=True, capture_output=True, text=True) - if not result.stdout.strip(): - print("Setting up git user name...") - run_command('git config user.name "GAIA Deployment"') - - result = subprocess.run("git config user.email", shell=True, capture_output=True, text=True) - if not result.stdout.strip(): - print("Setting up git user email...") - run_command('git config user.email "gaia-deployment@example.com"') - - # Set up the Hugging Face remote with token - print("Setting up git remote for Hugging Face...") - - # Check if remote exists - result = subprocess.run("git remote -v", shell=True, capture_output=True, text=True) - remote_output = result.stdout - - if "huggingface" in remote_output: - run_command("git remote remove huggingface") - - # Add the HF remote with token authentication - token_url = f"https://{token}@huggingface.co/spaces/{space_id}" - - # Don't print the token in the console output - print(f"Adding remote: huggingface -> https://****@huggingface.co/spaces/{space_id}") - return run_command(f"git remote add huggingface {token_url}") - -def create_gitignore(): - """Create .gitignore file with common patterns""" - patterns = [ - ".env", - "__pycache__/", - "*.py[cod]", - "*$py.class", - ".Python", - "env/", - "venv/", - ".venv", - ".coverage", - "*.log", - "deploy_with_token.py", - "deploy_to_huggingface_env.py" # Don't include this file with token handling - ] - - print("Creating .gitignore file...") - with open(".gitignore", "w") as f: - for pattern in patterns: - f.write(f"{pattern}\n") - - return True - -def commit_changes(): - """Stage and commit all changes""" - print("Staging files for commit...") - if not run_command("git add -A"): - return False - - print("Committing changes...") - return run_command('git commit -m "Deploy GAIA to Hugging Face Space"') - -def push_to_huggingface(): - """Push to Hugging Face Space""" - print("Pushing to Hugging Face Space...") - - # Try regular push first - if run_command("git push huggingface main"): - return True - - # If regular push fails, try force push - print("Regular push failed, attempting force push...") - if run_command("git push huggingface main --force"): - return True - - print("Force push failed. You may need to manually push:") - print("1. Ensure your token has write permissions") - print("2. Try: git push huggingface main --force") - return False - -def main(): - """Run deployment process""" - args = parse_args() - space_id = args.space - - # Get token from environment - token = get_token_from_env() - if not token: - print("ERROR: No Hugging Face token found in environment variables.") - print("Please set one of the following environment variables:") - print(" - HUGGINGFACE_TOKEN") - print(" - HF_TOKEN") - print(" - HUGGING_FACE_TOKEN") - print(" - HF_API_TOKEN") - print(" - HUGGINGFACE_API_TOKEN") - sys.exit(1) - - repo_url = f"https://huggingface.co/spaces/{space_id}" - - print("\n===== DEPLOYING TO HUGGING FACE SPACE =====") - print(f"Target Space: {repo_url}") - - # Ensure Git is installed - if not check_git_installed(): - print("Git is not installed. Please install Git and try again.") - sys.exit(1) - - # Setup steps - steps = [ - ("Setting up git", lambda: setup_git(space_id, token)), - ("Creating .gitignore", create_gitignore), - ("Committing changes", commit_changes), - ("Pushing to Hugging Face", push_to_huggingface) - ] - - for name, func in steps: - print(f"\n>> {name}...") - if not func(): - print(f"Step '{name}' failed. Deployment process halted.") - sys.exit(1) - - print("\n===== DEPLOYMENT COMPLETED SUCCESSFULLY =====") - print(f"Your project has been deployed to: {repo_url}") - print("Note: It may take a few minutes for the changes to be visible.") - -if __name__ == "__main__": - main() \ No newline at end of file diff --git a/deploy_with_token.py b/deploy_with_token.py deleted file mode 100644 index a20c05796315139ed14538817b4796c930553445..0000000000000000000000000000000000000000 --- a/deploy_with_token.py +++ /dev/null @@ -1,176 +0,0 @@ -""" -Token-Based Git Deployment Script for Hugging Face Spaces - -This script handles the git operations required to deploy to a Hugging Face Space -using token-based authentication. - -Usage: - python deploy_with_token.py --token YOUR_HF_TOKEN - -""" -import os -import subprocess -import sys -import argparse - -def parse_args(): - parser = argparse.ArgumentParser(description="Deploy to Hugging Face Space with token") - parser.add_argument("--token", help="Hugging Face API token", required=True) - parser.add_argument("--space", help="Space ID (username/space-name)", - default="JoachimVC/Final_Assignment_Template") - return parser.parse_args() - -def run_command(cmd, silent=False): - """Run a shell command and return output""" - if not silent: - print(f"Running: {cmd}") - try: - process = subprocess.Popen( - cmd, - shell=True, - stdout=subprocess.PIPE, - stderr=subprocess.PIPE, - text=True - ) - stdout, stderr = process.communicate() - - if process.returncode != 0: - print(f"Command failed with error code {process.returncode}") - print(f"Error: {stderr.strip()}") - return False - - if not silent and stdout.strip(): - print(stdout.strip()) - - return True - except Exception as e: - print(f"Failed to execute command: {str(e)}") - return False - -def check_git_installed(): - """Check if git is installed""" - print("Checking if git is installed...") - return run_command("git --version", silent=True) - -def setup_git(space_id, token): - """Initialize git if needed and configure remote with token""" - # Check if git is already initialized - if not os.path.exists(".git"): - print("Initializing git repository...") - if not run_command("git init"): - return False - - # Configure git user if not set - result = subprocess.run("git config user.name", shell=True, capture_output=True, text=True) - if not result.stdout.strip(): - print("Setting up git user name...") - run_command('git config user.name "GAIA Deployment"') - - result = subprocess.run("git config user.email", shell=True, capture_output=True, text=True) - if not result.stdout.strip(): - print("Setting up git user email...") - run_command('git config user.email "gaia-deployment@example.com"') - - # Set up the Hugging Face remote with token - print("Setting up git remote for Hugging Face...") - - # Check if remote exists - result = subprocess.run("git remote -v", shell=True, capture_output=True, text=True) - remote_output = result.stdout - - if "huggingface" in remote_output: - run_command("git remote remove huggingface") - - # Add the HF remote with token authentication - token_url = f"https://{token}@huggingface.co/spaces/{space_id}" - - # Don't print the token in the console output - print(f"Adding remote: huggingface -> https://****@huggingface.co/spaces/{space_id}") - return run_command(f"git remote add huggingface {token_url}") - -def create_gitignore(): - """Create .gitignore file with common patterns""" - patterns = [ - ".env", - "__pycache__/", - "*.py[cod]", - "*$py.class", - ".Python", - "env/", - "venv/", - ".venv", - ".coverage", - "*.log", - "deploy_with_token.py" # Don't include this file with token handling - ] - - print("Creating .gitignore file...") - with open(".gitignore", "w") as f: - for pattern in patterns: - f.write(f"{pattern}\n") - - return True - -def commit_changes(): - """Stage and commit all changes""" - print("Staging files for commit...") - if not run_command("git add -A"): - return False - - print("Committing changes...") - return run_command('git commit -m "Deploy GAIA to Hugging Face Space"') - -def push_to_huggingface(): - """Push to Hugging Face Space""" - print("Pushing to Hugging Face Space...") - - # Try regular push first - if run_command("git push huggingface main"): - return True - - # If regular push fails, try force push - print("Regular push failed, attempting force push...") - if run_command("git push huggingface main --force"): - return True - - print("Force push failed. You may need to manually push:") - print("1. Ensure your token has write permissions") - print("2. Try: git push huggingface main --force") - return False - -def main(): - """Run deployment process""" - args = parse_args() - token = args.token - space_id = args.space - - repo_url = f"https://huggingface.co/spaces/{space_id}" - - print("\n===== DEPLOYING TO HUGGING FACE SPACE =====") - print(f"Target Space: {repo_url}") - - # Ensure Git is installed - if not check_git_installed(): - print("Git is not installed. Please install Git and try again.") - sys.exit(1) - - # Setup steps - steps = [ - ("Setting up git", lambda: setup_git(space_id, token)), - ("Creating .gitignore", create_gitignore), - ("Committing changes", commit_changes), - ("Pushing to Hugging Face", push_to_huggingface) - ] - - for name, func in steps: - print(f"\n>> {name}...") - if not func(): - print(f"Step '{name}' failed. Deployment process halted.") - sys.exit(1) - - print("\n===== DEPLOYMENT COMPLETED SUCCESSFULLY =====") - print(f"Your project has been deployed to: {repo_url}") - print("Note: It may take a few minutes for the changes to be visible.") - -if __name__ == "__main__": - main() \ No newline at end of file diff --git a/deployment_checklist.md b/deployment_checklist.md deleted file mode 100644 index 085f42a5055b796b58e248d3b15efeb414ab958e..0000000000000000000000000000000000000000 --- a/deployment_checklist.md +++ /dev/null @@ -1,83 +0,0 @@ -# GAIA Agent Deployment Checklist for Hugging Face Space - -## Pre-Deployment Verification - -- [ ] Verify all required files are present - - [ ] app.py - - [ ] requirements.txt - - [ ] agent/agent.py, agent/config.py, agent/graph.py - - [ ] memory/supabase_memory.py, memory/supabase_memory_fix.py - - [ ] tools/ directory with all required tools - -- [ ] Verify requirements.txt contains all necessary dependencies - - [ ] openai - - [ ] gradio - - [ ] langchain - - [ ] supabase - - [ ] requests - - [ ] pandas - -- [ ] Verify app.py configuration - - [ ] OAuth handling is configured - - [ ] Gradio components are properly referenced - - [ ] Error handling is in place - -## Environment Variables Configuration - -- [ ] Set up required environment variables in Hugging Face Space: - - [ ] `OPENAI_API_KEY` - Required for language model access - - [ ] `SUPABASE_URL` - Required for memory functionality - - [ ] `SUPABASE_KEY` - Required for memory functionality - -- [ ] Set up optional environment variables: - - [ ] `SERPER_API_KEY` - For enhanced search capabilities - - [ ] `SPACE_ID` - Usually auto-set by Hugging Face - - [ ] `SPACE_HOST` - Usually auto-set by Hugging Face - -- [ ] Configure OAuth (for full functionality): - - [ ] Set `hf_oauth: true` in Space repository secrets - -## Deployment Process - -1. Push code to Hugging Face Space repository -2. Monitor build logs for any errors -3. Check if the Space is successfully built and running -4. Verify the interface loads correctly - -## Post-Deployment Verification - -- [ ] Verify the app loads without errors -- [ ] Check that the correct mode (full or demo) is active based on OAuth configuration -- [ ] Test the agent with a sample question -- [ ] Verify results display correctly in the interface -- [ ] Check integrated tools functionality: - - [ ] OpenAI integration - - [ ] Supabase memory integration - - [ ] Search tools - -## Troubleshooting Common Issues - -### OAuth Configuration Issues -- Symptom: "OAuth is required but OAUTH_CLIENT_ID environment variable is not set" error -- Solution: Set `hf_oauth: true` in Space repository secrets - -### OpenAI API Key Issues -- Symptom: "Error: OpenAI API key is required" or similar errors -- Solution: Verify OPENAI_API_KEY is correctly set in Space secrets - -### Supabase Connection Issues -- Symptom: "Error connecting to Supabase" or memory-related errors -- Solution: Verify SUPABASE_URL and SUPABASE_KEY are correctly set in Space secrets - -### Gradio Interface Issues -- Symptom: Interface not loading properly or component errors -- Solution: Check Space logs for specific component errors and update app.py accordingly - -## Rollback Plan - -If critical issues are encountered after deployment: - -1. Identify the specific component causing the issue from Space logs -2. If needed, revert to the previous working commit using Git -3. Fix issues locally and re-deploy with verified fixes -4. Consider temporarily enabling demo mode while fixing issues \ No newline at end of file diff --git a/deployment_readiness_report.json b/deployment_readiness_report.json deleted file mode 100644 index f3c3e89824ec5b715ee7896d9d72b1f0ceb3b351..0000000000000000000000000000000000000000 --- a/deployment_readiness_report.json +++ /dev/null @@ -1,38 +0,0 @@ -{ - "timestamp": "2025-05-01T09:14:06.477036", - "status": "not_ready", - "results": { - "file_structure": { - "passed": true, - "issues": [] - }, - "environment_variables": { - "passed": false, - "issues": [ - "OPENAI_API_KEY", - "SUPABASE_URL", - "SUPABASE_KEY" - ] - }, - "gradio_interface": { - "passed": false, - "issues": [ - "Error checking app.py: 'charmap' codec can't decode byte 0x9d in position 8062: character maps to " - ] - }, - "agent_configuration": { - "passed": true, - "issues": [] - }, - "requirements": { - "passed": true, - "issues": [] - } - }, - "recommendations": [ - "Set the required environment variable: OPENAI_API_KEY", - "Set the required environment variable: SUPABASE_URL", - "Set the required environment variable: SUPABASE_KEY", - "Fix Gradio interface issue: Error checking app.py: 'charmap' codec can't decode byte 0x9d in position 8062: character maps to " - ] -} \ No newline at end of file diff --git a/deployment_readiness_report.md b/deployment_readiness_report.md deleted file mode 100644 index 6c39fd1f046be54c9d7c48456ce0af1e977d2878..0000000000000000000000000000000000000000 --- a/deployment_readiness_report.md +++ /dev/null @@ -1,97 +0,0 @@ -# GAIA Deployment Readiness Report - -## 1. Deployment Status: ✅ SUCCESSFUL - -The GAIA agent system has been successfully deployed to Hugging Face Space at: -`https://huggingface.co/spaces/JoachimVC/Final_Assignment_Template` - -**Deployment Timestamp:** April 30, 2025 - -## 2. Environment Configuration - -| Category | Status | Details | -|----------|--------|---------| -| Code Files | ✅ Complete | All necessary code files have been uploaded | -| Environment Variables | ✅ Configured | All required API keys and configuration variables set | -| Secret Management | ✅ Secure | Using Hugging Face Secrets for sensitive credentials | - -### Environment Variables Configured: -- `OPENAI_API_KEY` ✅ -- `SERPER_API_KEY` ✅ -- `PERPLEXITY_API_KEY` ✅ -- `SUPABASE_URL` ✅ -- `SUPABASE_KEY` ✅ -- `HF_TOKEN` ✅ - -## 3. Component Integration Status - -| Component | Status | Verification | -|-----------|--------|-------------| -| OpenAI API | ✅ Working | Successfully connected to API and retrieved model list | -| Serper Search | ✅ Working | Successfully performed search query with expected results | -| Supabase Memory | ⚠️ Needs Verification | Environment variables set, but actual integration needs testing | -| DuckDuckGo Search | ⚠️ Needs Verification | Functionality available but needs end-to-end testing | -| Perplexity | ⚠️ Needs Verification | API key configured, but integration needs testing | -| Agent Graph Workflow | ⚠️ Needs Verification | Core functionality needs verification in the deployed environment | - -## 4. Cleanup Recommendations - -Several files and directories in the deployed space are not required for runtime operation and can be safely removed to optimize the environment. Refer to `huggingface_space_cleanup.md` for a detailed list of files that can be removed, including: - -- Test directories (`tests/`, `backup_tests/`) -- Development configuration (`.roo/`, `.roomodes/`, etc.) -- Deployment and cleaning scripts -- Non-essential documentation - -## 5. Verification Steps - -The following steps are recommended to complete verification of the deployed system: - -1. **Run Production Verification Script** - - Execute `verify_production.py` on the Hugging Face Space to test all components - - This script will generate a detailed report on component status - -2. **Test Conversation Memory** - - Initiate a multi-turn conversation that relies on memory - - Verify that the agent can recall information from previous interactions - -3. **Test Search Tool Integration** - - Ask a question requiring web search - - Verify that search results are properly integrated into responses - -4. **Performance Monitoring** - - Monitor response times for different query types - - Check for any memory leaks or increasing latency over time - -## 6. Recommendations - -1. **Memory Persistence** - - Verify Supabase table structure matches the required schema - - Check that memory persistence works across session restarts - -2. **Error Handling** - - Implement robust error handling for API failures - - Add automatic fallback to alternative models if primary model is unavailable - -3. **Monitoring** - - Set up periodic health checks to verify all integrations remain functional - - Implement usage tracking to monitor API consumption and costs - -4. **Documentation** - - Update user-facing documentation with examples of supported capabilities - - Provide clear guidance on limitations and expected behaviors - -## 7. Rollback Plan - -In case issues are discovered in the deployed system: - -1. Identify the specific component causing the issue -2. If possible, fix directly in the Hugging Face interface -3. For more complex issues, fix locally and redeploy -4. If critical failure occurs, consider reverting to previous stable version - -## Conclusion - -The GAIA system has been successfully deployed to Hugging Face Space with all required environment variables configured. Initial component testing shows successful integration with OpenAI and Serper APIs. Comprehensive testing of the Supabase memory integration and other components is recommended to ensure full functionality in the production environment. - -The deployment is currently in a good state for final verification and production use, with minor cleanup recommended to optimize the space. \ No newline at end of file diff --git a/deployment_verification.py b/deployment_verification.py deleted file mode 100644 index 9ff9b0cbd75c1181f0c346f7d7855805896a40e5..0000000000000000000000000000000000000000 --- a/deployment_verification.py +++ /dev/null @@ -1,327 +0,0 @@ -""" -Deployment verification script for GAIA assessment -Checks key components without OAuth requirements -""" -import os -import sys -import logging -import time -from typing import Dict, List, Any, Optional -import traceback - -# Configure logging -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("deployment_verification") - -class DeploymentVerifier: - """Verifies deployment readiness of the GAIA system""" - - def __init__(self): - self.results = {} - self.all_tests_passed = True - - def verify_environment_variables(self): - """Check required environment variables""" - logger.info("Checking environment variables...") - required_vars = ["OPENAI_API_KEY", "SUPABASE_URL", "SUPABASE_KEY"] - optional_vars = ["SERPER_API_KEY", "SPACE_ID", "SPACE_HOST"] - - missing = [] - for var in required_vars: - if not os.environ.get(var): - missing.append(var) - self.all_tests_passed = False - - if missing: - logger.error(f"❌ Missing required environment variables: {', '.join(missing)}") - self.results["environment_variables"] = False - else: - logger.info("✅ All required environment variables are set") - self.results["environment_variables"] = True - - # Check optional variables - for var in optional_vars: - if os.environ.get(var): - logger.info(f"✅ Optional variable {var} is set") - else: - logger.warning(f"⚠️ Optional variable {var} is not set") - - return self.results["environment_variables"] - - def verify_oauth_configuration(self): - """Check OAuth configuration for Hugging Face Space""" - logger.info("Checking OAuth configuration...") - - # For local testing, this will always be skipped/failed - space_host = os.environ.get("SPACE_HOST") - if not space_host: - logger.warning("⚠️ Not running in Hugging Face Space environment. OAuth check skipped.") - self.results["oauth_configuration"] = None - return None - - # Check if OAUTH_CLIENT_ID is set when in a Space - oauth_client_id = os.environ.get("OAUTH_CLIENT_ID") - if oauth_client_id: - logger.info("✅ OAUTH_CLIENT_ID is set") - self.results["oauth_configuration"] = True - else: - logger.error("❌ OAUTH_CLIENT_ID is not set") - logger.error("For deployment, ensure 'hf_oauth: true' is set in the Space metadata") - self.results["oauth_configuration"] = False - self.all_tests_passed = False - - return self.results["oauth_configuration"] - - def verify_agent_initialization(self): - """Verify that the GAIA agent can be initialized""" - logger.info("Verifying GAIA agent initialization...") - try: - from agent import GAIAAgent - - # Record start time - start_time = time.time() - - # Initialize the agent - agent = GAIAAgent() - - # Calculate initialization time - init_time = time.time() - start_time - - logger.info(f"✅ GAIA Agent initialized successfully in {init_time:.2f} seconds") - self.results["agent_initialization"] = True - return True - except Exception as e: - logger.error(f"❌ Failed to initialize GAIA agent: {e}") - logger.error(traceback.format_exc()) - self.results["agent_initialization"] = False - self.all_tests_passed = False - return False - - def verify_memory_system(self): - """Verify that the memory system works correctly""" - logger.info("Verifying memory system...") - try: - from memory.supabase_memory import SupabaseMemory - from agent.config import get_memory_config - - # Get memory configuration - memory_config = get_memory_config() - memory_config["enabled"] = True - - # Initialize memory - memory = SupabaseMemory(memory_config) - - # Test basic operations - test_key = f"test_key_{int(time.time())}" - test_value = {"message": "Test value", "timestamp": time.time()} - - # Store - memory.store(test_key, test_value) - logger.info(f"✅ Successfully stored test value with key: {test_key}") - - # Retrieve - retrieved = memory.retrieve(test_key) - - if retrieved and retrieved.get("message") == "Test value": - logger.info("✅ Successfully retrieved test value from memory") - self.results["memory_system"] = True - return True - else: - logger.error(f"❌ Retrieved value doesn't match expected. Got: {retrieved}") - self.results["memory_system"] = False - self.all_tests_passed = False - return False - except Exception as e: - logger.error(f"❌ Failed to verify memory system: {e}") - logger.error(traceback.format_exc()) - self.results["memory_system"] = False - self.all_tests_passed = False - return False - - def verify_tool_integrations(self): - """Verify that required tools are available and working""" - logger.info("Verifying tool integrations...") - tool_results = {} - - # Check DuckDuckGo - try: - from tools.web_tools import duckduckgo_search - - # Test with a simple query - test_query = "What is the capital of France?" - results = duckduckgo_search(test_query, num_results=1) - - if results and len(results) > 0: - logger.info("✅ DuckDuckGo search is working") - tool_results["duckduckgo"] = True - else: - logger.warning("⚠️ DuckDuckGo search returned no results") - tool_results["duckduckgo"] = False - self.all_tests_passed = False - except Exception as e: - logger.error(f"❌ DuckDuckGo search integration failed: {e}") - tool_results["duckduckgo"] = False - self.all_tests_passed = False - - # Check Serper if API key is available - serper_api_key = os.environ.get("SERPER_API_KEY") - if serper_api_key: - try: - from tools.web_tools import serper_search - - # Test with a simple query - test_query = "What is the capital of France?" - results = serper_search(test_query, num_results=1) - - if results and len(results) > 0: - logger.info("✅ Serper search is working") - tool_results["serper"] = True - else: - logger.warning("⚠️ Serper search returned no results") - tool_results["serper"] = False - except Exception as e: - logger.error(f"❌ Serper search integration failed: {e}") - tool_results["serper"] = False - # Don't fail the overall test for optional Serper - else: - logger.warning("⚠️ SERPER_API_KEY not set, skipping Serper verification") - tool_results["serper"] = None - - self.results["tool_integrations"] = tool_results - return any(result for result in tool_results.values() if result is True) - - def verify_agent_graph(self): - """Verify that the agent graph can be created""" - logger.info("Verifying agent graph...") - try: - from agent.graph import create_agent_graph - - # Create the agent graph - graph = create_agent_graph() - - if graph: - logger.info("✅ Agent graph created successfully") - self.results["agent_graph"] = True - return True - else: - logger.error("❌ Agent graph creation failed") - self.results["agent_graph"] = False - self.all_tests_passed = False - return False - except Exception as e: - logger.error(f"❌ Failed to verify agent graph: {e}") - logger.error(traceback.format_exc()) - self.results["agent_graph"] = False - self.all_tests_passed = False - return False - - def verify_gradio_app(self): - """Verify Gradio app can be initialized (without OAuth)""" - logger.info("Verifying Gradio app initialization...") - try: - import gradio as gr - - # Create a simple test app - with gr.Blocks() as demo: - gr.Markdown("# Test App") - gr.Button("Test Button") - - logger.info("✅ Gradio app initialization successful") - self.results["gradio_app"] = True - return True - except Exception as e: - logger.error(f"❌ Failed to initialize Gradio app: {e}") - logger.error(traceback.format_exc()) - self.results["gradio_app"] = False - self.all_tests_passed = False - return False - - def generate_deployment_recommendation(self): - """Generate deployment recommendation based on test results""" - logger.info("\n=== GENERATING DEPLOYMENT RECOMMENDATION ===") - - if self.all_tests_passed: - logger.info("✅ All critical tests passed!") - recommendation = """ - DEPLOYMENT RECOMMENDATION: READY FOR DEPLOYMENT - - Your system has passed all critical verification tests and is ready for deployment. - - For Hugging Face Space deployment, remember to: - 1. Set 'hf_oauth: true' in the Space metadata - 2. Ensure all environment variables are properly configured in the Space settings - """ - else: - logger.error("❌ Some tests failed!") - failed_tests = [test for test, result in self.results.items() - if result is False or (isinstance(result, dict) and not any(r for r in result.values()))] - - recommendation = f""" - DEPLOYMENT RECOMMENDATION: NOT READY FOR DEPLOYMENT - - The following issues need to be addressed before deployment: - {', '.join(failed_tests)} - - Please fix these issues and run the verification again. - """ - - logger.info(recommendation) - return recommendation - - def run_all_tests(self): - """Run all verification tests""" - logger.info("=== STARTING DEPLOYMENT VERIFICATION ===") - - # 1. Verify environment variables - self.verify_environment_variables() - - # 2. Verify OAuth configuration - self.verify_oauth_configuration() - - # 3. Verify agent initialization - self.verify_agent_initialization() - - # 4. Verify memory system - self.verify_memory_system() - - # 5. Verify tool integrations - self.verify_tool_integrations() - - # 6. Verify agent graph - self.verify_agent_graph() - - # 7. Verify Gradio app - self.verify_gradio_app() - - # Generate deployment recommendation - recommendation = self.generate_deployment_recommendation() - - # Print summary - logger.info("\n=== DEPLOYMENT VERIFICATION RESULTS ===") - for component, result in self.results.items(): - if isinstance(result, dict): - logger.info(f"{component}:") - for subcomponent, subresult in result.items(): - status = "✅ PASS" if subresult else "❌ FAIL" if subresult is False else "⚠️ SKIPPED" - logger.info(f" {subcomponent}: {status}") - else: - status = "✅ PASS" if result else "❌ FAIL" if result is False else "⚠️ SKIPPED" - logger.info(f"{component}: {status}") - - logger.info("\n" + recommendation) - - return self.all_tests_passed - -if __name__ == "__main__": - verifier = DeploymentVerifier() - success = verifier.run_all_tests() - - if success: - logger.info("\n✅ System is ready for deployment!") - sys.exit(0) - else: - logger.error("\n❌ System is NOT ready for deployment!") - sys.exit(1) \ No newline at end of file diff --git a/docs/all_apis_integration_testing.md b/docs/all_apis_integration_testing.md deleted file mode 100644 index b17d80da84d83d57004ecfad8f7dfcdc2ea027bd..0000000000000000000000000000000000000000 --- a/docs/all_apis_integration_testing.md +++ /dev/null @@ -1,170 +0,0 @@ -# All APIs Integration Testing - -This document provides information about the comprehensive integration test for all APIs in the GAIA agent. - -## Overview - -The integration test verifies that all APIs work together seamlessly in the GAIA agent. It tests: - -1. **OpenAI** - For language model capabilities -2. **Perplexity** - For enhanced search and reasoning -3. **Serper** - For Google search results -4. **DuckDuckGo** - For web search without API key requirements -5. **arXiv** - For academic and scientific paper search - -The test ensures that the agent can switch between different tools based on the task and maintain memory across different tool executions. - -## Prerequisites - -Before running the integration test, ensure you have the following environment variables set: - -- `OPENAI_API_KEY` (required) - Your OpenAI API key -- `SERPER_API_KEY` (optional) - Your Serper API key for Google search -- `PERPLEXITY_API_KEY` (optional) - Your Perplexity API key - -If optional API keys are not provided, the corresponding tests will be skipped. - -## Running the Test - -You can run the integration test using the provided runner script: - -```bash -python tests/run_all_apis_integration_test.py -``` - -### Command-line Options - -- `--verbose` or `-v`: Enable verbose logging for detailed output -- `--report-file` or `-r`: Specify a custom output file for the test report (default: `gaia_all_apis_integration_report.json`) - -Example with options: - -```bash -python tests/run_all_apis_integration_test.py --verbose --report-file my_report.json -``` - -## Test Components - -The integration test consists of several components: - -### 1. API Connection Tests - -Verifies that the agent can connect to each API service. For each API, it: -- Checks if the required API key is available -- Attempts to establish a connection -- Verifies that the connection is successful - -### 2. Individual Tool Execution Tests - -Tests each tool individually with simple queries to ensure they work correctly: -- DuckDuckGo search -- Serper search -- Perplexity search -- arXiv search -- Robust search (fallback mechanism) -- Merged search (combining results from multiple providers) - -### 3. Tool Switching Tests - -Verifies that the agent can switch between different tools based on the task: -- Scientific questions (should use arXiv and Perplexity) -- Factual questions (should use DuckDuckGo and Serper) -- Explanatory questions (should use Perplexity) - -### 4. Memory Integration Tests - -Tests the agent's ability to maintain memory across different tool executions: -- Processes a sequence of related questions -- Verifies that the conversation history is maintained -- Checks for cached results and intermediate results - -## Test Report - -After running the test, a comprehensive report is generated in JSON format. The report includes: - -- API connection results -- Tool execution results -- Complex query results -- Memory integration results -- Summary statistics - -The report is displayed in the console and saved to the specified file. - -## Interpreting Results - -The test results are displayed with the following indicators: - -- ✅ **PASS**: The test was successful -- ❌ **FAIL**: The test failed -- ⚠️ **SKIP**: The test was skipped (usually due to missing API keys) - -### Example Output - -``` -================================================================================ - GAIA ALL APIs INTEGRATION TEST REPORT -================================================================================ - --------------------------------------------------------------------------------- - SUMMARY --------------------------------------------------------------------------------- -Timestamp: 2025-04-30 18:00:00 -Total APIs Tested: 5 -Successful Connections: 4 -Successful Tool Executions: 6 -Successful Complex Queries: 3 -Memory Integration Success: True - --------------------------------------------------------------------------------- - API CONNECTIONS --------------------------------------------------------------------------------- -✅ PASS | openai: Successfully connected to OpenAI API -✅ PASS | perplexity: Successfully connected to Perplexity API -⚠️ SKIP | serper: Serper API key not available -✅ PASS | duckduckgo: Successfully connected to DuckDuckGo -✅ PASS | arxiv: Successfully connected to arXiv - -... -``` - -## Troubleshooting - -### Missing API Keys - -If you see warnings about missing API keys, ensure that the environment variables are properly set: - -```bash -export OPENAI_API_KEY=your_openai_api_key -export SERPER_API_KEY=your_serper_api_key -export PERPLEXITY_API_KEY=your_perplexity_api_key -``` - -### Connection Issues - -If API connection tests fail: -1. Check your internet connection -2. Verify that your API keys are valid and not expired -3. Check if the API service is experiencing downtime - -### Test Failures - -If specific tests fail: -1. Check the error message in the report -2. Look for detailed error logs if running with `--verbose` -3. Verify that the required dependencies are installed -4. Check if there are any rate limits or quotas on the APIs - -## Extending the Test - -To add more test cases or test additional APIs: - -1. Add new test methods to the `TestGAIAAllAPIsIntegration` class in `tests/test_gaia_all_apis_integration.py` -2. Update the report generation to include the new test results -3. Update this documentation to reflect the changes - -## Related Documentation - -- [OpenAI Integration Testing](openai_integration_testing.md) -- [Serper Integration](serper_integration.md) -- [DuckDuckGo Integration](duckduckgo_integration.md) -- [Multi-Search Integration](multi_search_integration.md) \ No newline at end of file diff --git a/docs/configuration.md b/docs/configuration.md deleted file mode 100644 index a83e9a0a375bb5b332f26d0a0e0a93ea43169b8c..0000000000000000000000000000000000000000 --- a/docs/configuration.md +++ /dev/null @@ -1,357 +0,0 @@ -# GAIA Agent Configuration Guide - -This document provides detailed information about the configuration options available for the GAIA agent. Understanding these options will help you customize the agent's behavior to suit your specific needs. - -## Configuration Overview - -The GAIA agent uses a hierarchical configuration structure with the following main sections: - -1. **Model Configuration**: Settings related to the language models used by the agent -2. **Tools Configuration**: Settings for the various tools used by the agent -3. **Memory Configuration**: Settings for the agent's memory components -4. **Agent Configuration**: General settings for the agent's behavior - -## Configuration Methods - -There are several ways to configure the GAIA agent: - -1. **Environment Variables**: Set configuration values using environment variables -2. **Configuration Files**: Use a `.env` file or JSON configuration files -3. **Direct Configuration**: Pass a configuration dictionary when initializing the agent - -## Model Configuration - -The model configuration controls which language models are used by the agent and how they behave. - -```python -model_config = { - "reasoning_model": "gpt-4o", # Model used for reasoning tasks (updated from gpt-4-turbo) - "tool_model": "gpt-3.5-turbo", # Model used for tool execution - "vision_model": "gpt-4o", # Model used for image analysis (updated from gpt-4-vision-preview) - "embedding_model": "text-embedding-3-large", # Model used for embeddings (updated from text-embedding-ada-002) - "temperature": 0.1, # Controls randomness (0.0 to 1.0) - "max_tokens": 4096, # Maximum number of tokens to generate - "top_p": 0.95, # Controls diversity via nucleus sampling - "frequency_penalty": 0.0, # Penalizes repeated tokens - "presence_penalty": 0.0 # Penalizes repeated topics -} -``` - -### Environment Variables - -``` -OPENAI_API_KEY=your-api-key -REASONING_MODEL=gpt-4o -TOOL_MODEL=gpt-3.5-turbo -VISION_MODEL=gpt-4o -EMBEDDING_MODEL=text-embedding-3-large -MODEL_TEMPERATURE=0.1 -MODEL_MAX_TOKENS=4096 -``` - -## Tools Configuration - -The tools configuration controls which tools are available to the agent and how they behave. - -```python -tools_config = { - "web_search": { - "enabled": True, - "providers": ["duckduckgo", "serper"], - "result_count": 5, - "timeout": 10 - }, - "web_content": { - "enabled": True, - "max_content_length": 10000, - "timeout": 15 - }, - "reasoning": { - "enabled": True, - "step_by_step": True, - "verification": True - }, - "multimodal": { - "enabled": True, - "image_analysis": True, - "chart_interpretation": True, - "document_parsing": True - }, - "arxiv": { - "enabled": True, - "max_results": 5, - "timeout": 10 - }, - "perplexity": { - "enabled": True, - "model": "sonar-medium-online", - "timeout": 15 - } -} -``` - -### Environment Variables - -``` -ENABLE_WEB_SEARCH=true -SEARCH_PROVIDERS=duckduckgo,serper -SEARCH_RESULT_COUNT=5 -SEARCH_TIMEOUT=10 -ENABLE_WEB_CONTENT=true -MAX_CONTENT_LENGTH=10000 -ENABLE_REASONING=true -ENABLE_MULTIMODAL=true -ENABLE_ARXIV=true -ARXIV_MAX_RESULTS=5 -ENABLE_PERPLEXITY=true -PERPLEXITY_MODEL=sonar-medium-online -``` - -## Memory Configuration - -The memory configuration controls how the agent stores and retrieves information. - -```python -memory_config = { - "supabase": { - "url": "https://tjamxhvvtnypbadvrkjq.supabase.co", - "key": "your-api-key", - "project_id": "tjamxhvvtnypbadvrkjq", - "project_name": "HF_GAIA_Assessment_AGT", - "table": "gaia_memory" - }, - "conversation_memory": { - "enabled": True, - "max_messages": 100, - "table": "gaia_memory_conversations" - }, - "result_cache": { - "enabled": True, - "ttl": 3600, # Time-to-live in seconds - "table": "gaia_memory_cache" - }, - "working_memory": { - "enabled": True, - "max_items": 50, - "table": "gaia_memory_working" - } -} -``` - -### Environment Variables - -``` -# Supabase Configuration -SUPABASE_URL=https://tjamxhvvtnypbadvrkjq.supabase.co -SUPABASE_KEY=your-api-key -SUPABASE_PROJECT_ID=tjamxhvvtnypbadvrkjq -SUPABASE_PROJECT_NAME=HF_GAIA_Assessment_AGT -MEMORY_TABLE_NAME=gaia_memory - -# Memory Settings -MEMORY_ENABLED=true -MEMORY_TTL=86400 -MEMORY_CACHE_SIZE=100 -MEMORY_SIMILARITY_THRESHOLD=0.85 -MEMORY_MAX_ENTRIES=1000 -``` - -## Agent Configuration - -The agent configuration controls the general behavior of the agent. - -```python -agent_config = { - "max_retries": 3, # Maximum number of retries for failed operations - "retry_delay": 1.0, # Delay between retries in seconds - "timeout": 300, # Overall timeout for processing a question - "max_iterations": 10, # Maximum number of iterations in the workflow - "return_error_message": False, # Whether to return error messages or raise exceptions - "verbose": False # Whether to enable verbose logging -} -``` - -### Environment Variables - -``` -AGENT_MAX_RETRIES=3 -AGENT_RETRY_DELAY=1.0 -AGENT_TIMEOUT=300 -AGENT_MAX_ITERATIONS=10 -RETURN_ERROR_MESSAGE=false -VERBOSE=false -``` - -## Complete Configuration Example - -Here's a complete example of a configuration dictionary that you can pass when initializing the agent: - -```python -config = { - "model": { - "reasoning_model": "gpt-4o", - "tool_model": "gpt-3.5-turbo", - "vision_model": "gpt-4o", - "embedding_model": "text-embedding-3-large", - "temperature": 0.1, - "max_tokens": 4096 - }, - "tools": { - "web_search": { - "enabled": True, - "providers": ["duckduckgo", "serper"], - "result_count": 5, - "timeout": 10 - }, - "web_content": { - "enabled": True, - "max_content_length": 10000, - "timeout": 15 - }, - "reasoning": { - "enabled": True, - "step_by_step": True, - "verification": True - }, - "multimodal": { - "enabled": True, - "image_analysis": True, - "chart_interpretation": True, - "document_parsing": True - }, - "arxiv": { - "enabled": True, - "max_results": 5, - "timeout": 10 - }, - "perplexity": { - "enabled": True, - "model": "sonar-medium-online", - "timeout": 15 - } - }, - "memory": { - "supabase": { - "url": "https://tjamxhvvtnypbadvrkjq.supabase.co", - "key": "your-api-key", - "project_id": "tjamxhvvtnypbadvrkjq", - "project_name": "HF_GAIA_Assessment_AGT", - "table": "gaia_memory" - }, - "conversation_memory": { - "enabled": True, - "max_messages": 100, - "table": "gaia_memory_conversations" - }, - "result_cache": { - "enabled": True, - "ttl": 3600, - "table": "gaia_memory_cache" - }, - "working_memory": { - "enabled": True, - "max_items": 50, - "table": "gaia_memory_working" - } - }, - "agent": { - "max_retries": 3, - "retry_delay": 1.0, - "timeout": 300, - "max_iterations": 10, - "return_error_message": False, - "verbose": False - } -} - -from agent import GAIAAgent -agent = GAIAAgent(config) -``` - -## Environment Variables File (.env) - -You can also use a `.env` file to configure the agent. Here's an example: - -``` -# API Keys and Endpoints -OPENAI_API_KEY=your-openai-api-key -SERPER_API_KEY=your-serper-api-key -PERPLEXITY_API_KEY=your-perplexity-api-key - -# Supabase Configuration -SUPABASE_URL=https://tjamxhvvtnypbadvrkjq.supabase.co -SUPABASE_KEY=your-supabase-api-key -SUPABASE_PROJECT_ID=tjamxhvvtnypbadvrkjq -SUPABASE_PROJECT_NAME=HF_GAIA_Assessment_AGT - -# Hugging Face Configuration -HF_TOKEN=your-huggingface-token - -# Model Configuration -REASONING_MODEL=gpt-4o -TOOL_MODEL=gpt-3.5-turbo -VISION_MODEL=gpt-4o -EMBEDDING_MODEL=text-embedding-3-large -MODEL_TEMPERATURE=0.1 -MODEL_MAX_TOKENS=4096 - -# Tools Configuration -ENABLE_WEB_SEARCH=true -SEARCH_PROVIDERS=duckduckgo,serper -SEARCH_RESULT_COUNT=5 -ENABLE_WEB_CONTENT=true -ENABLE_REASONING=true -ENABLE_MULTIMODAL=true -ENABLE_ARXIV=true -ENABLE_PERPLEXITY=true - -# Memory Configuration -ENABLE_CONVERSATION_MEMORY=true -ENABLE_RESULT_CACHE=true -ENABLE_WORKING_MEMORY=true - -# Agent Configuration -AGENT_MAX_RETRIES=3 -AGENT_TIMEOUT=300 -VERBOSE=false -``` - -## Dependencies - -The GAIA agent relies on several key dependencies: - -1. **langchain**: Core framework for building language model applications -2. **langchain_openai**: OpenAI integration for LangChain (separate package) -3. **langgraph**: Framework for building complex agent workflows -4. **supabase**: Python client for Supabase -5. **duckduckgo-search**: Client for DuckDuckGo search -6. **perplexity-python**: Client for Perplexity AI -7. **arxiv**: Client for arXiv search - -Make sure these dependencies are installed and up-to-date: - -```bash -pip install --upgrade langchain langchain_openai langgraph supabase duckduckgo-search perplexity-python arxiv -``` - -> **Note**: The `langchain_openai` package is now separate from the core `langchain` package. If you encounter import errors, make sure you have the latest version installed. - -## Configuration Precedence - -The agent uses the following precedence order when determining configuration values: - -1. Direct configuration passed to the constructor -2. Environment variables -3. Default values defined in the code - -This means that values passed directly to the constructor will override environment variables, which in turn override the default values. - -## Troubleshooting - -If you encounter configuration issues, check the [Troubleshooting Guide](troubleshooting.md) for common problems and solutions. - -## Next Steps - -For usage examples, see [Usage Examples](examples.md). - -For deployment instructions, see [Deployment Guide](deployment.md). \ No newline at end of file diff --git a/docs/connection_verification.md b/docs/connection_verification.md deleted file mode 100644 index 6e9943cd65f66fabf81d4dbc9a40abb1ed8d2463..0000000000000000000000000000000000000000 --- a/docs/connection_verification.md +++ /dev/null @@ -1,114 +0,0 @@ -# Connection Verification Guide - -This guide explains how to verify connections to external services used by the GAIA agent. - -## Overview - -The GAIA agent relies on several external services: - -1. **OpenAI API** - For language model access -2. **Perplexity API** - For web search and reasoning -3. **Supabase** - For memory storage -4. **Serper API** - For web search - -Proper connection to these services is essential for the agent to function correctly. - -## Verification Tools - -The project includes several tools to verify connections: - -1. `tests/verify_connections.py` - A comprehensive test script that verifies all connections -2. `direct_config_check.py` - A simple script to verify configuration values - -## Verifying All Connections - -To verify all connections at once, run: - -```bash -python -m tests.verify_connections -``` - -This script will: -- Check if all required API keys are set -- Test connections to each service -- Report success or failure for each connection - -## Verifying Individual Connections - -### OpenAI API - -To verify the OpenAI API connection: - -```bash -python -m tests.verify_connections --service openai -``` - -### Perplexity API - -To verify the Perplexity API connection: - -```bash -python -m tests.verify_connections --service perplexity -``` - -You can also use the dedicated Perplexity test: - -```bash -python -m tests.test_perplexity_sonar -``` - -### Supabase - -To verify the Supabase connection: - -```bash -python -m tests.verify_connections --service supabase -``` - -### Serper API - -To verify the Serper API connection: - -```bash -python -m tests.verify_connections --service serper -``` - -## Troubleshooting Connection Issues - -### API Key Issues - -If you encounter API key issues: - -1. Check that the key is correctly set in your `.env` file -2. Verify the key format (no extra spaces or characters) -3. Ensure the key has not expired -4. Check if the service has usage limits that you've exceeded - -### Network Issues - -If you encounter network issues: - -1. Check your internet connection -2. Verify that your firewall or network settings allow connections to the service -3. Check if the service is experiencing downtime - -### Configuration Issues - -If you encounter configuration issues: - -1. Run `python direct_config_check.py` to verify configuration values -2. Check that the configuration in `agent/config.py` is correct -3. Ensure environment variables are properly loaded - -## Setting Up API Keys - -All API keys should be stored in a `.env` file in the project root: - -``` -OPENAI_API_KEY=your_openai_key -PERPLEXITY_API_KEY=your_perplexity_key -SERPER_API_KEY=your_serper_key -SUPABASE_KEY=your_supabase_key -``` - -Never commit this file to version control. \ No newline at end of file diff --git a/docs/deployment.md b/docs/deployment.md deleted file mode 100644 index 2edfd1f7a3483e2c2ce43c8566723fd9342966cd..0000000000000000000000000000000000000000 --- a/docs/deployment.md +++ /dev/null @@ -1,303 +0,0 @@ -# GAIA Agent Deployment Guide - -This document provides instructions for deploying the GAIA agent in various environments, from local development to production settings. - -## Prerequisites - -Before deploying the GAIA agent, ensure you have: - -1. Python 3.8 or higher installed -2. All required dependencies installed -3. API keys for any external services (OpenAI, Serper, Supabase, etc.) -4. Sufficient resources for your expected workload - -## Local Deployment - -### Installation - -1. Clone the repository: - ```bash - git clone https://github.com/yourusername/gaia-agent.git - cd gaia-agent - ``` - -2. Create a virtual environment: - ```bash - python -m venv venv - source venv/bin/activate # On Windows: venv\Scripts\activate - ``` - -3. Install dependencies: - ```bash - pip install -r requirements.txt - ``` - -4. Create a `.env` file with your configuration: - ``` - OPENAI_API_KEY=your-openai-api-key - SERPER_API_KEY=your-serper-api-key - SUPABASE_URL=https://your-project-url.supabase.co - SUPABASE_KEY=your-supabase-api-key - ``` - -5. Run the application: - ```bash - python app.py - ``` - -### Testing - -Before deploying to production, run the test suite to ensure everything is working correctly: - -```bash -# Run all tests -python -m unittest discover tests - -# Run benchmark tests -python -m tests.benchmark --output benchmark_results.json -``` - -## Deployment Options - -### Hugging Face Spaces - -The GAIA agent is designed to be easily deployed to Hugging Face Spaces: - -1. Create a new Space on Hugging Face: - - Go to https://huggingface.co/spaces - - Click "Create new Space" - - Select "Gradio" as the SDK - - Choose a name for your Space - -2. Configure your Space: - - Add your API keys as secrets in the Space settings - - Set the Python version to 3.8 or higher - -3. Push your code to the Space: - ```bash - git remote add space https://huggingface.co/spaces/yourusername/your-space-name - git push space main - ``` - -4. Your agent will be automatically deployed and available at `https://huggingface.co/spaces/yourusername/your-space-name` - -### Docker Deployment - -For containerized deployment, you can use Docker: - -1. Create a Dockerfile: - ```dockerfile - FROM python:3.9-slim - - WORKDIR /app - - COPY requirements.txt . - RUN pip install --no-cache-dir -r requirements.txt - - COPY . . - - CMD ["python", "app.py"] - ``` - -2. Build the Docker image: - ```bash - docker build -t gaia-agent . - ``` - -3. Run the container: - ```bash - docker run -p 7860:7860 --env-file .env gaia-agent - ``` - -### Cloud Deployment - -#### AWS Elastic Beanstalk - -1. Install the EB CLI: - ```bash - pip install awsebcli - ``` - -2. Initialize your EB application: - ```bash - eb init -p python-3.8 gaia-agent - ``` - -3. Create an environment: - ```bash - eb create gaia-agent-env - ``` - -4. Configure environment variables: - ```bash - eb setenv OPENAI_API_KEY=your-openai-api-key SERPER_API_KEY=your-serper-api-key - ``` - -5. Deploy your application: - ```bash - eb deploy - ``` - -#### Google Cloud Run - -1. Build and push your Docker image to Google Container Registry: - ```bash - gcloud builds submit --tag gcr.io/your-project-id/gaia-agent - ``` - -2. Deploy to Cloud Run: - ```bash - gcloud run deploy gaia-agent \ - --image gcr.io/your-project-id/gaia-agent \ - --platform managed \ - --set-env-vars OPENAI_API_KEY=your-openai-api-key,SERPER_API_KEY=your-serper-api-key - ``` - -## Scaling Considerations - -### Memory Usage - -The GAIA agent can be memory-intensive, especially when processing complex questions or using large language models. Consider the following: - -- Monitor memory usage during benchmark testing -- Adjust `max_tokens` and other model parameters to control memory usage -- Consider using a memory cache for frequently asked questions - -### API Rate Limits - -Be aware of rate limits for external APIs: - -- OpenAI has rate limits based on your subscription tier -- Serper and other search APIs typically have daily or monthly quotas -- Implement retry logic with exponential backoff for rate limit errors - -### Cost Management - -Using external APIs can incur costs: - -- Monitor your API usage and costs -- Consider caching results to reduce API calls -- Use smaller or less expensive models for simpler tasks - -## Production Best Practices - -### Logging - -Implement comprehensive logging: - -```python -import logging - -logging.basicConfig( - level=logging.INFO, - format='%(asctime)s - %(name)s - %(levelname)s - %(message)s', - filename='gaia_agent.log' -) -``` - -### Monitoring - -Set up monitoring for: - -- API response times -- Error rates -- Memory and CPU usage -- Cost metrics - -### Security - -Protect sensitive information: - -- Never hardcode API keys in your code -- Use environment variables or a secure secrets manager -- Implement proper authentication for your API endpoints -- Validate and sanitize all user inputs - -### Backup and Recovery - -Implement backup and recovery procedures: - -- Regularly backup your Supabase database -- Create snapshots of your deployment environment -- Document recovery procedures - -## Continuous Integration/Continuous Deployment (CI/CD) - -Set up CI/CD pipelines to automate testing and deployment: - -### GitHub Actions Example - -Create a `.github/workflows/main.yml` file: - -```yaml -name: CI/CD Pipeline - -on: - push: - branches: [ main ] - pull_request: - branches: [ main ] - -jobs: - test: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v2 - - name: Set up Python - uses: actions/setup-python@v2 - with: - python-version: '3.9' - - name: Install dependencies - run: | - python -m pip install --upgrade pip - pip install -r requirements.txt - - name: Run tests - run: | - python -m unittest discover tests - - deploy: - needs: test - if: github.event_name == 'push' && github.ref == 'refs/heads/main' - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v2 - - name: Deploy to Hugging Face Spaces - uses: huggingface/huggingface-spaces-deploy-action@main - with: - space_id: ${{ secrets.SPACE_ID }} - token: ${{ secrets.HF_TOKEN }} -``` - -## Troubleshooting - -### Common Issues - -1. **API Key Errors**: - - Ensure all API keys are correctly set in your environment variables - - Check for typos or trailing spaces in your API keys - -2. **Memory Issues**: - - Reduce the `max_tokens` parameter in your model configuration - - Implement memory cleanup after processing each question - -3. **Timeout Errors**: - - Increase the timeout values in your configuration - - Optimize your agent's workflow to reduce processing time - -4. **Rate Limit Errors**: - - Implement retry logic with exponential backoff - - Consider upgrading your API subscription tier - -### Getting Help - -If you encounter issues not covered in this guide: - -1. Check the project's GitHub issues for similar problems -2. Consult the documentation for the specific components (LangChain, LangGraph, etc.) -3. Reach out to the community for assistance - -## Next Steps - -For usage examples, see [Usage Examples](examples.md). - -For configuration options, see [Configuration Guide](configuration.md). \ No newline at end of file diff --git a/docs/duckduckgo_integration.md b/docs/duckduckgo_integration.md deleted file mode 100644 index a84b7b7b187a89a2098d8821577a6fd5fa4bc740..0000000000000000000000000000000000000000 --- a/docs/duckduckgo_integration.md +++ /dev/null @@ -1,156 +0,0 @@ -# DuckDuckGo Integration for GAIA Agent - -This document describes how to set up and use the DuckDuckGo search integration with the GAIA agent. - -## Overview - -The GAIA agent now supports DuckDuckGo as an alternative search engine for web searches. This integration allows the agent to perform web searches without requiring an API key, making it a good fallback option when other search providers are unavailable or rate-limited. - -## Features - -- **No API Key Required**: DuckDuckGo search works without requiring an API key -- **Configurable Result Count**: Control how many search results are returned -- **Error Handling**: Robust error handling for various search scenarios -- **Standardized Results**: Results are formatted consistently with other search providers - -## Installation - -The DuckDuckGo integration requires the `duckduckgo-search` package: - -```bash -pip install duckduckgo-search -``` - -## Configuration - -The DuckDuckGo search tool can be configured in the GAIA agent's configuration: - -```python -# In agent/config.py -class GaiaConfig: - def __init__(self): - # ... - self.duckduckgo_enabled = True - self.duckduckgo_result_count = 5 - # ... -``` - -### Configuration Options - -| Option | Description | Default | -|--------|-------------|---------| -| `duckduckgo_enabled` | Enable/disable DuckDuckGo search | `True` | -| `duckduckgo_result_count` | Number of results to return | `5` | -| `duckduckgo_timeout` | Timeout in seconds for search requests | `30` | - -## Usage - -### Basic Usage - -```python -from tools.web_tools import create_duckduckgo_search - -# Create a DuckDuckGo search tool -search_tool = create_duckduckgo_search() - -# Perform a search -results = search_tool.search("python programming") - -# Process the results -for result in results: - print(f"Title: {result['title']}") - print(f"URL: {result['link']}") - print(f"Snippet: {result['snippet']}") - print("---") -``` - -### Integration with GAIA Agent - -The DuckDuckGo search tool is automatically integrated with the GAIA agent if enabled in the configuration: - -```python -from agent.agent import GaiaAgent -from agent.config import GaiaConfig - -# Create a configuration with DuckDuckGo enabled -config = GaiaConfig() -config.duckduckgo_enabled = True - -# Create a GAIA agent with the configuration -agent = GaiaAgent(config=config) - -# The agent will now use DuckDuckGo for web searches -response = agent.process("What is the latest version of Python?") -``` - -## Result Format - -The DuckDuckGo search tool returns results in the following format: - -```json -[ - { - "title": "Result title", - "link": "https://example.com/result", - "snippet": "A brief excerpt from the search result..." - }, - // More results... -] -``` - -## Fallback Mechanism - -The GAIA agent can be configured to use DuckDuckGo as a fallback when other search providers fail: - -```python -# In agent/config.py -class GaiaConfig: - def __init__(self): - # ... - self.search_fallback_order = ["serper", "duckduckgo", "perplexity"] - # ... -``` - -With this configuration, the agent will try Serper first, then DuckDuckGo if Serper fails, and finally Perplexity if both fail. - -## Testing - -You can test the DuckDuckGo integration using the provided test scripts: - -```bash -# Test just the DuckDuckGo search -python tests/test_duckduckgo_search.py - -# Test all search providers together -python tests/test_gaia_multi_search_providers.py -``` - -## Troubleshooting - -### Common Issues - -1. **No Results Returned**: DuckDuckGo may return no results for certain queries. Try reformulating the query. - -2. **Connection Errors**: If you're experiencing connection errors, check your internet connection and ensure that DuckDuckGo is accessible from your network. - -3. **Rate Limiting**: While DuckDuckGo doesn't require an API key, it may still rate-limit excessive requests. Consider adding delays between searches if you're making many requests. - -### Debugging - -To enable debug logging for the DuckDuckGo search tool: - -```python -import logging -logging.basicConfig(level=logging.DEBUG) -logger = logging.getLogger("tools.web_tools") -``` - -## Limitations - -- DuckDuckGo search may not provide as many results as commercial search APIs -- Some advanced search features (like filtering by date) may not be available -- Results may be less comprehensive than those from paid search providers - -## Conclusion - -The DuckDuckGo integration provides a reliable, no-cost alternative for web searches in the GAIA agent. It's particularly useful as a fallback option or for development and testing purposes. \ No newline at end of file diff --git a/docs/enhanced_web_search.md b/docs/enhanced_web_search.md deleted file mode 100644 index 42fc8a1cbcaba613947d9b482cdf5f1b6fce1bba..0000000000000000000000000000000000000000 --- a/docs/enhanced_web_search.md +++ /dev/null @@ -1,299 +0,0 @@ -# Enhanced Web Search - -The Enhanced Web Search system provides a comprehensive three-tier approach to web search, intelligently routing queries to the most appropriate search tools based on query type and available resources. This system is designed to handle a wide range of search needs, from simple factual queries to complex reasoning questions. - -## Overview - -The Enhanced Web Search system consists of three specialized search tools: - -1. **BrowserSearchTool** - For direct browser-based searches of any website -2. **LibrarySearchTool** - For searches using imported libraries like arXiv and DuckDuckGo -3. **ApiSearchTool** - For searches using API services like Perplexity and Serper - -These tools work together in a fallback chain, ensuring that if one tool fails or is unavailable, the system can fall back to another tool. This approach provides robust search capabilities that can adapt to different environments and query types. - -## Three-Tier Search Approach - -### 1. BrowserSearchTool - -The BrowserSearchTool enables direct browser-based searches of any website using the browser_action capability. - -**Purpose:** -- Directly interact with websites through a browser interface -- Access content that may be difficult to retrieve through APIs or libraries -- Provide visual context for search results - -**Advantages:** -- Can access any website, including those without APIs -- Provides visual context and can interact with dynamic content -- Works with a wide range of websites without requiring API keys - -**Use Cases:** -- Searching websites that don't have public APIs -- Accessing content behind login screens (with proper credentials) -- Interacting with dynamic web content -- Visual verification of search results - -**Example:** - -```python -from tools.web_tools import create_browser_search - -# Create the browser search tool -browser_tool = create_browser_search() - -# Search Wikipedia -results = browser_tool.search("quantum computing", source="wikipedia") - -# The results include instructions for using browser_action -print(results[0]["instructions"]) -``` - -### 2. LibrarySearchTool - -The LibrarySearchTool leverages imported Python libraries (DuckDuckGo and arXiv) for search operations, intelligently routing queries to the appropriate library based on query type. - -**Purpose:** -- Use established Python libraries for search operations -- Provide specialized academic search capabilities -- Offer general web search without requiring API keys - -**Advantages:** -- No API keys required -- Specialized academic search through arXiv -- General web search through DuckDuckGo -- Works offline with locally installed libraries - -**Use Cases:** -- Academic research and paper searches -- General web searches when API keys aren't available -- Environments with limited internet access but installed libraries - -**Example:** - -```python -from tools.web_tools import create_library_search - -# Create the library search tool -library_tool = create_library_search() - -# For academic queries, it automatically uses arXiv -academic_results = library_tool.search("recent papers on quantum computing") - -# For general queries, it automatically uses DuckDuckGo -general_results = library_tool.search("weather in New York") -``` - -### 3. ApiSearchTool - -The ApiSearchTool uses external API services (Perplexity and Serper) for search operations, intelligently selecting between them based on the query type. - -**Purpose:** -- Leverage powerful commercial search APIs -- Provide high-quality results for complex queries -- Offer fast and reliable search capabilities - -**Advantages:** -- High-quality results for complex queries through Perplexity -- Fast and reliable search through Serper's Google search API -- Structured and consistent results - -**Use Cases:** -- Complex, reasoning-based queries that benefit from Perplexity's capabilities -- Factual and simple queries that can be handled by Serper -- Production environments with API access - -**Example:** - -```python -from tools.web_tools import create_api_search - -# Create the API search tool -api_tool = create_api_search() - -# For complex queries, it automatically uses Perplexity -complex_results = api_tool.search("Explain quantum computing in detail and its potential impact on cryptography") - -# For simple queries, it automatically uses Serper -simple_results = api_tool.search("weather in New York") -``` - -## Fallback Chain - -The three search tools work together in a fallback chain, ensuring robust search capabilities even when some tools are unavailable or fail. The EnhancedWebSearchTool manages this fallback chain. - -**Fallback Process:** - -1. First, try the most appropriate tool based on the query type -2. If that tool fails or is unavailable, try the next most appropriate tool -3. Continue until a successful result is found or all tools have been tried - -**Example:** - -```python -from tools.web_tools import create_enhanced_web_search - -# Create the enhanced web search tool -enhanced_tool = create_enhanced_web_search() - -# Search for information -# This will automatically select the most appropriate tool -# and fall back to others if needed -results = enhanced_tool.search("Who nominated the only Featured Article on English Wikipedia about a dinosaur that was promoted in November 2016?") -``` - -## Query Type Detection - -Each tool includes specialized methods to detect specific types of queries: - -- **BrowserSearchTool**: Detects website-specific queries (e.g., "wikipedia article about...") -- **LibrarySearchTool**: Detects academic queries for arXiv vs. general queries for DuckDuckGo -- **ApiSearchTool**: Detects complex queries for Perplexity vs. simple queries for Serper - -These detection methods ensure that queries are routed to the most appropriate search tool. - -## Implementation Details - -### BrowserSearchTool - -The BrowserSearchTool provides search templates for common websites: - -```python -self.search_templates = { - "wikipedia": "https://en.wikipedia.org/wiki/Special:Search?search={query}", - "arxiv": "https://arxiv.org/search/?query={query}&searchtype=all", - "google": "https://www.google.com/search?q={query}", - "youtube": "https://www.youtube.com/results?search_query={query}", - # Additional templates... -} -``` - -It can detect the most appropriate source based on the query content and provides specific instructions for using browser_action with each source. - -### LibrarySearchTool - -The LibrarySearchTool includes methods to detect academic queries: - -```python -def _is_academic_query(self, query: str) -> bool: - # Check for academic keywords, fields, and citation patterns - # ... - return has_academic_keyword or has_academic_field or has_citation_pattern -``` - -It uses the DuckDuckGo library for general queries and the arXiv library for academic queries, formatting the results consistently. - -### ApiSearchTool - -The ApiSearchTool includes methods to detect complex queries: - -```python -def _is_complex_query(self, query: str) -> bool: - # Check for question words, complex indicators, and query length - # ... - return has_question_word or has_complex_indicator or is_long_query -``` - -It uses the Perplexity API for complex queries and the Serper API for simple queries, formatting the results consistently. - -## Configuration - -Each tool can be configured through environment variables and configuration files: - -### ApiSearchTool - -- **PERPLEXITY_API_KEY**: API key for Perplexity -- **SERPER_API_KEY**: API key for Serper - -### LibrarySearchTool - -- No API keys required, but the DuckDuckGo and arXiv libraries must be installed - -### BrowserSearchTool - -- No API keys required, but browser_action capability must be available - -## Usage - -### Basic Usage - -```python -from tools.web_tools import create_enhanced_web_search - -# Create the enhanced web search tool -enhanced_tool = create_enhanced_web_search() - -# Search for information -results = enhanced_tool.search("Who nominated the only Featured Article on English Wikipedia about a dinosaur that was promoted in November 2016?") - -# Process the results -for result in results: - print(f"Title: {result.get('title')}") - print(f"Snippet: {result.get('snippet')}") - print(f"Source: {result.get('source')}") - print(f"Relevance: {result.get('relevance_score')}") - print("---") -``` - -### Using with the Tool Registry - -```python -from agent.tool_registry import create_default_registry, unified_search - -# Create the registry -registry = create_default_registry() - -# Search using unified_search -results = unified_search(registry, "How many studio albums were published by Mercedes Sosa between 2000 and 2009?") - -# Process the results -for result in results["results"]: - print(f"Title: {result.get('title')}") - print(f"Snippet: {result.get('snippet')}") - print(f"Source: {result.get('source')}") - print(f"Relevance: {result.get('relevance_score')}") - print("---") -``` - -## Testing - -You can test the Enhanced Web Search system using the provided test scripts: - -```bash -# Run all tests -python tests/run_gaia_web_search_tests.py - -# Run only direct tests (without registry) -python tests/run_gaia_web_search_tests.py --direct-only - -# Run only registry tests -python tests/run_gaia_web_search_tests.py --registry-only - -# Enable verbose logging -python tests/run_gaia_web_search_tests.py --verbose -``` - -## Examples - -See the following example scripts for complete examples of how to use the Enhanced Web Search system: - -- `examples/enhanced_web_search_example.py` - Example of using the EnhancedWebSearchTool -- `examples/multi_search_example.py` - Example of using multiple search tools together - -## Requirements - -- Perplexity API key (set as `PERPLEXITY_API_KEY` environment variable) for ApiSearchTool -- Serper API key (set as `SERPER_API_KEY` environment variable) for ApiSearchTool -- DuckDuckGo and arXiv libraries installed for LibrarySearchTool -- Browser_action capability for BrowserSearchTool - -## Conclusion - -The Enhanced Web Search system provides a robust, flexible approach to web search through its three-tier architecture. By intelligently routing queries to the most appropriate search tools and implementing a fallback chain, it ensures reliable search capabilities across a wide range of environments and query types. - -This system significantly improves search performance by leveraging the strengths of each search approach: -- Direct browser-based searches for website-specific content -- Library-based searches for academic and general queries without API keys -- API-based searches for high-quality results on complex queries - -Together, these tools provide a comprehensive solution for web search needs in various contexts and environments. \ No newline at end of file diff --git a/docs/examples.md b/docs/examples.md deleted file mode 100644 index 5c946cef44fee0f3b86b47d2b54697d387c604e2..0000000000000000000000000000000000000000 --- a/docs/examples.md +++ /dev/null @@ -1,259 +0,0 @@ -# GAIA Agent Usage Examples - -This document provides examples of how to use the GAIA agent in different scenarios. These examples demonstrate the agent's capabilities and how to integrate it into your applications. - -## Basic Usage - -### Initializing the Agent - -```python -from agent import GAIAAgent - -# Initialize with default configuration -agent = GAIAAgent() - -# Or with custom configuration -custom_config = { - "model": { - "reasoning_model": "gpt-4", - "temperature": 0.2, - "max_tokens": 1000 - }, - "tools": { - "web_search": { - "enabled": True, - "providers": ["duckduckgo", "serper"] - } - } -} -agent = GAIAAgent(custom_config) -``` - -### Asking Questions - -```python -# Ask a simple factual question -answer = agent("What is the capital of France?") -print(f"Answer: {answer}") - -# Ask a more complex question requiring reasoning -answer = agent("If a train travels at 120 km/h for 2.5 hours, how far does it travel?") -print(f"Answer: {answer}") - -# Ask a question requiring web search -answer = agent("What were the key announcements at the most recent Apple WWDC?") -print(f"Answer: {answer}") -``` - -## Advanced Usage - -### Using Memory Components - -```python -# Get a snapshot of the agent's memory -memory_snapshot = agent.get_memory_snapshot() -print(f"Conversation history: {memory_snapshot['conversation']}") -print(f"Cached results: {memory_snapshot['cached_results']}") - -# Clear the agent's memory -agent.clear_memory() -``` - -### Error Handling - -```python -try: - answer = agent("What is the meaning of life?") - print(f"Answer: {answer}") -except Exception as e: - print(f"Error: {e}") - - # You can also configure the agent to return error messages instead of raising exceptions - agent_with_error_messages = GAIAAgent({ - "agent": { - "return_error_message": True - } - }) - - answer = agent_with_error_messages("What is the meaning of life?") - if answer.startswith("Error:"): - print(f"Agent encountered an error: {answer}") - else: - print(f"Answer: {answer}") -``` - -## Integration Examples - -### Integration with Gradio - -```python -import gradio as gr -from agent import GAIAAgent - -# Initialize the agent -agent = GAIAAgent() - -# Define the Gradio interface -def process_question(question): - try: - answer = agent(question) - return answer - except Exception as e: - return f"Error: {e}" - -# Create the interface -demo = gr.Interface( - fn=process_question, - inputs=gr.Textbox(lines=2, placeholder="Enter your question here..."), - outputs=gr.Textbox(label="Answer"), - title="GAIA Agent Demo", - description="Ask a question and get an answer from the GAIA agent." -) - -# Launch the interface -demo.launch() -``` - -### Integration with FastAPI - -```python -from fastapi import FastAPI, HTTPException -from pydantic import BaseModel -from agent import GAIAAgent - -# Initialize the agent -agent = GAIAAgent() - -# Create the FastAPI app -app = FastAPI(title="GAIA Agent API") - -# Define the request model -class QuestionRequest(BaseModel): - question: str - -# Define the response model -class AnswerResponse(BaseModel): - answer: str - processing_time: float - -# Define the endpoint -@app.post("/ask", response_model=AnswerResponse) -async def ask_question(request: QuestionRequest): - try: - import time - start_time = time.time() - answer = agent(request.question) - processing_time = time.time() - start_time - return AnswerResponse(answer=answer, processing_time=processing_time) - except Exception as e: - raise HTTPException(status_code=500, detail=str(e)) - -# Run with: uvicorn api:app --reload -``` - -## Benchmark Testing - -```python -from tests.benchmark import run_benchmark, SAMPLE_QUESTIONS - -# Run the benchmark with default settings -result = run_benchmark(SAMPLE_QUESTIONS) - -# Print the summary -result.print_summary() - -# Save the results to a file -result.save_to_file("benchmark_results.json") - -# Run with custom configuration -custom_config = { - "model": { - "reasoning_model": "gpt-4", - "temperature": 0.1 - } -} -result = run_benchmark(SAMPLE_QUESTIONS, agent_config=custom_config) -``` - -## Enhanced Web Search Examples - -The GAIA agent includes enhanced web search tools specifically designed to handle certain GAIA assessment questions. The following examples demonstrate how to use these tools directly. - -### Running the Enhanced Web Search Example - -```bash -# Run all enhanced web search examples -python examples/enhanced_web_search_example.py - -# Run only the Wikipedia example -python examples/enhanced_web_search_example.py --wikipedia - -# Run only the DuckDuckGo example -python examples/enhanced_web_search_example.py --duckduckgo - -# Run only the Universe Today example -python examples/enhanced_web_search_example.py --universe-today - -# Run only the Malko Competition example -python examples/enhanced_web_search_example.py --malko -``` - -### Using the Enhanced Web Search Tools in Your Code - -```python -from tools.web_tools import WikipediaSearchTool, DuckDuckGoSearchTool, SerperSearchTool, WebContentExtractor - -# Example 1: Using the WikipediaSearchTool to find information about featured articles -wiki_tool = WikipediaSearchTool() -results = wiki_tool.search("Who nominated the only Featured Article on English Wikipedia about a dinosaur that was promoted in November 2016?") -for result in results: - print(f"Title: {result.get('title', '')}") - print(f"Snippet: {result.get('snippet', '')}") - print() - -# Example 2: Using the DuckDuckGoSearchTool to find information about music albums -ddg_tool = DuckDuckGoSearchTool() -results = ddg_tool.search("How many studio albums were published by Mercedes Sosa between 2000 and 2009 (included)?") -for result in results: - print(f"Title: {result.get('title', '')}") - print(f"Snippet: {result.get('snippet', '')}") - print() - -# Example 3: Using the WebContentExtractor to extract specific information from web pages -extractor = WebContentExtractor() -content = extractor.extract_content("https://universetoday.com/example-article") -extracted_info = content.get("extracted_info", {}) -if "nasa_award_numbers" in extracted_info: - print(f"NASA Award Numbers: {extracted_info['nasa_award_numbers']}") - -# Example 4: Using the SerperSearchTool to find information about competitions -serper_tool = SerperSearchTool() -results = serper_tool.search("Who won the Malko Competition for Young Conductors in 2021?") -for result in results: - print(f"Title: {result.get('title', '')}") - print(f"Snippet: {result.get('snippet', '')}") - print() -``` - -For more detailed information on the enhanced web search tools, see [Enhanced Web Search](enhanced_web_search.md). - -## Command-Line Usage - -You can also run the agent from the command line using the provided scripts: - -```bash -# Run the Gradio interface -python app.py - -# Run benchmark tests -python -m tests.benchmark --output benchmark_results.json - -# Run benchmark tests with custom questions -python -m tests.benchmark --questions custom_questions.json --output benchmark_results.json -``` - -## Next Steps - -For more detailed information on configuration options, see [Configuration Guide](configuration.md). - -For deployment instructions, see [Deployment Guide](deployment.md). \ No newline at end of file diff --git a/docs/gaia_memory_integration.md b/docs/gaia_memory_integration.md deleted file mode 100644 index c97d4f3ad68161c3f5778fc347a8c371a61a9efa..0000000000000000000000000000000000000000 --- a/docs/gaia_memory_integration.md +++ /dev/null @@ -1,155 +0,0 @@ -# GAIA Memory Integration with Supabase - -This document describes the integration of Supabase as a persistent memory store for the GAIA agent. The integration enables the agent to maintain conversation history, cache results, and store intermediate data across multiple sessions. - -## Overview - -The GAIA agent uses Supabase as a backend database to store various types of memory: - -1. **Conversation Memory**: Stores the history of interactions between users and the agent -2. **Result Cache**: Caches tool results and answers to avoid redundant computations -3. **Working Memory**: Stores intermediate results during agent execution - -This memory architecture allows the agent to: -- Maintain context across multiple turns in a conversation -- Remember previous interactions even after the agent is restarted -- Build upon previous knowledge in follow-up questions -- Optimize performance by caching expensive operations - -## Architecture - -The memory system consists of several components: - -### SupabaseMemory - -The core memory implementation that handles the connection to Supabase and provides basic CRUD operations: - -- `store(key, value, metadata, ttl)`: Store a value in memory -- `retrieve(key, default)`: Retrieve a value from memory -- `delete(key)`: Delete a value from memory -- `clear()`: Clear all values from memory -- `list_keys(prefix)`: List all keys in memory - -### ConversationMemory - -A specialized memory interface for storing conversation history: - -- `add_message(role, content, metadata)`: Add a message to the conversation history -- `get_messages(limit, before, after)`: Get messages from the conversation history -- `clear()`: Clear the conversation history - -### ResultCache - -A specialized memory interface for caching tool results and answers: - -- `cache_result(tool_name, input_hash, result)`: Cache a tool result -- `get_result(tool_name, input_hash)`: Get a cached result -- `invalidate(key)`: Invalidate a cached result -- `clear()`: Clear all cached results - -### WorkingMemory - -A specialized memory interface for storing intermediate results during agent execution: - -- `store_intermediate_result(key, value, metadata)`: Store an intermediate result -- `get_intermediate_result(key, default)`: Get an intermediate result -- `list_intermediate_results()`: List all intermediate results -- `clear()`: Clear all intermediate results - -## Database Schema - -The Supabase database uses the following tables: - -### gaia_memory - -The main memory table with the following columns: - -- `id`: UUID primary key -- `key`: Text, unique key for the memory entry -- `value`: Text, JSON-serialized value -- `metadata`: JSONB, additional metadata -- `created_at`: Timestamp with timezone, creation time -- `expires_at`: Timestamp with timezone, expiration time (optional) - -## Configuration - -The memory system is configured through the `agent/config.py` file: - -```python -# Memory Settings -MEMORY_ENABLED = True -MEMORY_TABLE_NAME = "gaia_memory" -MEMORY_TTL = 86400 # 24 hours in seconds -MEMORY_CACHE_SIZE = 100 -MEMORY_SIMILARITY_THRESHOLD = 0.85 -MEMORY_MAX_ENTRIES = 1000 -``` - -## Usage - -### Initializing the Agent with Memory - -```python -from agent.agent import GAIAAgent - -# Create an agent with memory enabled -agent = GAIAAgent({ - "memory": { - "enabled": True, - "table_name": "gaia_memory", - "conversation_id": "unique-conversation-id" - } -}) -``` - -### Using Memory Across Sessions - -The memory system allows the agent to maintain context across multiple sessions: - -```python -# Session 1 -agent1 = GAIAAgent({"memory": {"conversation_id": "conversation-123"}}) -answer1 = agent1("What is the capital of France?") - -# Session 2 (could be a different process or even a different day) -agent2 = GAIAAgent({"memory": {"conversation_id": "conversation-123"}}) -answer2 = agent2("What is the population of that city?") # Agent remembers "that city" refers to Paris -``` - -## Testing - -The integration can be tested using the `tests/test_gaia_with_memory.py` script: - -```bash -# Test memory persistence -python tests/test_gaia_with_memory.py --test memory - -# Test API integration with memory -python tests/test_gaia_with_memory.py --test api - -# Run all tests -python tests/test_gaia_with_memory.py --test all -``` - -## Error Handling - -The memory system is designed to be resilient to failures: - -1. If Supabase is unavailable, the agent falls back to in-memory storage -2. If a memory entry expires, it is automatically removed from the system -3. If a memory operation fails, the error is logged and the agent continues operation - -## Security Considerations - -1. Supabase credentials are stored as environment variables -2. The memory system uses Supabase's built-in security features -3. Sensitive data should be encrypted before storage -4. Memory entries can have a time-to-live (TTL) to automatically expire - -## Future Improvements - -1. Implement vector embeddings for semantic search -2. Add memory compression to reduce storage requirements -3. Implement memory prioritization to focus on relevant information -4. Add support for multi-user memory isolation -5. Implement memory summarization for long conversations \ No newline at end of file diff --git a/docs/index.md b/docs/index.md deleted file mode 100644 index 3f8a0f4178cc5b61f0977768fa01b91efc59f2f9..0000000000000000000000000000000000000000 --- a/docs/index.md +++ /dev/null @@ -1,65 +0,0 @@ -# GAIA Agent Documentation - -Welcome to the GAIA Agent documentation. This comprehensive guide provides detailed information about the GAIA agent, its components, configuration options, and usage examples. - -## Table of Contents - -### Getting Started -- [Project Overview](../README.md): Introduction to the GAIA agent project -- [Configuration Guide](configuration.md): Detailed information about configuration options -- [Deployment Guide](deployment.md): Instructions for deploying the agent in various environments - -### Core Components -- [Agent Architecture](agent_architecture.md): Overview of the agent's architecture and components -- [Memory System](gaia_memory_integration.md): Information about the agent's memory system -- [Tool Integration](tool_integration.md): Guide to the agent's tool integration system - -### API Integrations -- [OpenAI Integration](openai_integration_testing.md): Guide for testing OpenAI integration with newer models -- [LangChain OpenAI Integration](langchain_openai_integration.md): Guide for the langchain_openai package integration -- [Serper Integration](serper_integration.md): Guide for using the Serper API for Google search -- [DuckDuckGo Integration](duckduckgo_integration.md): Guide for using DuckDuckGo search -- [Multi-Search Integration](multi_search_integration.md): Guide for using multiple search providers -- [Enhanced Web Search](enhanced_web_search.md): Guide for the enhanced web search tools for GAIA assessment -- [Perplexity Integration](perplexity_integration.md): Guide for using Perplexity AI - -### Memory Integrations -- [Supabase Memory Integration](supabase_memory_integration.md): Guide for setting up and using Supabase memory - -### Testing and Verification -- [Connection Verification](connection_verification.md): Guide for verifying connections to external services -- [All APIs Integration Testing](all_apis_integration_testing.md): Guide for testing all API integrations together - -### Examples and Tutorials -- [Usage Examples](examples.md): Examples of how to use the GAIA agent in different scenarios - -### Troubleshooting -- [Troubleshooting Guide](troubleshooting.md): Solutions for common issues and problems - -## Quick Links - -- [GitHub Repository](https://github.com/your-repo): Source code repository -- [Issue Tracker](https://github.com/your-repo/issues): Report issues and request features -- [Hugging Face Agents Course](https://huggingface.co/learn/agents): Learn more about the Hugging Face Agents Course - -## Recent Updates - -- **May 1, 2025**: Added enhanced web search tools for GAIA assessment questions -- **April 30, 2025**: Added troubleshooting guide and langchain_openai integration documentation -- **April 25, 2025**: Updated model configurations to use GPT-4o -- **April 20, 2025**: Added multi-search integration documentation -- **April 15, 2025**: Added Perplexity integration documentation -- **April 10, 2025**: Initial documentation release - -## Contributing - -If you'd like to contribute to the documentation, please follow these steps: - -1. Fork the repository -2. Create a new branch for your changes -3. Make your changes -4. Submit a pull request - -## License - -This documentation is part of the Hugging Face Agents Course and follows its licensing terms. \ No newline at end of file diff --git a/docs/langchain_openai_integration.md b/docs/langchain_openai_integration.md deleted file mode 100644 index 6af4c35a61b655eb2bfed17557d6e68d38bb4c73..0000000000000000000000000000000000000000 --- a/docs/langchain_openai_integration.md +++ /dev/null @@ -1,227 +0,0 @@ -# LangChain OpenAI Integration Guide - -This document provides detailed information about the integration between LangChain and OpenAI in the GAIA agent, focusing on the `langchain_openai` package. - -## Overview - -The GAIA agent uses the `langchain_openai` package to interact with OpenAI's language models. This package provides a clean interface for using OpenAI models within the LangChain framework, enabling features like: - -- Chat completions with various OpenAI models -- Embeddings generation -- Function calling -- Streaming responses -- Vision capabilities - -## Recent Changes - -As of recent updates to the LangChain ecosystem, the OpenAI integration has been moved from the core `langchain` package to a separate `langchain_openai` package. This change was made to: - -1. Modularize the LangChain ecosystem -2. Allow for more frequent updates to specific integrations -3. Reduce the size of the core package -4. Enable more flexible dependency management - -## Installation - -To use the `langchain_openai` package, you need to install it separately: - -```bash -pip install langchain_openai -``` - -For the GAIA agent, this dependency is included in the `requirements.txt` file, so it should be installed automatically when you run: - -```bash -pip install -r requirements.txt -``` - -If you encounter issues, try upgrading to the latest version: - -```bash -pip install --upgrade langchain_openai -``` - -## Usage in GAIA Agent - -The GAIA agent uses the `langchain_openai` package in several components: - -### 1. Agent Graph (agent/graph.py) - -```python -from langchain_openai import ChatOpenAI - -# Initialize the model -model = ChatOpenAI( - model=get_model_config().get("reasoning_model", "gpt-4o"), - temperature=get_model_config().get("temperature", 0.1), - max_tokens=get_model_config().get("max_tokens", 4096) -) -``` - -### 2. Reasoning Tools (tools/reasoning_tools.py) - -```python -from langchain_openai import ChatOpenAI - -# Initialize the model for reasoning -reasoning_model = ChatOpenAI( - model=get_model_config().get("reasoning_model", "gpt-4o"), - temperature=get_model_config().get("temperature", 0.1) -) -``` - -### 3. Multimodal Tools (tools/multimodal_tools.py) - -```python -from langchain_openai import ChatOpenAI - -# Initialize the model for vision tasks -vision_model = ChatOpenAI( - model=get_model_config().get("vision_model", "gpt-4o"), - temperature=get_model_config().get("temperature", 0.1) -) -``` - -## Common Issues and Solutions - -### 1. Import Errors - -If you encounter import errors like: - -``` -ImportError: No module named 'langchain_openai' -``` - -or - -``` -ImportError: cannot import name 'ChatOpenAI' from 'langchain_openai' -``` - -**Solutions:** - -1. Ensure the package is installed: - ```bash - pip install langchain_openai - ``` - -2. Check for conflicts with older versions: - ```bash - pip uninstall langchain-openai # Note the hyphen - pip install langchain_openai # Note the underscore - ``` - -3. Update your import statements: - ```python - # Old import (no longer works) - # from langchain.chat_models import ChatOpenAI - - # New import (correct) - from langchain_openai import ChatOpenAI - ``` - -### 2. Version Compatibility - -If you encounter compatibility issues between `langchain`, `langchain_openai`, and other LangChain packages: - -**Solution:** -```bash -pip install --upgrade langchain langchain_openai langgraph -``` - -This ensures all LangChain-related packages are updated to compatible versions. - -### 3. API Key Issues - -If you encounter authentication errors: - -**Solution:** -Ensure your OpenAI API key is correctly set: -```python -import os -from dotenv import load_dotenv - -load_dotenv() - -# Check if the API key is set -api_key = os.getenv("OPENAI_API_KEY") -if not api_key: - raise ValueError("OPENAI_API_KEY environment variable is not set") -``` - -## Model Configuration - -The GAIA agent uses the following OpenAI models by default: - -| Purpose | Model | Environment Variable | -|---------|-------|----------------------| -| Reasoning | gpt-4o | REASONING_MODEL | -| Tool Execution | gpt-3.5-turbo | TOOL_MODEL | -| Vision | gpt-4o | VISION_MODEL | -| Embeddings | text-embedding-3-large | EMBEDDING_MODEL | - -You can customize these models by setting the appropriate environment variables or by passing a configuration dictionary when initializing the agent. - -## Best Practices - -1. **Always specify the model version**: This ensures consistent behavior even when OpenAI releases new model versions. - ```python - ChatOpenAI(model="gpt-4o") # Specific model version - ``` - -2. **Set appropriate temperature**: Lower values (0.0-0.3) for factual tasks, higher values (0.5-0.8) for creative tasks. - ```python - ChatOpenAI(model="gpt-4o", temperature=0.1) # Low temperature for factual responses - ``` - -3. **Handle rate limits gracefully**: Implement exponential backoff and retry logic. - ```python - from tenacity import retry, stop_after_attempt, wait_exponential - - @retry(stop=stop_after_attempt(3), wait=wait_exponential(multiplier=1, min=2, max=10)) - def query_model(prompt): - model = ChatOpenAI(model="gpt-4o") - return model.invoke(prompt) - ``` - -4. **Use streaming for long responses**: This provides a better user experience for long-running queries. - ```python - model = ChatOpenAI(model="gpt-4o", streaming=True) - for chunk in model.stream("Tell me about AI"): - print(chunk.content, end="", flush=True) - ``` - -5. **Set max_tokens appropriately**: This prevents unnecessarily long responses and reduces token usage. - ```python - ChatOpenAI(model="gpt-4o", max_tokens=1000) - ``` - -## Testing - -The GAIA agent includes tests for OpenAI integration in `tests/test_openai_integration.py`. These tests verify: - -1. Connection to the OpenAI API -2. Compatibility with newer models like GPT-4o -3. Integration with the GAIA agent - -To run these tests: - -```bash -python tests/test_openai_integration.py -``` - -For more comprehensive testing: - -```bash -python tests/run_openai_tests.py -``` - -## Troubleshooting - -If you encounter issues with the `langchain_openai` integration, check the [Troubleshooting Guide](troubleshooting.md) for common problems and solutions. - -## Related Documentation - -- [OpenAI API Documentation](https://platform.openai.com/docs/api-reference) -- [LangChain Documentation](https://python.langchain.com/docs/get_started/introduction) -- [LangChain OpenAI Integration Documentation](https://python.langchain.com/docs/integrations/llms/openai) \ No newline at end of file diff --git a/docs/multi_search_integration.md b/docs/multi_search_integration.md deleted file mode 100644 index f31515f33dd37094370d24f2b0eb677b4698acbc..0000000000000000000000000000000000000000 --- a/docs/multi_search_integration.md +++ /dev/null @@ -1,318 +0,0 @@ -# Multi-Search Provider Integration for GAIA Agent - -This document explains how to integrate and use multiple search providers with the GAIA agent, including: - -1. DuckDuckGo -2. Serper (Google Search API) -3. Perplexity -4. arXiv - -## Overview - -The GAIA agent can leverage multiple search providers to get comprehensive and diverse search results. Each provider has its strengths: - -- **DuckDuckGo**: Privacy-focused search with no API key required -- **Serper**: Google search results with structured data -- **Perplexity**: AI-powered search with reasoning capabilities -- **arXiv**: Academic paper search for scientific research - -## Configuration - -### Environment Variables - -Set up the necessary API keys in your environment: - -```bash -# Required for Serper -export SERPER_API_KEY="your_serper_api_key" - -# Required for Perplexity -export PERPLEXITY_API_KEY="your_perplexity_api_key" - -# DuckDuckGo and arXiv don't require API keys -``` - -### Tool Configuration - -The search providers are configured in `agent/config.py`: - -```python -# Tool Parameters (hardcoded) -WEB_SEARCH_RESULT_COUNT = 5 -WEB_SEARCH_TIMEOUT = 10 -DUCKDUCKGO_TIMEOUT = 10 -DUCKDUCKGO_MAX_RESULTS = 5 -PERPLEXITY_TIMEOUT = 30 -PERPLEXITY_MODEL = "sonar-reasoning" -ARXIV_MAX_RESULTS = 3 -``` - -## Using Search Providers - -### Creating Search Tool Instances - -```python -# Import the necessary tools -from tools.web_tools import create_duckduckgo_search, create_serper_search -from tools.perplexity_tool import PerplexityTool -from tools.arxiv_tool import create_arxiv_search - -# Create DuckDuckGo search tool (no API key required) -duckduckgo_tool = create_duckduckgo_search() - -# Create Serper search tool (requires API key in environment) -serper_tool = create_serper_search() - -# Create Perplexity tool (requires API key in environment) -perplexity_tool = PerplexityTool() - -# Create arXiv search tool (no API key required) -arxiv_tool = create_arxiv_search() -``` - -### Performing Searches - -Each search provider has a similar interface: - -```python -# DuckDuckGo search -duckduckgo_results = duckduckgo_tool.search("your search query") -# Returns a list of dictionaries with 'title', 'link', and 'snippet' keys - -# Serper search -serper_results = serper_tool.search("your search query") -# Returns a list of dictionaries with 'title', 'link', and 'snippet' keys - -# Perplexity search -perplexity_results = perplexity_tool.search("your search query") -# Returns a dictionary with 'content', 'citations', and other metadata - -# arXiv search -arxiv_results = arxiv_tool.search("your search query") -# Returns a list of dictionaries with paper information - -# arXiv get paper by ID -paper = arxiv_tool.get_paper_by_id("2303.08774") -# Returns a dictionary with detailed paper information - -# arXiv search by category -category_results = arxiv_tool.search_by_category("cs.AI") -# Returns a list of dictionaries with papers in the specified category -``` - -## Integrating with LangGraph - -To integrate these search providers with the LangGraph-based GAIA agent, you need to modify the `execute_tool` function in `agent/graph.py`: - -```python -# Create a tool registry -tool_registry = { - "duckduckgo_search": create_duckduckgo_search(), - "serper_search": create_serper_search(), - "perplexity_search": PerplexityTool(), - "arxiv_search": create_arxiv_search(), - "arxiv_get_paper": create_arxiv_search(), - "arxiv_search_category": create_arxiv_search() -} - -def execute_tool(state: AgentState) -> AgentState: - # Get the current step from the plan - current_step = state["plan"][state["current_step"]] - tool_name = current_step.get("tool") - tool_input = current_step.get("tool_input", {}) - - logger.debug(f"Executing tool {tool_name} for step {current_step['step_number']}") - - try: - # Get the tool from the registry - tool = tool_registry.get(tool_name) - if not tool: - raise Exception(f"Tool {tool_name} not found in registry") - - # Execute the tool based on its type - if tool_name in ["duckduckgo_search", "serper_search"]: - result = tool.search(tool_input.get("query", "")) - elif tool_name == "perplexity_search": - result = tool.search(tool_input.get("query", "")) - elif tool_name == "arxiv_search": - result = tool.search(tool_input.get("query", ""), tool_input.get("max_results")) - elif tool_name == "arxiv_get_paper": - result = tool.get_paper_by_id(tool_input.get("paper_id", "")) - elif tool_name == "arxiv_search_category": - result = tool.search_by_category(tool_input.get("category", ""), tool_input.get("max_results")) - else: - # Generic tool execution - result = tool.run(tool_input) - - # Create a tool result - tool_result = { - "tool_name": tool_name, - "success": True, - "result": result, - "error": None - } - - # Update the state - new_state = state.copy() - new_state["tool_results"] = state.get("tool_results", []) + [tool_result] - new_state["current_step"] += 1 - - return new_state - except Exception as e: - # Handle error... - # (rest of the function remains the same) - } -``` - -## Comparison of Search Providers - -| Provider | API Key Required | Result Format | Strengths | Limitations | -|-------------|------------------|------------------------------|--------------------------------------------|--------------------------------------------| -| DuckDuckGo | No | List of result dictionaries | Privacy-focused, No API key | Sometimes less comprehensive | -| Serper | Yes | List of result dictionaries | Google results, Structured data | Requires API key, Usage limits | -| Perplexity | Yes | Content with citations | AI-powered, Reasoning capabilities | Requires API key, Slower response time | -| arXiv | No | List of paper dictionaries | Academic papers, Scientific research | Limited to academic content | - -## Best Practices - -1. **Fallback Mechanism**: Implement a fallback mechanism to try alternative search providers if one fails. - -```python -def robust_search(query): - # Try DuckDuckGo first (no API key required) - try: - results = duckduckgo_tool.search(query) - if results: - return {"provider": "duckduckgo", "results": results} - except Exception as e: - logger.warning(f"DuckDuckGo search failed: {str(e)}") - - # Try Serper as fallback - try: - results = serper_tool.search(query) - if results: - return {"provider": "serper", "results": results} - except Exception as e: - logger.warning(f"Serper search failed: {str(e)}") - - # Try Perplexity as third option - try: - results = perplexity_tool.search(query) - if results: - return {"provider": "perplexity", "results": results} - except Exception as e: - logger.warning(f"Perplexity search failed: {str(e)}") - - # Try arXiv as last resort - try: - results = arxiv_tool.search(query) - if results: - return {"provider": "arxiv", "results": results} - except Exception as e: - logger.warning(f"arXiv search failed: {str(e)}") - - # All providers failed - raise Exception("All search providers failed") -``` - -2. **Query Optimization**: Different search providers may respond better to different query formats. - -3. **Result Merging**: For comprehensive results, consider merging results from multiple providers. - -```python -def merged_search(query): - results = { - "duckduckgo": [], - "serper": [], - "perplexity": None, - "arxiv": [] - } - - # Collect results from all providers - try: - results["duckduckgo"] = duckduckgo_tool.search(query) - except Exception: - pass - - try: - results["serper"] = serper_tool.search(query) - except Exception: - pass - - try: - results["perplexity"] = perplexity_tool.search(query) - except Exception: - pass - - # Try arXiv - try: - results["arxiv"] = arxiv_tool.search(query) - except Exception: - pass - - # Merge and deduplicate results - merged = [] - seen_urls = set() - - # Add DuckDuckGo results - for result in results["duckduckgo"]: - if result["link"] not in seen_urls: - merged.append(result) - seen_urls.add(result["link"]) - - # Add Serper results - for result in results["serper"]: - if result["link"] not in seen_urls: - merged.append(result) - seen_urls.add(result["link"]) - - # Add arXiv results - arxiv_results = [] - for result in results["arxiv"]: - if "url" in result and result["url"] not in seen_urls: - # Convert arXiv result to standard format - formatted_result = { - "title": result.get("title", ""), - "link": result.get("url", ""), - "snippet": result.get("summary", "")[:200] + "..." if result.get("summary") else "" - } - merged.append(formatted_result) - seen_urls.add(result["url"]) - arxiv_results.append(result) # Keep original format for context - - # Add Perplexity content as context - perplexity_content = results["perplexity"].get("content") if results["perplexity"] else None - - return { - "merged_results": merged, - "perplexity_context": perplexity_content, - "arxiv_context": arxiv_results - } -``` - -## Testing - -To test the integration of all search providers, run: - -```bash -python tests/test_duckduckgo_integration.py -python tests/test_arxiv_integration.py -``` - -These tests verify: -1. Direct connection to each search provider -2. Proper functioning of each search tool -3. Error handling -4. Using multiple search providers together -5. Integration with the GAIA agent - -## Conclusion - -By integrating multiple search providers, the GAIA agent can provide more robust and comprehensive search capabilities. The agent can fall back to alternative providers if one fails, and can leverage the strengths of each provider for different types of queries: - -- Use DuckDuckGo for general web searches without API key requirements -- Use Serper for more comprehensive Google search results -- Use Perplexity for AI-powered reasoning and analysis -- Use arXiv for academic and scientific research papers - -This multi-provider approach ensures the GAIA agent can handle a wide range of search queries and provide the most relevant information to users. \ No newline at end of file diff --git a/docs/openai_integration_testing.md b/docs/openai_integration_testing.md deleted file mode 100644 index 359bef902cac19345f841855d2faeaf589b5b23d..0000000000000000000000000000000000000000 --- a/docs/openai_integration_testing.md +++ /dev/null @@ -1,138 +0,0 @@ -# Testing OpenAI Integration with Newer Models - -This document provides instructions for testing the OpenAI integration with newer models like GPT-4o in the GAIA agent. - -## Overview - -The GAIA agent has been updated to support newer OpenAI models, including: - -- **GPT-4o**: The latest multimodal model that replaces both GPT-4-turbo and GPT-4-vision-preview -- **text-embedding-3-large**: The latest embedding model that replaces text-embedding-ada-002 - -These test scripts verify that the configuration changes are working properly and that the GAIA agent can use these newer models effectively. - -## Prerequisites - -Before running the tests, make sure you have: - -1. Set up your environment variables with the necessary API keys: - - `OPENAI_API_KEY`: Your OpenAI API key - - `PERPLEXITY_API_KEY`: Your Perplexity API key (for multi-provider tests) - -2. Installed all required dependencies: - ```bash - pip install -r requirements.txt - ``` - -## Test Scripts - -### 1. OpenAI Integration Test - -This script tests the OpenAI integration with newer models. - -**File**: `tests/test_openai_integration.py` - -**Features**: -- Verifies connection to the OpenAI API -- Tests sending requests to newer models (GPT-4o, etc.) -- Tests vision capabilities with GPT-4o -- Optionally tests the GAIA agent with OpenAI models - -**Usage**: -```bash -python tests/test_openai_integration.py [--verbose] [--output FILENAME] [--test-agent] -``` - -**Options**: -- `--verbose`, `-v`: Enable verbose logging -- `--output`, `-o`: Specify output file for test results (default: openai_test_results.json) -- `--test-agent`, `-a`: Test the GAIA agent with OpenAI models - -### 2. GAIA Multi-Provider Test - -This script tests the GAIA agent's ability to use both OpenAI and Perplexity tools together. - -**File**: `tests/test_gaia_multi_provider.py` - -**Features**: -- Tests OpenAI-only questions -- Tests Perplexity-only questions -- Tests questions that require both providers -- Tests memory components with multi-provider data - -**Usage**: -```bash -python tests/test_gaia_multi_provider.py [--verbose] [--output FILENAME] -``` - -**Options**: -- `--verbose`, `-v`: Enable verbose logging -- `--output`, `-o`: Specify output file for test results (default: gaia_multi_provider_test_report.json) - -## Test Results - -Both test scripts generate detailed reports in JSON format, which include: - -- Connection status -- Model test results -- Vision test results (for OpenAI) -- Agent test results -- Memory component test results -- Summary statistics - -The reports are saved to the specified output file and also printed to the console. - -## Interpreting Results - -### OpenAI Integration Test - -A successful test should show: -- ✅ Connection to OpenAI API established -- ✅ All model tests passed -- ✅ Vision test passed (if applicable) -- ✅ Agent test passed (if enabled) - -### GAIA Multi-Provider Test - -A successful test should show: -- ✅ Both API keys available -- ✅ OpenAI-only questions answered correctly -- ✅ Perplexity-only questions answered correctly -- ✅ Combined questions answered correctly -- ✅ All memory components working - -## Troubleshooting - -### Common Issues - -1. **API Key Issues**: - - Make sure your API keys are set correctly in your environment variables - - Check that your API keys have not expired - - Verify that your API keys have access to the models being tested - -2. **Model Availability**: - - Some newer models may not be available to all accounts - - Check the OpenAI dashboard to see which models you have access to - -3. **Rate Limiting**: - - If you encounter rate limiting errors, try running the tests with fewer questions - - Add delays between requests by modifying the test scripts - -4. **Memory Issues**: - - If memory tests fail, check your Supabase configuration - - Verify that the memory tables exist and have the correct schema - -## Next Steps - -After verifying that the OpenAI integration works with newer models, you can: - -1. Update your application to use the newer models -2. Optimize your prompts for the newer models -3. Implement additional features that leverage the capabilities of the newer models -4. Monitor performance and costs to ensure efficient usage - -## Additional Resources - -- [OpenAI API Documentation](https://platform.openai.com/docs/api-reference) -- [GPT-4o Documentation](https://platform.openai.com/docs/models/gpt-4o) -- [Perplexity API Documentation](https://docs.perplexity.ai/) \ No newline at end of file diff --git a/docs/serper_integration.md b/docs/serper_integration.md deleted file mode 100644 index 428af211a7d53d878f963dc8c4921b6fef554aa8..0000000000000000000000000000000000000000 --- a/docs/serper_integration.md +++ /dev/null @@ -1,153 +0,0 @@ -# Serper API Integration - -This document provides information about the Serper API integration in the GAIA agent, including configuration, usage, and testing procedures. - -## Overview - -Serper is a Google Search API that our GAIA agent uses for web searches. It provides a reliable way to access Google search results programmatically, which enhances the agent's ability to retrieve up-to-date information from the web. - -## Configuration - -The Serper API integration requires the following configuration: - -1. **API Key**: Set the `SERPER_API_KEY` environment variable with your Serper API key. -2. **API URL**: The API endpoint is configured in `agent/config.py` as `SERPER_API_URL`. - -Example configuration in `.env` file: - -``` -SERPER_API_KEY=your_serper_api_key_here -``` - -## Testing Results - -We have successfully tested the Serper API integration and confirmed that it works correctly. The key findings from our testing are: - -1. **API Connection**: The Serper API connection is working correctly. We can successfully connect to the API and receive a 200 status code. -2. **Search Functionality**: The search functionality works as expected. We can send search queries and receive relevant results. -3. **Response Format**: The API returns results in the expected format, with organic search results containing title, link, and snippet information. - -### Sample Response - -Here's an example of the response format from the Serper API: - -```json -{ - "organic": [ - { - "title": "Artificial Intelligence News - ScienceDaily", - "link": "https://www.sciencedaily.com/news/computers_math/artificial_intelligence/", - "snippet": "Latest Headlines • AI Tool Outperforms Most Doctors On Exams • Brain-Inspired AI Breakthrough • Robot See, Robot Do • Explainable AI for Ship Navigation Raises ..." - }, - { - "title": "...", - "link": "...", - "snippet": "..." - } - ] -} -``` - -### Environment Configuration - -The Serper API key must be properly configured in the `.env` file. Our testing confirmed that the key is correctly loaded from the `.env` file and used in API requests. - -### Integration with GAIA Agent - -The Serper API is integrated with the GAIA agent through the `SerperSearchTool` class in `tools/web_tools.py`. This integration allows the agent to use the Serper API for web searches as part of its capabilities. - -## Implementation - -The Serper API is integrated through the `SerperSearchTool` class in `tools/web_tools.py`. This class: - -1. Initializes with the Serper API key and URL -2. Provides a `search` method that accepts a query string -3. Formats the request with appropriate headers and payload -4. Processes the response to extract relevant search results -5. Returns the results in a standardized format - -## Usage - -### Basic Usage - -```python -from tools.web_tools import create_serper_search - -# Create a Serper search tool instance -serper_tool = create_serper_search() - -# Perform a search -results = serper_tool.search("artificial intelligence latest developments") - -# Process the results -for result in results: - print(f"Title: {result['title']}") - print(f"Link: {result['link']}") - print(f"Snippet: {result['snippet']}") - print() -``` - -### Integration with GAIA Agent - -```python -from agent.agent import GaiaAgent -from tools.web_tools import create_serper_search - -# Create a GAIA agent instance -agent = GaiaAgent() - -# Create and register the Serper search tool -serper_tool = create_serper_search() -agent.register_tool("serper_search", serper_tool.search) - -# Run the agent with a query that would use the Serper tool -response = agent.run( - "What are the latest developments in artificial intelligence?", - tools=["serper_search"] -) - -print(response) -``` - -## Testing - -We have implemented several tests to verify the Serper API integration: - -1. **Standalone API Test**: `tests/test_serper_api.py` - Tests the direct connection to the Serper API. -2. **Unit Tests**: `tests/test_tools.py` - Contains unit tests for the `SerperSearchTool` class. -3. **Integration Tests**: `tests/test_gaia_serper_integration.py` - Tests the integration of Serper with the GAIA agent. -4. **Connection Verification**: `tests/verify_connections.py` - Includes a function to verify the Serper API connection. - -### Running the Tests - -To test the Serper API connection: - -```bash -python tests/test_serper_api.py -``` - -To run the integration tests: - -```bash -python tests/test_gaia_serper_integration.py -``` - -To verify all connections including Serper: - -```bash -python tests/verify_connections.py -``` - -## Troubleshooting - -Common issues and their solutions: - -1. **API Key Not Set**: Ensure the `SERPER_API_KEY` environment variable is set correctly. -2. **Connection Timeout**: Check your internet connection and try again later. -3. **Rate Limiting**: Serper API has rate limits. Check your usage and consider upgrading your plan if necessary. -4. **Invalid Response Format**: If the API response format changes, update the `SerperSearchTool.search` method to handle the new format. - -## References - -- [Serper API Documentation](https://serper.dev/docs) -- [GAIA Agent Documentation](./configuration.md) \ No newline at end of file diff --git a/docs/supabase_memory_integration.md b/docs/supabase_memory_integration.md deleted file mode 100644 index a19fad073778b7b496378c22c0fdca3b22023005..0000000000000000000000000000000000000000 --- a/docs/supabase_memory_integration.md +++ /dev/null @@ -1,175 +0,0 @@ -# Supabase Memory Integration for GAIA Agent - -This document provides an overview of the Supabase memory integration for the GAIA agent, including setup instructions, table structure, and testing procedures. - -## Overview - -The GAIA agent uses Supabase as a persistent memory storage solution. This allows the agent to maintain memory across different sessions and provides a reliable way to store conversation history, cache results, and track intermediate results during agent execution. - -The memory implementation consists of the following components: - -1. **SupabaseMemory**: Base memory implementation using Supabase for storage -2. **ConversationMemory**: Memory implementation for storing conversation history -3. **ResultCache**: Memory implementation for caching tool results and answers -4. **WorkingMemory**: Memory implementation for storing intermediate results during agent execution - -## Supabase Setup - -### Prerequisites - -- A Supabase account -- A Supabase project -- Supabase API key with appropriate permissions - -### Environment Variables - -The following environment variables need to be set: - -``` -SUPABASE_KEY=your_supabase_api_key -``` - -The Supabase URL and project details are configured in `agent/config.py`: - -```python -# Supabase Configuration -SUPABASE_URL = "https://tjamxhvvtnypbadvrkjq.supabase.co" -SUPABASE_PROJECT_ID = "tjamxhvvtnypbadvrkjq" -SUPABASE_PROJECT_NAME = "HF_GAIA_Assessment_AGT" -``` - -### Table Structure - -The Supabase integration uses the following tables: - -1. **gaia_memory**: Main memory table for storing key-value pairs -2. **gaia_memory_conversations**: Conversation history table -3. **gaia_memory_cache**: Cache table for storing tool results and answers -4. **gaia_memory_working**: Working memory table for storing intermediate results -5. **test_connection**: Test table for verifying connection - -You can create these tables by running the SQL statements in `supabase_tables.sql` in the Supabase SQL Editor. - -## Memory Implementation - -### SupabaseMemory - -The `SupabaseMemory` class provides the base memory implementation using Supabase for storage. It supports the following operations: - -- `store(key, value, metadata, ttl)`: Store a value in memory -- `retrieve(key, default)`: Retrieve a value from memory -- `delete(key)`: Delete a value from memory -- `clear()`: Clear all values from memory -- `list_keys(prefix)`: List all keys in memory - -### ConversationMemory - -The `ConversationMemory` class provides memory implementation for storing conversation history. It supports the following operations: - -- `add_message(role, content, metadata)`: Add a message to the conversation history -- `get_messages(limit, before, after)`: Get messages from the conversation history -- `clear()`: Clear the conversation history - -### ResultCache - -The `ResultCache` class provides memory implementation for caching tool results and answers. It supports the following operations: - -- `cache_result(query, result, metadata, ttl)`: Cache a result -- `get_result(query, default)`: Get a cached result -- `invalidate(key)`: Invalidate a cached result -- `clear()`: Clear all cached results - -### WorkingMemory - -The `WorkingMemory` class provides memory implementation for storing intermediate results during agent execution. It supports the following operations: - -- `store_intermediate_result(key, value, metadata, ttl)`: Store an intermediate result -- `get_intermediate_result(key, default)`: Get an intermediate result -- `list_intermediate_results()`: List all intermediate results -- `clear()`: Clear all intermediate results - -## Testing - -### Test Scripts - -The following test scripts are available for testing the Supabase memory integration: - -1. **test_supabase_connection.py**: Tests the connection to Supabase and verifies that the required tables exist -2. **test_supabase_memory.py**: Tests the Supabase memory module implementation -3. **test_gaia_supabase_integration.py**: Tests the integration of the GAIA agent with Supabase memory -4. **run_supabase_integration_tests.py**: Runs all the Supabase integration tests and documents the results - -### Running the Tests - -To run all the Supabase integration tests, use the following command: - -```bash -python tests/run_supabase_integration_tests.py -``` - -For verbose output, add the `--verbose` flag: - -```bash -python tests/run_supabase_integration_tests.py --verbose -``` - -To specify a custom output directory for the test results, use the `--output-dir` option: - -```bash -python tests/run_supabase_integration_tests.py --output-dir custom_docs -``` - -### Test Results - -The test results are documented in a markdown file in the specified output directory (default: `docs`). The file includes: - -- Test summary with date and Supabase project details -- Results for each test with status (passed/failed) -- Detailed test output in collapsible sections -- Overall assessment and recommendations - -## Troubleshooting - -If the tests fail, check the following: - -1. **Supabase Connection**: - - Verify that the Supabase URL and API key are correctly set - - Check that the Supabase project is active - - Ensure that the API key has the necessary permissions - -2. **Table Structure**: - - Verify that all required tables exist in the Supabase database - - Check that the tables have the correct structure - - Ensure that the indexes are properly created - -3. **Memory Configuration**: - - Check that memory is enabled in the configuration - - Verify that the table names are correctly set - - Ensure that the TTL and cache size settings are appropriate - -4. **Agent Integration**: - - Check that the agent is correctly using the memory components - - Verify that the conversation ID and session ID are properly set - - Ensure that the agent is correctly processing questions - -## Best Practices - -1. **Memory Management**: - - Regularly clear unused memory to prevent excessive storage usage - - Use appropriate TTL values for different types of memory entries - - Monitor memory usage and adjust cache sizes as needed - -2. **Error Handling**: - - Implement proper error handling for memory operations - - Gracefully handle connection issues and retry operations when appropriate - - Log memory-related errors for debugging purposes - -3. **Security**: - - Use a secure API key with appropriate permissions - - Do not store sensitive information in memory - - Regularly rotate API keys for security - -4. **Performance**: - - Use efficient queries to minimize latency - - Implement caching to reduce database load - - Monitor query performance and optimize as needed \ No newline at end of file diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md deleted file mode 100644 index f19f7fa1c4cfc7fddd836b7aaf9335e9c979bd5a..0000000000000000000000000000000000000000 --- a/docs/troubleshooting.md +++ /dev/null @@ -1,281 +0,0 @@ -# Troubleshooting Guide for GAIA Agent - -This guide addresses common issues you might encounter when setting up, configuring, or running the GAIA agent. It provides solutions and workarounds for these issues. - -## Table of Contents - -- [Dependency Issues](#dependency-issues) - - [langchain_openai Issues](#langchain_openai-issues) - - [Other Dependency Issues](#other-dependency-issues) -- [API Connection Issues](#api-connection-issues) - - [OpenAI API Issues](#openai-api-issues) - - [Serper API Issues](#serper-api-issues) - - [Perplexity API Issues](#perplexity-api-issues) -- [Supabase Issues](#supabase-issues) - - [Connection Issues](#connection-issues) - - [Table Structure Issues](#table-structure-issues) -- [Memory Issues](#memory-issues) -- [Agent Execution Issues](#agent-execution-issues) -- [Testing Issues](#testing-issues) - -## Dependency Issues - -### langchain_openai Issues - -The `langchain_openai` package is a critical dependency for the GAIA agent, as it provides the interface to OpenAI's language models. Here are common issues and their solutions: - -#### ImportError: No module named 'langchain_openai' - -This error occurs when the `langchain_openai` package is not installed or not in your Python path. - -**Solution:** -```bash -pip install langchain_openai -``` - -If you already have it installed, try upgrading to the latest version: -```bash -pip install --upgrade langchain_openai -``` - -#### AttributeError: module 'langchain_openai' has no attribute 'ChatOpenAI' - -This error can occur if you have an outdated version of `langchain_openai` or if there's a conflict with other packages. - -**Solutions:** - -1. Upgrade to the latest version: - ```bash - pip install --upgrade langchain_openai - ``` - -2. Check for conflicts with older versions of langchain: - ```bash - pip uninstall langchain-openai # Note the hyphen instead of underscore - pip install langchain_openai # Install the correct package - ``` - -3. Verify your import statement: - ```python - # Correct import - from langchain_openai import ChatOpenAI - - # Incorrect import (old style) - # from langchain.chat_models import ChatOpenAI - ``` - -#### Version Compatibility Issues - -If you're experiencing compatibility issues between `langchain`, `langchain_openai`, and other LangChain packages: - -**Solution:** -```bash -pip install --upgrade langchain langchain_openai langgraph -``` - -This ensures all LangChain-related packages are updated to compatible versions. - -### Other Dependency Issues - -#### Missing Dependencies - -If you encounter errors about missing dependencies: - -**Solution:** -```bash -pip install -r requirements.txt -``` - -#### Conflicting Dependencies - -If you have conflicting dependencies: - -**Solution:** -```bash -# Create a fresh virtual environment -python -m venv fresh_env -source fresh_env/bin/activate # On Windows: fresh_env\Scripts\activate -pip install -r requirements.txt -``` - -## API Connection Issues - -### OpenAI API Issues - -#### Authentication Error - -If you see an error like `AuthenticationError: Incorrect API key provided`: - -**Solutions:** -1. Verify your API key is correct and active in your OpenAI account -2. Check that the environment variable is set correctly: - ```bash - # In your .env file - OPENAI_API_KEY=your_actual_api_key - ``` -3. Try setting the API key directly in your code (for testing only): - ```python - import os - os.environ["OPENAI_API_KEY"] = "your_actual_api_key" - ``` - -#### Rate Limit Exceeded - -If you encounter rate limit errors: - -**Solutions:** -1. Implement exponential backoff and retry logic -2. Reduce the frequency of API calls -3. Consider upgrading your OpenAI plan for higher rate limits - -### Serper API Issues - -#### Authentication Error - -If you encounter authentication errors with Serper: - -**Solution:** -Verify your Serper API key is correctly set in the environment variables: -```bash -# In your .env file -SERPER_API_KEY=your_serper_api_key -``` - -#### No Results Returned - -If Serper searches return no results: - -**Solutions:** -1. Check your query formatting -2. Verify that Serper is available and operational -3. Try using an alternative search provider like DuckDuckGo - -### Perplexity API Issues - -#### Authentication Error - -If you encounter authentication errors with Perplexity: - -**Solution:** -Verify your Perplexity API key is correctly set in the environment variables: -```bash -# In your .env file -PERPLEXITY_API_KEY=your_perplexity_api_key -``` - -#### Model Not Available - -If you encounter errors about a model not being available: - -**Solution:** -Check the available models in your Perplexity account and update the configuration accordingly. - -## Supabase Issues - -### Connection Issues - -#### Authentication Error - -If you encounter authentication errors with Supabase: - -**Solution:** -Verify your Supabase URL and API key are correctly set: -```bash -# In your .env file -SUPABASE_URL=https://your-project-id.supabase.co -SUPABASE_KEY=your_supabase_api_key -``` - -#### Connection Timeout - -If you encounter connection timeouts: - -**Solutions:** -1. Check your internet connection -2. Verify that your Supabase project is active -3. Implement retry logic with exponential backoff - -### Table Structure Issues - -#### Missing Tables - -If you encounter errors about missing tables: - -**Solution:** -Run the SQL statements in `supabase_tables.sql` in the Supabase SQL Editor to create the required tables. - -#### Incorrect Table Structure - -If you encounter errors about incorrect table structure: - -**Solution:** -Compare your table structure with the expected structure in `supabase_tables.sql` and make any necessary adjustments. - -## Memory Issues - -#### Memory Not Persisting - -If the agent's memory is not persisting between sessions: - -**Solutions:** -1. Verify that memory is enabled in the configuration -2. Check that the Supabase connection is working correctly -3. Verify that the memory tables exist and have the correct structure - -#### Memory Retrieval Issues - -If the agent is unable to retrieve memory: - -**Solutions:** -1. Check the memory key format -2. Verify that the TTL (time-to-live) settings are appropriate -3. Check for errors in the Supabase logs - -## Agent Execution Issues - -#### Agent Not Responding - -If the agent is not responding or taking too long: - -**Solutions:** -1. Check the timeout settings in the configuration -2. Verify that all required APIs are available -3. Check for errors in the logs - -#### Incorrect Answers - -If the agent is providing incorrect answers: - -**Solutions:** -1. Check the question format -2. Verify that the agent is using the correct tools -3. Check the answer extraction logic - -## Testing Issues - -#### Tests Failing - -If tests are failing: - -**Solutions:** -1. Check that all required environment variables are set -2. Verify that all required APIs are available -3. Check for errors in the test logs - -#### Test Timeouts - -If tests are timing out: - -**Solutions:** -1. Increase the timeout settings -2. Run tests individually to identify the problematic test -3. Check for resource-intensive operations that might be causing the timeout - -## Additional Resources - -If you're still experiencing issues after trying the solutions in this guide, consider the following resources: - -1. Check the [GitHub repository](https://github.com/your-repo) for open issues or create a new one -2. Consult the [LangChain documentation](https://python.langchain.com/docs/get_started/introduction) -3. Refer to the [OpenAI API documentation](https://platform.openai.com/docs/api-reference) -4. Check the [Supabase documentation](https://supabase.com/docs) \ No newline at end of file diff --git a/env_template.txt b/env_template.txt deleted file mode 100644 index e4fe86d706ae82c79c40a4faced61cb2bb458210..0000000000000000000000000000000000000000 --- a/env_template.txt +++ /dev/null @@ -1,67 +0,0 @@ -# GAIA Environment Variables Template -# Copy these to Hugging Face Space secrets - -# === REQUIRED ENVIRONMENT VARIABLES === - -# OpenAI API key for model access -OPENAI_API_KEY=**** - -# URL for your Supabase instance -SUPABASE_URL=https://tjamxhvvtnypbadvrkjq.supabase.co - -# Service key for Supabase access -SUPABASE_KEY=**** - -# Serper.dev API key for web search -SERPER_API_KEY=**** - -# Default OpenAI model to use -DEFAULT_MODEL=gpt-4-turbo - -# OpenAI embedding model to use -EMBEDDING_MODEL=text-embedding-ada-002 - -# Fallback model if default is unavailable -FALLBACK_MODEL=gpt-3.5-turbo - -# Enable/disable memory functionality -MEMORY_ENABLED=True - -# Supabase table for memory storage -MEMORY_TABLE_NAME=gaia_memory - -# Timeout for DuckDuckGo searches in seconds -DUCKDUCKGO_TIMEOUT=10 - -# Maximum number of results from DuckDuckGo -DUCKDUCKGO_MAX_RESULTS=5 - -# === OPTIONAL ENVIRONMENT VARIABLES === - -# Custom API base URL for OpenAI (optional) -OPENAI_API_BASE=https://api.openai.com/v1 - -# Temperature for model responses (0.0-1.0) -MODEL_TEMPERATURE=0.1 - -# Maximum tokens in model responses -MODEL_MAX_TOKENS=4096 - -# Number of results to return from web searches -WEB_SEARCH_RESULT_COUNT=5 - -# Time-to-live for memory entries in seconds -MEMORY_TTL=86400 - -# Size of memory cache -MEMORY_CACHE_SIZE=100 - -# Maximum iterations for agent execution -MAX_ITERATIONS=10 - -# Enable verbose logging -VERBOSE=False - -# Logging level (DEBUG, INFO, WARNING, ERROR) -LOG_LEVEL=INFO - diff --git a/examples/enhanced_web_search_example.py b/examples/enhanced_web_search_example.py deleted file mode 100644 index 008b3af52563a9e5ba1bac6d60ea589c4aa0c465..0000000000000000000000000000000000000000 --- a/examples/enhanced_web_search_example.py +++ /dev/null @@ -1,84 +0,0 @@ -""" -Example script demonstrating how to use the EnhancedWebSearchTool for GAIA assessment questions. - -This script shows how to use the EnhancedWebSearchTool to search for information -related to specific GAIA assessment questions. -""" - -import os -import sys -import logging -import json -from typing import Dict, Any, List - -# Add the parent directory to the path so we can import the agent module -sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) - -from agent.tool_registry import create_default_registry, unified_search -from tools.web_tools import create_enhanced_web_search - -# Configure logging -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(name)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("enhanced_web_search_example") - -def main(): - """Run the enhanced web search example.""" - - # Check required environment variables - required_vars = ["PERPLEXITY_API_KEY"] - missing_vars = [var for var in required_vars if not os.environ.get(var)] - - if missing_vars: - logger.error(f"Missing required environment variables: {', '.join(missing_vars)}") - logger.error("Please set these environment variables before running the example.") - sys.exit(1) - - # Create the registry - logger.info("Creating tool registry...") - registry = create_default_registry() - - # GAIA assessment questions - questions = [ - "How many studio albums were published by Mercedes Sosa between 2000 and 2009 (included)?", - "Who nominated the only Featured Article on English Wikipedia about a dinosaur that was promoted in November 2016?", - "On June 6, 2023, an article by Carolyn Collins Petersen was published in Universe Today. This article mentions a team that produced a paper about their observations, linked at the bottom of the article. Find this paper. Under what NASA award number was the work performed by R. G. Arendt supported by?", - "Who won the Malko Competition for Young Conductors in 2021?" - ] - - # Search for each question - for i, question in enumerate(questions): - logger.info(f"\n\nQuestion {i+1}: {question}") - - # Search using unified_search - logger.info("Searching using unified_search...") - results = unified_search(registry, question) - - # Print the results - if results and results.get("results"): - logger.info(f"Found {len(results['results'])} results") - logger.info(f"Providers used: {results.get('metadata', {}).get('providers_used', [])}") - - # Print the top result - if results["results"]: - top_result = results["results"][0] - logger.info("\nTop result:") - logger.info(f"Title: {top_result.get('title', 'N/A')}") - logger.info(f"Source: {top_result.get('source', 'N/A')}") - logger.info(f"Relevance score: {top_result.get('relevance_score', 'N/A')}") - logger.info(f"Snippet: {top_result.get('snippet', 'N/A')}") - - # If we have Perplexity content, print it - perplexity_content = results.get("metadata", {}).get("perplexity_content") - if perplexity_content: - logger.info("\nPerplexity content:") - logger.info(perplexity_content[:500] + "..." if len(perplexity_content) > 500 else perplexity_content) - else: - logger.warning("No results found") - - logger.info("\nExample completed successfully!") - -if __name__ == "__main__": - main() \ No newline at end of file diff --git a/examples/multi_search_example.py b/examples/multi_search_example.py deleted file mode 100644 index af8000b3cd5bd085577b82b9e6475f0a8259b36e..0000000000000000000000000000000000000000 --- a/examples/multi_search_example.py +++ /dev/null @@ -1,143 +0,0 @@ -""" -Example script demonstrating how to use multiple search providers with the GAIA agent. - -This script shows how to: -1. Create a tool registry with all search providers -2. Perform searches using individual providers -3. Use the robust search function that tries multiple providers -4. Use the merged search function that combines results from multiple providers - -Usage: - python examples/multi_search_example.py -""" - -import os -import sys -import logging -from typing import Dict, Any, List - -sys.path.append('.') - -from agent.tool_registry import ( - ToolRegistry, - create_default_registry, - robust_search, - merged_search -) - -logging.basicConfig( - level=logging.INFO, - format='%(asctime)s - %(name)s - %(levelname)s - %(message)s' -) -logger = logging.getLogger("multi_search_example") - -def print_search_results(results: List[Dict[str, Any]], max_results: int = 3) -> None: - """ - Print search results in a readable format. - - Args: - results: List of search result dictionaries - max_results: Maximum number of results to print - """ - for i, result in enumerate(results[:max_results]): - snippet = result.get('snippet', 'No snippet') - - if len(results) > max_results: - -def print_perplexity_results(results: Dict[str, Any]) -> None: - """ - Print Perplexity search results in a readable format. - - Args: - results: Perplexity search result dictionary - """ - content = results.get("content", "No content") - - citations = results.get("citations", []) - if citations: - for i, citation in enumerate(citations[:3]): - - if len(citations) > 3: - -def main() -> None: - """Run the multi-search example.""" - - serper_api_key = os.environ.get("SERPER_API_KEY") - perplexity_api_key = os.environ.get("PERPLEXITY_API_KEY") - - - registry = create_default_registry() - - query = "What is the latest version of Python and when was it released?" - - - if registry.get_tool("duckduckgo_search"): - try: - results = registry.execute_tool("duckduckgo_search", query=query) - print_search_results(results) - except Exception as e: - else: - - if registry.get_tool("serper_search"): - try: - results = registry.execute_tool("serper_search", query=query) - print_search_results(results) - except Exception as e: - else: - - if registry.get_tool("perplexity_search"): - try: - results = registry.execute_tool("perplexity_search", query=query) - print_perplexity_results(results) - except Exception as e: - else: - - - try: - result = robust_search(registry, query) - print_search_results(result['results']) - except Exception as e: - - - try: - result = merged_search(registry, query) - - print_search_results(result['merged_results']) - - if result["perplexity_context"]: - else: - except Exception as e: - - - try: - academic_query = "large language models" - - if registry.get_tool("arxiv_search"): - try: - results = registry.execute_tool("arxiv_search", query=academic_query, max_results=3) - - for i, paper in enumerate(results): - summary = paper['summary'] - except Exception as e: - else: - - if registry.get_tool("arxiv_get_paper"): - try: - paper_id = "2303.08774" # "A Survey of Large Language Models" paper - - paper = registry.execute_tool("arxiv_get_paper", paper_id=paper_id) - except Exception as e: - - if registry.get_tool("arxiv_search_category"): - try: - category = "cs.AI" # Artificial Intelligence - - results = registry.execute_tool("arxiv_search_category", category=category, max_results=2) - - for i, paper in enumerate(results): - except Exception as e: - except Exception as e: - - -if __name__ == "__main__": - main() \ No newline at end of file diff --git a/examples/unified_search_example.py b/examples/unified_search_example.py deleted file mode 100644 index ad5e66bb8163ef30e3919a83f35949246e982668..0000000000000000000000000000000000000000 --- a/examples/unified_search_example.py +++ /dev/null @@ -1,373 +0,0 @@ -#!/usr/bin/env python -""" -Unified Search Example - -This script demonstrates how to use all three search tools: -1. BrowserSearchTool for direct browser-based searches -2. LibrarySearchTool for searches using imported libraries -3. ApiSearchTool for searches using external API services - -The example includes: -- Importing and initializing each tool -- Example queries for each tool type (academic, complex reasoning, simple factual) -- Demonstrating how to use them individually and together -- Showing how to interpret and process the results -""" - -import os -import json -import logging -from typing import Dict, Any, List, Optional - -# Configure logging -logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s') -logger = logging.getLogger("unified_search_example") - -# Import the tool registry and search tools -from agent.tool_registry import ToolRegistry, create_default_registry -from tools.web_tools import ( - LibrarySearchTool, - ApiSearchTool, - create_library_search, - create_api_search -) -from tools.multimodal_tools import BrowserSearchTool, create_browser_search_tool - -def print_section(title: str) -> None: - """Print a section title with formatting.""" - print("\n" + "=" * 80) - print(f" {title} ".center(80, "=")) - print("=" * 80 + "\n") - -def print_results(results: List[Dict[str, Any]], max_snippet_length: int = 150) -> None: - """ - Print search results in a readable format. - - Args: - results: List of search result dictionaries - max_snippet_length: Maximum length for snippet display - """ - if not results: - print("No results found.") - return - - for i, result in enumerate(results): - print(f"Result {i+1}:") - print(f" Title: {result.get('title', 'No title')}") - print(f" Link: {result.get('link', 'No link')}") - - # Format snippet for display - snippet = result.get('snippet', 'No snippet') - if len(snippet) > max_snippet_length: - snippet = snippet[:max_snippet_length] + "..." - print(f" Snippet: {snippet}") - - # Print source and relevance if available - if 'source' in result: - print(f" Source: {result['source']}") - if 'relevance_score' in result: - print(f" Relevance: {result['relevance_score']}") - - print() - -def example_browser_search() -> None: - """ - Demonstrate the BrowserSearchTool for direct browser-based searches. - - This tool is useful for: - - Visual content that needs to be seen directly - - Interactive exploration of search results - - Searching specific websites with their native interfaces - """ - print_section("BrowserSearchTool Example") - - # Initialize the BrowserSearchTool - browser_tool = create_browser_search_tool() - print("BrowserSearchTool initialized") - - # Example 1: General web search - print("\n1. General web search (Google):") - query = "latest advancements in quantum computing" - results = browser_tool.search(query, site="google") - print_results(results) - print("To use these results, you would use the browser_action tool to open the URL and interact with the search results directly.") - - # Example 2: Academic search on arXiv - print("\n2. Academic search on arXiv:") - academic_query = "transformer models in natural language processing" - results = browser_tool.search(academic_query, site="arxiv") - print_results(results) - print("The browser_action tool would allow you to view papers, download PDFs, and explore citations.") - - # Example 3: Wikipedia search for factual information - print("\n3. Wikipedia search for factual information:") - factual_query = "history of artificial intelligence" - results = browser_tool.search(factual_query, site="wikipedia") - print_results(results) - print("Using browser_action would let you navigate through Wikipedia articles and follow links to related topics.") - - # Example 4: Direct URL visit - print("\n4. Direct URL visit:") - url = "https://arxiv.org/abs/2303.08774" # Example arXiv paper - result = browser_tool.direct_visit(url) - print(f"Title: {result.get('title')}") - print(f"Link: {result.get('link')}") - print(f"Instructions: {result.get('instructions')}") - -def example_library_search() -> None: - """ - Demonstrate the LibrarySearchTool for searches using imported libraries. - - This tool is useful for: - - Programmatic searches without requiring a browser - - Automatically routing between general and academic sources - - Getting structured results that can be processed programmatically - """ - print_section("LibrarySearchTool Example") - - # Initialize the LibrarySearchTool - library_tool = create_library_search() - print("LibrarySearchTool initialized") - - # Example 1: General factual query (routes to DuckDuckGo) - print("\n1. General factual query (routes to DuckDuckGo):") - factual_query = "population of Tokyo in 2023" - print(f"Query: '{factual_query}'") - print("Analyzing query type...") - is_academic = library_tool._is_academic_query(factual_query) - print(f"Is academic query? {is_academic}") - print("Routing to appropriate search provider...") - results = library_tool.search(factual_query) - print_results(results) - - # Example 2: Academic query (routes to arXiv) - print("\n2. Academic query (routes to arXiv):") - academic_query = "recent papers on large language models" - print(f"Query: '{academic_query}'") - print("Analyzing query type...") - is_academic = library_tool._is_academic_query(academic_query) - print(f"Is academic query? {is_academic}") - print("Routing to appropriate search provider...") - results = library_tool.search(academic_query) - print_results(results) - - # Example 3: Processing and analyzing results - print("\n3. Processing and analyzing results:") - query = "climate change impacts" - results = library_tool.search(query) - - # Demonstrate how to extract and use specific information - if results: - # Count sources - sources = {} - for result in results: - source = result.get('source', 'unknown') - sources[source] = sources.get(source, 0) + 1 - - print(f"Query: '{query}'") - print(f"Total results: {len(results)}") - print(f"Results by source: {sources}") - - # Find highest relevance result - if any('relevance_score' in r for r in results): - most_relevant = max(results, key=lambda r: r.get('relevance_score', 0)) - print(f"Most relevant result: {most_relevant.get('title')} (Score: {most_relevant.get('relevance_score')})") - - # Extract all unique domains from results - domains = set() - for result in results: - link = result.get('link', '') - if link.startswith('http'): - from urllib.parse import urlparse - domain = urlparse(link).netloc - domains.add(domain) - - print(f"Unique domains in results: {domains}") - -def example_api_search() -> None: - """ - Demonstrate the ApiSearchTool for searches using external API services. - - This tool is useful for: - - Complex reasoning queries that benefit from AI-powered search - - Getting high-quality, synthesized results for complex topics - - Accessing real-time information from search engines - - Note: Requires API keys for Perplexity and/or Serper - """ - print_section("ApiSearchTool Example") - - # Check if API keys are available - perplexity_key = os.environ.get("PERPLEXITY_API_KEY") - serper_key = os.environ.get("SERPER_API_KEY") - - if not perplexity_key and not serper_key: - print("Warning: No API keys found for Perplexity or Serper.") - print("Set PERPLEXITY_API_KEY and/or SERPER_API_KEY environment variables to use ApiSearchTool.") - print("Continuing with limited functionality...\n") - - # Initialize the ApiSearchTool - api_tool = create_api_search() - print("ApiSearchTool initialized") - - # Example 1: Complex reasoning query (routes to Perplexity if available) - print("\n1. Complex reasoning query (routes to Perplexity if available):") - complex_query = "Explain the implications of quantum computing on modern cryptography" - print(f"Query: '{complex_query}'") - print("Analyzing query complexity...") - is_complex = api_tool._is_complex_query(complex_query) - print(f"Is complex query? {is_complex}") - print("Routing to appropriate API service...") - - try: - results = api_tool.search(complex_query) - print_results(results) - - # Show how to access full content if available - if results and 'full_content' in results[0]: - print("Full content is available for this result.") - print("First 200 characters of full content:") - print(results[0]['full_content'][:200] + "...") - except Exception as e: - print(f"Error: {str(e)}") - print("This may be due to missing API keys or API service issues.") - - # Example 2: Simple factual query (routes to Serper if available) - print("\n2. Simple factual query (routes to Serper if available):") - factual_query = "height of Mount Everest" - print(f"Query: '{factual_query}'") - print("Analyzing query complexity...") - is_complex = api_tool._is_complex_query(factual_query) - print(f"Is complex query? {is_complex}") - print("Routing to appropriate API service...") - - try: - results = api_tool.search(factual_query) - print_results(results) - except Exception as e: - print(f"Error: {str(e)}") - print("This may be due to missing API keys or API service issues.") - - # Example 3: Working with citations (if available from Perplexity) - print("\n3. Working with citations (if available from Perplexity):") - citation_query = "recent breakthroughs in fusion energy research" - - try: - results = api_tool.search(citation_query) - - for result in results: - if 'citations' in result: - print(f"Result includes {len(result['citations'])} citations:") - for i, citation in enumerate(result['citations'][:3]): # Show first 3 citations - print(f" Citation {i+1}: {citation.get('title', 'No title')} - {citation.get('url', 'No URL')}") - except Exception as e: - print(f"Error: {str(e)}") - print("This may be due to missing API keys or API service issues.") - -def example_unified_search() -> None: - """ - Demonstrate how to use all three search tools together through the tool registry. - - This example shows: - - How to use the tool registry to access all search tools - - How to choose the right tool based on query type - - How to combine results from multiple tools - """ - print_section("Unified Search Example") - - # Create a tool registry with all search tools - registry = create_default_registry() - print("Tool registry created with the following search tools:") - search_tools = [tool for tool in registry.list_tools() if 'search' in tool] - for tool in search_tools: - print(f"- {tool}") - - # Example 1: Using the registry to execute specific tools - print("\n1. Using the registry to execute specific tools:") - query = "renewable energy technologies" - - print(f"\nQuery: '{query}'") - print("Executing with different search tools:") - - # Try different search tools and compare results - results_by_tool = {} - for tool_name in ['duckduckgo_search', 'library_search', 'enhanced_web_search']: - if registry.get_tool(tool_name): - try: - print(f"\nExecuting {tool_name}...") - results = registry.execute_tool(tool_name, query=query) - results_by_tool[tool_name] = results - print(f"Got {len(results)} results from {tool_name}") - except Exception as e: - print(f"Error with {tool_name}: {str(e)}") - - # Example 2: Using the unified_search function from tool_registry - print("\n2. Using the unified_search function:") - from agent.tool_registry import unified_search - - query = "impact of artificial intelligence on healthcare" - print(f"\nQuery: '{query}'") - print("Executing unified_search...") - - try: - unified_results = unified_search(registry, query) - print(f"Got results from the following providers: {unified_results['metadata']['providers_used']}") - print(f"Total results: {len(unified_results['results'])}") - print("\nTop 2 results:") - print_results(unified_results['results'][:2]) - - # Show execution times - print("Execution times:") - for provider, time in unified_results['metadata']['execution_times'].items(): - print(f"- {provider}: {time:.2f} seconds") - except Exception as e: - print(f"Error with unified_search: {str(e)}") - - # Example 3: Combining and ranking results from multiple tools - print("\n3. Combining and ranking results from multiple tools:") - - # Function to combine results from multiple tools - def combine_and_rank_results(results_by_tool): - all_results = [] - for tool, results in results_by_tool.items(): - # Add tool name to each result if not already present - for result in results: - if 'source' not in result: - result['source'] = tool - all_results.append(result) - - # Sort by relevance score if available - if all_results and 'relevance_score' in all_results[0]: - all_results.sort(key=lambda x: x.get('relevance_score', 0), reverse=True) - - return all_results - - combined_results = combine_and_rank_results(results_by_tool) - print(f"Combined {len(combined_results)} results from {len(results_by_tool)} tools") - print("\nTop 3 results after combining and ranking:") - print_results(combined_results[:3]) - -def main(): - """Run all examples.""" - print_section("Unified Search Examples") - print("This script demonstrates how to use all three search tools:") - print("1. BrowserSearchTool - For direct browser-based searches") - print("2. LibrarySearchTool - For searches using imported libraries") - print("3. ApiSearchTool - For searches using external API services") - - # Run individual examples - example_browser_search() - example_library_search() - example_api_search() - example_unified_search() - - print_section("Conclusion") - print("This example demonstrated how to use all three search tools:") - print("1. BrowserSearchTool - Best for visual content and interactive exploration") - print("2. LibrarySearchTool - Best for programmatic searches without API keys") - print("3. ApiSearchTool - Best for complex reasoning queries with high-quality results") - print("\nThese tools can be used individually or together through the tool registry.") - print("Choose the right tool based on your specific search needs and available resources.") - -if __name__ == "__main__": - main() \ No newline at end of file diff --git a/final_deployment_guide.md b/final_deployment_guide.md deleted file mode 100644 index 69fc410e1b075408f0dbc09de4b48ecf487dce22..0000000000000000000000000000000000000000 --- a/final_deployment_guide.md +++ /dev/null @@ -1,150 +0,0 @@ -# Complete GAIA Agent Deployment Guide for Hugging Face Space - -This comprehensive guide addresses all issues encountered when deploying the GAIA agent to Hugging Face Space: - -1. OAuth configuration error -2. Dependency conflicts in requirements.txt -3. Runtime error with missing import - -## Step 1: Fix OAuth Configuration - -1. Navigate to your Space on Hugging Face -2. Click on the "Files" tab -3. Open **README.md** -4. Edit the file to include this YAML frontmatter at the top: - -```yaml ---- -title: GAIA Agent Assessment -emoji: 🤖 -colorFrom: blue -colorTo: green -sdk: gradio -sdk_version: 3.50.2 -app_file: app.py -pinned: false -license: mit -hf_oauth: true # This enables OAuth! ---- -``` - -5. Click "Commit changes" - -## Step 2: Fix Dependency Conflicts - -1. Navigate to your Space's "Files" tab -2. Open **requirements.txt** -3. Replace with this compatible configuration: - -``` -# Fixed requirements with langchain ecosystem compatibility -# Main dependencies -gradio==3.50.2 -requests==2.31.0 - -# Langchain ecosystem - specific versions known to work together -langchain==0.1.0 # Updated to ensure compatibility with langchain-core -langchain-core==0.1.17 # Version compatible with langchain 0.1.0 -langchain_openai==0.0.5 # Updated to work with newer langchain -langgraph==0.0.24 # Updated to work with newer langchain packages - -# OpenAI -openai==1.6.1 # This version works with langchain_openai 0.0.5 - -# HTTP Libraries -httpx==0.24.1 # Compatible with supabase 1.2.0 and openai 1.6.1 - -# Database and storage -supabase==1.2.0 - -# Search tools - downgraded to avoid httpx conflict -duckduckgo-search==3.8.3 - -# Parsing and content handling -beautifulsoup4==4.12.2 -pillow==10.0.1 -arxiv==1.4.8 -python-dotenv==1.0.0 - -# Additional dependencies -pydantic==1.10.13 -numpy==1.24.4 -pandas==2.0.3 -``` - -4. Click "Commit changes" - -## Step 3: Fix Runtime Import Error - -1. Navigate to your Space's "Files" tab -2. Open **agent/graph.py** -3. Find and modify line 22: - - FROM: `from langgraph.prebuilt import ToolNode` - - TO: `# Removed problematic import: from langgraph.prebuilt import ToolNode` -4. Click "Commit changes" - -Alternatively, you can upload and run the fix_graph_imports.py script: - -1. Upload fix_graph_imports.py to your Space -2. In the Space "Settings" tab, under "Shell", run: - ``` - python fix_graph_imports.py - ``` - -## Step 4: Configure Environment Variables - -1. Go to the "Settings" tab of your Space -2. Scroll to "Repository secrets" -3. Add the following environment variables: - - `OPENAI_API_KEY`: Your OpenAI API key - - `SUPABASE_URL`: Your Supabase project URL - - `SUPABASE_KEY`: Your Supabase API key - - `SERPER_API_KEY`: (Optional) Your Serper API key - -## Step 5: Restart Your Space - -1. Still in "Settings", scroll to "Factory reboot" -2. Click "Restart Space" to apply all changes - -## Verification Steps - -After restarting: -1. Check the build logs to confirm successful installation -2. Verify the app loads correctly in the browser -3. Test the OAuth login functionality -4. Run a test evaluation to ensure all components work - -## Summary of Issues Fixed - -### 1. OAuth Configuration Issue -- **Problem**: Hugging Face Space requires OAuth for user identification in GAIA assessment -- **Solution**: Added `hf_oauth: true` to Space metadata in README.md - -### 2. Dependency Conflicts -- **Problem**: Multiple package version conflicts, particularly with httpx: - - supabase 1.2.0 requires httpx<0.25.0,>=0.24.0 - - openai 1.6.1 requires httpx<1,>=0.23.0 - - duckduckgo-search 3.9.5 requires httpx>=0.25.1 -- **Solution**: Updated package versions to be compatible, downgraded certain packages - -### 3. Runtime Import Error -- **Problem**: `ModuleNotFoundError: No module named 'langchain.tools.render'` -- **Solution**: Removed the unused import `from langgraph.prebuilt import ToolNode` - and updated langchain ecosystem packages to compatible versions - -## Troubleshooting Common Issues - -### If Dependency Conflicts Persist -Try these alternative approaches: -1. **Further simplify dependencies**: - - Remove optional packages temporarily - - Add them back one by one to identify conflicts - -2. **Use separate requirements files**: - - Create a base requirements.txt with core dependencies - - Use post-installation scripts to install conflicting packages separately - -### If Runtime Errors Continue -1. Check the Space logs for specific error messages -2. Look for other imports that might be causing issues -3. Consider temporarily commenting out features until you identify the problematic component \ No newline at end of file diff --git a/final_verification_summary.md b/final_verification_summary.md deleted file mode 100644 index 0119f731646dcfb7a2f861cf2cd94ca503369acb..0000000000000000000000000000000000000000 --- a/final_verification_summary.md +++ /dev/null @@ -1,85 +0,0 @@ -# GAIA Agent Final Verification Summary - -## Pre-Deployment Verification Process - -This summary documents the comprehensive verification process completed prior to deploying the GAIA Agent to Hugging Face Space. - -## Components Verified - -| Component | Status | Notes | -|-----------|--------|-------| -| Gradio Configuration | ✅ Fixed | Updated from 3.50.2 to 5.25.2 in README.md | -| Package Dependencies | ✅ Fixed | Removed unavailable `perplexity-python` dependency | -| Code Syntax | ✅ Fixed | Repaired syntax errors in app.py and verify_gaia_assessment.py | -| Environment Variables | ⚠️ Required | Documented required environment variables for HF Space | -| API Integration | 🔄 Pending | Requires valid API keys for verification | -| Memory Integration | 🔄 Pending | Requires Supabase connection for verification | - -## Issues Addressed - -### Build Issues -1. ✅ Updated Gradio version in README.md to match requirements.txt (5.25.2) -2. ✅ Fixed dependency issue with non-existent `perplexity-python` package -3. ✅ Fixed syntax errors in application code (app.py) -4. ✅ Fixed syntax errors in verification script (verify_gaia_assessment.py) - -### Runtime Issues -1. ⚠️ Environment variables need to be configured in Hugging Face Space -2. 🔄 API integrations need to be tested after deployment - -## Deployment Preparation - -The following resources have been created to facilitate deployment: - -1. **Production Verification Report** (`production_verification_report.md`) - - Detailed documentation of issues found and fixed - - Requirements for proper functioning of the application - -2. **Deployment Instructions** (`hf_deployment_instructions.md`) - - Step-by-step guide for deploying to Hugging Face Space - - Instructions for configuring environment variables - -3. **Deployment Script** (`deploy_to_hf.py`) - - Automated project structure validation - - Generation of deployment commands - - Template environment variable configuration - -## Required Post-Deployment Tests - -After deploying to Hugging Face Space with proper environment variables, the following tests should be performed: - -1. **OpenAI Integration Test** - ```python - python test_openai_minimal.py - ``` - -2. **Supabase Integration Test** - ```python - python test_supabase_minimal.py - ``` - -3. **Serper Integration Test** - ```python - python test_serper_minimal.py - ``` - -4. **Full Verification Test** - ```python - python verify_gaia_assessment.py --verbose - ``` - -## Performance Considerations - -- The agent leverages caching to improve response times for repeated questions -- Full functionality requires all API integrations to be working correctly -- Memory components increase response quality but may increase latency - -## Security Considerations - -- All API keys must be stored as Hugging Face Space secrets -- No credentials should be hardcoded in the codebase -- Memory contents should be regularly audited for sensitive information - -## Conclusion - -The GAIA Agent is ready for deployment to Hugging Face Space. All identified build and syntax issues have been fixed. The application will require proper environment variable configuration to function correctly. Post-deployment testing is recommended to ensure all components work as expected in the production environment. \ No newline at end of file diff --git a/fix_encoding.py b/fix_encoding.py deleted file mode 100644 index 021accc1554d48a0b137bfa7296830d204d99ca8..0000000000000000000000000000000000000000 --- a/fix_encoding.py +++ /dev/null @@ -1,87 +0,0 @@ -""" -Script to find and fix Unicode characters in app.py -""" -import re -import os - -def find_unicode_chars(file_path): - try: - with open(file_path, 'r', encoding='utf-8') as f: - content = f.read() - - # Find all non-ASCII characters - unicode_chars = set(re.findall(r'[^\x00-\x7F]', content)) - - if unicode_chars: - print(f"Found {len(unicode_chars)} Unicode characters in {file_path}:") - for char in unicode_chars: - print(f" '{char}' (U+{ord(char):04X})") - return unicode_chars - else: - print(f"No Unicode characters found in {file_path}") - return None - except Exception as e: - print(f"Error reading {file_path}: {e}") - return None - -def replace_unicode_chars(file_path, output_path): - try: - with open(file_path, 'r', encoding='utf-8') as f: - content = f.read() - - # Replace common Unicode characters with ASCII alternatives - replacements = { - '✓': '[PASS]', - '✗': '[FAIL]', - '❌': '[ERROR]', - '✅': '[SUCCESS]', - '⚠️': '[WARNING]', - '→': '->', - '←': '<-', - '…': '...', - '–': '-', - '—': '--', - ''': "'", - ''': "'", - '"': '"', - '"': '"', - '•': '*', - '©': '(c)', - '®': '(R)', - '™': '(TM)', - '°': ' degrees', - '±': '+/-', - '×': 'x', - '÷': '/', - '≤': '<=', - '≥': '>=' - } - - for char, replacement in replacements.items(): - content = content.replace(char, replacement) - - # Replace any remaining non-ASCII characters with their Unicode escape sequence - def replace_non_ascii(match): - char = match.group(0) - return f"[U+{ord(char):04X}]" - - content = re.sub(r'[^\x00-\x7F]', replace_non_ascii, content) - - with open(output_path, 'w', encoding='utf-8') as f: - f.write(content) - - print(f"Fixed file saved to {output_path}") - return True - except Exception as e: - print(f"Error fixing {file_path}: {e}") - return False - -if __name__ == "__main__": - input_file = "app.py" - output_file = "app_fixed.py" - - print(f"Checking {input_file} for Unicode characters...") - find_unicode_chars(input_file) - - print(f"\nReplacing Unicode characters in {input_file}...") - replace_unicode_chars(input_file, output_file) \ No newline at end of file diff --git a/fix_graph_imports.py b/fix_graph_imports.py deleted file mode 100644 index 1aae4d3231f96494ee7b90cffe275f67fdcba3a9..0000000000000000000000000000000000000000 --- a/fix_graph_imports.py +++ /dev/null @@ -1,64 +0,0 @@ -#!/usr/bin/env python -""" -Fix for the ModuleNotFoundError: No module named 'langchain.tools.render' error. - -This script patches the agent/graph.py file to remove the unused import that's -causing the runtime error. -""" - -import os -import re -import sys -import logging - -logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s') -logger = logging.getLogger(__name__) - -def patch_graph_file(): - """Patch the agent/graph.py file to remove the problematic import.""" - graph_file = "agent/graph.py" - - if not os.path.exists(graph_file): - logger.error(f"Could not find {graph_file}") - return False - - try: - with open(graph_file, "r") as f: - content = f.read() - - # Create a backup - backup_file = f"{graph_file}.bak" - with open(backup_file, "w") as f: - f.write(content) - logger.info(f"Created backup at {backup_file}") - - # Remove the problematic import - pattern = r"from langgraph\.prebuilt import ToolNode\n" - updated_content = re.sub(pattern, "# Removed problematic import: from langgraph.prebuilt import ToolNode\n", content) - - with open(graph_file, "w") as f: - f.write(updated_content) - - logger.info(f"Successfully patched {graph_file}") - return True - - except Exception as e: - logger.error(f"Error patching file: {str(e)}") - return False - -def main(): - """Main function to run the patch.""" - logger.info("Starting graph.py import patch...") - - if patch_graph_file(): - logger.info("Patch applied successfully!") - logger.info("This fixes the 'ModuleNotFoundError: No module named 'langchain.tools.render'' error") - logger.info("by removing an unused import from agent/graph.py") - else: - logger.error("Failed to apply patch") - return 1 - - return 0 - -if __name__ == "__main__": - sys.exit(main()) \ No newline at end of file diff --git a/gaia_agent.log b/gaia_agent.log deleted file mode 100644 index 0234db0232c0c1aec1a215fa4cb00c47b594180b..0000000000000000000000000000000000000000 --- a/gaia_agent.log +++ /dev/null @@ -1,1993 +0,0 @@ -2025-04-30 17:20:40,146 - test_duckduckgo - INFO - DuckDuckGo search tool initialized -2025-04-30 17:20:40,148 - test_duckduckgo - INFO - Successfully imported duckduckgo-search package -2025-04-30 17:20:40,161 - test_duckduckgo - INFO - Successfully created DDGS instance -2025-04-30 17:20:40,868 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 200 -2025-04-30 17:20:40,870 - test_duckduckgo - INFO - Successfully retrieved 2 results from DuckDuckGo -2025-04-30 17:20:40,870 - test_duckduckgo - INFO - Result 1: SQL Fiddle - Online SQL Compiler for learning & practice - https://sqlfiddle.com/ -2025-04-30 17:20:40,870 - test_duckduckgo - INFO - Result 2: DB Fiddle - SQL Database Playground - https://www.db-fiddle.com/ -2025-04-30 17:20:40,870 - test_duckduckgo - INFO - DuckDuckGo search tool initialized -2025-04-30 17:20:41,630 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 200 -2025-04-30 17:20:41,633 - test_duckduckgo - INFO - Successfully retrieved 5 results using DuckDuckGoSearchTool -2025-04-30 17:20:41,633 - test_duckduckgo - INFO - Result format is correct: dict_keys(['title', 'link', 'snippet']) -2025-04-30 17:20:41,633 - test_duckduckgo - INFO - First result: Welcome to Python.org - https://www.python.org/ -2025-04-30 17:20:41,634 - test_duckduckgo - INFO - DuckDuckGo search tool initialized -2025-04-30 17:20:41,634 - duckduckgo_search.DDGS - INFO - Error to search using lite backend: keywords is mandatory -2025-04-30 17:20:41,634 - duckduckgo_search.DDGS - INFO - Error to search using html backend: keywords is mandatory -2025-04-30 17:20:41,634 - gaia_agent.tools.web - ERROR - Error searching DuckDuckGo: keywords is mandatory -2025-04-30 17:20:41,636 - gaia_agent.tools.web - ERROR - Traceback (most recent call last): - File "c:\Users\joach\OneDrive - VISCONSULT\00_VISCONSULT\05_AI_and_Digitalization\02_AI_Agents\02_HF_Agents_Course\99_Final_Assignment\tools\web_tools.py", line 131, in search - results = list(ddgs.text( - ~~~~~~~~~^ - query, - ^^^^^^ - max_results=self.max_results, - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - timelimit=self.ddg_timeout - ^^^^^^^^^^^^^^^^^^^^^^^^^^ - )) - ^ - File "C:\Users\joach\AppData\Roaming\Python\Python313\site-packages\duckduckgo_search\duckduckgo_search.py", line 184, in text - raise DuckDuckGoSearchException(err) -duckduckgo_search.exceptions.DuckDuckGoSearchException: keywords is mandatory - -2025-04-30 17:20:41,714 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 302 -2025-04-30 17:20:41,715 - duckduckgo_search.DDGS - INFO - Error to search using lite backend: https://lite.duckduckgo.com/lite/ return None. params=None content=None data={'q': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', 'kl': 'wt-wt', 'df': 10} -2025-04-30 17:20:42,550 - primp - INFO - response: https://html.duckduckgo.com/html 302 -2025-04-30 17:20:42,550 - duckduckgo_search.DDGS - INFO - Error to search using html backend: https://html.duckduckgo.com/html return None. params=None content=None data={'q': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', 'b': '', 'kl': 'wt-wt', 'df': 10} -2025-04-30 17:20:42,551 - gaia_agent.tools.web - ERROR - Error searching DuckDuckGo: https://html.duckduckgo.com/html return None. params=None content=None data={'q': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', 'b': '', 'kl': 'wt-wt', 'df': 10} -2025-04-30 17:20:42,552 - gaia_agent.tools.web - ERROR - Traceback (most recent call last): - File "c:\Users\joach\OneDrive - VISCONSULT\00_VISCONSULT\05_AI_and_Digitalization\02_AI_Agents\02_HF_Agents_Course\99_Final_Assignment\tools\web_tools.py", line 131, in search - results = list(ddgs.text( - ~~~~~~~~~^ - query, - ^^^^^^ - max_results=self.max_results, - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - timelimit=self.ddg_timeout - ^^^^^^^^^^^^^^^^^^^^^^^^^^ - )) - ^ - File "C:\Users\joach\AppData\Roaming\Python\Python313\site-packages\duckduckgo_search\duckduckgo_search.py", line 184, in text - raise DuckDuckGoSearchException(err) -duckduckgo_search.exceptions.DuckDuckGoSearchException: https://html.duckduckgo.com/html return None. params=None content=None data={'q': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', 'b': '', 'kl': 'wt-wt', 'df': 10} - -2025-04-30 17:20:42,552 - test_duckduckgo - WARNING - Long query caused an error: DuckDuckGo search failed: https://html.duckduckgo.com/html return None. params=None content=None data={'q': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', 'b': '', 'kl': 'wt-wt', 'df': 10} -2025-04-30 17:20:42,552 - test_duckduckgo - INFO - DuckDuckGo search tool initialized -2025-04-30 17:20:43,397 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 200 -2025-04-30 17:20:43,400 - test_duckduckgo - INFO - Result count configuration works, got 3 results -2025-04-30 17:21:04,285 - test_duckduckgo - INFO - DuckDuckGo search tool initialized -2025-04-30 17:21:04,286 - test_duckduckgo - INFO - Successfully imported duckduckgo-search package -2025-04-30 17:21:04,295 - test_duckduckgo - INFO - Successfully created DDGS instance -2025-04-30 17:21:05,087 - primp - INFO - response: https://html.duckduckgo.com/html 200 -2025-04-30 17:21:05,090 - test_duckduckgo - INFO - Successfully retrieved 2 results from DuckDuckGo -2025-04-30 17:21:05,090 - test_duckduckgo - INFO - Result 1: SQL Fiddle - Online SQL Compiler for learning & practice - https://sqlfiddle.com/ -2025-04-30 17:21:05,090 - test_duckduckgo - INFO - Result 2: SQL Test - https://sqltest.net/ -2025-04-30 17:21:05,090 - test_duckduckgo - INFO - DuckDuckGo search tool initialized -2025-04-30 17:21:05,915 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 200 -2025-04-30 17:21:05,917 - test_duckduckgo - INFO - Successfully retrieved 5 results using DuckDuckGoSearchTool -2025-04-30 17:21:05,918 - test_duckduckgo - INFO - Result format is correct: dict_keys(['title', 'link', 'snippet']) -2025-04-30 17:21:05,918 - test_duckduckgo - INFO - First result: Welcome to Python.org - https://www.python.org/ -2025-04-30 17:21:05,918 - test_duckduckgo - INFO - DuckDuckGo search tool initialized -2025-04-30 17:21:05,918 - duckduckgo_search.DDGS - INFO - Error to search using lite backend: keywords is mandatory -2025-04-30 17:21:05,919 - duckduckgo_search.DDGS - INFO - Error to search using html backend: keywords is mandatory -2025-04-30 17:21:05,919 - gaia_agent.tools.web - ERROR - Error searching DuckDuckGo: keywords is mandatory -2025-04-30 17:21:05,921 - gaia_agent.tools.web - ERROR - Traceback (most recent call last): - File "c:\Users\joach\OneDrive - VISCONSULT\00_VISCONSULT\05_AI_and_Digitalization\02_AI_Agents\02_HF_Agents_Course\99_Final_Assignment\tools\web_tools.py", line 131, in search - results = list(ddgs.text( - ~~~~~~~~~^ - query, - ^^^^^^ - max_results=self.max_results, - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - timelimit=self.ddg_timeout - ^^^^^^^^^^^^^^^^^^^^^^^^^^ - )) - ^ - File "C:\Users\joach\AppData\Roaming\Python\Python313\site-packages\duckduckgo_search\duckduckgo_search.py", line 184, in text - raise DuckDuckGoSearchException(err) -duckduckgo_search.exceptions.DuckDuckGoSearchException: keywords is mandatory - -2025-04-30 17:21:06,024 - primp - INFO - response: https://html.duckduckgo.com/html 302 -2025-04-30 17:21:06,025 - duckduckgo_search.DDGS - INFO - Error to search using html backend: https://html.duckduckgo.com/html return None. params=None content=None data={'q': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', 'b': '', 'kl': 'wt-wt', 'df': 10} -2025-04-30 17:21:06,896 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 302 -2025-04-30 17:21:06,896 - duckduckgo_search.DDGS - INFO - Error to search using lite backend: https://lite.duckduckgo.com/lite/ return None. params=None content=None data={'q': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', 'kl': 'wt-wt', 'df': 10} -2025-04-30 17:21:06,896 - gaia_agent.tools.web - ERROR - Error searching DuckDuckGo: https://lite.duckduckgo.com/lite/ return None. params=None content=None data={'q': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', 'kl': 'wt-wt', 'df': 10} -2025-04-30 17:21:06,897 - gaia_agent.tools.web - ERROR - Traceback (most recent call last): - File "c:\Users\joach\OneDrive - VISCONSULT\00_VISCONSULT\05_AI_and_Digitalization\02_AI_Agents\02_HF_Agents_Course\99_Final_Assignment\tools\web_tools.py", line 131, in search - results = list(ddgs.text( - ~~~~~~~~~^ - query, - ^^^^^^ - max_results=self.max_results, - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - timelimit=self.ddg_timeout - ^^^^^^^^^^^^^^^^^^^^^^^^^^ - )) - ^ - File "C:\Users\joach\AppData\Roaming\Python\Python313\site-packages\duckduckgo_search\duckduckgo_search.py", line 184, in text - raise DuckDuckGoSearchException(err) -duckduckgo_search.exceptions.DuckDuckGoSearchException: https://lite.duckduckgo.com/lite/ return None. params=None content=None data={'q': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', 'kl': 'wt-wt', 'df': 10} - -2025-04-30 17:21:06,897 - test_duckduckgo - WARNING - Long query caused an error: DuckDuckGo search failed: https://lite.duckduckgo.com/lite/ return None. params=None content=None data={'q': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', 'kl': 'wt-wt', 'df': 10} -2025-04-30 17:21:06,898 - test_duckduckgo - INFO - DuckDuckGo search tool initialized -2025-04-30 17:21:07,735 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 200 -2025-04-30 17:21:07,737 - test_duckduckgo - INFO - Result count configuration works, got 3 results -2025-04-30 17:21:41,355 - test_duckduckgo - INFO - DuckDuckGo search tool initialized -2025-04-30 17:21:41,355 - test_duckduckgo - INFO - Successfully imported duckduckgo-search package -2025-04-30 17:21:41,367 - test_duckduckgo - INFO - Successfully created DDGS instance -2025-04-30 17:21:42,105 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 200 -2025-04-30 17:21:42,107 - test_duckduckgo - INFO - Successfully retrieved 2 results from DuckDuckGo -2025-04-30 17:21:42,107 - test_duckduckgo - INFO - Result 1: SQL Fiddle - Online SQL Compiler for learning & practice - https://sqlfiddle.com/ -2025-04-30 17:21:42,107 - test_duckduckgo - INFO - Result 2: Learn SQL - Online SQL Terminal - Practice SQL Querys - https://www.sql-practice.com/ -2025-04-30 17:21:42,107 - test_duckduckgo - INFO - DuckDuckGo search tool initialized -2025-04-30 17:21:42,878 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 200 -2025-04-30 17:21:42,880 - test_duckduckgo - INFO - Successfully retrieved 5 results using DuckDuckGoSearchTool -2025-04-30 17:21:42,880 - test_duckduckgo - INFO - Result format is correct: dict_keys(['title', 'link', 'snippet']) -2025-04-30 17:21:42,880 - test_duckduckgo - INFO - First result: Welcome to Python.org - https://www.python.org/ -2025-04-30 17:21:42,880 - test_duckduckgo - INFO - DuckDuckGo search tool initialized -2025-04-30 17:21:42,881 - duckduckgo_search.DDGS - INFO - Error to search using lite backend: keywords is mandatory -2025-04-30 17:21:42,881 - duckduckgo_search.DDGS - INFO - Error to search using html backend: keywords is mandatory -2025-04-30 17:21:42,881 - gaia_agent.tools.web - ERROR - Error searching DuckDuckGo: keywords is mandatory -2025-04-30 17:21:42,882 - gaia_agent.tools.web - ERROR - Traceback (most recent call last): - File "c:\Users\joach\OneDrive - VISCONSULT\00_VISCONSULT\05_AI_and_Digitalization\02_AI_Agents\02_HF_Agents_Course\99_Final_Assignment\tools\web_tools.py", line 131, in search - results = list(ddgs.text( - ~~~~~~~~~^ - query, - ^^^^^^ - max_results=self.max_results, - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - timelimit=self.ddg_timeout - ^^^^^^^^^^^^^^^^^^^^^^^^^^ - )) - ^ - File "C:\Users\joach\AppData\Roaming\Python\Python313\site-packages\duckduckgo_search\duckduckgo_search.py", line 184, in text - raise DuckDuckGoSearchException(err) -duckduckgo_search.exceptions.DuckDuckGoSearchException: keywords is mandatory - -2025-04-30 17:21:42,965 - primp - INFO - response: https://html.duckduckgo.com/html 302 -2025-04-30 17:21:42,965 - duckduckgo_search.DDGS - INFO - Error to search using html backend: https://html.duckduckgo.com/html return None. params=None content=None data={'q': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', 'b': '', 'kl': 'wt-wt', 'df': 10} -2025-04-30 17:21:43,794 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 302 -2025-04-30 17:21:43,795 - duckduckgo_search.DDGS - INFO - Error to search using lite backend: https://lite.duckduckgo.com/lite/ return None. params=None content=None data={'q': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', 'kl': 'wt-wt', 'df': 10} -2025-04-30 17:21:43,795 - gaia_agent.tools.web - ERROR - Error searching DuckDuckGo: https://lite.duckduckgo.com/lite/ return None. params=None content=None data={'q': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', 'kl': 'wt-wt', 'df': 10} -2025-04-30 17:21:43,796 - gaia_agent.tools.web - ERROR - Traceback (most recent call last): - File "c:\Users\joach\OneDrive - VISCONSULT\00_VISCONSULT\05_AI_and_Digitalization\02_AI_Agents\02_HF_Agents_Course\99_Final_Assignment\tools\web_tools.py", line 131, in search - results = list(ddgs.text( - ~~~~~~~~~^ - query, - ^^^^^^ - max_results=self.max_results, - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - timelimit=self.ddg_timeout - ^^^^^^^^^^^^^^^^^^^^^^^^^^ - )) - ^ - File "C:\Users\joach\AppData\Roaming\Python\Python313\site-packages\duckduckgo_search\duckduckgo_search.py", line 184, in text - raise DuckDuckGoSearchException(err) -duckduckgo_search.exceptions.DuckDuckGoSearchException: https://lite.duckduckgo.com/lite/ return None. params=None content=None data={'q': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', 'kl': 'wt-wt', 'df': 10} - -2025-04-30 17:21:43,796 - test_duckduckgo - WARNING - Long query caused an error: DuckDuckGo search failed: https://lite.duckduckgo.com/lite/ return None. params=None content=None data={'q': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', 'kl': 'wt-wt', 'df': 10} -2025-04-30 17:21:43,796 - test_duckduckgo - INFO - DuckDuckGo search tool initialized -2025-04-30 17:21:44,638 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 200 -2025-04-30 17:21:44,639 - test_duckduckgo - INFO - Result count configuration works, got 3 results -2025-04-30 17:22:29,072 - test_duckduckgo - INFO - DuckDuckGo search tool initialized -2025-04-30 17:22:29,073 - test_duckduckgo - INFO - Successfully imported duckduckgo-search package -2025-04-30 17:22:29,088 - test_duckduckgo - INFO - Successfully created DDGS instance -2025-04-30 17:22:29,857 - primp - INFO - response: https://html.duckduckgo.com/html 200 -2025-04-30 17:22:29,859 - test_duckduckgo - INFO - Successfully retrieved 2 results from DuckDuckGo -2025-04-30 17:22:29,860 - test_duckduckgo - INFO - Result 1: SQL Fiddle - Online SQL Compiler for learning & practice - https://sqlfiddle.com/ -2025-04-30 17:22:29,860 - test_duckduckgo - INFO - Result 2: DB Fiddle - SQL Database Playground - https://www.db-fiddle.com/ -2025-04-30 17:22:30,637 - primp - INFO - response: https://html.duckduckgo.com/html 200 -2025-04-30 17:22:30,642 - test_duckduckgo - INFO - Successfully retrieved 5 results using DuckDuckGoSearchTool -2025-04-30 17:22:30,642 - test_duckduckgo - INFO - Result format is correct: dict_keys(['title', 'link', 'snippet']) -2025-04-30 17:22:30,643 - test_duckduckgo - INFO - First result: Welcome to Python.org - https://www.python.org/ -2025-04-30 17:22:30,643 - duckduckgo_search.DDGS - INFO - Error to search using html backend: keywords is mandatory -2025-04-30 17:22:30,644 - duckduckgo_search.DDGS - INFO - Error to search using lite backend: keywords is mandatory -2025-04-30 17:22:30,644 - gaia_agent.tools.web - ERROR - Error searching DuckDuckGo: keywords is mandatory -2025-04-30 17:22:30,648 - gaia_agent.tools.web - ERROR - Traceback (most recent call last): - File "c:\Users\joach\OneDrive - VISCONSULT\00_VISCONSULT\05_AI_and_Digitalization\02_AI_Agents\02_HF_Agents_Course\99_Final_Assignment\tools\web_tools.py", line 131, in search - results = list(ddgs.text( - ~~~~~~~~~^ - query, - ^^^^^^ - max_results=self.max_results, - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - timelimit=self.ddg_timeout - ^^^^^^^^^^^^^^^^^^^^^^^^^^ - )) - ^ - File "C:\Users\joach\AppData\Roaming\Python\Python313\site-packages\duckduckgo_search\duckduckgo_search.py", line 184, in text - raise DuckDuckGoSearchException(err) -duckduckgo_search.exceptions.DuckDuckGoSearchException: keywords is mandatory - -2025-04-30 17:22:30,750 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 302 -2025-04-30 17:22:30,750 - duckduckgo_search.DDGS - INFO - Error to search using lite backend: https://lite.duckduckgo.com/lite/ return None. params=None content=None data={'q': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', 'kl': 'wt-wt', 'df': 10} -2025-04-30 17:22:31,585 - primp - INFO - response: https://html.duckduckgo.com/html 302 -2025-04-30 17:22:31,585 - duckduckgo_search.DDGS - INFO - Error to search using html backend: https://html.duckduckgo.com/html return None. params=None content=None data={'q': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', 'b': '', 'kl': 'wt-wt', 'df': 10} -2025-04-30 17:22:31,585 - gaia_agent.tools.web - ERROR - Error searching DuckDuckGo: https://html.duckduckgo.com/html return None. params=None content=None data={'q': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', 'b': '', 'kl': 'wt-wt', 'df': 10} -2025-04-30 17:22:31,587 - gaia_agent.tools.web - ERROR - Traceback (most recent call last): - File "c:\Users\joach\OneDrive - VISCONSULT\00_VISCONSULT\05_AI_and_Digitalization\02_AI_Agents\02_HF_Agents_Course\99_Final_Assignment\tools\web_tools.py", line 131, in search - results = list(ddgs.text( - ~~~~~~~~~^ - query, - ^^^^^^ - max_results=self.max_results, - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - timelimit=self.ddg_timeout - ^^^^^^^^^^^^^^^^^^^^^^^^^^ - )) - ^ - File "C:\Users\joach\AppData\Roaming\Python\Python313\site-packages\duckduckgo_search\duckduckgo_search.py", line 184, in text - raise DuckDuckGoSearchException(err) -duckduckgo_search.exceptions.DuckDuckGoSearchException: https://html.duckduckgo.com/html return None. params=None content=None data={'q': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', 'b': '', 'kl': 'wt-wt', 'df': 10} - -2025-04-30 17:22:31,587 - test_duckduckgo - WARNING - Long query caused an error: DuckDuckGo search failed: https://html.duckduckgo.com/html return None. params=None content=None data={'q': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', 'b': '', 'kl': 'wt-wt', 'df': 10} -2025-04-30 17:22:32,364 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 200 -2025-04-30 17:22:32,367 - test_duckduckgo - INFO - Result count configuration works, got 3 results -2025-04-30 17:27:27,316 - gaia_agent - INFO - Initializing GAIAAgent -2025-04-30 17:27:27,317 - gaia_agent - ERROR - Error initializing memory components: 'memory' -2025-04-30 17:27:27,317 - gaia_agent - WARNING - Continuing without persistent memory -2025-04-30 17:27:27,317 - gaia_agent - INFO - GAIAAgent initialized successfully -2025-04-30 17:27:27,317 - test_multi_search - INFO - Test setup complete with all available search providers -2025-04-30 17:27:28,097 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 200 -2025-04-30 17:27:28,100 - test_multi_search - INFO - DuckDuckGo search successful: 5 results for 'latest Python version' -2025-04-30 17:27:28,100 - test_multi_search - INFO - First result: Download Python | Python.org - https://www.python.org/downloads/ -2025-04-30 17:27:28,831 - test_multi_search - INFO - Serper search successful: 2 results for 'latest Python version' -2025-04-30 17:27:28,831 - test_multi_search - INFO - First result: Download Python | Python.org - https://www.python.org/downloads/ -2025-04-30 17:27:35,796 - test_multi_search - INFO - Perplexity search successful for 'latest Python version' -2025-04-30 17:27:35,797 - test_multi_search - ERROR - Perplexity search failed: slice(None, 100, None) -2025-04-30 17:28:14,154 - gaia_agent - INFO - Initializing GAIAAgent -2025-04-30 17:28:14,155 - gaia_agent - ERROR - Error initializing memory components: 'memory' -2025-04-30 17:28:14,155 - gaia_agent - WARNING - Continuing without persistent memory -2025-04-30 17:28:14,155 - gaia_agent - INFO - GAIAAgent initialized successfully -2025-04-30 17:28:14,155 - test_multi_search - INFO - Test setup complete with all available search providers -2025-04-30 17:28:14,910 - primp - INFO - response: https://html.duckduckgo.com/html 200 -2025-04-30 17:28:14,912 - test_multi_search - INFO - DuckDuckGo search successful: 5 results for 'latest Python version' -2025-04-30 17:28:14,912 - test_multi_search - INFO - First result: Download Python | Python.org - https://www.python.org/downloads/ -2025-04-30 17:28:15,660 - test_multi_search - INFO - Serper search successful: 2 results for 'latest Python version' -2025-04-30 17:28:15,661 - test_multi_search - INFO - First result: Download Python | Python.org - https://www.python.org/downloads/ -2025-04-30 17:28:22,938 - test_multi_search - INFO - Perplexity search successful for 'latest Python version' -2025-04-30 17:28:22,940 - test_multi_search - INFO - Result snippet: -Okay, the user is asking about the latest Python version. Let me check the search results pr... -2025-04-30 17:28:23,742 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 200 -2025-04-30 17:28:23,745 - test_multi_search - INFO - DuckDuckGo found 5 results -2025-04-30 17:28:24,723 - test_multi_search - INFO - Serper found 2 results -2025-04-30 17:28:31,286 - test_multi_search - INFO - Perplexity search successful, content length: 1932 -2025-04-30 17:28:31,286 - test_multi_search - INFO - Successfully tested 3 out of 3 providers -2025-04-30 17:28:31,286 - test_multi_search - INFO - Comparing results from different providers: -2025-04-30 17:28:31,286 - test_multi_search - INFO - Duckduckgo first result: Download Python | Python.org -2025-04-30 17:28:31,286 - test_multi_search - INFO - Serper first result: Download Python | Python.org -2025-04-30 17:28:31,286 - test_multi_search - ERROR - GAIA agent with search providers failed: 'GAIAAgent' object has no attribute 'add_tool' -2025-04-30 17:35:24,020 - test_duckduckgo_integration - INFO - DuckDuckGo search tool initialized -2025-04-30 17:35:24,021 - gaia_agent - INFO - Initializing GAIAAgent -2025-04-30 17:35:24,022 - gaia_agent - ERROR - Error initializing memory components: 'memory' -2025-04-30 17:35:24,022 - gaia_agent - WARNING - Continuing without persistent memory -2025-04-30 17:35:24,022 - gaia_agent - INFO - GAIAAgent initialized successfully -2025-04-30 17:35:24,023 - test_duckduckgo_integration - INFO - GAIA agent initialized -2025-04-30 17:35:24,023 - test_duckduckgo_integration - INFO - Successfully imported duckduckgo-search package -2025-04-30 17:35:24,034 - test_duckduckgo_integration - INFO - Successfully created DDGS instance -2025-04-30 17:35:24,771 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 200 -2025-04-30 17:35:24,774 - test_duckduckgo_integration - INFO - Successfully retrieved 2 results from DuckDuckGo -2025-04-30 17:35:24,774 - test_duckduckgo_integration - INFO - Result 1: SQL Fiddle - Online SQL Compiler for learning & practice - https://sqlfiddle.com/ -2025-04-30 17:35:24,775 - test_duckduckgo_integration - INFO - Result 2: SQL Test - https://sqltest.net/ -2025-04-30 17:35:25,531 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 200 -2025-04-30 17:35:25,535 - test_duckduckgo_integration - INFO - Successfully retrieved 5 results using DuckDuckGoSearchTool -2025-04-30 17:35:25,535 - test_duckduckgo_integration - INFO - Result format is correct: dict_keys(['title', 'link', 'snippet']) -2025-04-30 17:35:25,536 - test_duckduckgo_integration - INFO - First result: Welcome to Python.org - https://www.python.org/ -2025-04-30 17:35:25,536 - duckduckgo_search.DDGS - INFO - Error to search using lite backend: keywords is mandatory -2025-04-30 17:35:25,536 - duckduckgo_search.DDGS - INFO - Error to search using html backend: keywords is mandatory -2025-04-30 17:35:25,537 - gaia_agent.tools.web - ERROR - Error searching DuckDuckGo: keywords is mandatory -2025-04-30 17:35:25,539 - gaia_agent.tools.web - ERROR - Traceback (most recent call last): - File "c:\Users\joach\OneDrive - VISCONSULT\00_VISCONSULT\05_AI_and_Digitalization\02_AI_Agents\02_HF_Agents_Course\99_Final_Assignment\tools\web_tools.py", line 131, in search - results = list(ddgs.text( - ~~~~~~~~~^ - query, - ^^^^^^ - max_results=self.max_results, - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - timelimit=self.ddg_timeout - ^^^^^^^^^^^^^^^^^^^^^^^^^^ - )) - ^ - File "C:\Users\joach\AppData\Roaming\Python\Python313\site-packages\duckduckgo_search\duckduckgo_search.py", line 184, in text - raise DuckDuckGoSearchException(err) -duckduckgo_search.exceptions.DuckDuckGoSearchException: keywords is mandatory - -2025-04-30 17:35:25,620 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 302 -2025-04-30 17:35:25,622 - duckduckgo_search.DDGS - INFO - Error to search using lite backend: https://lite.duckduckgo.com/lite/ return None. params=None content=None data={'q': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', 'kl': 'wt-wt', 'df': 10} -2025-04-30 17:35:26,477 - primp - INFO - response: https://html.duckduckgo.com/html 302 -2025-04-30 17:35:26,477 - duckduckgo_search.DDGS - INFO - Error to search using html backend: https://html.duckduckgo.com/html return None. params=None content=None data={'q': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', 'b': '', 'kl': 'wt-wt', 'df': 10} -2025-04-30 17:35:26,477 - gaia_agent.tools.web - ERROR - Error searching DuckDuckGo: https://html.duckduckgo.com/html return None. params=None content=None data={'q': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', 'b': '', 'kl': 'wt-wt', 'df': 10} -2025-04-30 17:35:26,478 - gaia_agent.tools.web - ERROR - Traceback (most recent call last): - File "c:\Users\joach\OneDrive - VISCONSULT\00_VISCONSULT\05_AI_and_Digitalization\02_AI_Agents\02_HF_Agents_Course\99_Final_Assignment\tools\web_tools.py", line 131, in search - results = list(ddgs.text( - ~~~~~~~~~^ - query, - ^^^^^^ - max_results=self.max_results, - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - timelimit=self.ddg_timeout - ^^^^^^^^^^^^^^^^^^^^^^^^^^ - )) - ^ - File "C:\Users\joach\AppData\Roaming\Python\Python313\site-packages\duckduckgo_search\duckduckgo_search.py", line 184, in text - raise DuckDuckGoSearchException(err) -duckduckgo_search.exceptions.DuckDuckGoSearchException: https://html.duckduckgo.com/html return None. params=None content=None data={'q': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', 'b': '', 'kl': 'wt-wt', 'df': 10} - -2025-04-30 17:35:26,478 - test_duckduckgo_integration - WARNING - Long query caused an error: DuckDuckGo search failed: https://html.duckduckgo.com/html return None. params=None content=None data={'q': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', 'b': '', 'kl': 'wt-wt', 'df': 10} -2025-04-30 17:35:27,241 - primp - INFO - response: https://html.duckduckgo.com/html 200 -2025-04-30 17:35:27,244 - test_duckduckgo_integration - INFO - Result count configuration works, got 3 results -2025-04-30 17:35:27,244 - test_duckduckgo_integration - INFO - Testing integration with LangGraph-based GAIA agent -2025-04-30 17:35:27,244 - test_duckduckgo_integration - INFO - - To integrate DuckDuckGo with the LangGraph-based GAIA agent: - - 1. Modify the execute_tool function in agent/graph.py to use real tools: - - Create a tool registry that maps tool names to tool instances - - Look up the tool in the registry based on the tool name - - Call the tool with the provided input - - Return the actual result - - 2. Register the DuckDuckGo tool in the tool registry: - - Create a DuckDuckGoSearchTool instance - - Add it to the tool registry with a name like "duckduckgo_search" - - 3. Update the agent configuration to include DuckDuckGo: - - Add DuckDuckGo configuration to the tool_config in agent/config.py - - Ensure the tool is available in the LangGraph workflow - - Example code for execute_tool function: - - ```python - def execute_tool(state: AgentState) -> AgentState: - # Get the current step from the plan - current_step = state["plan"][state["current_step"]] - tool_name = current_step.get("tool") - tool_input = current_step.get("tool_input", {}) - - logger.debug(f"Executing tool {tool_name} for step {current_step['step_number']}") - - try: - # Get the tool from the registry - tool = tool_registry.get(tool_name) - if not tool: - raise Exception(f"Tool {tool_name} not found in registry") - - # Execute the tool - if tool_name == "duckduckgo_search": - result = tool.search(tool_input.get("query", "")) - else: - # Generic tool execution - result = tool.run(tool_input) - - # Create a tool result - tool_result = { - "tool_name": tool_name, - "success": True, - "result": result, - "error": None - } - - # Update the state - new_state = state.copy() - new_state["tool_results"] = state.get("tool_results", []) + [tool_result] - new_state["current_step"] += 1 - - return new_state - except Exception as e: - # Handle error... - # (rest of the function remains the same) - } - ``` - -2025-04-30 17:35:27,244 - test_duckduckgo_integration - INFO - Testing DuckDuckGo alongside other search providers -2025-04-30 17:35:27,244 - test_duckduckgo_integration - INFO - Serper search tool initialized -2025-04-30 17:35:27,244 - test_duckduckgo_integration - INFO - Perplexity search tool initialized -2025-04-30 17:35:28,023 - primp - INFO - response: https://html.duckduckgo.com/html 200 -2025-04-30 17:35:28,025 - test_duckduckgo_integration - INFO - DuckDuckGo found 3 results -2025-04-30 17:35:28,776 - test_duckduckgo_integration - INFO - Serper found 2 results -2025-04-30 17:35:36,224 - test_duckduckgo_integration - INFO - Perplexity search successful, content length: 1915 -2025-04-30 17:35:36,224 - test_duckduckgo_integration - INFO - Successfully tested 3 out of 3 providers -2025-04-30 17:35:36,224 - test_duckduckgo_integration - INFO - Comparing results from different providers: -2025-04-30 17:35:36,224 - test_duckduckgo_integration - INFO - Duckduckgo first result: Download Python | Python.org -2025-04-30 17:35:36,224 - test_duckduckgo_integration - INFO - Serper first result: Download Python | Python.org -2025-04-30 17:38:41,039 - gaia_agent.tool_registry - INFO - Created default tool registry with 3 tools -2025-04-30 17:38:41,954 - primp - INFO - response: https://html.duckduckgo.com/html 200 -2025-04-30 17:38:53,630 - primp - INFO - response: https://html.duckduckgo.com/html 200 -2025-04-30 17:38:54,512 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 200 -2025-04-30 17:49:03,294 - gaia_agent.tools.arxiv - WARNING - arXiv package not installed. Install with: pip install arxiv -2025-04-30 17:49:03,295 - test_arxiv_integration - INFO - arXiv search tool initialized -2025-04-30 17:49:03,295 - test_arxiv_integration - INFO - DuckDuckGo search tool initialized -2025-04-30 17:49:03,296 - test_arxiv_integration - INFO - Serper search tool initialized -2025-04-30 17:49:03,296 - test_arxiv_integration - INFO - Perplexity search tool initialized -2025-04-30 17:49:52,796 - test_arxiv_integration - INFO - arXiv search tool initialized -2025-04-30 17:49:52,797 - test_arxiv_integration - INFO - DuckDuckGo search tool initialized -2025-04-30 17:49:52,797 - test_arxiv_integration - INFO - Serper search tool initialized -2025-04-30 17:49:52,797 - test_arxiv_integration - INFO - Perplexity search tool initialized -2025-04-30 17:49:52,798 - test_arxiv_integration - INFO - Successfully imported arxiv package -2025-04-30 17:49:52,798 - arxiv - INFO - Requesting page (first: True, try: 0): https://export.arxiv.org/api/query?search_query=large+language+models&id_list=&sortBy=relevance&sortOrder=descending&start=0&max_results=100 -2025-04-30 17:49:54,345 - arxiv - INFO - Got first page: 100 of 1257521 total results -2025-04-30 17:49:54,346 - test_arxiv_integration - INFO - Successfully retrieved 2 results from arXiv -2025-04-30 17:49:54,347 - test_arxiv_integration - INFO - Result format is correct: dict_keys(['title', 'authors', 'summary', 'published', 'url', 'pdf_url', 'categories', 'comment', 'journal_ref', 'doi']) -2025-04-30 17:49:54,347 - test_arxiv_integration - INFO - First result: Lost in Translation: Large Language Models in Non-English Content Analysis by Gabriel Nicholas, Aliya Bhatia -2025-04-30 17:49:54,347 - arxiv - INFO - Requesting page (first: True, try: 0): https://export.arxiv.org/api/query?search_query=&id_list=2303.08774&sortBy=relevance&sortOrder=descending&start=0&max_results=100 -2025-04-30 17:49:54,951 - arxiv - INFO - Got first page: 1 of 1 total results -2025-04-30 17:49:54,952 - test_arxiv_integration - INFO - Successfully retrieved paper with ID 2303.08774 -2025-04-30 17:49:54,952 - test_arxiv_integration - INFO - Result format is correct: dict_keys(['title', 'authors', 'summary', 'published', 'url', 'pdf_url', 'categories', 'comment', 'journal_ref', 'doi']) -2025-04-30 17:49:54,952 - test_arxiv_integration - INFO - Paper: GPT-4 Technical Report by OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff Belgum, Irwan Bello, Jake Berdine, Gabriel Bernadett-Shapiro, Christopher Berner, Lenny Bogdonoff, Oleg Boiko, Madelaine Boyd, Anna-Luisa Brakman, Greg Brockman, Tim Brooks, Miles Brundage, Kevin Button, Trevor Cai, Rosie Campbell, Andrew Cann, Brittany Carey, Chelsea Carlson, Rory Carmichael, Brooke Chan, Che Chang, Fotis Chantzis, Derek Chen, Sully Chen, Ruby Chen, Jason Chen, Mark Chen, Ben Chess, Chester Cho, Casey Chu, Hyung Won Chung, Dave Cummings, Jeremiah Currier, Yunxing Dai, Cory Decareaux, Thomas Degry, Noah Deutsch, Damien Deville, Arka Dhar, David Dohan, Steve Dowling, Sheila Dunning, Adrien Ecoffet, Atty Eleti, Tyna Eloundou, David Farhi, Liam Fedus, Niko Felix, Simón Posada Fishman, Juston Forte, Isabella Fulford, Leo Gao, Elie Georges, Christian Gibson, Vik Goel, Tarun Gogineni, Gabriel Goh, Rapha Gontijo-Lopes, Jonathan Gordon, Morgan Grafstein, Scott Gray, Ryan Greene, Joshua Gross, Shixiang Shane Gu, Yufei Guo, Chris Hallacy, Jesse Han, Jeff Harris, Yuchen He, Mike Heaton, Johannes Heidecke, Chris Hesse, Alan Hickey, Wade Hickey, Peter Hoeschele, Brandon Houghton, Kenny Hsu, Shengli Hu, Xin Hu, Joost Huizinga, Shantanu Jain, Shawn Jain, Joanne Jang, Angela Jiang, Roger Jiang, Haozhun Jin, Denny Jin, Shino Jomoto, Billie Jonn, Heewoo Jun, Tomer Kaftan, \u0141ukasz Kaiser, Ali Kamali, Ingmar Kanitscheider, Nitish Shirish Keskar, Tabarak Khan, Logan Kilpatrick, Jong Wook Kim, Christina Kim, Yongjik Kim, Jan Hendrik Kirchner, Jamie Kiros, Matt Knight, Daniel Kokotajlo, \u0141ukasz Kondraciuk, Andrew Kondrich, Aris Konstantinidis, Kyle Kosic, Gretchen Krueger, Vishal Kuo, Michael Lampe, Ikai Lan, Teddy Lee, Jan Leike, Jade Leung, Daniel Levy, Chak Ming Li, Rachel Lim, Molly Lin, Stephanie Lin, Mateusz Litwin, Theresa Lopez, Ryan Lowe, Patricia Lue, Anna Makanju, Kim Malfacini, Sam Manning, Todor Markov, Yaniv Markovski, Bianca Martin, Katie Mayer, Andrew Mayne, Bob McGrew, Scott Mayer McKinney, Christine McLeavey, Paul McMillan, Jake McNeil, David Medina, Aalok Mehta, Jacob Menick, Luke Metz, Andrey Mishchenko, Pamela Mishkin, Vinnie Monaco, Evan Morikawa, Daniel Mossing, Tong Mu, Mira Murati, Oleg Murk, David Mély, Ashvin Nair, Reiichiro Nakano, Rajeev Nayak, Arvind Neelakantan, Richard Ngo, Hyeonwoo Noh, Long Ouyang, Cullen O'Keefe, Jakub Pachocki, Alex Paino, Joe Palermo, Ashley Pantuliano, Giambattista Parascandolo, Joel Parish, Emy Parparita, Alex Passos, Mikhail Pavlov, Andrew Peng, Adam Perelman, Filipe de Avila Belbute Peres, Michael Petrov, Henrique Ponde de Oliveira Pinto, Michael, Pokorny, Michelle Pokrass, Vitchyr H. Pong, Tolly Powell, Alethea Power, Boris Power, Elizabeth Proehl, Raul Puri, Alec Radford, Jack Rae, Aditya Ramesh, Cameron Raymond, Francis Real, Kendra Rimbach, Carl Ross, Bob Rotsted, Henri Roussez, Nick Ryder, Mario Saltarelli, Ted Sanders, Shibani Santurkar, Girish Sastry, Heather Schmidt, David Schnurr, John Schulman, Daniel Selsam, Kyla Sheppard, Toki Sherbakov, Jessica Shieh, Sarah Shoker, Pranav Shyam, Szymon Sidor, Eric Sigler, Maddie Simens, Jordan Sitkin, Katarina Slama, Ian Sohl, Benjamin Sokolowsky, Yang Song, Natalie Staudacher, Felipe Petroski Such, Natalie Summers, Ilya Sutskever, Jie Tang, Nikolas Tezak, Madeleine B. Thompson, Phil Tillet, Amin Tootoonchian, Elizabeth Tseng, Preston Tuggle, Nick Turley, Jerry Tworek, Juan Felipe Cerón Uribe, Andrea Vallone, Arun Vijayvergiya, Chelsea Voss, Carroll Wainwright, Justin Jay Wang, Alvin Wang, Ben Wang, Jonathan Ward, Jason Wei, CJ Weinmann, Akila Welihinda, Peter Welinder, Jiayi Weng, Lilian Weng, Matt Wiethoff, Dave Willner, Clemens Winter, Samuel Wolrich, Hannah Wong, Lauren Workman, Sherwin Wu, Jeff Wu, Michael Wu, Kai Xiao, Tao Xu, Sarah Yoo, Kevin Yu, Qiming Yuan, Wojciech Zaremba, Rowan Zellers, Chong Zhang, Marvin Zhang, Shengjia Zhao, Tianhao Zheng, Juntang Zhuang, William Zhuk, Barret Zoph -2025-04-30 17:49:54,953 - arxiv - INFO - Requesting page (first: True, try: 0): https://export.arxiv.org/api/query?search_query=cat%3Acs.AI&id_list=&sortBy=relevance&sortOrder=descending&start=0&max_results=100 -2025-04-30 17:49:56,023 - arxiv - INFO - Got first page: 100 of 123927 total results -2025-04-30 17:49:56,025 - test_arxiv_integration - INFO - Successfully retrieved 2 results from category cs.AI -2025-04-30 17:49:56,025 - test_arxiv_integration - INFO - Result format is correct: dict_keys(['title', 'authors', 'summary', 'published', 'url', 'pdf_url', 'categories', 'comment', 'journal_ref', 'doi']) -2025-04-30 17:49:56,025 - test_arxiv_integration - INFO - First result: Dynamic Backtracking by M. L. Ginsberg -2025-04-30 17:49:56,026 - arxiv - INFO - Requesting page (first: True, try: 0): https://export.arxiv.org/api/query?search_query=&id_list=invalid_id&sortBy=relevance&sortOrder=descending&start=0&max_results=100 -2025-04-30 17:49:56,572 - arxiv - INFO - Sleeping: 2.999923 seconds -2025-04-30 17:49:59,572 - arxiv - INFO - Requesting page (first: True, try: 1): https://export.arxiv.org/api/query?search_query=&id_list=invalid_id&sortBy=relevance&sortOrder=descending&start=0&max_results=100 -2025-04-30 17:50:00,123 - arxiv - INFO - Sleeping: 2.999937 seconds -2025-04-30 17:50:03,123 - arxiv - INFO - Requesting page (first: True, try: 2): https://export.arxiv.org/api/query?search_query=&id_list=invalid_id&sortBy=relevance&sortOrder=descending&start=0&max_results=100 -2025-04-30 17:50:03,722 - arxiv - INFO - Sleeping: 2.999984 seconds -2025-04-30 17:50:06,722 - arxiv - INFO - Requesting page (first: True, try: 3): https://export.arxiv.org/api/query?search_query=&id_list=invalid_id&sortBy=relevance&sortOrder=descending&start=0&max_results=100 -2025-04-30 17:50:07,268 - gaia_agent.tools.arxiv - ERROR - Error getting arXiv paper invalid_id: Page request resulted in HTTP 400 (https://export.arxiv.org/api/query?search_query=&id_list=invalid_id&sortBy=relevance&sortOrder=descending&start=0&max_results=100) -2025-04-30 17:50:07,275 - gaia_agent.tools.arxiv - ERROR - Traceback (most recent call last): - File "C:\Users\joach\AppData\Roaming\Python\Python313\site-packages\arxiv\__init__.py", line 648, in _parse_feed - return self.__try_parse_feed(url, first_page=first_page, try_index=_try_index) - ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - File "C:\Users\joach\AppData\Roaming\Python\Python313\site-packages\arxiv\__init__.py", line 685, in __try_parse_feed - raise HTTPError(url, try_index, resp.status_code) -arxiv.HTTPError: Page request resulted in HTTP 400 (https://export.arxiv.org/api/query?search_query=&id_list=invalid_id&sortBy=relevance&sortOrder=descending&start=0&max_results=100) - -During handling of the above exception, another exception occurred: - -Traceback (most recent call last): - File "C:\Users\joach\AppData\Roaming\Python\Python313\site-packages\arxiv\__init__.py", line 648, in _parse_feed - return self.__try_parse_feed(url, first_page=first_page, try_index=_try_index) - ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - File "C:\Users\joach\AppData\Roaming\Python\Python313\site-packages\arxiv\__init__.py", line 685, in __try_parse_feed - raise HTTPError(url, try_index, resp.status_code) -arxiv.HTTPError: Page request resulted in HTTP 400 (https://export.arxiv.org/api/query?search_query=&id_list=invalid_id&sortBy=relevance&sortOrder=descending&start=0&max_results=100) - -During handling of the above exception, another exception occurred: - -Traceback (most recent call last): - File "C:\Users\joach\AppData\Roaming\Python\Python313\site-packages\arxiv\__init__.py", line 648, in _parse_feed - return self.__try_parse_feed(url, first_page=first_page, try_index=_try_index) - ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - File "C:\Users\joach\AppData\Roaming\Python\Python313\site-packages\arxiv\__init__.py", line 685, in __try_parse_feed - raise HTTPError(url, try_index, resp.status_code) -arxiv.HTTPError: Page request resulted in HTTP 400 (https://export.arxiv.org/api/query?search_query=&id_list=invalid_id&sortBy=relevance&sortOrder=descending&start=0&max_results=100) - -During handling of the above exception, another exception occurred: - -Traceback (most recent call last): - File "c:\Users\joach\OneDrive - VISCONSULT\00_VISCONSULT\05_AI_and_Digitalization\02_AI_Agents\02_HF_Agents_Course\99_Final_Assignment\tools\arxiv_tool.py", line 147, in get_paper_by_id - results = list(client.results(search)) - File "C:\Users\joach\AppData\Roaming\Python\Python313\site-packages\arxiv\__init__.py", line 601, in _results - feed = self._parse_feed(page_url, first_page=True) - File "C:\Users\joach\AppData\Roaming\Python\Python313\site-packages\arxiv\__init__.py", line 656, in _parse_feed - return self._parse_feed(url, first_page=first_page, _try_index=_try_index + 1) - ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - File "C:\Users\joach\AppData\Roaming\Python\Python313\site-packages\arxiv\__init__.py", line 656, in _parse_feed - return self._parse_feed(url, first_page=first_page, _try_index=_try_index + 1) - ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - File "C:\Users\joach\AppData\Roaming\Python\Python313\site-packages\arxiv\__init__.py", line 656, in _parse_feed - return self._parse_feed(url, first_page=first_page, _try_index=_try_index + 1) - ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - File "C:\Users\joach\AppData\Roaming\Python\Python313\site-packages\arxiv\__init__.py", line 658, in _parse_feed - raise err - File "C:\Users\joach\AppData\Roaming\Python\Python313\site-packages\arxiv\__init__.py", line 648, in _parse_feed - return self.__try_parse_feed(url, first_page=first_page, try_index=_try_index) - ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - File "C:\Users\joach\AppData\Roaming\Python\Python313\site-packages\arxiv\__init__.py", line 685, in __try_parse_feed - raise HTTPError(url, try_index, resp.status_code) -arxiv.HTTPError: Page request resulted in HTTP 400 (https://export.arxiv.org/api/query?search_query=&id_list=invalid_id&sortBy=relevance&sortOrder=descending&start=0&max_results=100) - -2025-04-30 17:50:07,275 - arxiv - INFO - Requesting page (first: True, try: 0): https://export.arxiv.org/api/query?search_query=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa&id_list=&sortBy=relevance&sortOrder=descending&start=0&max_results=100 -2025-04-30 17:50:07,811 - arxiv - INFO - Got empty first page; stopping generation -2025-04-30 17:50:07,811 - test_arxiv_integration - INFO - Long query handled successfully, got 0 results -2025-04-30 17:50:07,811 - test_arxiv_integration - INFO - Testing arXiv alongside other search providers -2025-04-30 17:50:07,812 - arxiv - INFO - Requesting page (first: True, try: 0): https://export.arxiv.org/api/query?search_query=large+language+models&id_list=&sortBy=relevance&sortOrder=descending&start=0&max_results=100 -2025-04-30 17:50:09,040 - arxiv - INFO - Got first page: 100 of 1257521 total results -2025-04-30 17:50:09,041 - test_arxiv_integration - INFO - arXiv found 2 results -2025-04-30 17:50:09,842 - primp - INFO - response: https://html.duckduckgo.com/html 200 -2025-04-30 17:50:09,847 - test_arxiv_integration - INFO - DuckDuckGo found 5 results -2025-04-30 17:50:10,573 - test_arxiv_integration - INFO - Serper found 5 results -2025-04-30 17:50:17,460 - test_arxiv_integration - INFO - Perplexity search successful, content length: 2490 -2025-04-30 17:50:17,460 - test_arxiv_integration - INFO - Successfully tested 4 out of 4 providers -2025-04-30 17:50:17,460 - test_arxiv_integration - INFO - Comparing results from different providers: -2025-04-30 17:50:17,460 - test_arxiv_integration - INFO - arXiv first result: Lost in Translation: Large Language Models in Non-English Content Analysis -2025-04-30 17:50:17,460 - test_arxiv_integration - INFO - DuckDuckGo first result: Large language model - Wikipedia -2025-04-30 17:50:17,460 - test_arxiv_integration - INFO - Serper first result: Large language model - Wikipedia -2025-04-30 17:50:41,300 - gaia_agent.tool_registry - INFO - Created default tool registry with 6 tools -2025-04-30 17:50:42,234 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 200 -2025-04-30 17:50:50,968 - primp - INFO - response: https://html.duckduckgo.com/html 200 -2025-04-30 17:50:51,942 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 200 -2025-04-30 17:52:33,622 - gaia_agent.tool_registry - INFO - Created default tool registry with 6 tools -2025-04-30 17:52:34,476 - primp - INFO - response: https://html.duckduckgo.com/html 200 -2025-04-30 17:52:43,892 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 200 -2025-04-30 17:52:44,711 - primp - INFO - response: https://html.duckduckgo.com/html 200 -2025-04-30 17:52:52,717 - arxiv - INFO - Requesting page (first: True, try: 0): https://export.arxiv.org/api/query?search_query=large+language+models&id_list=&sortBy=relevance&sortOrder=descending&start=0&max_results=100 -2025-04-30 17:52:53,358 - arxiv - INFO - Got first page: 10 of 1257521 total results -2025-04-30 17:52:53,359 - arxiv - INFO - Requesting page (first: True, try: 0): https://export.arxiv.org/api/query?search_query=&id_list=2303.08774&sortBy=relevance&sortOrder=descending&start=0&max_results=100 -2025-04-30 17:52:53,950 - arxiv - INFO - Got first page: 1 of 1 total results -2025-04-30 17:52:53,952 - arxiv - INFO - Requesting page (first: True, try: 0): https://export.arxiv.org/api/query?search_query=cat%3Acs.AI&id_list=&sortBy=relevance&sortOrder=descending&start=0&max_results=100 -2025-04-30 17:52:55,070 - arxiv - INFO - Got first page: 100 of 123927 total results -2025-05-01 20:16:12,637 - gaia_agent.tool_registry - INFO - Created default tool registry with 6 tools -2025-05-01 20:16:13,644 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 200 -2025-05-01 20:29:34,912 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 200 -2025-05-01 20:33:03,581 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 200 -2025-05-01 20:35:56,366 - gaia_agent.tool_registry - INFO - Created default tool registry with 7 tools -2025-05-01 20:35:56,368 - gaia_agent.tests.phase1_production - INFO - Testing assessment analysis... -2025-05-01 20:35:56,368 - gaia_agent.tests.phase1_production - INFO - Assessment analysis test completed successfully -2025-05-01 20:35:56,369 - gaia_agent.tool_registry - INFO - Created default tool registry with 7 tools -2025-05-01 20:35:56,369 - gaia_agent.tests.phase1_production - INFO - Generating Hugging Face update report... -2025-05-01 20:35:56,370 - gaia_agent.tests.phase1_production - INFO - Hugging Face update report generated successfully -2025-05-01 20:35:56,370 - gaia_agent.tool_registry - INFO - Created default tool registry with 7 tools -2025-05-01 20:35:56,371 - gaia_agent.tests.phase1_production - INFO - Testing Mercedes Sosa search... -2025-05-01 20:35:57,714 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 200 -2025-05-01 20:35:57,722 - gaia_agent.tool_registry - INFO - Created default tool registry with 7 tools -2025-05-01 20:35:57,723 - gaia_agent.tests.phase1_production - INFO - Testing Wikipedia featured article search... -2025-05-01 20:35:58,527 - primp - INFO - response: https://html.duckduckgo.com/html 200 -2025-05-01 20:35:59,515 - gaia_agent.tests.phase1_production - INFO - Wikipedia featured article search test completed successfully -2025-05-01 20:36:58,835 - gaia_agent.tool_registry - INFO - Created default tool registry with 7 tools -2025-05-01 20:36:58,836 - gaia_agent.tests.phase1_production - INFO - Testing assessment analysis... -2025-05-01 20:36:58,837 - gaia_agent.tests.phase1_production - INFO - Assessment analysis test completed successfully -2025-05-01 20:36:58,837 - gaia_agent.tool_registry - INFO - Created default tool registry with 7 tools -2025-05-01 20:36:58,838 - gaia_agent.tests.phase1_production - INFO - Generating Hugging Face update report... -2025-05-01 20:36:58,838 - gaia_agent.tests.phase1_production - INFO - Hugging Face update report generated successfully -2025-05-01 20:36:58,838 - gaia_agent.tool_registry - INFO - Created default tool registry with 7 tools -2025-05-01 20:36:58,838 - gaia_agent.tests.phase1_production - INFO - Testing Mercedes Sosa search... -2025-05-01 20:36:59,872 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 200 -2025-05-01 20:37:00,912 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 200 -2025-05-01 20:37:09,149 - arxiv - INFO - Requesting page (first: True, try: 0): https://export.arxiv.org/api/query?search_query=How+many+studio+albums+were+published+by+Mercedes+Sosa+between+2000+and+2009+%28included%29%3F&id_list=&sortBy=relevance&sortOrder=descending&start=0&max_results=100 -2025-05-01 20:37:09,964 - arxiv - INFO - Got empty first page; stopping generation -2025-05-01 20:37:09,965 - gaia_agent.tests.phase1_production - INFO - Mercedes Sosa search test completed successfully -2025-05-01 20:37:09,966 - gaia_agent.tool_registry - INFO - Created default tool registry with 7 tools -2025-05-01 20:37:09,966 - gaia_agent.tests.phase1_production - INFO - Testing Wikipedia featured article search... -2025-05-01 20:37:10,730 - primp - INFO - response: https://html.duckduckgo.com/html 200 -2025-05-01 20:37:11,746 - gaia_agent.tests.phase1_production - INFO - Wikipedia featured article search test completed successfully -2025-05-01 21:01:19,712 - gaia_agent.tools.multimodal - WARNING - youtube_transcript_api not installed. Install with: pip install youtube-transcript-api -2025-05-01 21:01:20,499 - gaia_agent.tools.multimodal - WARNING - youtube_transcript_api not installed. Install with: pip install youtube-transcript-api -2025-05-01 21:01:21,314 - gaia_agent.tools.multimodal - WARNING - youtube_transcript_api not installed. Install with: pip install youtube-transcript-api -2025-05-01 21:01:22,100 - gaia_agent.tools.multimodal - WARNING - youtube_transcript_api not installed. Install with: pip install youtube-transcript-api -2025-05-01 21:01:22,971 - gaia_agent.tools.multimodal - WARNING - youtube_transcript_api not installed. Install with: pip install youtube-transcript-api -2025-05-01 21:01:22,971 - gaia_agent.tool_registry - INFO - Created default tool registry with 8 tools -2025-05-01 21:01:23,740 - gaia_agent.tools.multimodal - WARNING - youtube_transcript_api not installed. Install with: pip install youtube-transcript-api -2025-05-01 21:01:24,508 - gaia_agent.tools.multimodal - WARNING - youtube_transcript_api not installed. Install with: pip install youtube-transcript-api -2025-05-01 21:03:02,779 - gaia_agent.tool_registry - INFO - Created default tool registry with 8 tools -2025-05-01 21:03:03,706 - gaia_agent.tools.multimodal - ERROR - Transcripts are disabled for this video: Transcripts are disabled -2025-05-01 21:03:06,242 - httpx - INFO - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK" -2025-05-01 21:04:18,090 - gaia_agent.tool_registry - INFO - Created default tool registry with 8 tools -2025-05-01 21:04:18,995 - gaia_agent.tools.multimodal - ERROR - Error extracting transcript: Transcripts are disabled -2025-05-01 21:04:18,998 - gaia_agent.tools.multimodal - ERROR - Traceback (most recent call last): - File "c:\Users\joach\OneDrive - VISCONSULT\00_VISCONSULT\05_AI_and_Digitalization\02_AI_Agents\02_HF_Agents_Course\99_Final_Assignment\tools\multimodal_tools.py", line 644, in extract_transcript - transcript_list = YouTubeTranscriptApi.get_transcript(video_id) - File "C:\Python313\Lib\unittest\mock.py", line 1169, in __call__ - return self._mock_call(*args, **kwargs) - ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^ - File "C:\Python313\Lib\unittest\mock.py", line 1173, in _mock_call - return self._execute_mock_call(*args, **kwargs) - ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^ - File "C:\Python313\Lib\unittest\mock.py", line 1228, in _execute_mock_call - raise effect -TestYouTubeVideoTool.test_transcript_error_handling..TranscriptsDisabled: Transcripts are disabled - -2025-05-01 21:04:18,998 - gaia_agent.tools.multimodal - ERROR - Error extracting transcript: No transcript found -2025-05-01 21:04:18,999 - gaia_agent.tools.multimodal - ERROR - Traceback (most recent call last): - File "c:\Users\joach\OneDrive - VISCONSULT\00_VISCONSULT\05_AI_and_Digitalization\02_AI_Agents\02_HF_Agents_Course\99_Final_Assignment\tools\multimodal_tools.py", line 644, in extract_transcript - transcript_list = YouTubeTranscriptApi.get_transcript(video_id) - File "C:\Python313\Lib\unittest\mock.py", line 1169, in __call__ - return self._mock_call(*args, **kwargs) - ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^ - File "C:\Python313\Lib\unittest\mock.py", line 1173, in _mock_call - return self._execute_mock_call(*args, **kwargs) - ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^ - File "C:\Python313\Lib\unittest\mock.py", line 1228, in _execute_mock_call - raise effect -TestYouTubeVideoTool.test_transcript_error_handling..NoTranscriptFound: No transcript found - -2025-05-01 21:04:21,597 - httpx - INFO - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK" -2025-05-01 21:04:23,045 - httpx - INFO - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK" -2025-05-01 21:04:25,851 - httpx - INFO - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK" -2025-05-01 21:05:11,861 - gaia_agent.tool_registry - INFO - Created default tool registry with 8 tools -2025-05-01 21:05:12,717 - gaia_agent.tools.multimodal - ERROR - Error extracting transcript: Transcripts are disabled -2025-05-01 21:05:12,719 - gaia_agent.tools.multimodal - ERROR - Traceback (most recent call last): - File "c:\Users\joach\OneDrive - VISCONSULT\00_VISCONSULT\05_AI_and_Digitalization\02_AI_Agents\02_HF_Agents_Course\99_Final_Assignment\tools\multimodal_tools.py", line 644, in extract_transcript - transcript_list = YouTubeTranscriptApi.get_transcript(video_id) - File "C:\Python313\Lib\unittest\mock.py", line 1169, in __call__ - return self._mock_call(*args, **kwargs) - ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^ - File "C:\Python313\Lib\unittest\mock.py", line 1173, in _mock_call - return self._execute_mock_call(*args, **kwargs) - ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^ - File "C:\Python313\Lib\unittest\mock.py", line 1228, in _execute_mock_call - raise effect -TestYouTubeVideoTool.test_transcript_error_handling..TranscriptsDisabled: Transcripts are disabled - -2025-05-01 21:05:12,720 - gaia_agent.tools.multimodal - ERROR - Error extracting transcript: No transcript found -2025-05-01 21:05:12,721 - gaia_agent.tools.multimodal - ERROR - Traceback (most recent call last): - File "c:\Users\joach\OneDrive - VISCONSULT\00_VISCONSULT\05_AI_and_Digitalization\02_AI_Agents\02_HF_Agents_Course\99_Final_Assignment\tools\multimodal_tools.py", line 644, in extract_transcript - transcript_list = YouTubeTranscriptApi.get_transcript(video_id) - File "C:\Python313\Lib\unittest\mock.py", line 1169, in __call__ - return self._mock_call(*args, **kwargs) - ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^ - File "C:\Python313\Lib\unittest\mock.py", line 1173, in _mock_call - return self._execute_mock_call(*args, **kwargs) - ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^ - File "C:\Python313\Lib\unittest\mock.py", line 1228, in _execute_mock_call - raise effect -TestYouTubeVideoTool.test_transcript_error_handling..NoTranscriptFound: No transcript found - -2025-05-01 21:05:16,560 - httpx - INFO - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK" -2025-05-01 21:12:10,356 - gaia_agent.tools.multimodal - ERROR - Error extracting transcript: -Could not retrieve a transcript for the video https://www.youtube.com/watch?v=ThisIsNotAValidVideoID! This is most likely caused by: - -The video is no longer available - -If you are sure that the described cause is not responsible for this error and that a transcript should be retrievable, please create an issue at https://github.com/jdepoix/youtube-transcript-api/issues. Please add which version of youtube_transcript_api you are using and provide the information needed to replicate the error. Also make sure that there are no open issues which already describe your problem! -2025-05-01 21:12:10,359 - gaia_agent.tools.multimodal - ERROR - Traceback (most recent call last): - File "c:\Users\joach\OneDrive - VISCONSULT\00_VISCONSULT\05_AI_and_Digitalization\02_AI_Agents\02_HF_Agents_Course\99_Final_Assignment\tools\multimodal_tools.py", line 644, in extract_transcript - transcript_list = YouTubeTranscriptApi.get_transcript(video_id) - File "C:\Users\joach\AppData\Roaming\Python\Python313\site-packages\youtube_transcript_api\_api.py", line 306, in get_transcript - cls.list_transcripts(video_id, proxies, cookies) - ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - File "C:\Users\joach\AppData\Roaming\Python\Python313\site-packages\youtube_transcript_api\_api.py", line 207, in list_transcripts - return ytt_api.list(video_id) - ~~~~~~~~~~~~^^^^^^^^^^ - File "C:\Users\joach\AppData\Roaming\Python\Python313\site-packages\youtube_transcript_api\_api.py", line 135, in list - return self._fetcher.fetch(video_id) - ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^ - File "C:\Users\joach\AppData\Roaming\Python\Python313\site-packages\youtube_transcript_api\_transcripts.py", line 352, in fetch - self._fetch_captions_json(video_id), - ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^ - File "C:\Users\joach\AppData\Roaming\Python\Python313\site-packages\youtube_transcript_api\_transcripts.py", line 357, in _fetch_captions_json - return self._extract_captions_json( - ~~~~~~~~~~~~~~~~~~~~~~~~~~~^ - self._fetch_video_html(video_id), video_id - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - ) - ^ - File "C:\Users\joach\AppData\Roaming\Python\Python313\site-packages\youtube_transcript_api\_transcripts.py", line 380, in _extract_captions_json - self._assert_playability(video_data.get("playabilityStatus"), video_id) - ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - File "C:\Users\joach\AppData\Roaming\Python\Python313\site-packages\youtube_transcript_api\_transcripts.py", line 408, in _assert_playability - raise VideoUnavailable(video_id) -youtube_transcript_api._errors.VideoUnavailable: -Could not retrieve a transcript for the video https://www.youtube.com/watch?v=ThisIsNotAValidVideoID! This is most likely caused by: - -The video is no longer available - -If you are sure that the described cause is not responsible for this error and that a transcript should be retrievable, please create an issue at https://github.com/jdepoix/youtube-transcript-api/issues. Please add which version of youtube_transcript_api you are using and provide the information needed to replicate the error. Also make sure that there are no open issues which already describe your problem! - -2025-05-01 21:12:12,887 - gaia_agent.tool_registry - INFO - Created default tool registry with 8 tools -2025-05-01 21:12:15,682 - httpx - INFO - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK" -2025-05-01 21:56:42,308 - youtube_direct_test - INFO - Starting YouTube direct testing script -2025-05-01 21:56:42,309 - youtube_direct_test - INFO - === Testing URL: https://www.youtube.com/watch?v=L1vXCYZAYYM === -2025-05-01 21:56:42,309 - youtube_direct_test - INFO - Testing YouTube URL: https://www.youtube.com/watch?v=L1vXCYZAYYM -2025-05-01 21:56:43,149 - youtube_direct_test - INFO - Successfully extracted video ID: L1vXCYZAYYM -2025-05-01 21:56:43,910 - gaia_agent.tools.multimodal - ERROR - Transcripts are disabled for this video: -Could not retrieve a transcript for the video https://www.youtube.com/watch?v=L1vXCYZAYYM! This is most likely caused by: - -Subtitles are disabled for this video - -If you are sure that the described cause is not responsible for this error and that a transcript should be retrievable, please create an issue at https://github.com/jdepoix/youtube-transcript-api/issues. Please add which version of youtube_transcript_api you are using and provide the information needed to replicate the error. Also make sure that there are no open issues which already describe your problem! -2025-05-01 21:56:43,911 - youtube_direct_test - ERROR - Error in transcript extraction: Transcripts are disabled for this video -2025-05-01 21:56:43,911 - youtube_direct_test - ERROR - \u274c FAILURE: https://www.youtube.com/watch?v=L1vXCYZAYYM -2025-05-01 21:56:43,913 - youtube_direct_test - ERROR - Stage: transcript_extraction -2025-05-01 21:56:43,913 - youtube_direct_test - ERROR - Error: Transcripts are disabled for this video -2025-05-01 21:56:43,913 - youtube_direct_test - ERROR - Error type: TranscriptsDisabled -2025-05-01 21:56:43,914 - youtube_direct_test - INFO - ================================================================================ -2025-05-01 21:56:43,914 - youtube_direct_test - INFO - === Testing URL: https://www.youtube.com/watch?v=1htKBjuUWec === -2025-05-01 21:56:43,914 - youtube_direct_test - INFO - Testing YouTube URL: https://www.youtube.com/watch?v=1htKBjuUWec -2025-05-01 21:56:44,654 - youtube_direct_test - INFO - Successfully extracted video ID: 1htKBjuUWec -2025-05-01 21:56:47,485 - httpx - INFO - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK" -2025-05-01 21:56:47,493 - youtube_direct_test - INFO - Successfully extracted transcript for video: Teal'c coffee first time -2025-05-01 21:56:47,493 - youtube_direct_test - INFO - \u2705 SUCCESS: https://www.youtube.com/watch?v=1htKBjuUWec -2025-05-01 21:56:47,493 - youtube_direct_test - INFO - Title: Teal'c coffee first time -2025-05-01 21:56:47,494 - youtube_direct_test - INFO - Transcript length: 175 characters -2025-05-01 21:56:47,494 - youtube_direct_test - INFO - Transcript sample: [00:00] Wow this coffee's great I was just -[00:03] thinking that -[00:05] yeah is that cinnamon chicory -[00:17] tea oak -[00:21] [Music] -[00:24] isn't that hot -[00:26] extremely... -2025-05-01 21:56:47,494 - youtube_direct_test - INFO - ================================================================================ -2025-05-01 21:56:47,495 - youtube_direct_test - INFO - Detailed results saved to youtube_test_results.json -2025-05-01 21:56:47,495 - youtube_direct_test - INFO - Test Summary: 1/2 tests passed -2025-05-01 21:57:51,230 - youtube_direct_test - INFO - Starting YouTube direct testing script -2025-05-01 21:57:51,230 - youtube_direct_test - INFO - === Testing URL: https://www.youtube.com/watch?v=L1vXCYZAYYM === -2025-05-01 21:57:51,231 - youtube_direct_test - INFO - Testing YouTube URL: https://www.youtube.com/watch?v=L1vXCYZAYYM -2025-05-01 21:57:52,034 - youtube_direct_test - INFO - Successfully extracted video ID: L1vXCYZAYYM -2025-05-01 21:57:52,760 - gaia_agent.tools.multimodal - ERROR - Transcripts are disabled for this video: -Could not retrieve a transcript for the video https://www.youtube.com/watch?v=L1vXCYZAYYM! This is most likely caused by: - -Subtitles are disabled for this video - -If you are sure that the described cause is not responsible for this error and that a transcript should be retrievable, please create an issue at https://github.com/jdepoix/youtube-transcript-api/issues. Please add which version of youtube_transcript_api you are using and provide the information needed to replicate the error. Also make sure that there are no open issues which already describe your problem! -2025-05-01 21:57:52,761 - youtube_direct_test - ERROR - Error in transcript extraction: Transcripts are disabled for this video -2025-05-01 21:57:52,762 - youtube_direct_test - ERROR - \u274c FAILURE: https://www.youtube.com/watch?v=L1vXCYZAYYM -2025-05-01 21:57:52,762 - youtube_direct_test - ERROR - Stage: transcript_extraction -2025-05-01 21:57:52,762 - youtube_direct_test - ERROR - Error: Transcripts are disabled for this video -2025-05-01 21:57:52,763 - youtube_direct_test - ERROR - Error type: TranscriptsDisabled -2025-05-01 21:57:52,763 - youtube_direct_test - INFO - ================================================================================ -2025-05-01 21:57:52,763 - youtube_direct_test - INFO - === Testing URL: https://www.youtube.com/watch?v=1htKBjuUWec === -2025-05-01 21:57:52,763 - youtube_direct_test - INFO - Testing YouTube URL: https://www.youtube.com/watch?v=1htKBjuUWec -2025-05-01 21:57:53,507 - youtube_direct_test - INFO - Successfully extracted video ID: 1htKBjuUWec -2025-05-01 21:57:57,408 - httpx - INFO - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK" -2025-05-01 21:57:57,416 - youtube_direct_test - INFO - Successfully extracted transcript for video: Teal'c coffee first time -2025-05-01 21:57:57,416 - youtube_direct_test - INFO - \u2705 SUCCESS: https://www.youtube.com/watch?v=1htKBjuUWec -2025-05-01 21:57:57,416 - youtube_direct_test - INFO - Title: Teal'c coffee first time -2025-05-01 21:57:57,417 - youtube_direct_test - INFO - Transcript length: 175 characters -2025-05-01 21:57:57,417 - youtube_direct_test - INFO - Transcript sample: [00:00] Wow this coffee's great I was just -[00:03] thinking that -[00:05] yeah is that cinnamon chicory -[00:17] tea oak -[00:21] [Music] -[00:24] isn't that hot -[00:26] extremely... -2025-05-01 21:57:57,417 - youtube_direct_test - INFO - ================================================================================ -2025-05-01 21:57:57,418 - youtube_direct_test - INFO - Detailed results saved to youtube_test_results.json -2025-05-01 21:57:57,419 - youtube_direct_test - INFO - Test Summary: 1/2 tests passed -2025-05-01 21:58:37,071 - youtube_direct_test - INFO - Starting YouTube direct testing script -2025-05-01 21:58:37,072 - youtube_direct_test - INFO - === Testing URL: https://www.youtube.com/watch?v=L1vXCYZAYYM === -2025-05-01 21:58:37,072 - youtube_direct_test - INFO - Testing YouTube URL: https://www.youtube.com/watch?v=L1vXCYZAYYM -2025-05-01 21:58:37,910 - youtube_direct_test - INFO - Successfully extracted video ID: L1vXCYZAYYM -2025-05-01 21:58:38,713 - gaia_agent.tools.multimodal - ERROR - Transcripts are disabled for this video: -Could not retrieve a transcript for the video https://www.youtube.com/watch?v=L1vXCYZAYYM! This is most likely caused by: - -Subtitles are disabled for this video - -If you are sure that the described cause is not responsible for this error and that a transcript should be retrievable, please create an issue at https://github.com/jdepoix/youtube-transcript-api/issues. Please add which version of youtube_transcript_api you are using and provide the information needed to replicate the error. Also make sure that there are no open issues which already describe your problem! -2025-05-01 21:58:38,715 - youtube_direct_test - ERROR - Error in transcript extraction: Transcripts are disabled for this video -2025-05-01 21:58:38,715 - youtube_direct_test - ERROR - \u274c FAILURE: https://www.youtube.com/watch?v=L1vXCYZAYYM -2025-05-01 21:58:38,715 - youtube_direct_test - ERROR - Stage: transcript_extraction -2025-05-01 21:58:38,715 - youtube_direct_test - ERROR - Error: Transcripts are disabled for this video -2025-05-01 21:58:38,715 - youtube_direct_test - ERROR - Error type: TranscriptsDisabled -2025-05-01 21:58:38,715 - youtube_direct_test - INFO - ================================================================================ -2025-05-01 21:58:38,715 - youtube_direct_test - INFO - === Testing URL: https://www.youtube.com/watch?v=1htKBjuUWec === -2025-05-01 21:58:38,715 - youtube_direct_test - INFO - Testing YouTube URL: https://www.youtube.com/watch?v=1htKBjuUWec -2025-05-01 21:58:39,474 - youtube_direct_test - INFO - Successfully extracted video ID: 1htKBjuUWec -2025-05-01 21:58:41,624 - httpx - INFO - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK" -2025-05-01 21:58:41,632 - youtube_direct_test - INFO - Successfully extracted transcript for video: Teal'c coffee first time -2025-05-01 21:58:41,632 - youtube_direct_test - INFO - \u2705 SUCCESS: https://www.youtube.com/watch?v=1htKBjuUWec -2025-05-01 21:58:41,632 - youtube_direct_test - INFO - Title: Teal'c coffee first time -2025-05-01 21:58:41,632 - youtube_direct_test - INFO - Transcript length: 175 characters -2025-05-01 21:58:41,632 - youtube_direct_test - INFO - Transcript sample: [00:00] Wow this coffee's great I was just -[00:03] thinking that -[00:05] yeah is that cinnamon chicory -[00:17] tea oak -[00:21] [Music] -[00:24] isn't that hot -[00:26] extremely... -2025-05-01 21:58:41,632 - youtube_direct_test - INFO - ================================================================================ -2025-05-01 21:58:41,633 - youtube_direct_test - INFO - Detailed results saved to youtube_test_results.json -2025-05-01 21:58:41,633 - youtube_direct_test - INFO - Test Summary: 1/2 tests passed -2025-05-01 22:00:00,106 - youtube_direct_test - INFO - Starting YouTube direct testing script -2025-05-01 22:00:00,106 - youtube_direct_test - INFO - === Testing URL: https://www.youtube.com/watch?v=L1vXCYZAYYM === -2025-05-01 22:00:00,107 - youtube_direct_test - INFO - Testing YouTube URL: https://www.youtube.com/watch?v=L1vXCYZAYYM -2025-05-01 22:00:00,954 - youtube_direct_test - INFO - Successfully extracted video ID: L1vXCYZAYYM -2025-05-01 22:00:01,663 - gaia_agent.tools.multimodal - ERROR - Transcripts are disabled for this video: -Could not retrieve a transcript for the video https://www.youtube.com/watch?v=L1vXCYZAYYM! This is most likely caused by: - -Subtitles are disabled for this video - -If you are sure that the described cause is not responsible for this error and that a transcript should be retrievable, please create an issue at https://github.com/jdepoix/youtube-transcript-api/issues. Please add which version of youtube_transcript_api you are using and provide the information needed to replicate the error. Also make sure that there are no open issues which already describe your problem! -2025-05-01 22:00:01,664 - youtube_direct_test - ERROR - Error in transcript extraction: Transcripts are disabled for this video -2025-05-01 22:01:02,496 - youtube_direct_test - INFO - Starting YouTube direct testing script -2025-05-01 22:01:02,497 - youtube_direct_test - INFO - === Testing URL: https://www.youtube.com/watch?v=L1vXCYZAYYM === -2025-05-01 22:01:02,497 - youtube_direct_test - INFO - Testing YouTube URL: https://www.youtube.com/watch?v=L1vXCYZAYYM -2025-05-01 22:01:03,286 - youtube_direct_test - INFO - Successfully extracted video ID: L1vXCYZAYYM -2025-05-01 22:01:04,055 - gaia_agent.tools.multimodal - ERROR - Transcripts are disabled for this video: -Could not retrieve a transcript for the video https://www.youtube.com/watch?v=L1vXCYZAYYM! This is most likely caused by: - -Subtitles are disabled for this video - -If you are sure that the described cause is not responsible for this error and that a transcript should be retrievable, please create an issue at https://github.com/jdepoix/youtube-transcript-api/issues. Please add which version of youtube_transcript_api you are using and provide the information needed to replicate the error. Also make sure that there are no open issues which already describe your problem! -2025-05-01 22:01:04,056 - youtube_direct_test - ERROR - Error in transcript extraction: Transcripts are disabled for this video -2025-05-01 22:01:04,056 - youtube_direct_test - ERROR - FAILURE: https://www.youtube.com/watch?v=L1vXCYZAYYM -2025-05-01 22:01:04,056 - youtube_direct_test - ERROR - Stage: transcript_extraction -2025-05-01 22:01:04,056 - youtube_direct_test - ERROR - Error: Transcripts are disabled for this video -2025-05-01 22:01:04,056 - youtube_direct_test - ERROR - Error type: TranscriptsDisabled -2025-05-01 22:01:04,056 - youtube_direct_test - INFO - ================================================================================ -2025-05-01 22:01:04,056 - youtube_direct_test - INFO - === Testing URL: https://www.youtube.com/watch?v=1htKBjuUWec === -2025-05-01 22:01:04,056 - youtube_direct_test - INFO - Testing YouTube URL: https://www.youtube.com/watch?v=1htKBjuUWec -2025-05-01 22:01:04,806 - youtube_direct_test - INFO - Successfully extracted video ID: 1htKBjuUWec -2025-05-01 22:01:06,973 - httpx - INFO - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK" -2025-05-01 22:01:07,372 - youtube_direct_test - INFO - Successfully extracted transcript for video: Teal'c coffee first time -2025-05-01 22:01:07,372 - youtube_direct_test - INFO - SUCCESS: https://www.youtube.com/watch?v=1htKBjuUWec -2025-05-01 22:01:07,372 - youtube_direct_test - INFO - Title: Teal'c coffee first time -2025-05-01 22:01:07,372 - youtube_direct_test - INFO - Transcript length: 175 characters -2025-05-01 22:01:07,372 - youtube_direct_test - INFO - Transcript sample: [00:00] Wow this coffee's great I was just -[00:03] thinking that -[00:05] yeah is that cinnamon chicory -[00:17] tea oak -[00:21] [Music] -[00:24] isn't that hot -[00:26] extremely... -2025-05-01 22:01:07,372 - youtube_direct_test - INFO - ================================================================================ -2025-05-01 22:01:07,373 - youtube_direct_test - INFO - Detailed results saved to youtube_test_results.json -2025-05-01 22:01:07,373 - youtube_direct_test - INFO - Test Summary: 1/2 tests passed -2025-05-01 22:01:59,053 - youtube_direct_test - INFO - Starting YouTube direct testing script -2025-05-01 22:01:59,054 - youtube_direct_test - INFO - === Testing URL: https://www.youtube.com/watch?v=L1vXCYZAYYM === -2025-05-01 22:01:59,054 - youtube_direct_test - INFO - Testing YouTube URL: https://www.youtube.com/watch?v=L1vXCYZAYYM -2025-05-01 22:01:59,819 - youtube_direct_test - INFO - Successfully extracted video ID: L1vXCYZAYYM -2025-05-01 22:02:00,590 - gaia_agent.tools.multimodal - ERROR - Transcripts are disabled for this video: -Could not retrieve a transcript for the video https://www.youtube.com/watch?v=L1vXCYZAYYM! This is most likely caused by: - -Subtitles are disabled for this video - -If you are sure that the described cause is not responsible for this error and that a transcript should be retrievable, please create an issue at https://github.com/jdepoix/youtube-transcript-api/issues. Please add which version of youtube_transcript_api you are using and provide the information needed to replicate the error. Also make sure that there are no open issues which already describe your problem! -2025-05-01 22:02:00,592 - youtube_direct_test - ERROR - Error in transcript extraction: Transcripts are disabled for this video -2025-05-01 22:02:00,592 - youtube_direct_test - ERROR - FAILURE: https://www.youtube.com/watch?v=L1vXCYZAYYM -2025-05-01 22:02:00,592 - youtube_direct_test - ERROR - Stage: transcript_extraction -2025-05-01 22:02:00,592 - youtube_direct_test - ERROR - Error: Transcripts are disabled for this video -2025-05-01 22:02:00,592 - youtube_direct_test - ERROR - Error type: TranscriptsDisabled -2025-05-01 22:02:00,592 - youtube_direct_test - INFO - ================================================================================ -2025-05-01 22:02:00,592 - youtube_direct_test - INFO - === Testing URL: https://www.youtube.com/watch?v=1htKBjuUWec === -2025-05-01 22:02:00,592 - youtube_direct_test - INFO - Testing YouTube URL: https://www.youtube.com/watch?v=1htKBjuUWec -2025-05-01 22:02:01,329 - youtube_direct_test - INFO - Successfully extracted video ID: 1htKBjuUWec -2025-05-01 22:02:05,811 - httpx - INFO - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK" -2025-05-01 22:02:05,819 - youtube_direct_test - INFO - Successfully extracted transcript for video: Teal'c coffee first time -2025-05-01 22:02:05,819 - youtube_direct_test - INFO - SUCCESS: https://www.youtube.com/watch?v=1htKBjuUWec -2025-05-01 22:02:05,819 - youtube_direct_test - INFO - Title: Teal'c coffee first time -2025-05-01 22:02:05,819 - youtube_direct_test - INFO - Transcript length: 175 characters -2025-05-01 22:02:05,819 - youtube_direct_test - INFO - Transcript sample: [00:00] Wow this coffee's great I was just -[00:03] thinking that -[00:05] yeah is that cinnamon chicory -[00:17] tea oak -[00:21] [Music] -[00:24] isn't that hot -[00:26] extremely... -2025-05-01 22:02:05,819 - youtube_direct_test - INFO - ================================================================================ -2025-05-01 22:02:05,820 - youtube_direct_test - INFO - Detailed results saved to youtube_test_results.json -2025-05-01 22:02:05,820 - youtube_direct_test - INFO - Test Summary: 1/2 tests passed -2025-05-01 22:12:13,508 - youtube_direct_test - INFO - Starting YouTube direct testing script -2025-05-01 22:12:13,508 - youtube_direct_test - INFO - === Testing URL: https://www.youtube.com/watch?v=L1vXCYZAYYM === -2025-05-01 22:12:13,509 - youtube_direct_test - INFO - Testing YouTube URL: https://www.youtube.com/watch?v=L1vXCYZAYYM -2025-05-01 22:12:14,343 - gaia_agent.tools.multimodal - WARNING - youtube_transcript_api not installed. Install with: pip install youtube-transcript-api -2025-05-01 22:12:14,344 - youtube_direct_test - INFO - Successfully extracted video ID: L1vXCYZAYYM -2025-05-01 22:12:14,344 - gaia_agent.tools.multimodal - ERROR - youtube_transcript_api not installed. Install with: pip install youtube-transcript-api -2025-05-01 22:12:14,344 - youtube_direct_test - ERROR - Error in transcript extraction: youtube_transcript_api not installed. Install with: pip install youtube-transcript-api -2025-05-01 22:12:14,344 - youtube_direct_test - ERROR - FAILURE: https://www.youtube.com/watch?v=L1vXCYZAYYM -2025-05-01 22:12:14,345 - youtube_direct_test - ERROR - Stage: transcript_extraction -2025-05-01 22:12:14,345 - youtube_direct_test - ERROR - Error: youtube_transcript_api not installed. Install with: pip install youtube-transcript-api -2025-05-01 22:12:14,345 - youtube_direct_test - ERROR - Error type: ImportError -2025-05-01 22:12:14,345 - youtube_direct_test - INFO - ================================================================================ -2025-05-01 22:12:14,346 - youtube_direct_test - INFO - === Testing URL: https://www.youtube.com/watch?v=1htKBjuUWec === -2025-05-01 22:12:14,346 - youtube_direct_test - INFO - Testing YouTube URL: https://www.youtube.com/watch?v=1htKBjuUWec -2025-05-01 22:12:15,297 - gaia_agent.tools.multimodal - WARNING - youtube_transcript_api not installed. Install with: pip install youtube-transcript-api -2025-05-01 22:12:15,297 - youtube_direct_test - INFO - Successfully extracted video ID: 1htKBjuUWec -2025-05-01 22:12:15,297 - gaia_agent.tools.multimodal - ERROR - youtube_transcript_api not installed. Install with: pip install youtube-transcript-api -2025-05-01 22:12:15,298 - youtube_direct_test - ERROR - Error in transcript extraction: youtube_transcript_api not installed. Install with: pip install youtube-transcript-api -2025-05-01 22:12:15,298 - youtube_direct_test - ERROR - FAILURE: https://www.youtube.com/watch?v=1htKBjuUWec -2025-05-01 22:12:15,298 - youtube_direct_test - ERROR - Stage: transcript_extraction -2025-05-01 22:12:15,298 - youtube_direct_test - ERROR - Error: youtube_transcript_api not installed. Install with: pip install youtube-transcript-api -2025-05-01 22:12:15,298 - youtube_direct_test - ERROR - Error type: ImportError -2025-05-01 22:12:15,298 - youtube_direct_test - INFO - ================================================================================ -2025-05-01 22:12:15,299 - youtube_direct_test - INFO - Detailed results saved to youtube_test_results.json -2025-05-01 22:12:15,299 - youtube_direct_test - INFO - Test Summary: 0/2 tests passed -2025-05-01 22:13:56,499 - youtube_direct_test - INFO - Starting YouTube direct testing script -2025-05-01 22:13:56,500 - youtube_direct_test - INFO - === Testing URL: https://www.youtube.com/watch?v=L1vXCYZAYYM === -2025-05-01 22:13:56,500 - youtube_direct_test - INFO - Testing YouTube URL: https://www.youtube.com/watch?v=L1vXCYZAYYM -2025-05-01 22:13:57,358 - youtube_direct_test - INFO - Successfully extracted video ID: L1vXCYZAYYM -2025-05-01 22:13:57,359 - gaia_agent.tools.multimodal - INFO - Extracted video ID 'L1vXCYZAYYM' from URL: https://www.youtube.com/watch?v=L1vXCYZAYYM -2025-05-01 22:13:57,359 - gaia_agent.tools.multimodal - INFO - Getting metadata for video ID: L1vXCYZAYYM -2025-05-01 22:13:57,514 - gaia_agent.tools.multimodal - INFO - Attempting to get transcript for video L1vXCYZAYYM with language auto -2025-05-01 22:13:58,277 - gaia_agent.tools.multimodal - WARNING - Transcript unavailable (attempt 1): -Could not retrieve a transcript for the video https://www.youtube.com/watch?v=L1vXCYZAYYM! This is most likely caused by: - -Subtitles are disabled for this video - -If you are sure that the described cause is not responsible for this error and that a transcript should be retrievable, please create an issue at https://github.com/jdepoix/youtube-transcript-api/issues. Please add which version of youtube_transcript_api you are using and provide the information needed to replicate the error. Also make sure that there are no open issues which already describe your problem! -2025-05-01 22:13:58,278 - gaia_agent.tools.multimodal - ERROR - Transcripts are disabled for this video: -Could not retrieve a transcript for the video https://www.youtube.com/watch?v=L1vXCYZAYYM! This is most likely caused by: - -Subtitles are disabled for this video - -If you are sure that the described cause is not responsible for this error and that a transcript should be retrievable, please create an issue at https://github.com/jdepoix/youtube-transcript-api/issues. Please add which version of youtube_transcript_api you are using and provide the information needed to replicate the error. Also make sure that there are no open issues which already describe your problem! -2025-05-01 22:13:58,279 - youtube_direct_test - ERROR - Error in transcript extraction: Transcripts are disabled for this video: -Could not retrieve a transcript for the video https://www.youtube.com/watch?v=L1vXCYZAYYM! This is most likely caused by: - -Subtitles are disabled for this video - -If you are sure that the described cause is not responsible for this error and that a transcript should be retrievable, please create an issue at https://github.com/jdepoix/youtube-transcript-api/issues. Please add which version of youtube_transcript_api you are using and provide the information needed to replicate the error. Also make sure that there are no open issues which already describe your problem! -2025-05-01 22:13:58,281 - youtube_direct_test - ERROR - FAILURE: https://www.youtube.com/watch?v=L1vXCYZAYYM -2025-05-01 22:13:58,281 - youtube_direct_test - ERROR - Stage: transcript_extraction -2025-05-01 22:13:58,281 - youtube_direct_test - ERROR - Error: Transcripts are disabled for this video: -Could not retrieve a transcript for the video https://www.youtube.com/watch?v=L1vXCYZAYYM! This is most likely caused by: - -Subtitles are disabled for this video - -If you are sure that the described cause is not responsible for this error and that a transcript should be retrievable, please create an issue at https://github.com/jdepoix/youtube-transcript-api/issues. Please add which version of youtube_transcript_api you are using and provide the information needed to replicate the error. Also make sure that there are no open issues which already describe your problem! -2025-05-01 22:13:58,281 - youtube_direct_test - ERROR - Error type: TranscriptsDisabled -2025-05-01 22:13:58,281 - youtube_direct_test - INFO - ================================================================================ -2025-05-01 22:13:58,282 - youtube_direct_test - INFO - === Testing URL: https://www.youtube.com/watch?v=1htKBjuUWec === -2025-05-01 22:13:58,282 - youtube_direct_test - INFO - Testing YouTube URL: https://www.youtube.com/watch?v=1htKBjuUWec -2025-05-01 22:13:59,168 - youtube_direct_test - INFO - Successfully extracted video ID: 1htKBjuUWec -2025-05-01 22:13:59,168 - gaia_agent.tools.multimodal - INFO - Extracted video ID '1htKBjuUWec' from URL: https://www.youtube.com/watch?v=1htKBjuUWec -2025-05-01 22:13:59,168 - gaia_agent.tools.multimodal - INFO - Getting metadata for video ID: 1htKBjuUWec -2025-05-01 22:13:59,252 - gaia_agent.tools.multimodal - INFO - Attempting to get transcript for video 1htKBjuUWec with language auto -2025-05-01 22:13:59,970 - gaia_agent.tools.multimodal - INFO - Processing transcript to identify speakers -2025-05-01 22:14:03,819 - httpx - INFO - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK" -2025-05-01 22:14:03,825 - gaia_agent.tools.multimodal - INFO - Successfully processed transcript with speaker identification -2025-05-01 22:14:03,825 - youtube_direct_test - INFO - Successfully extracted transcript for video: Teal'c coffee first time -2025-05-01 22:14:03,825 - youtube_direct_test - INFO - SUCCESS: https://www.youtube.com/watch?v=1htKBjuUWec -2025-05-01 22:14:03,825 - youtube_direct_test - INFO - Title: Teal'c coffee first time -2025-05-01 22:14:03,825 - youtube_direct_test - INFO - Transcript length: 175 characters -2025-05-01 22:14:03,825 - youtube_direct_test - INFO - Transcript sample: [00:00] Wow this coffee's great I was just -[00:03] thinking that -[00:05] yeah is that cinnamon chicory -[00:17] tea oak -[00:21] [Music] -[00:24] isn't that hot -[00:26] extremely... -2025-05-01 22:14:03,825 - youtube_direct_test - INFO - ================================================================================ -2025-05-01 22:14:03,826 - youtube_direct_test - INFO - Detailed results saved to youtube_test_results.json -2025-05-01 22:14:03,826 - youtube_direct_test - INFO - Test Summary: 1/2 tests passed -2025-05-01 22:14:29,057 - gaia_agent.tools.multimodal - INFO - Using provided video ID: ThisIsNotAValidVideoID -2025-05-01 22:14:29,058 - gaia_agent.tools.multimodal - WARNING - Video ID 'ThisIsNotAValidVideoID' has unusual format -2025-05-01 22:14:29,058 - gaia_agent.tools.multimodal - INFO - Getting metadata for video ID: ThisIsNotAValidVideoID -2025-05-01 22:14:29,167 - gaia_agent.tools.multimodal - WARNING - Failed to get metadata: HTTP 400 -2025-05-01 22:14:29,168 - gaia_agent.tools.multimodal - WARNING - Metadata attempt 1 returned unknown title, retrying... -2025-05-01 22:14:31,168 - gaia_agent.tools.multimodal - INFO - Getting metadata for video ID: ThisIsNotAValidVideoID -2025-05-01 22:14:31,275 - gaia_agent.tools.multimodal - WARNING - Failed to get metadata: HTTP 400 -2025-05-01 22:14:31,276 - gaia_agent.tools.multimodal - WARNING - Metadata attempt 2 returned unknown title, retrying... -2025-05-01 22:14:33,277 - gaia_agent.tools.multimodal - INFO - Getting metadata for video ID: ThisIsNotAValidVideoID -2025-05-01 22:14:33,364 - gaia_agent.tools.multimodal - WARNING - Failed to get metadata: HTTP 400 -2025-05-01 22:14:33,365 - gaia_agent.tools.multimodal - INFO - Attempting to get transcript for video ThisIsNotAValidVideoID with language auto -2025-05-01 22:14:33,583 - gaia_agent.tools.multimodal - WARNING - Transcript unavailable (attempt 1): -Could not retrieve a transcript for the video https://www.youtube.com/watch?v=ThisIsNotAValidVideoID! This is most likely caused by: - -The video is no longer available - -If you are sure that the described cause is not responsible for this error and that a transcript should be retrievable, please create an issue at https://github.com/jdepoix/youtube-transcript-api/issues. Please add which version of youtube_transcript_api you are using and provide the information needed to replicate the error. Also make sure that there are no open issues which already describe your problem! -2025-05-01 22:14:33,583 - gaia_agent.tools.multimodal - ERROR - Video is unavailable: -Could not retrieve a transcript for the video https://www.youtube.com/watch?v=ThisIsNotAValidVideoID! This is most likely caused by: - -The video is no longer available - -If you are sure that the described cause is not responsible for this error and that a transcript should be retrievable, please create an issue at https://github.com/jdepoix/youtube-transcript-api/issues. Please add which version of youtube_transcript_api you are using and provide the information needed to replicate the error. Also make sure that there are no open issues which already describe your problem! -2025-05-01 22:14:36,094 - gaia_agent.tool_registry - INFO - Created default tool registry with 8 tools -2025-05-01 22:14:36,925 - gaia_agent.tools.multimodal - INFO - Extracted video ID '1htKBjuUWec' from URL: https://www.youtube.com/watch?v=1htKBjuUWec -2025-05-01 22:14:36,926 - gaia_agent.tools.multimodal - INFO - Getting metadata for video ID: 1htKBjuUWec -2025-05-01 22:14:37,041 - gaia_agent.tools.multimodal - INFO - Attempting to get transcript for video 1htKBjuUWec with language auto -2025-05-01 22:14:37,683 - gaia_agent.tools.multimodal - INFO - Processing transcript to identify speakers -2025-05-01 22:14:39,818 - httpx - INFO - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK" -2025-05-01 22:14:39,826 - gaia_agent.tools.multimodal - INFO - Successfully processed transcript with speaker identification -2025-05-02 09:18:34,852 - gaia_agent.tools.multimodal - INFO - Using HybridYouTubeVideoTool for combined transcript extraction and browser viewing -2025-05-02 09:18:36,877 - gaia_agent.tool_registry - INFO - Created default tool registry with 8 tools -2025-05-02 09:18:36,878 - gaia_assessment_web_search - INFO - Running search for query: 'What albums did Mercedes Sosa release between 2000 and 2009?' with format: unified -2025-05-02 09:18:38,099 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 200 -2025-05-02 09:18:38,114 - gaia_assessment_web_search - INFO - Running search for query: 'What albums did Mercedes Sosa release between 2000 and 2009?' with format: robust -2025-05-02 09:18:39,211 - primp - INFO - response: https://html.duckduckgo.com/html 200 -2025-05-02 09:18:39,218 - gaia_assessment_web_search - INFO - Running search for query: 'What albums did Mercedes Sosa release between 2000 and 2009?' with format: merged -2025-05-02 09:18:40,287 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 200 -2025-05-02 09:18:40,292 - gaia_assessment_web_search - INFO - Running search for query: 'Who nominated the Spinosaurus article for featured status on Wikipedia?' with format: unified -2025-05-02 09:18:40,514 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 200 -2025-05-02 09:18:41,343 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 200 -2025-05-02 09:18:41,347 - gaia_assessment_web_search - INFO - Running search for query: 'Who nominated the Spinosaurus article for featured status on Wikipedia?' with format: robust -2025-05-02 09:18:41,548 - primp - INFO - response: https://html.duckduckgo.com/html 202 -2025-05-02 09:18:41,548 - duckduckgo_search.DDGS - INFO - Error to search using html backend: https://html.duckduckgo.com/html 202 Ratelimit -2025-05-02 09:18:42,327 - primp - INFO - response: https://html.duckduckgo.com/html 200 -2025-05-02 09:18:42,501 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 202 -2025-05-02 09:18:42,502 - duckduckgo_search.DDGS - INFO - Error to search using lite backend: https://lite.duckduckgo.com/lite/ 202 Ratelimit -2025-05-02 09:18:42,502 - gaia_agent.tools.web - WARNING - DuckDuckGo Wikipedia search failed: https://lite.duckduckgo.com/lite/ 202 Ratelimit -2025-05-02 09:18:42,925 - gaia_assessment_web_search - INFO - Running search for query: 'Who nominated the Spinosaurus article for featured status on Wikipedia?' with format: merged -2025-05-02 09:18:43,130 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 202 -2025-05-02 09:18:43,131 - duckduckgo_search.DDGS - INFO - Error to search using lite backend: https://lite.duckduckgo.com/lite/ 202 Ratelimit -2025-05-02 09:18:43,840 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 200 -2025-05-02 09:18:44,113 - primp - INFO - response: https://html.duckduckgo.com/html 202 -2025-05-02 09:18:44,114 - duckduckgo_search.DDGS - INFO - Error to search using html backend: https://html.duckduckgo.com/html 202 Ratelimit -2025-05-02 09:18:44,114 - gaia_agent.tools.web - WARNING - DuckDuckGo Wikipedia search failed: https://html.duckduckgo.com/html 202 Ratelimit -2025-05-02 09:18:44,423 - gaia_assessment_web_search - INFO - Running search for query: 'What is the NASA award number mentioned in the Universe Today article about exoplanet research?' with format: unified -2025-05-02 09:18:44,589 - primp - INFO - response: https://html.duckduckgo.com/html 202 -2025-05-02 09:18:44,590 - duckduckgo_search.DDGS - INFO - Error to search using html backend: https://html.duckduckgo.com/html 202 Ratelimit -2025-05-02 09:18:45,512 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 202 -2025-05-02 09:18:45,513 - duckduckgo_search.DDGS - INFO - Error to search using lite backend: https://lite.duckduckgo.com/lite/ 202 Ratelimit -2025-05-02 09:18:45,513 - gaia_agent.tools.web - ERROR - Error searching DuckDuckGo: https://lite.duckduckgo.com/lite/ 202 Ratelimit -2025-05-02 09:18:45,517 - gaia_agent.tools.web - ERROR - Traceback (most recent call last): - File "c:\Users\joach\OneDrive - VISCONSULT\00_VISCONSULT\05_AI_and_Digitalization\02_AI_Agents\02_HF_Agents_Course\99_Final_Assignment\tools\web_tools.py", line 192, in search - results = list(ddgs.text( - ~~~~~~~~~^ - query, - ^^^^^^ - max_results=self.max_results, - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - timelimit=self.ddg_timeout - ^^^^^^^^^^^^^^^^^^^^^^^^^^ - )) - ^ - File "C:\Users\joach\AppData\Roaming\Python\Python313\site-packages\duckduckgo_search\duckduckgo_search.py", line 184, in text - raise DuckDuckGoSearchException(err) -duckduckgo_search.exceptions.DuckDuckGoSearchException: https://lite.duckduckgo.com/lite/ 202 Ratelimit - -2025-05-02 09:18:45,517 - gaia_agent.tools.web - INFO - Returning empty results due to DuckDuckGo search failure -2025-05-02 09:18:45,527 - gaia_assessment_web_search - INFO - Running search for query: 'What is the NASA award number mentioned in the Universe Today article about exoplanet research?' with format: robust -2025-05-02 09:18:45,763 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 202 -2025-05-02 09:18:45,763 - duckduckgo_search.DDGS - INFO - Error to search using lite backend: https://lite.duckduckgo.com/lite/ 202 Ratelimit -2025-05-02 09:18:46,670 - primp - INFO - response: https://html.duckduckgo.com/html 202 -2025-05-02 09:18:46,671 - duckduckgo_search.DDGS - INFO - Error to search using html backend: https://html.duckduckgo.com/html 202 Ratelimit -2025-05-02 09:18:46,671 - gaia_agent.tools.web - ERROR - Error searching DuckDuckGo: https://html.duckduckgo.com/html 202 Ratelimit -2025-05-02 09:18:46,672 - gaia_agent.tools.web - ERROR - Traceback (most recent call last): - File "c:\Users\joach\OneDrive - VISCONSULT\00_VISCONSULT\05_AI_and_Digitalization\02_AI_Agents\02_HF_Agents_Course\99_Final_Assignment\tools\web_tools.py", line 192, in search - results = list(ddgs.text( - ~~~~~~~~~^ - query, - ^^^^^^ - max_results=self.max_results, - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - timelimit=self.ddg_timeout - ^^^^^^^^^^^^^^^^^^^^^^^^^^ - )) - ^ - File "C:\Users\joach\AppData\Roaming\Python\Python313\site-packages\duckduckgo_search\duckduckgo_search.py", line 184, in text - raise DuckDuckGoSearchException(err) -duckduckgo_search.exceptions.DuckDuckGoSearchException: https://html.duckduckgo.com/html 202 Ratelimit - -2025-05-02 09:18:46,672 - gaia_agent.tools.web - INFO - Returning empty results due to DuckDuckGo search failure -2025-05-02 09:18:46,673 - gaia_assessment_web_search - INFO - Running search for query: 'What is the NASA award number mentioned in the Universe Today article about exoplanet research?' with format: merged -2025-05-02 09:18:46,834 - primp - INFO - response: https://html.duckduckgo.com/html 202 -2025-05-02 09:18:46,835 - duckduckgo_search.DDGS - INFO - Error to search using html backend: https://html.duckduckgo.com/html 202 Ratelimit -2025-05-02 09:18:47,789 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 202 -2025-05-02 09:18:47,789 - duckduckgo_search.DDGS - INFO - Error to search using lite backend: https://lite.duckduckgo.com/lite/ 202 Ratelimit -2025-05-02 09:18:47,789 - gaia_agent.tools.web - ERROR - Error searching DuckDuckGo: https://lite.duckduckgo.com/lite/ 202 Ratelimit -2025-05-02 09:18:47,790 - gaia_agent.tools.web - ERROR - Traceback (most recent call last): - File "c:\Users\joach\OneDrive - VISCONSULT\00_VISCONSULT\05_AI_and_Digitalization\02_AI_Agents\02_HF_Agents_Course\99_Final_Assignment\tools\web_tools.py", line 192, in search - results = list(ddgs.text( - ~~~~~~~~~^ - query, - ^^^^^^ - max_results=self.max_results, - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - timelimit=self.ddg_timeout - ^^^^^^^^^^^^^^^^^^^^^^^^^^ - )) - ^ - File "C:\Users\joach\AppData\Roaming\Python\Python313\site-packages\duckduckgo_search\duckduckgo_search.py", line 184, in text - raise DuckDuckGoSearchException(err) -duckduckgo_search.exceptions.DuckDuckGoSearchException: https://lite.duckduckgo.com/lite/ 202 Ratelimit - -2025-05-02 09:18:47,790 - gaia_agent.tools.web - INFO - Returning empty results due to DuckDuckGo search failure -2025-05-02 09:18:47,790 - gaia_assessment_web_search - INFO - Running search for query: 'Who are the recent recipients of the Malko Competition?' with format: unified -2025-05-02 09:18:47,951 - primp - INFO - response: https://html.duckduckgo.com/html 202 -2025-05-02 09:18:47,951 - duckduckgo_search.DDGS - INFO - Error to search using html backend: https://html.duckduckgo.com/html 202 Ratelimit -2025-05-02 09:18:48,867 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 202 -2025-05-02 09:18:48,868 - duckduckgo_search.DDGS - INFO - Error to search using lite backend: https://lite.duckduckgo.com/lite/ 202 Ratelimit -2025-05-02 09:18:48,868 - gaia_agent.tools.web - ERROR - Error searching DuckDuckGo: https://lite.duckduckgo.com/lite/ 202 Ratelimit -2025-05-02 09:18:48,869 - gaia_agent.tools.web - ERROR - Traceback (most recent call last): - File "c:\Users\joach\OneDrive - VISCONSULT\00_VISCONSULT\05_AI_and_Digitalization\02_AI_Agents\02_HF_Agents_Course\99_Final_Assignment\tools\web_tools.py", line 192, in search - results = list(ddgs.text( - ~~~~~~~~~^ - query, - ^^^^^^ - max_results=self.max_results, - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - timelimit=self.ddg_timeout - ^^^^^^^^^^^^^^^^^^^^^^^^^^ - )) - ^ - File "C:\Users\joach\AppData\Roaming\Python\Python313\site-packages\duckduckgo_search\duckduckgo_search.py", line 184, in text - raise DuckDuckGoSearchException(err) -duckduckgo_search.exceptions.DuckDuckGoSearchException: https://lite.duckduckgo.com/lite/ 202 Ratelimit - -2025-05-02 09:18:48,869 - gaia_agent.tools.web - INFO - Returning empty results due to DuckDuckGo search failure -2025-05-02 09:18:48,870 - gaia_assessment_web_search - INFO - Running search for query: 'Who are the recent recipients of the Malko Competition?' with format: robust -2025-05-02 09:18:49,037 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 202 -2025-05-02 09:18:49,037 - duckduckgo_search.DDGS - INFO - Error to search using lite backend: https://lite.duckduckgo.com/lite/ 202 Ratelimit -2025-05-02 09:18:49,933 - primp - INFO - response: https://html.duckduckgo.com/html 202 -2025-05-02 09:18:49,933 - duckduckgo_search.DDGS - INFO - Error to search using html backend: https://html.duckduckgo.com/html 202 Ratelimit -2025-05-02 09:18:49,934 - gaia_agent.tools.web - ERROR - Error searching DuckDuckGo: https://html.duckduckgo.com/html 202 Ratelimit -2025-05-02 09:18:49,934 - gaia_agent.tools.web - ERROR - Traceback (most recent call last): - File "c:\Users\joach\OneDrive - VISCONSULT\00_VISCONSULT\05_AI_and_Digitalization\02_AI_Agents\02_HF_Agents_Course\99_Final_Assignment\tools\web_tools.py", line 192, in search - results = list(ddgs.text( - ~~~~~~~~~^ - query, - ^^^^^^ - max_results=self.max_results, - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - timelimit=self.ddg_timeout - ^^^^^^^^^^^^^^^^^^^^^^^^^^ - )) - ^ - File "C:\Users\joach\AppData\Roaming\Python\Python313\site-packages\duckduckgo_search\duckduckgo_search.py", line 184, in text - raise DuckDuckGoSearchException(err) -duckduckgo_search.exceptions.DuckDuckGoSearchException: https://html.duckduckgo.com/html 202 Ratelimit - -2025-05-02 09:18:49,934 - gaia_agent.tools.web - INFO - Returning empty results due to DuckDuckGo search failure -2025-05-02 09:18:49,935 - gaia_assessment_web_search - INFO - Running search for query: 'Who are the recent recipients of the Malko Competition?' with format: merged -2025-05-02 09:18:50,083 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 202 -2025-05-02 09:18:50,083 - duckduckgo_search.DDGS - INFO - Error to search using lite backend: https://lite.duckduckgo.com/lite/ 202 Ratelimit -2025-05-02 09:18:51,002 - primp - INFO - response: https://html.duckduckgo.com/html 202 -2025-05-02 09:18:51,003 - duckduckgo_search.DDGS - INFO - Error to search using html backend: https://html.duckduckgo.com/html 202 Ratelimit -2025-05-02 09:18:51,003 - gaia_agent.tools.web - ERROR - Error searching DuckDuckGo: https://html.duckduckgo.com/html 202 Ratelimit -2025-05-02 09:18:51,004 - gaia_agent.tools.web - ERROR - Traceback (most recent call last): - File "c:\Users\joach\OneDrive - VISCONSULT\00_VISCONSULT\05_AI_and_Digitalization\02_AI_Agents\02_HF_Agents_Course\99_Final_Assignment\tools\web_tools.py", line 192, in search - results = list(ddgs.text( - ~~~~~~~~~^ - query, - ^^^^^^ - max_results=self.max_results, - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - timelimit=self.ddg_timeout - ^^^^^^^^^^^^^^^^^^^^^^^^^^ - )) - ^ - File "C:\Users\joach\AppData\Roaming\Python\Python313\site-packages\duckduckgo_search\duckduckgo_search.py", line 184, in text - raise DuckDuckGoSearchException(err) -duckduckgo_search.exceptions.DuckDuckGoSearchException: https://html.duckduckgo.com/html 202 Ratelimit - -2025-05-02 09:18:51,004 - gaia_agent.tools.web - INFO - Returning empty results due to DuckDuckGo search failure -2025-05-02 09:21:36,654 - gaia_agent.tools.multimodal - INFO - Using HybridYouTubeVideoTool for combined transcript extraction and browser viewing -2025-05-02 09:21:38,243 - gaia_agent.tool_registry - INFO - Created default tool registry with 8 tools -2025-05-02 09:22:16,957 - gaia_agent.tools.multimodal - INFO - Using HybridYouTubeVideoTool for combined transcript extraction and browser viewing -2025-05-02 09:22:18,472 - gaia_agent.tool_registry - INFO - Created default tool registry with 8 tools -2025-05-02 09:22:19,325 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 200 -2025-05-02 09:22:19,474 - primp - INFO - response: https://html.duckduckgo.com/html 200 -2025-05-02 09:22:19,478 - gaia_agent.tools.multimodal - INFO - Using HybridYouTubeVideoTool for combined transcript extraction and browser viewing -2025-05-02 09:22:20,959 - gaia_agent.tool_registry - INFO - Created default tool registry with 8 tools -2025-05-02 09:22:21,864 - primp - INFO - response: https://html.duckduckgo.com/html 200 -2025-05-02 09:22:22,693 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 200 -2025-05-02 09:22:23,630 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 200 -2025-05-02 09:22:23,633 - gaia_agent.tools.multimodal - INFO - Using HybridYouTubeVideoTool for combined transcript extraction and browser viewing -2025-05-02 09:22:25,168 - gaia_agent.tool_registry - INFO - Created default tool registry with 8 tools -2025-05-02 09:22:25,311 - primp - INFO - response: https://html.duckduckgo.com/html 202 -2025-05-02 09:22:25,311 - duckduckgo_search.DDGS - INFO - Error to search using html backend: https://html.duckduckgo.com/html 202 Ratelimit -2025-05-02 09:22:26,210 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 202 -2025-05-02 09:22:26,210 - duckduckgo_search.DDGS - INFO - Error to search using lite backend: https://lite.duckduckgo.com/lite/ 202 Ratelimit -2025-05-02 09:22:26,210 - gaia_agent.tools.web - ERROR - Error searching DuckDuckGo: https://lite.duckduckgo.com/lite/ 202 Ratelimit -2025-05-02 09:22:26,212 - gaia_agent.tools.web - ERROR - Traceback (most recent call last): - File "c:\Users\joach\OneDrive - VISCONSULT\00_VISCONSULT\05_AI_and_Digitalization\02_AI_Agents\02_HF_Agents_Course\99_Final_Assignment\tools\web_tools.py", line 192, in search - results = list(ddgs.text( - ~~~~~~~~~^ - query, - ^^^^^^ - max_results=self.max_results, - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - timelimit=self.ddg_timeout - ^^^^^^^^^^^^^^^^^^^^^^^^^^ - )) - ^ - File "C:\Users\joach\AppData\Roaming\Python\Python313\site-packages\duckduckgo_search\duckduckgo_search.py", line 184, in text - raise DuckDuckGoSearchException(err) -duckduckgo_search.exceptions.DuckDuckGoSearchException: https://lite.duckduckgo.com/lite/ 202 Ratelimit - -2025-05-02 09:22:26,212 - gaia_agent.tools.web - INFO - Returning empty results due to DuckDuckGo search failure -2025-05-02 09:22:37,989 - gaia_agent.tools.multimodal - INFO - Using HybridYouTubeVideoTool for combined transcript extraction and browser viewing -2025-05-02 09:22:39,691 - gaia_agent.tool_registry - INFO - Created default tool registry with 8 tools -2025-05-02 09:22:39,691 - gaia_agent.tools.multimodal - INFO - Using HybridYouTubeVideoTool for combined transcript extraction and browser viewing -2025-05-02 09:22:41,269 - gaia_agent.tool_registry - INFO - Created default tool registry with 8 tools -2025-05-02 09:22:41,276 - gaia_agent.tools.multimodal - INFO - Using HybridYouTubeVideoTool for combined transcript extraction and browser viewing -2025-05-02 09:22:43,017 - gaia_agent.tool_registry - INFO - Created default tool registry with 8 tools -2025-05-02 09:49:27,834 - run_gaia_web_search_tests - INFO - Running direct tests... -2025-05-02 09:50:30,000 - run_gaia_web_search_tests - INFO - Running direct tests... -2025-05-02 09:50:30,001 - gaia_agent.tools.web - INFO - Added Perplexity tool to EnhancedWebSearchTool -2025-05-02 09:50:30,002 - gaia_agent.tools.web - INFO - Added Wikipedia tool to EnhancedWebSearchTool -2025-05-02 09:50:30,002 - gaia_agent.tools.web - INFO - Added Serper tool to EnhancedWebSearchTool -2025-05-02 09:50:30,002 - gaia_agent.tools.web - INFO - Added DuckDuckGo tool to EnhancedWebSearchTool -2025-05-02 09:50:30,003 - test_gaia_web_search - INFO - Testing question: mercedes_sosa -2025-05-02 09:50:30,003 - test_gaia_web_search - INFO - Query: How many studio albums were published by Mercedes Sosa between 2000 and 2009 (included)? -2025-05-02 09:50:30,003 - gaia_agent.tools.web - INFO - Using Perplexity for query: How many studio albums were published by Mercedes Sosa between 2000 and 2009 (included)? -2025-05-02 09:50:38,288 - test_gaia_web_search - INFO - Found 1 results -2025-05-02 09:50:38,288 - test_gaia_web_search - INFO - Found 4/4 expected keywords -2025-05-02 09:50:38,288 - test_gaia_web_search - INFO - Success percentage: 100.00% -2025-05-02 09:50:38,288 - test_gaia_web_search - INFO - Testing question: wikipedia_dinosaur -2025-05-02 09:50:38,288 - test_gaia_web_search - INFO - Query: Who nominated the only Featured Article on English Wikipedia about a dinosaur that was promoted in November 2016? -2025-05-02 09:50:38,288 - gaia_agent.tools.web - INFO - Using Perplexity for query: Who nominated the only Featured Article on English Wikipedia about a dinosaur that was promoted in November 2016? -2025-05-02 09:50:47,453 - test_gaia_web_search - INFO - Found 1 results -2025-05-02 09:50:47,455 - test_gaia_web_search - INFO - Found 5/5 expected keywords -2025-05-02 09:50:47,456 - test_gaia_web_search - INFO - Success percentage: 100.00% -2025-05-02 09:50:47,456 - test_gaia_web_search - INFO - Testing question: universe_today -2025-05-02 09:50:47,457 - test_gaia_web_search - INFO - Query: On June 6, 2023, an article by Carolyn Collins Petersen was published in Universe Today. This article mentions a team that produced a paper about their observations, linked at the bottom of the article. Find this paper. Under what NASA award number was the work performed by R. G. Arendt supported by? -2025-05-02 09:50:47,457 - gaia_agent.tools.web - INFO - Using Perplexity for query: On June 6, 2023, an article by Carolyn Collins Petersen was published in Universe Today. This article mentions a team that produced a paper about their observations, linked at the bottom of the article. Find this paper. Under what NASA award number was the work performed by R. G. Arendt supported by? -2025-05-02 09:50:56,762 - test_gaia_web_search - INFO - Found 1 results -2025-05-02 09:50:56,762 - test_gaia_web_search - INFO - Found 4/4 expected keywords -2025-05-02 09:50:56,762 - test_gaia_web_search - INFO - Success percentage: 100.00% -2025-05-02 09:50:56,762 - test_gaia_web_search - INFO - Testing question: malko_competition -2025-05-02 09:50:56,763 - test_gaia_web_search - INFO - Query: Who won the Malko Competition for Young Conductors in 2021? -2025-05-02 09:50:56,763 - gaia_agent.tools.web - INFO - Using Perplexity for query: Who won the Malko Competition for Young Conductors in 2021? -2025-05-02 09:51:03,855 - test_gaia_web_search - INFO - Found 1 results -2025-05-02 09:51:03,855 - test_gaia_web_search - INFO - Found 3/4 expected keywords -2025-05-02 09:51:03,855 - test_gaia_web_search - INFO - Success percentage: 75.00% -2025-05-02 09:51:03,856 - test_gaia_web_search - INFO - - -=== TEST SUMMARY === -2025-05-02 09:51:03,856 - test_gaia_web_search - INFO - Successfully answered 4/4 questions -2025-05-02 09:51:03,856 - test_gaia_web_search - INFO - \u2705 SUCCESS - mercedes_sosa -2025-05-02 09:51:03,858 - test_gaia_web_search - INFO - \u2705 SUCCESS - wikipedia_dinosaur -2025-05-02 09:51:03,858 - test_gaia_web_search - INFO - \u2705 SUCCESS - universe_today -2025-05-02 09:51:03,858 - test_gaia_web_search - INFO - \u2705 SUCCESS - malko_competition -2025-05-02 09:51:03,860 - test_gaia_web_search - INFO - Results saved to gaia_web_search_results.json -2025-05-02 09:51:03,861 - run_gaia_web_search_tests - INFO - -Tests completed successfully! -2025-05-02 09:57:50,946 - enhanced_web_search_example - INFO - Creating tool registry... -2025-05-02 09:57:50,947 - gaia_agent.tools.web - INFO - Added Perplexity tool to EnhancedWebSearchTool -2025-05-02 09:57:50,947 - gaia_agent.tools.web - INFO - Added Wikipedia tool to EnhancedWebSearchTool -2025-05-02 09:57:50,947 - gaia_agent.tools.web - INFO - Added Serper tool to EnhancedWebSearchTool -2025-05-02 09:57:50,948 - gaia_agent.tools.web - INFO - Added DuckDuckGo tool to EnhancedWebSearchTool -2025-05-02 09:57:50,948 - gaia_agent.tool_registry - INFO - Registered Enhanced Web Search tool -2025-05-02 09:57:50,948 - gaia_agent.tools.multimodal - INFO - Using HybridYouTubeVideoTool for combined transcript extraction and browser viewing -2025-05-02 09:57:52,568 - gaia_agent.tool_registry - INFO - Created default tool registry with 9 tools -2025-05-02 09:57:52,568 - enhanced_web_search_example - INFO - - -Question 1: How many studio albums were published by Mercedes Sosa between 2000 and 2009 (included)? -2025-05-02 09:57:52,568 - enhanced_web_search_example - INFO - Searching using unified_search... -2025-05-02 09:57:52,568 - gaia_agent.tool_registry - INFO - Using enhanced web search for query: How many studio albums were published by Mercedes Sosa between 2000 and 2009 (included)? -2025-05-02 09:57:52,568 - gaia_agent.tools.web - INFO - Using Perplexity for query: How many studio albums were published by Mercedes Sosa between 2000 and 2009 (included)? -2025-05-02 09:58:00,753 - gaia_agent.tool_registry - INFO - Enhanced web search returned high-quality results, skipping other tools -2025-05-02 09:58:00,754 - enhanced_web_search_example - INFO - Found 1 results -2025-05-02 09:58:00,754 - enhanced_web_search_example - INFO - Providers used: ['enhanced_web_search'] -2025-05-02 09:58:00,754 - enhanced_web_search_example - INFO - -Top result: -2025-05-02 09:58:00,754 - enhanced_web_search_example - INFO - Title: Perplexity AI Search Result -2025-05-02 09:58:00,754 - enhanced_web_search_example - INFO - Source: perplexity -2025-05-02 09:58:00,754 - enhanced_web_search_example - INFO - Relevance score: 10.0 -2025-05-02 09:58:00,754 - enhanced_web_search_example - INFO - Snippet: -Okay, let's tackle this question: How many studio albums were published by Mercedes Sosa between 2000 and 2009, inclusive? First, I need to go through the provided search results to find the relevant information. - -Starting with search result [4], which lists albums from Album of The Year. The entries here include "Misa Criolla" (2000), "Al Despertar" (1998), "Escondido En Mi País" (1996), "Gestos De Amor" (1994), "Sino" (1992), "De Mí" (1991), "Mercedes Sosa 87" (1987), "Vivir" (1987), "... -2025-05-02 09:58:00,754 - enhanced_web_search_example - INFO - - -Question 2: Who nominated the only Featured Article on English Wikipedia about a dinosaur that was promoted in November 2016? -2025-05-02 09:58:00,754 - enhanced_web_search_example - INFO - Searching using unified_search... -2025-05-02 09:58:00,754 - gaia_agent.tool_registry - INFO - Using enhanced web search for query: Who nominated the only Featured Article on English Wikipedia about a dinosaur that was promoted in November 2016? -2025-05-02 09:58:00,754 - gaia_agent.tools.web - INFO - Using Perplexity for query: Who nominated the only Featured Article on English Wikipedia about a dinosaur that was promoted in November 2016? -2025-05-02 09:58:08,887 - gaia_agent.tool_registry - INFO - Enhanced web search returned high-quality results, skipping other tools -2025-05-02 09:58:08,888 - enhanced_web_search_example - INFO - Found 1 results -2025-05-02 09:58:08,888 - enhanced_web_search_example - INFO - Providers used: ['enhanced_web_search'] -2025-05-02 09:58:08,888 - enhanced_web_search_example - INFO - -Top result: -2025-05-02 09:58:08,888 - enhanced_web_search_example - INFO - Title: Perplexity AI Search Result -2025-05-02 09:58:08,888 - enhanced_web_search_example - INFO - Source: perplexity -2025-05-02 09:58:08,888 - enhanced_web_search_example - INFO - Relevance score: 10.0 -2025-05-02 09:58:08,888 - enhanced_web_search_example - INFO - Snippet: -Okay, let's tackle this query step by step. The user is asking who nominated the only Featured Article on English Wikipedia about a dinosaur that was promoted in November 2016. - -First, I need to parse the question correctly. The key points here are: Featured Article, English Wikipedia, dinosaur-related, promoted in November 2016, and the nominator. - -Looking at the search results provided, let's go through each one to find relevant information. - -Search result [1] is from Wikipedia's Fea... -2025-05-02 09:58:08,888 - enhanced_web_search_example - INFO - - -Question 3: On June 6, 2023, an article by Carolyn Collins Petersen was published in Universe Today. This article mentions a team that produced a paper about their observations, linked at the bottom of the article. Find this paper. Under what NASA award number was the work performed by R. G. Arendt supported by? -2025-05-02 09:58:08,888 - enhanced_web_search_example - INFO - Searching using unified_search... -2025-05-02 09:58:08,888 - gaia_agent.tool_registry - INFO - Using enhanced web search for query: On June 6, 2023, an article by Carolyn Collins Petersen was published in Universe Today. This article mentions a team that produced a paper about their observations, linked at the bottom of the article. Find this paper. Under what NASA award number was the work performed by R. G. Arendt supported by? -2025-05-02 09:58:08,889 - gaia_agent.tools.web - INFO - Using Perplexity for query: On June 6, 2023, an article by Carolyn Collins Petersen was published in Universe Today. This article mentions a team that produced a paper about their observations, linked at the bottom of the article. Find this paper. Under what NASA award number was the work performed by R. G. Arendt supported by? -2025-05-02 09:58:17,161 - gaia_agent.tool_registry - INFO - Enhanced web search returned high-quality results, skipping other tools -2025-05-02 09:58:17,161 - enhanced_web_search_example - INFO - Found 1 results -2025-05-02 09:58:17,161 - enhanced_web_search_example - INFO - Providers used: ['enhanced_web_search'] -2025-05-02 09:58:17,161 - enhanced_web_search_example - INFO - -Top result: -2025-05-02 09:58:17,161 - enhanced_web_search_example - INFO - Title: Perplexity AI Search Result -2025-05-02 09:58:17,161 - enhanced_web_search_example - INFO - Source: perplexity -2025-05-02 09:58:17,161 - enhanced_web_search_example - INFO - Relevance score: 10.0 -2025-05-02 09:58:17,161 - enhanced_web_search_example - INFO - Snippet: -Okay, let's tackle this query step by step. The user is asking about an article by Carolyn Collins Petersen published on June 6, 2023, in Universe Today. Specifically, they want to find the paper mentioned at the bottom of that article and determine the NASA award number under which the work by R. G. Arendt was supported. - -First, I need to check the search results provided. Looking at the results, [1] and [2] both mention the article from June 6, 2023, about mysterious filaments in the M... -2025-05-02 09:58:17,161 - enhanced_web_search_example - INFO - - -Question 4: Who won the Malko Competition for Young Conductors in 2021? -2025-05-02 09:58:17,161 - enhanced_web_search_example - INFO - Searching using unified_search... -2025-05-02 09:58:17,161 - gaia_agent.tool_registry - INFO - Using enhanced web search for query: Who won the Malko Competition for Young Conductors in 2021? -2025-05-02 09:58:17,161 - gaia_agent.tools.web - INFO - Using Perplexity for query: Who won the Malko Competition for Young Conductors in 2021? -2025-05-02 09:58:24,934 - gaia_agent.tool_registry - INFO - Enhanced web search returned high-quality results, skipping other tools -2025-05-02 09:58:24,934 - enhanced_web_search_example - INFO - Found 1 results -2025-05-02 09:58:24,934 - enhanced_web_search_example - INFO - Providers used: ['enhanced_web_search'] -2025-05-02 09:58:24,934 - enhanced_web_search_example - INFO - -Top result: -2025-05-02 09:58:24,934 - enhanced_web_search_example - INFO - Title: Perplexity AI Search Result -2025-05-02 09:58:24,934 - enhanced_web_search_example - INFO - Source: perplexity -2025-05-02 09:58:24,934 - enhanced_web_search_example - INFO - Relevance score: 10.0 -2025-05-02 09:58:24,934 - enhanced_web_search_example - INFO - Snippet: -Okay, let me tackle this query step by step. The user is asking who won the Malko Competition for Young Conductors in 2021. I need to provide a comprehensive answer with relevant facts and proper citations. - -First, I'll check the search results provided. Looking at result , the Wikipedia list shows Dmitry Matvienko from Belarus as the 2021 winner. That's a clear indication. Then, result [1] from the official Malko Competition website also confirms this, listing Dmitry Matvienko for 2021 ... -2025-05-02 09:58:24,934 - enhanced_web_search_example - INFO - -Example completed successfully! -2025-05-02 10:05:23,247 - run_gaia_web_search_tests - INFO - Running direct tests... -2025-05-02 10:05:23,248 - gaia_agent.tools.web - INFO - Added Perplexity tool to EnhancedWebSearchTool -2025-05-02 10:05:23,249 - gaia_agent.tools.multimodal - INFO - Using HybridYouTubeVideoTool for combined transcript extraction and browser viewing -2025-05-02 10:05:25,062 - gaia_agent.tools.web - INFO - Added YouTube tool to EnhancedWebSearchTool -2025-05-02 10:05:25,063 - gaia_agent.tools.web - INFO - Added Wikipedia tool to EnhancedWebSearchTool -2025-05-02 10:05:25,063 - gaia_agent.tools.web - INFO - Added Serper tool to EnhancedWebSearchTool -2025-05-02 10:05:25,063 - gaia_agent.tools.web - INFO - Added DuckDuckGo tool to EnhancedWebSearchTool -2025-05-02 10:05:25,064 - test_gaia_web_search - INFO - Testing question: mercedes_sosa -2025-05-02 10:05:25,064 - test_gaia_web_search - INFO - Query: How many studio albums were published by Mercedes Sosa between 2000 and 2009 (included)? -2025-05-02 10:05:25,064 - gaia_agent.tools.web - INFO - Using Perplexity for query: How many studio albums were published by Mercedes Sosa between 2000 and 2009 (included)? -2025-05-02 10:05:33,200 - test_gaia_web_search - INFO - Found 1 results -2025-05-02 10:05:33,201 - test_gaia_web_search - INFO - Found 4/4 expected keywords -2025-05-02 10:05:33,201 - test_gaia_web_search - INFO - Success percentage: 100.00% -2025-05-02 10:05:33,202 - test_gaia_web_search - INFO - Testing question: wikipedia_dinosaur -2025-05-02 10:05:33,202 - test_gaia_web_search - INFO - Query: Who nominated the only Featured Article on English Wikipedia about a dinosaur that was promoted in November 2016? -2025-05-02 10:05:33,202 - gaia_agent.tools.web - INFO - Using Perplexity for query: Who nominated the only Featured Article on English Wikipedia about a dinosaur that was promoted in November 2016? -2025-05-02 10:05:40,864 - test_gaia_web_search - INFO - Found 1 results -2025-05-02 10:05:40,865 - test_gaia_web_search - INFO - Found 3/5 expected keywords -2025-05-02 10:05:40,865 - test_gaia_web_search - INFO - Success percentage: 60.00% -2025-05-02 10:05:40,865 - test_gaia_web_search - INFO - Testing question: universe_today -2025-05-02 10:05:40,865 - test_gaia_web_search - INFO - Query: On June 6, 2023, an article by Carolyn Collins Petersen was published in Universe Today. This article mentions a team that produced a paper about their observations, linked at the bottom of the article. Find this paper. Under what NASA award number was the work performed by R. G. Arendt supported by? -2025-05-02 10:05:40,866 - gaia_agent.tools.web - INFO - Using Perplexity for query: On June 6, 2023, an article by Carolyn Collins Petersen was published in Universe Today. This article mentions a team that produced a paper about their observations, linked at the bottom of the article. Find this paper. Under what NASA award number was the work performed by R. G. Arendt supported by? -2025-05-02 10:05:49,772 - test_gaia_web_search - INFO - Found 1 results -2025-05-02 10:05:49,773 - test_gaia_web_search - INFO - Found 4/4 expected keywords -2025-05-02 10:05:49,773 - test_gaia_web_search - INFO - Success percentage: 100.00% -2025-05-02 10:05:49,773 - test_gaia_web_search - INFO - Testing question: malko_competition -2025-05-02 10:05:49,773 - test_gaia_web_search - INFO - Query: Who won the Malko Competition for Young Conductors in 2021? -2025-05-02 10:05:49,774 - gaia_agent.tools.web - INFO - Using Perplexity for query: Who won the Malko Competition for Young Conductors in 2021? -2025-05-02 10:05:57,505 - test_gaia_web_search - INFO - Found 1 results -2025-05-02 10:05:57,506 - test_gaia_web_search - INFO - Found 4/4 expected keywords -2025-05-02 10:05:57,507 - test_gaia_web_search - INFO - Success percentage: 100.00% -2025-05-02 10:05:57,507 - test_gaia_web_search - INFO - - -=== TEST SUMMARY === -2025-05-02 10:05:57,507 - test_gaia_web_search - INFO - Successfully answered 4/4 questions -2025-05-02 10:05:57,507 - test_gaia_web_search - INFO - \u2705 SUCCESS - mercedes_sosa -2025-05-02 10:05:57,507 - test_gaia_web_search - INFO - \u2705 SUCCESS - wikipedia_dinosaur -2025-05-02 10:05:57,507 - test_gaia_web_search - INFO - \u2705 SUCCESS - universe_today -2025-05-02 10:05:57,507 - test_gaia_web_search - INFO - \u2705 SUCCESS - malko_competition -2025-05-02 10:05:57,509 - test_gaia_web_search - INFO - Results saved to gaia_web_search_results.json -2025-05-02 10:05:57,509 - run_gaia_web_search_tests - INFO - -Tests completed successfully! -2025-05-02 10:06:21,316 - enhanced_web_search_example - INFO - Creating tool registry... -2025-05-02 10:06:21,317 - gaia_agent.tools.web - INFO - Added Perplexity tool to EnhancedWebSearchTool -2025-05-02 10:06:21,317 - gaia_agent.tools.multimodal - INFO - Using HybridYouTubeVideoTool for combined transcript extraction and browser viewing -2025-05-02 10:06:23,072 - gaia_agent.tools.web - INFO - Added YouTube tool to EnhancedWebSearchTool -2025-05-02 10:06:23,073 - gaia_agent.tools.web - INFO - Added Wikipedia tool to EnhancedWebSearchTool -2025-05-02 10:06:23,073 - gaia_agent.tools.web - INFO - Added Serper tool to EnhancedWebSearchTool -2025-05-02 10:06:23,073 - gaia_agent.tools.web - INFO - Added DuckDuckGo tool to EnhancedWebSearchTool -2025-05-02 10:06:23,074 - gaia_agent.tool_registry - INFO - Registered Enhanced Web Search tool -2025-05-02 10:06:23,074 - gaia_agent.tools.multimodal - INFO - Using HybridYouTubeVideoTool for combined transcript extraction and browser viewing -2025-05-02 10:06:24,688 - gaia_agent.tool_registry - INFO - Created default tool registry with 9 tools -2025-05-02 10:06:24,688 - enhanced_web_search_example - INFO - - -Question 1: How many studio albums were published by Mercedes Sosa between 2000 and 2009 (included)? -2025-05-02 10:06:24,689 - enhanced_web_search_example - INFO - Searching using unified_search... -2025-05-02 10:06:24,689 - gaia_agent.tool_registry - INFO - Using enhanced web search for query: How many studio albums were published by Mercedes Sosa between 2000 and 2009 (included)? -2025-05-02 10:06:24,689 - gaia_agent.tools.web - INFO - Using Perplexity for query: How many studio albums were published by Mercedes Sosa between 2000 and 2009 (included)? -2025-05-02 10:06:32,901 - gaia_agent.tool_registry - INFO - Enhanced web search returned high-quality results, skipping other tools -2025-05-02 10:06:32,902 - enhanced_web_search_example - INFO - Found 1 results -2025-05-02 10:06:32,902 - enhanced_web_search_example - INFO - Providers used: ['enhanced_web_search'] -2025-05-02 10:06:32,902 - enhanced_web_search_example - INFO - -Top result: -2025-05-02 10:06:32,903 - enhanced_web_search_example - INFO - Title: Perplexity AI Search Result -2025-05-02 10:06:32,903 - enhanced_web_search_example - INFO - Source: perplexity -2025-05-02 10:06:32,904 - enhanced_web_search_example - INFO - Relevance score: 10.0 -2025-05-02 10:06:32,904 - enhanced_web_search_example - INFO - Snippet: -Okay, let's tackle this question: How many studio albums did Mercedes Sosa release between 2000 and 2009, inclusive? Alright, first I need to go through the search results provided to find the necessary information. - -Starting with the search results. The first three links (1, 2, 3) are from AllMusic, Discogs, and Musicboard. These sites likely list her discography. Then there's the Album of the Year page (4) which has a section on her albums. The fifth link is a World Music Central profi... -2025-05-02 10:06:32,904 - enhanced_web_search_example - INFO - - -Question 2: Who nominated the only Featured Article on English Wikipedia about a dinosaur that was promoted in November 2016? -2025-05-02 10:06:32,905 - enhanced_web_search_example - INFO - Searching using unified_search... -2025-05-02 10:06:32,905 - gaia_agent.tool_registry - INFO - Using enhanced web search for query: Who nominated the only Featured Article on English Wikipedia about a dinosaur that was promoted in November 2016? -2025-05-02 10:06:32,905 - gaia_agent.tools.web - INFO - Using Perplexity for query: Who nominated the only Featured Article on English Wikipedia about a dinosaur that was promoted in November 2016? -2025-05-02 10:06:41,088 - gaia_agent.tool_registry - INFO - Enhanced web search returned high-quality results, skipping other tools -2025-05-02 10:06:41,089 - enhanced_web_search_example - INFO - Found 1 results -2025-05-02 10:06:41,089 - enhanced_web_search_example - INFO - Providers used: ['enhanced_web_search'] -2025-05-02 10:06:41,089 - enhanced_web_search_example - INFO - -Top result: -2025-05-02 10:06:41,089 - enhanced_web_search_example - INFO - Title: Perplexity AI Search Result -2025-05-02 10:06:41,089 - enhanced_web_search_example - INFO - Source: perplexity -2025-05-02 10:06:41,089 - enhanced_web_search_example - INFO - Relevance score: 10.0 -2025-05-02 10:06:41,089 - enhanced_web_search_example - INFO - Snippet: -Okay, let's tackle this query step by step. The user is asking, "Who nominated the only Featured Article on English Wikipedia about a dinosaur that was promoted in November 2016?" Alright, so first, I need to identify the dinosaur article in question and its nominator. - -Looking at the search results provided, result [1] is about the Giganotosaurus article's nomination for Featured Article status. The URL mentions the Featured Article Candidates page for Giganotosaurus, dated around Septe... -2025-05-02 10:06:41,089 - enhanced_web_search_example - INFO - - -Question 3: On June 6, 2023, an article by Carolyn Collins Petersen was published in Universe Today. This article mentions a team that produced a paper about their observations, linked at the bottom of the article. Find this paper. Under what NASA award number was the work performed by R. G. Arendt supported by? -2025-05-02 10:06:41,089 - enhanced_web_search_example - INFO - Searching using unified_search... -2025-05-02 10:06:41,089 - gaia_agent.tool_registry - INFO - Using enhanced web search for query: On June 6, 2023, an article by Carolyn Collins Petersen was published in Universe Today. This article mentions a team that produced a paper about their observations, linked at the bottom of the article. Find this paper. Under what NASA award number was the work performed by R. G. Arendt supported by? -2025-05-02 10:06:41,089 - gaia_agent.tools.web - INFO - Using Perplexity for query: On June 6, 2023, an article by Carolyn Collins Petersen was published in Universe Today. This article mentions a team that produced a paper about their observations, linked at the bottom of the article. Find this paper. Under what NASA award number was the work performed by R. G. Arendt supported by? -2025-05-02 10:06:49,897 - gaia_agent.tool_registry - INFO - Enhanced web search returned high-quality results, skipping other tools -2025-05-02 10:06:49,897 - enhanced_web_search_example - INFO - Found 1 results -2025-05-02 10:06:49,897 - enhanced_web_search_example - INFO - Providers used: ['enhanced_web_search'] -2025-05-02 10:06:49,897 - enhanced_web_search_example - INFO - -Top result: -2025-05-02 10:06:49,897 - enhanced_web_search_example - INFO - Title: Perplexity AI Search Result -2025-05-02 10:06:49,897 - enhanced_web_search_example - INFO - Source: perplexity -2025-05-02 10:06:49,897 - enhanced_web_search_example - INFO - Relevance score: 10.0 -2025-05-02 10:06:49,897 - enhanced_web_search_example - INFO - Snippet: -Okay, let me try to figure this out. The user is asking about a paper linked in a specific article by Carolyn Collins Petersen from June 6, 2023. The article talks about mysterious filaments at the Milky Way's center and mentions a team with a paper. They want to know the NASA award number for R. G. Arendt's work. - -First, I need to locate the article. The search results show [1] is the article from Universe Today on June 6, 2023, by Carolyn Collins Petersen about the filaments. The user ... -2025-05-02 10:06:49,897 - enhanced_web_search_example - INFO - - -Question 4: Who won the Malko Competition for Young Conductors in 2021? -2025-05-02 10:06:49,897 - enhanced_web_search_example - INFO - Searching using unified_search... -2025-05-02 10:06:49,897 - gaia_agent.tool_registry - INFO - Using enhanced web search for query: Who won the Malko Competition for Young Conductors in 2021? -2025-05-02 10:06:49,897 - gaia_agent.tools.web - INFO - Using Perplexity for query: Who won the Malko Competition for Young Conductors in 2021? -2025-05-02 10:06:58,575 - gaia_agent.tool_registry - INFO - Enhanced web search returned high-quality results, skipping other tools -2025-05-02 10:06:58,575 - enhanced_web_search_example - INFO - Found 1 results -2025-05-02 10:06:58,575 - enhanced_web_search_example - INFO - Providers used: ['enhanced_web_search'] -2025-05-02 10:06:58,575 - enhanced_web_search_example - INFO - -Top result: -2025-05-02 10:06:58,575 - enhanced_web_search_example - INFO - Title: Perplexity AI Search Result -2025-05-02 10:06:58,575 - enhanced_web_search_example - INFO - Source: perplexity -2025-05-02 10:06:58,575 - enhanced_web_search_example - INFO - Relevance score: 10.0 -2025-05-02 10:06:58,575 - enhanced_web_search_example - INFO - Snippet: -Okay, let's tackle this query. The user is asking who won the Malko Competition for Young Conductors in 2021. I need to look through the provided search results to find the correct information. - -First, I'll check each search result one by one. Result lists the winners with their years and nationalities. For 2021, it shows Dmitry Matvienko from Belarus. That's a clear indication. - -Result [1] from the Malko Competition's official site also lists 2021 as the year Dmitry Matvienko won, wit... -2025-05-02 10:06:58,575 - enhanced_web_search_example - INFO - -Example completed successfully! -2025-05-02 10:11:42,976 - run_gaia_web_search_tests - INFO - Running direct tests... -2025-05-02 10:11:42,977 - gaia_agent.tools.web - INFO - Added Perplexity tool to EnhancedWebSearchTool -2025-05-02 10:11:42,978 - gaia_agent.tools.multimodal - INFO - Using HybridYouTubeVideoTool for combined transcript extraction and browser viewing -2025-05-02 10:11:44,877 - gaia_agent.tools.web - INFO - Added YouTube tool to EnhancedWebSearchTool -2025-05-02 10:11:44,878 - gaia_agent.tools.web - INFO - Added Wikipedia tool to EnhancedWebSearchTool -2025-05-02 10:11:44,878 - gaia_agent.tools.web - INFO - Added Serper tool to EnhancedWebSearchTool -2025-05-02 10:11:44,878 - gaia_agent.tools.web - INFO - Added DuckDuckGo tool to EnhancedWebSearchTool -2025-05-02 10:11:44,878 - test_gaia_web_search - INFO - Testing question: mercedes_sosa -2025-05-02 10:11:44,879 - test_gaia_web_search - INFO - Query: How many studio albums were published by Mercedes Sosa between 2000 and 2009 (included)? -2025-05-02 10:11:44,879 - gaia_agent.tools.web - INFO - Using Perplexity for query: How many studio albums were published by Mercedes Sosa between 2000 and 2009 (included)? -2025-05-02 10:11:53,721 - test_gaia_web_search - INFO - Found 1 results -2025-05-02 10:11:53,722 - test_gaia_web_search - INFO - Found 4/4 expected keywords -2025-05-02 10:11:53,723 - test_gaia_web_search - INFO - Success percentage: 100.00% -2025-05-02 10:11:53,723 - test_gaia_web_search - INFO - Testing question: wikipedia_dinosaur -2025-05-02 10:11:53,723 - test_gaia_web_search - INFO - Query: Who nominated the only Featured Article on English Wikipedia about a dinosaur that was promoted in November 2016? -2025-05-02 10:11:53,724 - gaia_agent.tools.web - INFO - Using Perplexity for query: Who nominated the only Featured Article on English Wikipedia about a dinosaur that was promoted in November 2016? -2025-05-02 10:12:02,138 - test_gaia_web_search - INFO - Found 1 results -2025-05-02 10:12:02,138 - test_gaia_web_search - INFO - Found 3/5 expected keywords -2025-05-02 10:12:02,139 - test_gaia_web_search - INFO - Success percentage: 60.00% -2025-05-02 10:12:02,139 - test_gaia_web_search - INFO - Testing question: universe_today -2025-05-02 10:12:02,139 - test_gaia_web_search - INFO - Query: On June 6, 2023, an article by Carolyn Collins Petersen was published in Universe Today. This article mentions a team that produced a paper about their observations, linked at the bottom of the article. Find this paper. Under what NASA award number was the work performed by R. G. Arendt supported by? -2025-05-02 10:12:02,139 - gaia_agent.tools.web - INFO - Using Perplexity for query: On June 6, 2023, an article by Carolyn Collins Petersen was published in Universe Today. This article mentions a team that produced a paper about their observations, linked at the bottom of the article. Find this paper. Under what NASA award number was the work performed by R. G. Arendt supported by? -2025-05-02 10:12:10,339 - test_gaia_web_search - INFO - Found 1 results -2025-05-02 10:12:10,340 - test_gaia_web_search - INFO - Found 4/4 expected keywords -2025-05-02 10:12:10,340 - test_gaia_web_search - INFO - Success percentage: 100.00% -2025-05-02 10:12:10,340 - test_gaia_web_search - INFO - Testing question: malko_competition -2025-05-02 10:12:10,340 - test_gaia_web_search - INFO - Query: Who won the Malko Competition for Young Conductors in 2021? -2025-05-02 10:12:10,340 - gaia_agent.tools.web - INFO - Using Perplexity for query: Who won the Malko Competition for Young Conductors in 2021? -2025-05-02 10:12:19,507 - test_gaia_web_search - INFO - Found 1 results -2025-05-02 10:12:19,508 - test_gaia_web_search - INFO - Found 4/4 expected keywords -2025-05-02 10:12:19,508 - test_gaia_web_search - INFO - Success percentage: 100.00% -2025-05-02 10:12:19,508 - test_gaia_web_search - INFO - - -=== TEST SUMMARY === -2025-05-02 10:12:19,508 - test_gaia_web_search - INFO - Successfully answered 4/4 questions -2025-05-02 10:12:19,508 - test_gaia_web_search - INFO - \u2705 SUCCESS - mercedes_sosa -2025-05-02 10:12:19,509 - test_gaia_web_search - INFO - \u2705 SUCCESS - wikipedia_dinosaur -2025-05-02 10:12:19,509 - test_gaia_web_search - INFO - \u2705 SUCCESS - universe_today -2025-05-02 10:12:19,509 - test_gaia_web_search - INFO - \u2705 SUCCESS - malko_competition -2025-05-02 10:12:19,511 - test_gaia_web_search - INFO - Results saved to gaia_web_search_results.json -2025-05-02 10:12:19,511 - run_gaia_web_search_tests - INFO - -Tests completed successfully! -2025-05-02 10:12:30,332 - enhanced_web_search_example - INFO - Creating tool registry... -2025-05-02 10:12:30,333 - gaia_agent.tools.web - INFO - Added Perplexity tool to EnhancedWebSearchTool -2025-05-02 10:12:30,333 - gaia_agent.tools.multimodal - INFO - Using HybridYouTubeVideoTool for combined transcript extraction and browser viewing -2025-05-02 10:12:31,933 - gaia_agent.tools.web - INFO - Added YouTube tool to EnhancedWebSearchTool -2025-05-02 10:12:31,933 - gaia_agent.tools.web - INFO - Added Wikipedia tool to EnhancedWebSearchTool -2025-05-02 10:12:31,933 - gaia_agent.tools.web - INFO - Added Serper tool to EnhancedWebSearchTool -2025-05-02 10:12:31,934 - gaia_agent.tools.web - INFO - Added DuckDuckGo tool to EnhancedWebSearchTool -2025-05-02 10:12:31,934 - gaia_agent.tool_registry - INFO - Registered Enhanced Web Search tool -2025-05-02 10:12:31,934 - gaia_agent.tools.multimodal - INFO - Using HybridYouTubeVideoTool for combined transcript extraction and browser viewing -2025-05-02 10:12:33,585 - gaia_agent.tool_registry - INFO - Created default tool registry with 9 tools -2025-05-02 10:12:33,585 - enhanced_web_search_example - INFO - - -Question 1: How many studio albums were published by Mercedes Sosa between 2000 and 2009 (included)? -2025-05-02 10:12:33,586 - enhanced_web_search_example - INFO - Searching using unified_search... -2025-05-02 10:12:33,586 - gaia_agent.tool_registry - INFO - Using enhanced web search for query: How many studio albums were published by Mercedes Sosa between 2000 and 2009 (included)? -2025-05-02 10:12:33,586 - gaia_agent.tools.web - INFO - Using Perplexity for query: How many studio albums were published by Mercedes Sosa between 2000 and 2009 (included)? -2025-05-02 10:12:50,441 - gaia_agent.tool_registry - INFO - Enhanced web search returned high-quality results, skipping other tools -2025-05-02 10:12:50,441 - enhanced_web_search_example - INFO - Found 1 results -2025-05-02 10:12:50,441 - enhanced_web_search_example - INFO - Providers used: ['enhanced_web_search'] -2025-05-02 10:12:50,441 - enhanced_web_search_example - INFO - -Top result: -2025-05-02 10:12:50,441 - enhanced_web_search_example - INFO - Title: Perplexity AI Search Result -2025-05-02 10:12:50,441 - enhanced_web_search_example - INFO - Source: perplexity -2025-05-02 10:12:50,441 - enhanced_web_search_example - INFO - Relevance score: 10.0 -2025-05-02 10:12:50,441 - enhanced_web_search_example - INFO - Snippet: -Okay, let's tackle this question: How many studio albums were published by Mercedes Sosa between 2000 and 2009, inclusive? - -First, I need to parse the search results provided. The user wants to know specifically about studio albums, so I should exclude compilations, live albums, and EPs unless they're considered studio in the context. Let me go through each search result to find relevant information. - -Looking at the AllMusic link [1], the page probably lists her discography, but without... -2025-05-02 10:12:50,441 - enhanced_web_search_example - INFO - - -Question 2: Who nominated the only Featured Article on English Wikipedia about a dinosaur that was promoted in November 2016? -2025-05-02 10:12:50,441 - enhanced_web_search_example - INFO - Searching using unified_search... -2025-05-02 10:12:50,441 - gaia_agent.tool_registry - INFO - Using enhanced web search for query: Who nominated the only Featured Article on English Wikipedia about a dinosaur that was promoted in November 2016? -2025-05-02 10:12:50,441 - gaia_agent.tools.web - INFO - Using Perplexity for query: Who nominated the only Featured Article on English Wikipedia about a dinosaur that was promoted in November 2016? -2025-05-02 10:13:00,363 - gaia_agent.tool_registry - INFO - Enhanced web search returned high-quality results, skipping other tools -2025-05-02 10:13:00,363 - enhanced_web_search_example - INFO - Found 1 results -2025-05-02 10:13:00,363 - enhanced_web_search_example - INFO - Providers used: ['enhanced_web_search'] -2025-05-02 10:13:00,363 - enhanced_web_search_example - INFO - -Top result: -2025-05-02 10:13:00,364 - enhanced_web_search_example - INFO - Title: Perplexity AI Search Result -2025-05-02 10:13:00,364 - enhanced_web_search_example - INFO - Source: perplexity -2025-05-02 10:13:00,364 - enhanced_web_search_example - INFO - Relevance score: 10.0 -2025-05-02 10:13:00,364 - enhanced_web_search_example - INFO - Snippet: -Okay, so the user is asking who nominated the only Featured Article about a dinosaur that got promoted in November 2016. Let me start by looking through the provided search results to find the relevant information. - -Looking at the first result, it's about the Giganotosaurus article's nomination for Featured Article status. The URL mentions "archive1" and the content says the nominator was FunkMonk on September 30, 2016. But wait, the promotion date is November 2016, so maybe that's when ... -2025-05-02 10:13:00,364 - enhanced_web_search_example - INFO - - -Question 3: On June 6, 2023, an article by Carolyn Collins Petersen was published in Universe Today. This article mentions a team that produced a paper about their observations, linked at the bottom of the article. Find this paper. Under what NASA award number was the work performed by R. G. Arendt supported by? -2025-05-02 10:13:00,364 - enhanced_web_search_example - INFO - Searching using unified_search... -2025-05-02 10:13:00,364 - gaia_agent.tool_registry - INFO - Using enhanced web search for query: On June 6, 2023, an article by Carolyn Collins Petersen was published in Universe Today. This article mentions a team that produced a paper about their observations, linked at the bottom of the article. Find this paper. Under what NASA award number was the work performed by R. G. Arendt supported by? -2025-05-02 10:13:00,364 - gaia_agent.tools.web - INFO - Using Perplexity for query: On June 6, 2023, an article by Carolyn Collins Petersen was published in Universe Today. This article mentions a team that produced a paper about their observations, linked at the bottom of the article. Find this paper. Under what NASA award number was the work performed by R. G. Arendt supported by? -2025-05-02 10:13:14,897 - gaia_agent.tool_registry - INFO - Enhanced web search returned high-quality results, skipping other tools -2025-05-02 10:13:14,897 - enhanced_web_search_example - INFO - Found 1 results -2025-05-02 10:13:14,897 - enhanced_web_search_example - INFO - Providers used: ['enhanced_web_search'] -2025-05-02 10:13:14,897 - enhanced_web_search_example - INFO - -Top result: -2025-05-02 10:13:14,897 - enhanced_web_search_example - INFO - Title: Perplexity AI Search Result -2025-05-02 10:13:14,897 - enhanced_web_search_example - INFO - Source: perplexity -2025-05-02 10:13:14,897 - enhanced_web_search_example - INFO - Relevance score: 10.0 -2025-05-02 10:13:14,897 - enhanced_web_search_example - INFO - Snippet: -Okay, let's tackle this query step by step. The user wants to find a paper mentioned in an article by Carolyn Collins Petersen from June 6, 2023, published on Universe Today. Specifically, they need to know the NASA award number under which R. G. Arendt's work was supported. - -First, I need to check the provided search results. The article is mentioned in search result [1], which links to the Universe Today post. The user says the paper is linked at the bottom of the article, but the sear... -2025-05-02 10:13:14,897 - enhanced_web_search_example - INFO - - -Question 4: Who won the Malko Competition for Young Conductors in 2021? -2025-05-02 10:13:14,897 - enhanced_web_search_example - INFO - Searching using unified_search... -2025-05-02 10:13:14,898 - gaia_agent.tool_registry - INFO - Using enhanced web search for query: Who won the Malko Competition for Young Conductors in 2021? -2025-05-02 10:13:14,898 - gaia_agent.tools.web - INFO - Using Perplexity for query: Who won the Malko Competition for Young Conductors in 2021? -2025-05-02 10:13:24,121 - gaia_agent.tool_registry - INFO - Enhanced web search returned high-quality results, skipping other tools -2025-05-02 10:13:24,121 - enhanced_web_search_example - INFO - Found 1 results -2025-05-02 10:13:24,121 - enhanced_web_search_example - INFO - Providers used: ['enhanced_web_search'] -2025-05-02 10:13:24,121 - enhanced_web_search_example - INFO - -Top result: -2025-05-02 10:13:24,121 - enhanced_web_search_example - INFO - Title: Perplexity AI Search Result -2025-05-02 10:13:24,122 - enhanced_web_search_example - INFO - Source: perplexity -2025-05-02 10:13:24,122 - enhanced_web_search_example - INFO - Relevance score: 10.0 -2025-05-02 10:13:24,122 - enhanced_web_search_example - INFO - Snippet: -Okay, let's tackle this query about the 2021 Malko Competition winner. The user wants to know who won, so I need to focus on the 2021 results. Looking through the search results, the first thing I notice is that multiple sources mention Dmitry Matvienko as the winner. - -From result , the table lists 2021 as Belarus for the recipient, which aligns with what's in [3] and [5]. The Gramophone article [3] is particularly detailed, stating that Matvienko is from Belarus and won the competition ... -2025-05-02 10:13:24,122 - enhanced_web_search_example - INFO - -Example completed successfully! -2025-05-02 10:17:39,964 - run_gaia_web_search_tests - INFO - Running direct tests... -2025-05-02 10:17:39,965 - gaia_agent.tools.web - INFO - Added Perplexity tool to EnhancedWebSearchTool -2025-05-02 10:17:39,965 - gaia_agent.tools.multimodal - INFO - Using HybridYouTubeVideoTool for combined transcript extraction and browser viewing -2025-05-02 10:17:41,878 - gaia_agent.tools.web - INFO - Added YouTube tool to EnhancedWebSearchTool -2025-05-02 10:17:41,878 - gaia_agent.tools.web - INFO - Added Wikipedia tool to EnhancedWebSearchTool -2025-05-02 10:17:41,879 - gaia_agent.tools.web - INFO - Added Serper tool to EnhancedWebSearchTool -2025-05-02 10:17:41,879 - gaia_agent.tools.web - INFO - Added DuckDuckGo tool to EnhancedWebSearchTool -2025-05-02 10:17:41,879 - test_gaia_web_search - INFO - Testing question: mercedes_sosa -2025-05-02 10:17:41,879 - test_gaia_web_search - INFO - Query: How many studio albums were published by Mercedes Sosa between 2000 and 2009 (included)? -2025-05-02 10:17:41,879 - gaia_agent.tools.web - INFO - Using Perplexity for query: How many studio albums were published by Mercedes Sosa between 2000 and 2009 (included)? -2025-05-02 10:17:51,413 - test_gaia_web_search - INFO - Found 1 results -2025-05-02 10:17:51,414 - test_gaia_web_search - INFO - Found 4/4 expected keywords -2025-05-02 10:17:51,414 - test_gaia_web_search - INFO - Success percentage: 100.00% -2025-05-02 10:17:51,414 - test_gaia_web_search - INFO - Testing question: wikipedia_dinosaur -2025-05-02 10:17:51,415 - test_gaia_web_search - INFO - Query: Who nominated the only Featured Article on English Wikipedia about a dinosaur that was promoted in November 2016? -2025-05-02 10:17:51,415 - gaia_agent.tools.web - INFO - Using Perplexity for query: Who nominated the only Featured Article on English Wikipedia about a dinosaur that was promoted in November 2016? -2025-05-02 10:17:59,938 - test_gaia_web_search - INFO - Found 1 results -2025-05-02 10:17:59,939 - test_gaia_web_search - INFO - Found 4/5 expected keywords -2025-05-02 10:17:59,939 - test_gaia_web_search - INFO - Success percentage: 80.00% -2025-05-02 10:17:59,939 - test_gaia_web_search - INFO - Testing question: universe_today -2025-05-02 10:17:59,940 - test_gaia_web_search - INFO - Query: On June 6, 2023, an article by Carolyn Collins Petersen was published in Universe Today. This article mentions a team that produced a paper about their observations, linked at the bottom of the article. Find this paper. Under what NASA award number was the work performed by R. G. Arendt supported by? -2025-05-02 10:17:59,940 - gaia_agent.tools.web - INFO - Using Perplexity for query: On June 6, 2023, an article by Carolyn Collins Petersen was published in Universe Today. This article mentions a team that produced a paper about their observations, linked at the bottom of the article. Find this paper. Under what NASA award number was the work performed by R. G. Arendt supported by? -2025-05-02 10:18:09,035 - test_gaia_web_search - INFO - Found 1 results -2025-05-02 10:18:09,036 - test_gaia_web_search - INFO - Found 4/4 expected keywords -2025-05-02 10:18:09,036 - test_gaia_web_search - INFO - Success percentage: 100.00% -2025-05-02 10:18:09,036 - test_gaia_web_search - INFO - Testing question: malko_competition -2025-05-02 10:18:09,036 - test_gaia_web_search - INFO - Query: Who won the Malko Competition for Young Conductors in 2021? -2025-05-02 10:18:09,036 - gaia_agent.tools.web - INFO - Using Perplexity for query: Who won the Malko Competition for Young Conductors in 2021? -2025-05-02 10:18:28,118 - test_gaia_web_search - INFO - Found 1 results -2025-05-02 10:18:28,118 - test_gaia_web_search - INFO - Found 3/4 expected keywords -2025-05-02 10:18:28,119 - test_gaia_web_search - INFO - Success percentage: 75.00% -2025-05-02 10:18:28,119 - test_gaia_web_search - INFO - - -=== TEST SUMMARY === -2025-05-02 10:18:28,119 - test_gaia_web_search - INFO - Successfully answered 4/4 questions -2025-05-02 10:18:28,119 - test_gaia_web_search - INFO - \u2705 SUCCESS - mercedes_sosa -2025-05-02 10:18:28,120 - test_gaia_web_search - INFO - \u2705 SUCCESS - wikipedia_dinosaur -2025-05-02 10:18:28,120 - test_gaia_web_search - INFO - \u2705 SUCCESS - universe_today -2025-05-02 10:18:28,120 - test_gaia_web_search - INFO - \u2705 SUCCESS - malko_competition -2025-05-02 10:18:28,121 - test_gaia_web_search - INFO - Results saved to gaia_web_search_results.json -2025-05-02 10:18:28,122 - run_gaia_web_search_tests - INFO - -Tests completed successfully! -2025-05-02 10:18:42,218 - enhanced_web_search_example - INFO - Creating tool registry... -2025-05-02 10:18:42,218 - gaia_agent.tools.web - INFO - Added Perplexity tool to EnhancedWebSearchTool -2025-05-02 10:18:42,218 - gaia_agent.tools.multimodal - INFO - Using HybridYouTubeVideoTool for combined transcript extraction and browser viewing -2025-05-02 10:18:43,896 - gaia_agent.tools.web - INFO - Added YouTube tool to EnhancedWebSearchTool -2025-05-02 10:18:43,896 - gaia_agent.tools.web - INFO - Added Wikipedia tool to EnhancedWebSearchTool -2025-05-02 10:18:43,896 - gaia_agent.tools.web - INFO - Added Serper tool to EnhancedWebSearchTool -2025-05-02 10:18:43,896 - gaia_agent.tools.web - INFO - Added DuckDuckGo tool to EnhancedWebSearchTool -2025-05-02 10:18:43,896 - gaia_agent.tool_registry - INFO - Registered Enhanced Web Search tool -2025-05-02 10:18:43,896 - gaia_agent.tools.multimodal - INFO - Using HybridYouTubeVideoTool for combined transcript extraction and browser viewing -2025-05-02 10:18:45,439 - gaia_agent.tool_registry - INFO - Created default tool registry with 9 tools -2025-05-02 10:18:45,439 - enhanced_web_search_example - INFO - - -Question 1: How many studio albums were published by Mercedes Sosa between 2000 and 2009 (included)? -2025-05-02 10:18:45,439 - enhanced_web_search_example - INFO - Searching using unified_search... -2025-05-02 10:18:45,439 - gaia_agent.tool_registry - INFO - Using enhanced web search for query: How many studio albums were published by Mercedes Sosa between 2000 and 2009 (included)? -2025-05-02 10:18:45,439 - gaia_agent.tools.web - INFO - Using Perplexity for query: How many studio albums were published by Mercedes Sosa between 2000 and 2009 (included)? -2025-05-02 10:19:01,794 - gaia_agent.tool_registry - INFO - Enhanced web search returned high-quality results, skipping other tools -2025-05-02 10:19:01,794 - enhanced_web_search_example - INFO - Found 1 results -2025-05-02 10:19:01,794 - enhanced_web_search_example - INFO - Providers used: ['enhanced_web_search'] -2025-05-02 10:19:01,794 - enhanced_web_search_example - INFO - -Top result: -2025-05-02 10:19:01,794 - enhanced_web_search_example - INFO - Title: Perplexity AI Search Result -2025-05-02 10:19:01,794 - enhanced_web_search_example - INFO - Source: perplexity -2025-05-02 10:19:01,794 - enhanced_web_search_example - INFO - Relevance score: 10.0 -2025-05-02 10:19:01,794 - enhanced_web_search_example - INFO - Snippet: -Okay, let's tackle this question about Mercedes Sosa's studio albums between 2000 and 2009. The user wants to know how many she released in that decade, including 2000 and 2009. - -First, I need to look through the provided search results to find her discography. The search results mention a few sources: AllMusic, Discogs, Musicboard, Album of the Year, and World Music Central. Let's go through each one. - -Looking at result [4], Album of the Year lists her albums. There's a section with rel... -2025-05-02 10:19:01,794 - enhanced_web_search_example - INFO - - -Question 2: Who nominated the only Featured Article on English Wikipedia about a dinosaur that was promoted in November 2016? -2025-05-02 10:19:01,794 - enhanced_web_search_example - INFO - Searching using unified_search... -2025-05-02 10:19:01,794 - gaia_agent.tool_registry - INFO - Using enhanced web search for query: Who nominated the only Featured Article on English Wikipedia about a dinosaur that was promoted in November 2016? -2025-05-02 10:19:01,795 - gaia_agent.tools.web - INFO - Using Perplexity for query: Who nominated the only Featured Article on English Wikipedia about a dinosaur that was promoted in November 2016? -2025-05-02 10:19:14,265 - gaia_agent.tool_registry - INFO - Enhanced web search returned high-quality results, skipping other tools -2025-05-02 10:19:14,265 - enhanced_web_search_example - INFO - Found 1 results -2025-05-02 10:19:14,265 - enhanced_web_search_example - INFO - Providers used: ['enhanced_web_search'] -2025-05-02 10:19:14,265 - enhanced_web_search_example - INFO - -Top result: -2025-05-02 10:19:14,265 - enhanced_web_search_example - INFO - Title: Perplexity AI Search Result -2025-05-02 10:19:14,265 - enhanced_web_search_example - INFO - Source: perplexity -2025-05-02 10:19:14,265 - enhanced_web_search_example - INFO - Relevance score: 10.0 -2025-05-02 10:19:14,265 - enhanced_web_search_example - INFO - Snippet: -Okay, let's tackle this query step by step. The user is asking who nominated the only Featured Article on English Wikipedia about a dinosaur that was promoted in November 2016. - -First, I need to understand what a Featured Article (FA) is. From what I know, FAs on Wikipedia are articles that have been vetted through a process to ensure they meet high standards of quality, accuracy, and comprehensiveness. The nomination process involves editors proposing articles and others reviewing them.... -2025-05-02 10:19:14,265 - enhanced_web_search_example - INFO - - -Question 3: On June 6, 2023, an article by Carolyn Collins Petersen was published in Universe Today. This article mentions a team that produced a paper about their observations, linked at the bottom of the article. Find this paper. Under what NASA award number was the work performed by R. G. Arendt supported by? -2025-05-02 10:19:14,265 - enhanced_web_search_example - INFO - Searching using unified_search... -2025-05-02 10:19:14,265 - gaia_agent.tool_registry - INFO - Using enhanced web search for query: On June 6, 2023, an article by Carolyn Collins Petersen was published in Universe Today. This article mentions a team that produced a paper about their observations, linked at the bottom of the article. Find this paper. Under what NASA award number was the work performed by R. G. Arendt supported by? -2025-05-02 10:19:14,265 - gaia_agent.tools.web - INFO - Using Perplexity for query: On June 6, 2023, an article by Carolyn Collins Petersen was published in Universe Today. This article mentions a team that produced a paper about their observations, linked at the bottom of the article. Find this paper. Under what NASA award number was the work performed by R. G. Arendt supported by? -2025-05-02 10:19:25,681 - gaia_agent.tool_registry - INFO - Enhanced web search returned high-quality results, skipping other tools -2025-05-02 10:19:25,681 - enhanced_web_search_example - INFO - Found 1 results -2025-05-02 10:19:25,681 - enhanced_web_search_example - INFO - Providers used: ['enhanced_web_search'] -2025-05-02 10:19:25,681 - enhanced_web_search_example - INFO - -Top result: -2025-05-02 10:19:25,681 - enhanced_web_search_example - INFO - Title: Perplexity AI Search Result -2025-05-02 10:19:25,681 - enhanced_web_search_example - INFO - Source: perplexity -2025-05-02 10:19:25,681 - enhanced_web_search_example - INFO - Relevance score: 10.0 -2025-05-02 10:19:25,681 - enhanced_web_search_example - INFO - Snippet: -Okay, let's tackle this query step by step. The user is asking about a specific article by Carolyn Collins Petersen published on June 6, 2023, in Universe Today. The article mentions a team that produced a paper linked at the bottom, and the user wants to find that paper to identify the NASA award number under which R. G. Arendt's work was supported. - -First, I need to check the provided search results. The first result [1] is the actual article from Universe Today dated June 6, 2023, tit... -2025-05-02 10:19:25,681 - enhanced_web_search_example - INFO - - -Question 4: Who won the Malko Competition for Young Conductors in 2021? -2025-05-02 10:19:25,681 - enhanced_web_search_example - INFO - Searching using unified_search... -2025-05-02 10:19:25,681 - gaia_agent.tool_registry - INFO - Using enhanced web search for query: Who won the Malko Competition for Young Conductors in 2021? -2025-05-02 10:19:25,681 - gaia_agent.tools.web - INFO - Using Perplexity for query: Who won the Malko Competition for Young Conductors in 2021? -2025-05-02 10:19:37,689 - gaia_agent.tool_registry - INFO - Enhanced web search returned high-quality results, skipping other tools -2025-05-02 10:19:37,689 - enhanced_web_search_example - INFO - Found 1 results -2025-05-02 10:19:37,689 - enhanced_web_search_example - INFO - Providers used: ['enhanced_web_search'] -2025-05-02 10:19:37,689 - enhanced_web_search_example - INFO - -Top result: -2025-05-02 10:19:37,690 - enhanced_web_search_example - INFO - Title: Perplexity AI Search Result -2025-05-02 10:19:37,690 - enhanced_web_search_example - INFO - Source: perplexity -2025-05-02 10:19:37,690 - enhanced_web_search_example - INFO - Relevance score: 10.0 -2025-05-02 10:19:37,690 - enhanced_web_search_example - INFO - Snippet: -Okay, let's tackle this query about who won the Malko Competition in 2021. The user wants a comprehensive answer with facts and citations. First, I'll check the search results provided. - -Looking at the first result from Wikipedia, it lists Dmitry Matvienko from Belarus as the 2021 winner. That's a key point. Then result [1] from the Malko Competition's official site confirms the same winner, Samuel Seungwon Lee for 2024, but the 2021 entry is Dmitry Matvienko. So that's consistent. - -Res... -2025-05-02 10:19:37,690 - enhanced_web_search_example - INFO - -Example completed successfully! -2025-05-02 12:18:18,057 - gaia_agent.tools.web - ERROR - Error searching arXiv: sequence item 0: expected str instance, MagicMock found -2025-05-02 12:18:18,059 - gaia_agent.tools.web - ERROR - Traceback (most recent call last): - File "c:\Users\joach\OneDrive - VISCONSULT\00_VISCONSULT\05_AI_and_Digitalization\02_AI_Agents\02_HF_Agents_Course\99_Final_Assignment\tools\web_tools.py", line 1004, in _search_arxiv - authors = ", ".join([author.name for author in paper.authors]) -TypeError: sequence item 0: expected str instance, MagicMock found - -2025-05-02 12:18:18,063 - gaia_agent.tools.web - INFO - Using arXiv for academic query: latest research papers on quantum computing -2025-05-02 12:18:18,064 - gaia_agent.tools.web - INFO - Using DuckDuckGo for general query: weather forecast for tomorrow -2025-05-02 12:20:32,541 - gaia_agent.tools.web - ERROR - Error searching arXiv: sequence item 0: expected str instance, MagicMock found -2025-05-02 12:20:32,543 - gaia_agent.tools.web - ERROR - Traceback (most recent call last): - File "c:\Users\joach\OneDrive - VISCONSULT\00_VISCONSULT\05_AI_and_Digitalization\02_AI_Agents\02_HF_Agents_Course\99_Final_Assignment\tools\web_tools.py", line 1004, in _search_arxiv - authors = ", ".join([author.name for author in paper.authors]) -TypeError: sequence item 0: expected str instance, MagicMock found - -2025-05-02 12:20:32,547 - gaia_agent.tools.web - INFO - Using arXiv for academic query: latest research papers on quantum computing -2025-05-02 12:20:32,548 - gaia_agent.tools.web - INFO - Using DuckDuckGo for general query: weather forecast for tomorrow -2025-05-02 12:21:04,387 - gaia_agent.tools.web - ERROR - Error searching arXiv: sequence item 0: expected str instance, MagicMock found -2025-05-02 12:21:04,389 - gaia_agent.tools.web - ERROR - Traceback (most recent call last): - File "c:\Users\joach\OneDrive - VISCONSULT\00_VISCONSULT\05_AI_and_Digitalization\02_AI_Agents\02_HF_Agents_Course\99_Final_Assignment\tools\web_tools.py", line 1004, in _search_arxiv - authors = ", ".join([author.name for author in paper.authors]) -TypeError: sequence item 0: expected str instance, MagicMock found - -2025-05-02 12:21:04,394 - gaia_agent.tools.web - INFO - Using arXiv for academic query: latest research papers on quantum computing -2025-05-02 12:21:04,395 - gaia_agent.tools.web - INFO - Using DuckDuckGo for general query: weather forecast for tomorrow -2025-05-02 12:23:41,771 - arxiv - INFO - Requesting page (first: True, try: 0): https://export.arxiv.org/api/query?search_query=quantum+computing&id_list=&sortBy=relevance&sortOrder=descending&start=0&max_results=100 -2025-05-02 12:23:43,138 - arxiv - INFO - Got first page: 100 of 1030824 total results -2025-05-02 12:23:44,176 - primp - INFO - response: https://html.duckduckgo.com/html 200 -2025-05-02 12:23:44,191 - gaia_agent.tools.web - INFO - Using arXiv for academic query: latest research papers on quantum computing -2025-05-02 12:23:44,192 - arxiv - INFO - Requesting page (first: True, try: 0): https://export.arxiv.org/api/query?search_query=latest+research+papers+on+quantum+computing&id_list=&sortBy=relevance&sortOrder=descending&start=0&max_results=100 -2025-05-02 12:23:45,114 - arxiv - INFO - Got first page: 1 of 2342331 total results -2025-05-02 12:23:45,114 - arxiv - INFO - Sleeping: 2.997759 seconds -2025-05-02 12:23:48,112 - arxiv - INFO - Requesting page (first: False, try: 0): https://export.arxiv.org/api/query?search_query=latest+research+papers+on+quantum+computing&id_list=&sortBy=relevance&sortOrder=descending&start=1&max_results=100 -2025-05-02 12:23:49,867 - gaia_agent.tools.web - INFO - Using DuckDuckGo for general query: weather forecast for tomorrow -2025-05-02 12:23:50,800 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 200 -2025-05-02 14:37:47,193 - arxiv - INFO - Requesting page (first: True, try: 0): https://export.arxiv.org/api/query?search_query=quantum+computing&id_list=&sortBy=relevance&sortOrder=descending&start=0&max_results=100 -2025-05-02 14:37:48,613 - arxiv - INFO - Got first page: 100 of 1030824 total results -2025-05-02 14:37:49,537 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 200 -2025-05-02 14:37:49,551 - gaia_agent.tools.web - INFO - Using arXiv for academic query: latest research papers on quantum computing -2025-05-02 14:37:49,551 - arxiv - INFO - Requesting page (first: True, try: 0): https://export.arxiv.org/api/query?search_query=latest+research+papers+on+quantum+computing&id_list=&sortBy=relevance&sortOrder=descending&start=0&max_results=100 -2025-05-02 14:37:51,295 - arxiv - INFO - Got first page: 100 of 2342331 total results -2025-05-02 14:37:51,297 - gaia_agent.tools.web - INFO - Using DuckDuckGo for general query: weather forecast for tomorrow -2025-05-02 14:37:52,119 - primp - INFO - response: https://html.duckduckgo.com/html 200 -2025-05-02 16:50:02,957 - gaia_agent.tools.web - INFO - Perplexity tool initialized successfully -2025-05-02 16:50:02,959 - gaia_agent.tools.web - INFO - Serper tool initialized successfully -2025-05-02 16:50:02,960 - gaia_agent.tools.web - INFO - Added ApiSearch tool to EnhancedWebSearchTool (optimized for GAIA assessment) -2025-05-02 16:50:02,961 - gaia_agent.tools.multimodal - INFO - Using HybridYouTubeVideoTool for combined transcript extraction and browser viewing -2025-05-02 16:50:04,934 - gaia_agent.tools.web - INFO - Added YouTube tool to EnhancedWebSearchTool -2025-05-02 16:50:04,934 - gaia_agent.tools.web - WARNING - WikipediaSearchTool is not implemented in this version -2025-05-02 16:50:04,935 - gaia_agent.tools.web - WARNING - Wikipedia tool not available, skipping -2025-05-02 16:50:04,935 - gaia_agent.tools.web - INFO - Added LibrarySearch tool to EnhancedWebSearchTool -2025-05-02 16:50:04,935 - gaia_agent.tool_registry - INFO - Registered Enhanced Web Search tool -2025-05-02 16:50:04,936 - gaia_agent.tools.web - WARNING - WikipediaSearchTool is not implemented in this version -2025-05-02 16:50:04,936 - gaia_agent.tools.multimodal - INFO - Using HybridYouTubeVideoTool for combined transcript extraction and browser viewing -2025-05-02 16:50:06,688 - gaia_agent.tool_registry - INFO - Registered Browser Search tool -2025-05-02 16:50:06,688 - gaia_agent.tool_registry - INFO - Registered Library Search tool -2025-05-02 16:50:06,688 - gaia_agent.tools.web - INFO - Perplexity tool initialized successfully -2025-05-02 16:50:06,689 - gaia_agent.tools.web - INFO - Serper tool initialized successfully -2025-05-02 16:50:06,689 - gaia_agent.tool_registry - INFO - Registered API Search tool -2025-05-02 16:50:06,689 - gaia_agent.tool_registry - INFO - Created default tool registry with 12 tools -2025-05-02 16:50:06,689 - gaia_assessment_web_search - INFO - Running search for query: 'What albums did Mercedes Sosa release between 2000 and 2009?' with format: unified -2025-05-02 16:50:06,690 - gaia_assessment_web_search - ERROR - Error searching for 'What albums did Mercedes Sosa release between 2000 and 2009?' with format unified: cannot import name 'calculate_query_relevance' from 'tools.web_tools' (c:\Users\joach\OneDrive - VISCONSULT\00_VISCONSULT\05_AI_and_Digitalization\02_AI_Agents\02_HF_Agents_Course\99_Final_Assignment\tools\web_tools.py) -2025-05-02 16:50:06,690 - gaia_assessment_web_search - INFO - Running search for query: 'What albums did Mercedes Sosa release between 2000 and 2009?' with format: robust -2025-05-02 16:50:06,690 - gaia_assessment_web_search - ERROR - Error searching for 'What albums did Mercedes Sosa release between 2000 and 2009?' with format robust: cannot import name 'calculate_query_relevance' from 'tools.web_tools' (c:\Users\joach\OneDrive - VISCONSULT\00_VISCONSULT\05_AI_and_Digitalization\02_AI_Agents\02_HF_Agents_Course\99_Final_Assignment\tools\web_tools.py) -2025-05-02 16:50:06,690 - gaia_assessment_web_search - INFO - Running search for query: 'What albums did Mercedes Sosa release between 2000 and 2009?' with format: merged -2025-05-02 16:50:06,690 - gaia_assessment_web_search - ERROR - Error searching for 'What albums did Mercedes Sosa release between 2000 and 2009?' with format merged: cannot import name 'calculate_query_relevance' from 'tools.web_tools' (c:\Users\joach\OneDrive - VISCONSULT\00_VISCONSULT\05_AI_and_Digitalization\02_AI_Agents\02_HF_Agents_Course\99_Final_Assignment\tools\web_tools.py) -2025-05-02 16:50:06,690 - gaia_assessment_web_search - INFO - Running search for query: 'Who nominated the Spinosaurus article for featured status on Wikipedia?' with format: unified -2025-05-02 16:50:06,690 - gaia_assessment_web_search - ERROR - Error searching for 'Who nominated the Spinosaurus article for featured status on Wikipedia?' with format unified: cannot import name 'calculate_query_relevance' from 'tools.web_tools' (c:\Users\joach\OneDrive - VISCONSULT\00_VISCONSULT\05_AI_and_Digitalization\02_AI_Agents\02_HF_Agents_Course\99_Final_Assignment\tools\web_tools.py) -2025-05-02 16:50:06,690 - gaia_assessment_web_search - INFO - Running search for query: 'Who nominated the Spinosaurus article for featured status on Wikipedia?' with format: robust -2025-05-02 16:50:06,691 - gaia_assessment_web_search - ERROR - Error searching for 'Who nominated the Spinosaurus article for featured status on Wikipedia?' with format robust: cannot import name 'calculate_query_relevance' from 'tools.web_tools' (c:\Users\joach\OneDrive - VISCONSULT\00_VISCONSULT\05_AI_and_Digitalization\02_AI_Agents\02_HF_Agents_Course\99_Final_Assignment\tools\web_tools.py) -2025-05-02 16:50:06,691 - gaia_assessment_web_search - INFO - Running search for query: 'Who nominated the Spinosaurus article for featured status on Wikipedia?' with format: merged -2025-05-02 16:50:06,691 - gaia_assessment_web_search - ERROR - Error searching for 'Who nominated the Spinosaurus article for featured status on Wikipedia?' with format merged: cannot import name 'calculate_query_relevance' from 'tools.web_tools' (c:\Users\joach\OneDrive - VISCONSULT\00_VISCONSULT\05_AI_and_Digitalization\02_AI_Agents\02_HF_Agents_Course\99_Final_Assignment\tools\web_tools.py) -2025-05-02 16:50:06,691 - gaia_assessment_web_search - INFO - Running search for query: 'What is the NASA award number mentioned in the Universe Today article about exoplanet research?' with format: unified -2025-05-02 16:50:06,691 - gaia_assessment_web_search - ERROR - Error searching for 'What is the NASA award number mentioned in the Universe Today article about exoplanet research?' with format unified: cannot import name 'calculate_query_relevance' from 'tools.web_tools' (c:\Users\joach\OneDrive - VISCONSULT\00_VISCONSULT\05_AI_and_Digitalization\02_AI_Agents\02_HF_Agents_Course\99_Final_Assignment\tools\web_tools.py) -2025-05-02 16:50:06,691 - gaia_assessment_web_search - INFO - Running search for query: 'What is the NASA award number mentioned in the Universe Today article about exoplanet research?' with format: robust -2025-05-02 16:50:06,691 - gaia_assessment_web_search - ERROR - Error searching for 'What is the NASA award number mentioned in the Universe Today article about exoplanet research?' with format robust: cannot import name 'calculate_query_relevance' from 'tools.web_tools' (c:\Users\joach\OneDrive - VISCONSULT\00_VISCONSULT\05_AI_and_Digitalization\02_AI_Agents\02_HF_Agents_Course\99_Final_Assignment\tools\web_tools.py) -2025-05-02 16:50:06,691 - gaia_assessment_web_search - INFO - Running search for query: 'What is the NASA award number mentioned in the Universe Today article about exoplanet research?' with format: merged -2025-05-02 16:50:06,691 - gaia_assessment_web_search - ERROR - Error searching for 'What is the NASA award number mentioned in the Universe Today article about exoplanet research?' with format merged: cannot import name 'calculate_query_relevance' from 'tools.web_tools' (c:\Users\joach\OneDrive - VISCONSULT\00_VISCONSULT\05_AI_and_Digitalization\02_AI_Agents\02_HF_Agents_Course\99_Final_Assignment\tools\web_tools.py) -2025-05-02 16:50:06,691 - gaia_assessment_web_search - INFO - Running search for query: 'Who are the recent recipients of the Malko Competition?' with format: unified -2025-05-02 16:50:06,691 - gaia_assessment_web_search - ERROR - Error searching for 'Who are the recent recipients of the Malko Competition?' with format unified: cannot import name 'calculate_query_relevance' from 'tools.web_tools' (c:\Users\joach\OneDrive - VISCONSULT\00_VISCONSULT\05_AI_and_Digitalization\02_AI_Agents\02_HF_Agents_Course\99_Final_Assignment\tools\web_tools.py) -2025-05-02 16:50:06,691 - gaia_assessment_web_search - INFO - Running search for query: 'Who are the recent recipients of the Malko Competition?' with format: robust -2025-05-02 16:50:06,691 - gaia_assessment_web_search - ERROR - Error searching for 'Who are the recent recipients of the Malko Competition?' with format robust: cannot import name 'calculate_query_relevance' from 'tools.web_tools' (c:\Users\joach\OneDrive - VISCONSULT\00_VISCONSULT\05_AI_and_Digitalization\02_AI_Agents\02_HF_Agents_Course\99_Final_Assignment\tools\web_tools.py) -2025-05-02 16:50:06,691 - gaia_assessment_web_search - INFO - Running search for query: 'Who are the recent recipients of the Malko Competition?' with format: merged -2025-05-02 16:50:06,691 - gaia_assessment_web_search - ERROR - Error searching for 'Who are the recent recipients of the Malko Competition?' with format merged: cannot import name 'calculate_query_relevance' from 'tools.web_tools' (c:\Users\joach\OneDrive - VISCONSULT\00_VISCONSULT\05_AI_and_Digitalization\02_AI_Agents\02_HF_Agents_Course\99_Final_Assignment\tools\web_tools.py) -2025-05-02 16:51:36,259 - gaia_agent.tools.web - INFO - Perplexity tool initialized successfully -2025-05-02 16:51:36,260 - gaia_agent.tools.web - INFO - Serper tool initialized successfully -2025-05-02 16:51:36,260 - gaia_agent.tools.web - INFO - Added ApiSearch tool to EnhancedWebSearchTool (optimized for GAIA assessment) -2025-05-02 16:51:36,260 - gaia_agent.tools.multimodal - INFO - Using HybridYouTubeVideoTool for combined transcript extraction and browser viewing -2025-05-02 16:51:37,975 - gaia_agent.tools.web - INFO - Added YouTube tool to EnhancedWebSearchTool -2025-05-02 16:51:37,976 - gaia_agent.tools.web - WARNING - WikipediaSearchTool is not implemented in this version -2025-05-02 16:51:37,976 - gaia_agent.tools.web - WARNING - Wikipedia tool not available, skipping -2025-05-02 16:51:37,976 - gaia_agent.tools.web - INFO - Added LibrarySearch tool to EnhancedWebSearchTool -2025-05-02 16:51:37,976 - gaia_agent.tool_registry - INFO - Registered Enhanced Web Search tool -2025-05-02 16:51:37,977 - gaia_agent.tools.web - WARNING - WikipediaSearchTool is not implemented in this version -2025-05-02 16:51:37,977 - gaia_agent.tools.multimodal - INFO - Using HybridYouTubeVideoTool for combined transcript extraction and browser viewing -2025-05-02 16:51:39,583 - gaia_agent.tool_registry - INFO - Registered Browser Search tool -2025-05-02 16:51:39,583 - gaia_agent.tool_registry - INFO - Registered Library Search tool -2025-05-02 16:51:39,583 - gaia_agent.tools.web - INFO - Perplexity tool initialized successfully -2025-05-02 16:51:39,584 - gaia_agent.tools.web - INFO - Serper tool initialized successfully -2025-05-02 16:51:39,584 - gaia_agent.tool_registry - INFO - Registered API Search tool -2025-05-02 16:51:39,584 - gaia_agent.tool_registry - INFO - Created default tool registry with 12 tools -2025-05-02 16:51:39,584 - gaia_assessment_web_search - INFO - Running search for query: 'What albums did Mercedes Sosa release between 2000 and 2009?' with format: unified -2025-05-02 16:51:39,584 - gaia_agent.tool_registry - INFO - Using enhanced web search for query: What albums did Mercedes Sosa release between 2000 and 2009? -2025-05-02 16:51:39,584 - gaia_agent.tools.web - INFO - Using Serper API for query: What albums did Mercedes Sosa release between 2000 and 2009? -2025-05-02 16:51:40,825 - gaia_agent.tool_registry - INFO - Enhanced web search returned high-quality results, skipping other tools -2025-05-02 16:51:40,825 - gaia_assessment_web_search - INFO - Running search for query: 'What albums did Mercedes Sosa release between 2000 and 2009?' with format: robust -2025-05-02 16:51:40,825 - gaia_agent.tool_registry - INFO - Using enhanced web search for query: What albums did Mercedes Sosa release between 2000 and 2009? -2025-05-02 16:51:40,825 - gaia_agent.tools.web - INFO - Using Serper API for query: What albums did Mercedes Sosa release between 2000 and 2009? -2025-05-02 16:51:41,512 - gaia_agent.tool_registry - INFO - Enhanced web search returned high-quality results, skipping other tools -2025-05-02 16:51:41,513 - gaia_assessment_web_search - INFO - Running search for query: 'What albums did Mercedes Sosa release between 2000 and 2009?' with format: merged -2025-05-02 16:51:41,513 - gaia_agent.tool_registry - INFO - Using enhanced web search for query: What albums did Mercedes Sosa release between 2000 and 2009? -2025-05-02 16:51:41,513 - gaia_agent.tools.web - INFO - Using Serper API for query: What albums did Mercedes Sosa release between 2000 and 2009? -2025-05-02 16:51:42,319 - gaia_agent.tool_registry - INFO - Enhanced web search returned high-quality results, skipping other tools -2025-05-02 16:51:42,319 - gaia_assessment_web_search - INFO - Running search for query: 'Who nominated the Spinosaurus article for featured status on Wikipedia?' with format: unified -2025-05-02 16:51:42,320 - gaia_agent.tool_registry - INFO - Using enhanced web search for query: Who nominated the Spinosaurus article for featured status on Wikipedia? -2025-05-02 16:51:42,320 - gaia_agent.tools.web - INFO - Using Serper API for Wikipedia-specific query: Who nominated the Spinosaurus article for featured status on Wikipedia? -2025-05-02 16:51:43,750 - gaia_agent.tool_registry - INFO - Enhanced web search returned high-quality results, skipping other tools -2025-05-02 16:51:43,750 - gaia_assessment_web_search - INFO - Running search for query: 'Who nominated the Spinosaurus article for featured status on Wikipedia?' with format: robust -2025-05-02 16:51:43,751 - gaia_agent.tool_registry - INFO - Using enhanced web search for query: Who nominated the Spinosaurus article for featured status on Wikipedia? -2025-05-02 16:51:43,751 - gaia_agent.tools.web - INFO - Using Serper API for Wikipedia-specific query: Who nominated the Spinosaurus article for featured status on Wikipedia? -2025-05-02 16:51:44,550 - gaia_agent.tool_registry - INFO - Enhanced web search returned high-quality results, skipping other tools -2025-05-02 16:51:44,550 - gaia_assessment_web_search - INFO - Running search for query: 'Who nominated the Spinosaurus article for featured status on Wikipedia?' with format: merged -2025-05-02 16:51:44,550 - gaia_agent.tool_registry - INFO - Using enhanced web search for query: Who nominated the Spinosaurus article for featured status on Wikipedia? -2025-05-02 16:51:44,550 - gaia_agent.tools.web - INFO - Using Serper API for Wikipedia-specific query: Who nominated the Spinosaurus article for featured status on Wikipedia? -2025-05-02 16:51:45,446 - gaia_agent.tool_registry - INFO - Enhanced web search returned high-quality results, skipping other tools -2025-05-02 16:51:45,446 - gaia_assessment_web_search - INFO - Running search for query: 'What is the NASA award number mentioned in the Universe Today article about exoplanet research?' with format: unified -2025-05-02 16:51:45,446 - gaia_agent.tool_registry - INFO - Using enhanced web search for query: What is the NASA award number mentioned in the Universe Today article about exoplanet research? -2025-05-02 16:51:45,446 - gaia_agent.tools.web - INFO - Using Serper API for scientific article query: What is the NASA award number mentioned in the Universe Today article about exoplanet research? -2025-05-02 16:51:46,720 - gaia_agent.tool_registry - INFO - Enhanced web search returned high-quality results, skipping other tools -2025-05-02 16:51:46,720 - gaia_assessment_web_search - INFO - Running search for query: 'What is the NASA award number mentioned in the Universe Today article about exoplanet research?' with format: robust -2025-05-02 16:51:46,720 - gaia_agent.tool_registry - INFO - Using enhanced web search for query: What is the NASA award number mentioned in the Universe Today article about exoplanet research? -2025-05-02 16:51:46,720 - gaia_agent.tools.web - INFO - Using Serper API for scientific article query: What is the NASA award number mentioned in the Universe Today article about exoplanet research? -2025-05-02 16:51:47,500 - gaia_agent.tool_registry - INFO - Enhanced web search returned high-quality results, skipping other tools -2025-05-02 16:51:47,501 - gaia_assessment_web_search - INFO - Running search for query: 'What is the NASA award number mentioned in the Universe Today article about exoplanet research?' with format: merged -2025-05-02 16:51:47,501 - gaia_agent.tool_registry - INFO - Using enhanced web search for query: What is the NASA award number mentioned in the Universe Today article about exoplanet research? -2025-05-02 16:51:47,501 - gaia_agent.tools.web - INFO - Using Serper API for scientific article query: What is the NASA award number mentioned in the Universe Today article about exoplanet research? -2025-05-02 16:51:48,328 - gaia_agent.tool_registry - INFO - Enhanced web search returned high-quality results, skipping other tools -2025-05-02 16:51:48,329 - gaia_assessment_web_search - INFO - Running search for query: 'Who are the recent recipients of the Malko Competition?' with format: unified -2025-05-02 16:51:48,330 - gaia_agent.tool_registry - INFO - Using enhanced web search for query: Who are the recent recipients of the Malko Competition? -2025-05-02 16:51:48,330 - gaia_agent.tools.web - INFO - Using Serper API for query: Who are the recent recipients of the Malko Competition? -2025-05-02 16:51:49,017 - gaia_agent.tools.web - INFO - Using Serper API for query: Who are the recent recipients of the Malko Competition? -2025-05-02 16:51:49,018 - gaia_agent.tools.web - INFO - Using DuckDuckGo for general query: Who are the recent recipients of the Malko Competition? -2025-05-02 16:51:49,994 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 200 -2025-05-02 16:51:50,209 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 200 -2025-05-02 16:51:50,214 - gaia_assessment_web_search - INFO - Running search for query: 'Who are the recent recipients of the Malko Competition?' with format: robust -2025-05-02 16:51:50,215 - gaia_agent.tool_registry - INFO - Using enhanced web search for query: Who are the recent recipients of the Malko Competition? -2025-05-02 16:51:50,215 - gaia_agent.tools.web - INFO - Using Serper API for query: Who are the recent recipients of the Malko Competition? -2025-05-02 16:51:50,987 - gaia_agent.tools.web - INFO - Using Serper API for query: Who are the recent recipients of the Malko Competition? -2025-05-02 16:51:50,988 - gaia_agent.tools.web - INFO - Using DuckDuckGo for general query: Who are the recent recipients of the Malko Competition? -2025-05-02 16:51:51,891 - primp - INFO - response: https://html.duckduckgo.com/html 200 -2025-05-02 16:51:51,970 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 200 -2025-05-02 16:51:51,976 - gaia_assessment_web_search - INFO - Running search for query: 'Who are the recent recipients of the Malko Competition?' with format: merged -2025-05-02 16:51:51,976 - gaia_agent.tool_registry - INFO - Using enhanced web search for query: Who are the recent recipients of the Malko Competition? -2025-05-02 16:51:51,977 - gaia_agent.tools.web - INFO - Using Serper API for query: Who are the recent recipients of the Malko Competition? -2025-05-02 16:51:52,676 - gaia_agent.tools.web - INFO - Using Serper API for query: Who are the recent recipients of the Malko Competition? -2025-05-02 16:51:52,678 - gaia_agent.tools.web - INFO - Using DuckDuckGo for general query: Who are the recent recipients of the Malko Competition? -2025-05-02 16:51:53,508 - primp - INFO - response: https://html.duckduckgo.com/html 200 -2025-05-02 16:51:53,541 - primp - INFO - response: https://html.duckduckgo.com/html 200 -2025-05-02 16:53:05,460 - test_gaia_web_search - INFO - Testing EnhancedWebSearchTool directly... -2025-05-02 16:53:05,461 - gaia_agent.tools.web - INFO - Perplexity tool initialized successfully -2025-05-02 16:53:05,461 - gaia_agent.tools.web - INFO - Serper tool initialized successfully -2025-05-02 16:53:05,461 - gaia_agent.tools.web - INFO - Added ApiSearch tool to EnhancedWebSearchTool (optimized for GAIA assessment) -2025-05-02 16:53:05,461 - gaia_agent.tools.multimodal - INFO - Using HybridYouTubeVideoTool for combined transcript extraction and browser viewing -2025-05-02 16:53:07,042 - gaia_agent.tools.web - INFO - Added YouTube tool to EnhancedWebSearchTool -2025-05-02 16:53:07,042 - gaia_agent.tools.web - WARNING - WikipediaSearchTool is not implemented in this version -2025-05-02 16:53:07,042 - gaia_agent.tools.web - WARNING - Wikipedia tool not available, skipping -2025-05-02 16:53:07,042 - gaia_agent.tools.web - INFO - Added LibrarySearch tool to EnhancedWebSearchTool -2025-05-02 16:53:07,042 - test_gaia_web_search - INFO - Testing question: mercedes_sosa -2025-05-02 16:53:07,042 - test_gaia_web_search - INFO - Query: How many studio albums were published by Mercedes Sosa between 2000 and 2009 (included)? -2025-05-02 16:53:07,042 - gaia_agent.tools.web - INFO - Using Perplexity API for complex query: How many studio albums were published by Mercedes Sosa between 2000 and 2009 (included)? -2025-05-02 16:53:16,951 - test_gaia_web_search - INFO - Found 1 results -2025-05-02 16:53:16,951 - test_gaia_web_search - INFO - Found 4/4 expected keywords -2025-05-02 16:53:16,951 - test_gaia_web_search - INFO - Success percentage: 100.00% -2025-05-02 16:53:16,951 - test_gaia_web_search - INFO - Testing question: wikipedia_dinosaur -2025-05-02 16:53:16,951 - test_gaia_web_search - INFO - Query: Who nominated the only Featured Article on English Wikipedia about a dinosaur that was promoted in November 2016? -2025-05-02 16:53:16,951 - gaia_agent.tools.web - INFO - Using Serper API for Wikipedia-specific query: Who nominated the only Featured Article on English Wikipedia about a dinosaur that was promoted in November 2016? -2025-05-02 16:53:17,896 - test_gaia_web_search - INFO - Found 5 results -2025-05-02 16:53:17,896 - test_gaia_web_search - INFO - Found 5/5 expected keywords -2025-05-02 16:53:17,896 - test_gaia_web_search - INFO - Success percentage: 100.00% -2025-05-02 16:53:17,896 - test_gaia_web_search - INFO - Testing question: universe_today -2025-05-02 16:53:17,896 - test_gaia_web_search - INFO - Query: On June 6, 2023, an article by Carolyn Collins Petersen was published in Universe Today. This article mentions a team that produced a paper about their observations, linked at the bottom of the article. Find this paper. Under what NASA award number was the work performed by R. G. Arendt supported by? -2025-05-02 16:53:17,896 - gaia_agent.tools.web - INFO - Using Serper API for scientific article query: On June 6, 2023, an article by Carolyn Collins Petersen was published in Universe Today. This article mentions a team that produced a paper about their observations, linked at the bottom of the article. Find this paper. Under what NASA award number was the work performed by R. G. Arendt supported by? -2025-05-02 16:53:18,630 - test_gaia_web_search - INFO - Found 5 results -2025-05-02 16:53:18,630 - test_gaia_web_search - INFO - Found 3/4 expected keywords -2025-05-02 16:53:18,630 - test_gaia_web_search - INFO - Success percentage: 75.00% -2025-05-02 16:53:18,630 - test_gaia_web_search - INFO - Testing question: malko_competition -2025-05-02 16:53:18,630 - test_gaia_web_search - INFO - Query: Who won the Malko Competition for Young Conductors in 2021? -2025-05-02 16:53:18,630 - gaia_agent.tools.web - INFO - Using Serper API for query: Who won the Malko Competition for Young Conductors in 2021? -2025-05-02 16:53:19,723 - test_gaia_web_search - INFO - Found 5 results -2025-05-02 16:53:19,723 - test_gaia_web_search - INFO - Found 4/4 expected keywords -2025-05-02 16:53:19,723 - test_gaia_web_search - INFO - Success percentage: 100.00% -2025-05-02 16:53:19,723 - test_gaia_web_search - INFO - - -=== TEST SUMMARY === -2025-05-02 16:53:19,723 - test_gaia_web_search - INFO - Successfully answered 4/4 questions -2025-05-02 16:53:19,723 - test_gaia_web_search - INFO - \u2705 SUCCESS - mercedes_sosa -2025-05-02 16:53:19,724 - test_gaia_web_search - INFO - \u2705 SUCCESS - wikipedia_dinosaur -2025-05-02 16:53:19,724 - test_gaia_web_search - INFO - \u2705 SUCCESS - universe_today -2025-05-02 16:53:19,724 - test_gaia_web_search - INFO - \u2705 SUCCESS - malko_competition -2025-05-02 16:53:19,725 - test_gaia_web_search - INFO - Results saved to gaia_web_search_results.json -2025-05-02 16:53:19,725 - test_gaia_web_search - INFO - - -Testing EnhancedWebSearchTool through registry... -2025-05-02 16:53:19,725 - gaia_agent.tools.web - INFO - Perplexity tool initialized successfully -2025-05-02 16:53:19,726 - gaia_agent.tools.web - INFO - Serper tool initialized successfully -2025-05-02 16:53:19,726 - gaia_agent.tools.web - INFO - Added ApiSearch tool to EnhancedWebSearchTool (optimized for GAIA assessment) -2025-05-02 16:53:19,726 - gaia_agent.tools.multimodal - INFO - Using HybridYouTubeVideoTool for combined transcript extraction and browser viewing -2025-05-02 16:53:21,329 - gaia_agent.tools.web - INFO - Added YouTube tool to EnhancedWebSearchTool -2025-05-02 16:53:21,329 - gaia_agent.tools.web - WARNING - WikipediaSearchTool is not implemented in this version -2025-05-02 16:53:21,329 - gaia_agent.tools.web - WARNING - Wikipedia tool not available, skipping -2025-05-02 16:53:21,329 - gaia_agent.tools.web - INFO - Added LibrarySearch tool to EnhancedWebSearchTool -2025-05-02 16:53:21,329 - gaia_agent.tool_registry - INFO - Registered Enhanced Web Search tool -2025-05-02 16:53:21,329 - gaia_agent.tools.web - WARNING - WikipediaSearchTool is not implemented in this version -2025-05-02 16:53:21,329 - gaia_agent.tools.multimodal - INFO - Using HybridYouTubeVideoTool for combined transcript extraction and browser viewing -2025-05-02 16:53:22,934 - gaia_agent.tool_registry - INFO - Registered Browser Search tool -2025-05-02 16:53:22,934 - gaia_agent.tool_registry - INFO - Registered Library Search tool -2025-05-02 16:53:22,934 - gaia_agent.tools.web - INFO - Perplexity tool initialized successfully -2025-05-02 16:53:22,934 - gaia_agent.tools.web - INFO - Serper tool initialized successfully -2025-05-02 16:53:22,934 - gaia_agent.tool_registry - INFO - Registered API Search tool -2025-05-02 16:53:22,934 - gaia_agent.tool_registry - INFO - Created default tool registry with 12 tools -2025-05-02 16:53:22,934 - test_gaia_web_search - INFO - Testing question through registry: mercedes_sosa -2025-05-02 16:53:22,934 - test_gaia_web_search - INFO - Query: How many studio albums were published by Mercedes Sosa between 2000 and 2009 (included)? -2025-05-02 16:53:22,934 - gaia_agent.tool_registry - INFO - Using enhanced web search for query: How many studio albums were published by Mercedes Sosa between 2000 and 2009 (included)? -2025-05-02 16:53:22,935 - gaia_agent.tools.web - INFO - Using Perplexity API for complex query: How many studio albums were published by Mercedes Sosa between 2000 and 2009 (included)? -2025-05-02 16:53:32,783 - gaia_agent.tool_registry - INFO - Enhanced web search returned high-quality results, skipping other tools -2025-05-02 16:53:32,783 - test_gaia_web_search - INFO - Found 1 results -2025-05-02 16:53:32,783 - test_gaia_web_search - INFO - Found 4/4 expected keywords -2025-05-02 16:53:32,783 - test_gaia_web_search - INFO - Success percentage: 100.00% -2025-05-02 16:53:32,783 - test_gaia_web_search - INFO - Testing question through registry: wikipedia_dinosaur -2025-05-02 16:53:32,783 - test_gaia_web_search - INFO - Query: Who nominated the only Featured Article on English Wikipedia about a dinosaur that was promoted in November 2016? -2025-05-02 16:53:32,784 - gaia_agent.tool_registry - INFO - Using enhanced web search for query: Who nominated the only Featured Article on English Wikipedia about a dinosaur that was promoted in November 2016? -2025-05-02 16:53:32,784 - gaia_agent.tools.web - INFO - Using Serper API for Wikipedia-specific query: Who nominated the only Featured Article on English Wikipedia about a dinosaur that was promoted in November 2016? -2025-05-02 16:53:33,641 - gaia_agent.tool_registry - INFO - Enhanced web search returned high-quality results, skipping other tools -2025-05-02 16:53:33,642 - test_gaia_web_search - INFO - Found 5 results -2025-05-02 16:53:33,642 - test_gaia_web_search - INFO - Found 5/5 expected keywords -2025-05-02 16:53:33,642 - test_gaia_web_search - INFO - Success percentage: 100.00% -2025-05-02 16:53:33,642 - test_gaia_web_search - INFO - - -=== REGISTRY TEST SUMMARY === -2025-05-02 16:53:33,642 - test_gaia_web_search - INFO - Successfully answered 2/2 questions through registry -2025-05-02 16:53:33,642 - test_gaia_web_search - INFO - \u2705 SUCCESS - mercedes_sosa -2025-05-02 16:53:33,642 - test_gaia_web_search - INFO - Providers used: enhanced_web_search -2025-05-02 16:53:33,642 - test_gaia_web_search - INFO - \u2705 SUCCESS - wikipedia_dinosaur -2025-05-02 16:53:33,642 - test_gaia_web_search - INFO - Providers used: enhanced_web_search -2025-05-02 16:53:33,642 - test_gaia_web_search - INFO - - -=== OVERALL SUMMARY === -2025-05-02 16:53:33,642 - test_gaia_web_search - INFO - Direct testing: 4/4 questions successful -2025-05-02 16:53:33,643 - test_gaia_web_search - INFO - Registry testing: 2/2 questions successful -2025-05-02 16:55:31,639 - gaia_agent.tools.web - INFO - Perplexity tool initialized successfully -2025-05-02 16:55:31,640 - gaia_agent.tools.web - INFO - Serper tool initialized successfully -2025-05-02 16:55:31,640 - gaia_agent.tools.web - INFO - Added ApiSearch tool to EnhancedWebSearchTool (optimized for GAIA assessment) -2025-05-02 16:55:31,640 - gaia_agent.tools.multimodal - INFO - Using HybridYouTubeVideoTool for combined transcript extraction and browser viewing -2025-05-02 16:55:33,348 - gaia_agent.tools.web - INFO - Added YouTube tool to EnhancedWebSearchTool -2025-05-02 16:55:33,348 - gaia_agent.tools.web - WARNING - WikipediaSearchTool is not implemented in this version -2025-05-02 16:55:33,349 - gaia_agent.tools.web - WARNING - Wikipedia tool not available, skipping -2025-05-02 16:55:33,349 - gaia_agent.tools.web - INFO - Added LibrarySearch tool to EnhancedWebSearchTool -2025-05-02 16:55:33,349 - gaia_agent.tool_registry - INFO - Registered Enhanced Web Search tool -2025-05-02 16:55:33,349 - gaia_agent.tools.web - WARNING - WikipediaSearchTool is not implemented in this version -2025-05-02 16:55:33,350 - gaia_agent.tools.multimodal - INFO - Using HybridYouTubeVideoTool for combined transcript extraction and browser viewing -2025-05-02 16:55:34,945 - gaia_agent.tool_registry - INFO - Registered Browser Search tool -2025-05-02 16:55:34,946 - gaia_agent.tool_registry - INFO - Registered Library Search tool -2025-05-02 16:55:34,946 - gaia_agent.tools.web - INFO - Perplexity tool initialized successfully -2025-05-02 16:55:34,946 - gaia_agent.tools.web - INFO - Serper tool initialized successfully -2025-05-02 16:55:34,947 - gaia_agent.tool_registry - INFO - Registered API Search tool -2025-05-02 16:55:34,947 - gaia_agent.tool_registry - INFO - Created default tool registry with 12 tools -2025-05-02 16:55:34,947 - gaia_assessment_web_search - INFO - Running search for query: 'What albums did Mercedes Sosa release between 2000 and 2009?' with format: unified -2025-05-02 16:55:34,947 - gaia_agent.tool_registry - INFO - Using enhanced web search for query: What albums did Mercedes Sosa release between 2000 and 2009? -2025-05-02 16:55:34,947 - gaia_agent.tools.web - INFO - Using Serper API for query: What albums did Mercedes Sosa release between 2000 and 2009? -2025-05-02 16:55:35,772 - gaia_agent.tool_registry - INFO - Enhanced web search returned high-quality results, skipping other tools -2025-05-02 16:55:35,772 - gaia_assessment_web_search - INFO - Running search for query: 'What albums did Mercedes Sosa release between 2000 and 2009?' with format: robust -2025-05-02 16:55:35,772 - gaia_agent.tool_registry - INFO - Using enhanced web search for query: What albums did Mercedes Sosa release between 2000 and 2009? -2025-05-02 16:55:35,772 - gaia_agent.tools.web - INFO - Using Serper API for query: What albums did Mercedes Sosa release between 2000 and 2009? -2025-05-02 16:55:36,733 - gaia_agent.tool_registry - INFO - Enhanced web search returned high-quality results, skipping other tools -2025-05-02 16:55:36,733 - gaia_assessment_web_search - INFO - Running search for query: 'What albums did Mercedes Sosa release between 2000 and 2009?' with format: merged -2025-05-02 16:55:36,734 - gaia_agent.tool_registry - INFO - Using enhanced web search for query: What albums did Mercedes Sosa release between 2000 and 2009? -2025-05-02 16:55:36,734 - gaia_agent.tools.web - INFO - Using Serper API for query: What albums did Mercedes Sosa release between 2000 and 2009? -2025-05-02 16:55:37,568 - gaia_agent.tool_registry - INFO - Enhanced web search returned high-quality results, skipping other tools -2025-05-02 16:55:37,568 - gaia_assessment_web_search - INFO - Running search for query: 'Who nominated the Spinosaurus article for featured status on Wikipedia?' with format: unified -2025-05-02 16:55:37,568 - gaia_agent.tool_registry - INFO - Using enhanced web search for query: Who nominated the Spinosaurus article for featured status on Wikipedia? -2025-05-02 16:55:37,568 - gaia_agent.tools.web - INFO - Using Serper API for Wikipedia-specific query: Who nominated the Spinosaurus article for featured status on Wikipedia? -2025-05-02 16:55:38,477 - gaia_agent.tool_registry - INFO - Enhanced web search returned high-quality results, skipping other tools -2025-05-02 16:55:38,477 - gaia_assessment_web_search - INFO - Running search for query: 'Who nominated the Spinosaurus article for featured status on Wikipedia?' with format: robust -2025-05-02 16:55:38,477 - gaia_agent.tool_registry - INFO - Using enhanced web search for query: Who nominated the Spinosaurus article for featured status on Wikipedia? -2025-05-02 16:55:38,477 - gaia_agent.tools.web - INFO - Using Serper API for Wikipedia-specific query: Who nominated the Spinosaurus article for featured status on Wikipedia? -2025-05-02 16:55:39,190 - gaia_agent.tool_registry - INFO - Enhanced web search returned high-quality results, skipping other tools -2025-05-02 16:55:39,190 - gaia_assessment_web_search - INFO - Running search for query: 'Who nominated the Spinosaurus article for featured status on Wikipedia?' with format: merged -2025-05-02 16:55:39,190 - gaia_agent.tool_registry - INFO - Using enhanced web search for query: Who nominated the Spinosaurus article for featured status on Wikipedia? -2025-05-02 16:55:39,190 - gaia_agent.tools.web - INFO - Using Serper API for Wikipedia-specific query: Who nominated the Spinosaurus article for featured status on Wikipedia? -2025-05-02 16:55:39,977 - gaia_agent.tool_registry - INFO - Enhanced web search returned high-quality results, skipping other tools -2025-05-02 16:55:39,977 - gaia_assessment_web_search - INFO - Running search for query: 'What is the NASA award number mentioned in the Universe Today article about exoplanet research?' with format: unified -2025-05-02 16:55:39,977 - gaia_agent.tool_registry - INFO - Using enhanced web search for query: What is the NASA award number mentioned in the Universe Today article about exoplanet research? -2025-05-02 16:55:39,977 - gaia_agent.tools.web - INFO - Using Serper API for scientific article query: What is the NASA award number mentioned in the Universe Today article about exoplanet research? -2025-05-02 16:55:40,679 - gaia_agent.tool_registry - INFO - Enhanced web search returned high-quality results, skipping other tools -2025-05-02 16:55:40,680 - gaia_assessment_web_search - INFO - Running search for query: 'What is the NASA award number mentioned in the Universe Today article about exoplanet research?' with format: robust -2025-05-02 16:55:40,680 - gaia_agent.tool_registry - INFO - Using enhanced web search for query: What is the NASA award number mentioned in the Universe Today article about exoplanet research? -2025-05-02 16:55:40,680 - gaia_agent.tools.web - INFO - Using Serper API for scientific article query: What is the NASA award number mentioned in the Universe Today article about exoplanet research? -2025-05-02 16:55:41,440 - gaia_agent.tool_registry - INFO - Enhanced web search returned high-quality results, skipping other tools -2025-05-02 16:55:41,441 - gaia_assessment_web_search - INFO - Running search for query: 'What is the NASA award number mentioned in the Universe Today article about exoplanet research?' with format: merged -2025-05-02 16:55:41,441 - gaia_agent.tool_registry - INFO - Using enhanced web search for query: What is the NASA award number mentioned in the Universe Today article about exoplanet research? -2025-05-02 16:55:41,441 - gaia_agent.tools.web - INFO - Using Serper API for scientific article query: What is the NASA award number mentioned in the Universe Today article about exoplanet research? -2025-05-02 16:55:42,175 - gaia_agent.tool_registry - INFO - Enhanced web search returned high-quality results, skipping other tools -2025-05-02 16:55:42,175 - gaia_assessment_web_search - INFO - Running search for query: 'Who are the recent recipients of the Malko Competition?' with format: unified -2025-05-02 16:55:42,175 - gaia_agent.tool_registry - INFO - Using enhanced web search for query: Who are the recent recipients of the Malko Competition? -2025-05-02 16:55:42,175 - gaia_agent.tools.web - INFO - Using Serper API for query: Who are the recent recipients of the Malko Competition? -2025-05-02 16:55:43,112 - gaia_agent.tools.web - INFO - Using Serper API for query: Who are the recent recipients of the Malko Competition? -2025-05-02 16:55:43,114 - gaia_agent.tools.web - INFO - Using DuckDuckGo for general query: Who are the recent recipients of the Malko Competition? -2025-05-02 16:55:43,977 - primp - INFO - response: https://html.duckduckgo.com/html 200 -2025-05-02 16:55:44,058 - primp - INFO - response: https://html.duckduckgo.com/html 200 -2025-05-02 16:55:44,063 - gaia_assessment_web_search - INFO - Running search for query: 'Who are the recent recipients of the Malko Competition?' with format: robust -2025-05-02 16:55:44,063 - gaia_agent.tool_registry - INFO - Using enhanced web search for query: Who are the recent recipients of the Malko Competition? -2025-05-02 16:55:44,063 - gaia_agent.tools.web - INFO - Using Serper API for query: Who are the recent recipients of the Malko Competition? -2025-05-02 16:55:44,957 - gaia_agent.tools.web - INFO - Using Serper API for query: Who are the recent recipients of the Malko Competition? -2025-05-02 16:55:44,959 - gaia_agent.tools.web - INFO - Using DuckDuckGo for general query: Who are the recent recipients of the Malko Competition? -2025-05-02 16:55:45,755 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 200 -2025-05-02 16:55:45,782 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 200 -2025-05-02 16:55:45,937 - gaia_assessment_web_search - INFO - Running search for query: 'Who are the recent recipients of the Malko Competition?' with format: merged -2025-05-02 16:55:45,937 - gaia_agent.tool_registry - INFO - Using enhanced web search for query: Who are the recent recipients of the Malko Competition? -2025-05-02 16:55:45,937 - gaia_agent.tools.web - INFO - Using Serper API for query: Who are the recent recipients of the Malko Competition? -2025-05-02 16:55:46,696 - gaia_agent.tools.web - INFO - Using Serper API for query: Who are the recent recipients of the Malko Competition? -2025-05-02 16:55:46,697 - gaia_agent.tools.web - INFO - Using DuckDuckGo for general query: Who are the recent recipients of the Malko Competition? -2025-05-02 16:55:47,495 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 200 -2025-05-02 16:55:47,502 - primp - INFO - response: https://lite.duckduckgo.com/lite/ 200 diff --git a/gaia_assessment_results_20250501_090159.json b/gaia_assessment_results_20250501_090159.json deleted file mode 100644 index 5e07c33e33a14b2da1b43b36974ac0ea946da605..0000000000000000000000000000000000000000 --- a/gaia_assessment_results_20250501_090159.json +++ /dev/null @@ -1,103 +0,0 @@ -{ - "timestamp": "20250501_090159", - "tests": { - "simple_question": { - "question": "What is the capital of France?", - "error": "unhashable type: 'dict'", - "execution_time": 9.528660535812378, - "initial_state": { - "gaia_memory": 0, - "gaia_memory_working": 0, - "gaia_memory_conversations": 0, - "gaia_memory_cache": 0 - }, - "final_state": { - "gaia_memory": 0, - "gaia_memory_working": 0, - "gaia_memory_conversations": 0, - "gaia_memory_cache": 0 - } - }, - "cached_question": { - "question": "What is the capital of France?", - "error": "unhashable type: 'dict'", - "execution_time": 11.032195091247559, - "initial_state": { - "gaia_memory": 0, - "gaia_memory_working": 0, - "gaia_memory_conversations": 0, - "gaia_memory_cache": 0 - }, - "final_state": { - "gaia_memory": 0, - "gaia_memory_working": 0, - "gaia_memory_conversations": 0, - "gaia_memory_cache": 0 - } - }, - "follow_up_question": { - "question": "What is the population of that city?", - "error": "unhashable type: 'dict'", - "execution_time": 8.699026584625244, - "initial_state": { - "gaia_memory": 0, - "gaia_memory_working": 0, - "gaia_memory_conversations": 0, - "gaia_memory_cache": 0 - }, - "final_state": { - "gaia_memory": 0, - "gaia_memory_working": 0, - "gaia_memory_conversations": 0, - "gaia_memory_cache": 0 - } - }, - "complex_question": { - "question": "\n I need a comprehensive analysis of renewable energy trends. \n First, what are the top 3 renewable energy sources by global capacity? \n Second, which countries are leading in renewable energy adoption? \n Finally, what are the projected growth rates for solar and wind energy over the next decade?\n ", - "error": "unhashable type: 'dict'", - "execution_time": 9.387453079223633, - "initial_state": { - "gaia_memory": 0, - "gaia_memory_working": 0, - "gaia_memory_conversations": 0, - "gaia_memory_cache": 0 - }, - "final_state": { - "gaia_memory": 0, - "gaia_memory_working": 0, - "gaia_memory_conversations": 0, - "gaia_memory_cache": 0 - } - }, - "multi_turn_conversation": { - "error": "unhashable type: 'dict'", - "conversation": [], - "initial_state": { - "gaia_memory": 0, - "gaia_memory_working": 0, - "gaia_memory_conversations": 0, - "gaia_memory_cache": 0 - }, - "final_state": { - "gaia_memory": 0, - "gaia_memory_working": 0, - "gaia_memory_conversations": 0, - "gaia_memory_cache": 0 - } - }, - "working_memory_verification": { - "has_entries": false, - "keys": [], - "examples": {} - }, - "conversation_memory_verification": { - "has_entries": false, - "message_count": 0, - "messages": [] - }, - "cache_verification": { - "has_entries": false, - "entry_count": 0 - } - } -} \ No newline at end of file diff --git a/gaia_assessment_results_20250501_090940.json b/gaia_assessment_results_20250501_090940.json deleted file mode 100644 index 430f3e5fa95cee58b7178d3052616c8eeca9dba9..0000000000000000000000000000000000000000 --- a/gaia_assessment_results_20250501_090940.json +++ /dev/null @@ -1,103 +0,0 @@ -{ - "timestamp": "20250501_090940", - "tests": { - "simple_question": { - "question": "What is the capital of France?", - "error": "unhashable type: 'dict'", - "execution_time": 9.139986753463745, - "initial_state": { - "gaia_memory": 0, - "gaia_memory_working": 0, - "gaia_memory_conversations": 0, - "gaia_memory_cache": 0 - }, - "final_state": { - "gaia_memory": 0, - "gaia_memory_working": 0, - "gaia_memory_conversations": 0, - "gaia_memory_cache": 0 - } - }, - "cached_question": { - "question": "What is the capital of France?", - "error": "unhashable type: 'dict'", - "execution_time": 9.158259153366089, - "initial_state": { - "gaia_memory": 0, - "gaia_memory_working": 0, - "gaia_memory_conversations": 0, - "gaia_memory_cache": 0 - }, - "final_state": { - "gaia_memory": 0, - "gaia_memory_working": 0, - "gaia_memory_conversations": 0, - "gaia_memory_cache": 0 - } - }, - "follow_up_question": { - "question": "What is the population of that city?", - "error": "unhashable type: 'dict'", - "execution_time": 9.298568964004517, - "initial_state": { - "gaia_memory": 0, - "gaia_memory_working": 0, - "gaia_memory_conversations": 0, - "gaia_memory_cache": 0 - }, - "final_state": { - "gaia_memory": 0, - "gaia_memory_working": 0, - "gaia_memory_conversations": 0, - "gaia_memory_cache": 0 - } - }, - "complex_question": { - "question": "\n I need a comprehensive analysis of renewable energy trends. \n First, what are the top 3 renewable energy sources by global capacity? \n Second, which countries are leading in renewable energy adoption? \n Finally, what are the projected growth rates for solar and wind energy over the next decade?\n ", - "error": "unhashable type: 'dict'", - "execution_time": 9.107700824737549, - "initial_state": { - "gaia_memory": 0, - "gaia_memory_working": 0, - "gaia_memory_conversations": 0, - "gaia_memory_cache": 0 - }, - "final_state": { - "gaia_memory": 0, - "gaia_memory_working": 0, - "gaia_memory_conversations": 0, - "gaia_memory_cache": 0 - } - }, - "multi_turn_conversation": { - "error": "unhashable type: 'dict'", - "conversation": [], - "initial_state": { - "gaia_memory": 0, - "gaia_memory_working": 0, - "gaia_memory_conversations": 0, - "gaia_memory_cache": 0 - }, - "final_state": { - "gaia_memory": 0, - "gaia_memory_working": 0, - "gaia_memory_conversations": 0, - "gaia_memory_cache": 0 - } - }, - "working_memory_verification": { - "has_entries": false, - "keys": [], - "examples": {} - }, - "conversation_memory_verification": { - "has_entries": false, - "message_count": 0, - "messages": [] - }, - "cache_verification": { - "has_entries": false, - "entry_count": 0 - } - } -} \ No newline at end of file diff --git a/gaia_assessment_results_20250501_093320.json b/gaia_assessment_results_20250501_093320.json deleted file mode 100644 index 4a24821ef0101b9b6e85454b3f3af1bd6634fb45..0000000000000000000000000000000000000000 --- a/gaia_assessment_results_20250501_093320.json +++ /dev/null @@ -1,103 +0,0 @@ -{ - "timestamp": "20250501_093320", - "tests": { - "simple_question": { - "question": "What is the capital of France?", - "error": "unhashable type: 'dict'", - "execution_time": 8.94474482536316, - "initial_state": { - "gaia_memory": 0, - "gaia_memory_working": 0, - "gaia_memory_conversations": 0, - "gaia_memory_cache": 0 - }, - "final_state": { - "gaia_memory": 0, - "gaia_memory_working": 0, - "gaia_memory_conversations": 0, - "gaia_memory_cache": 0 - } - }, - "cached_question": { - "question": "What is the capital of France?", - "error": "unhashable type: 'dict'", - "execution_time": 8.460245847702026, - "initial_state": { - "gaia_memory": 0, - "gaia_memory_working": 0, - "gaia_memory_conversations": 0, - "gaia_memory_cache": 0 - }, - "final_state": { - "gaia_memory": 0, - "gaia_memory_working": 0, - "gaia_memory_conversations": 0, - "gaia_memory_cache": 0 - } - }, - "follow_up_question": { - "question": "What is the population of that city?", - "error": "unhashable type: 'dict'", - "execution_time": 8.62381649017334, - "initial_state": { - "gaia_memory": 0, - "gaia_memory_working": 0, - "gaia_memory_conversations": 0, - "gaia_memory_cache": 0 - }, - "final_state": { - "gaia_memory": 0, - "gaia_memory_working": 0, - "gaia_memory_conversations": 0, - "gaia_memory_cache": 0 - } - }, - "complex_question": { - "question": "\n I need a comprehensive analysis of renewable energy trends. \n First, what are the top 3 renewable energy sources by global capacity? \n Second, which countries are leading in renewable energy adoption? \n Finally, what are the projected growth rates for solar and wind energy over the next decade?\n ", - "error": "unhashable type: 'dict'", - "execution_time": 9.041590690612793, - "initial_state": { - "gaia_memory": 0, - "gaia_memory_working": 0, - "gaia_memory_conversations": 0, - "gaia_memory_cache": 0 - }, - "final_state": { - "gaia_memory": 0, - "gaia_memory_working": 0, - "gaia_memory_conversations": 0, - "gaia_memory_cache": 0 - } - }, - "multi_turn_conversation": { - "error": "unhashable type: 'dict'", - "conversation": [], - "initial_state": { - "gaia_memory": 0, - "gaia_memory_working": 0, - "gaia_memory_conversations": 0, - "gaia_memory_cache": 0 - }, - "final_state": { - "gaia_memory": 0, - "gaia_memory_working": 0, - "gaia_memory_conversations": 0, - "gaia_memory_cache": 0 - } - }, - "working_memory_verification": { - "has_entries": false, - "keys": [], - "examples": {} - }, - "conversation_memory_verification": { - "has_entries": false, - "message_count": 0, - "messages": [] - }, - "cache_verification": { - "has_entries": false, - "entry_count": 0 - } - } -} \ No newline at end of file diff --git a/gaia_web_search_results.json b/gaia_web_search_results.json deleted file mode 100644 index fcb11b48fb572bde23606e598cce9689a2dceec6..0000000000000000000000000000000000000000 --- a/gaia_web_search_results.json +++ /dev/null @@ -1,60 +0,0 @@ -{ - "mercedes_sosa": { - "success": true, - "success_percentage": 100.0, - "found_keywords": [ - "album", - "Mercedes Sosa", - "2009", - "2000" - ], - "missing_keywords": [], - "results_count": 1, - "query": "How many studio albums were published by Mercedes Sosa between 2000 and 2009 (included)?", - "top_result_snippet": "\nOkay, let's tackle this query step by step. The user wants a list of Mercedes Sosa's albums between 2000 and 2009, including release dates and record labels. Also, they need to know how many studio albums she released in that period.\n\nFirst, I'll go through each search result to find the rel..." - }, - "wikipedia_dinosaur": { - "success": true, - "success_percentage": 100.0, - "found_keywords": [ - "FunkMonk", - "Wikipedia", - "featured", - "2016", - "Giganotosaurus" - ], - "missing_keywords": [], - "results_count": 5, - "query": "Who nominated the only Featured Article on English Wikipedia about a dinosaur that was promoted in November 2016?", - "top_result_snippet": "The article was promoted by Ian Rose via FACBot (talk) 14:41, 19 November 2016 [1]. ... article is one of the most viewed dinosaur articles on Wikipedia. The ..." - }, - "universe_today": { - "success": true, - "success_percentage": 75.0, - "found_keywords": [ - "Universe Today", - "NASA", - "award" - ], - "missing_keywords": [ - "paper" - ], - "results_count": 5, - "query": "On June 6, 2023, an article by Carolyn Collins Petersen was published in Universe Today. This article mentions a team that produced a paper about their observations, linked at the bottom of the article. Find this paper. Under what NASA award number was the work performed by R. G. Arendt supported by?", - "top_result_snippet": "... Resources. NASA's Universe of Learning. Exoplanet Watch is funded by NASA's Universe of Learning, which is supported by NASA under award number ..." - }, - "malko_competition": { - "success": true, - "success_percentage": 100.0, - "found_keywords": [ - "Malko", - "2021", - "conductor", - "competition" - ], - "missing_keywords": [], - "results_count": 5, - "query": "Who won the Malko Competition for Young Conductors in 2021?", - "top_result_snippet": "Malko Competition 2021, Grand Finale: Dmitry Matvienko \u00b7 Malko Competition 2021, semi-final: Dmitry Matvienko, Stravinsky/Bystr\u00f6m \u00b7 Malko Competition 2021, 2nd ..." - } -} \ No newline at end of file diff --git a/generate_env_template.py b/generate_env_template.py deleted file mode 100644 index 05108e4478b5df98fbb508b2766eda9d5d14de9d..0000000000000000000000000000000000000000 --- a/generate_env_template.py +++ /dev/null @@ -1,314 +0,0 @@ -Generate Environment Variable Template for Hugging Face Space -Generate Variable -This script generates a template .env file with all required environment variables -for the GAIA project. a This temp.env latecwith all equiredbe used to set up secrHugging Face Space. -the rject. This teplat cabe used set up secrets in -Usage: - python generate_env_template.py - -Author: DevOps Team -Date: April 30, 2025 -""" - -import os -import logging -from pagure logging -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("env_generator") - -# Required environment variables -REQUIRED_ENV_VARS = [ - { -# Requi "d envimon: "O variablePI_API_KEY", -REQUIRED_ENV_VARS = [ - cp{ - "ne Ak: eOPENAI_API_KEYy, for model access", - xm "-key-here..":"OpAIAPI kymodlcss, - "xpl: "sk...youroeni-kyhere... - }, -{{ -m":"UPABASE_URL - "ASsrpio: "URL fo yo "eSuscbn: isac", - "xa":"https://yu-pjc-d.supae.co - }, -e:PE{ - cp"i "n me":y"SUPABASE_KEY",pabase access", - , "d scriptio ":""Servicee": "SERPSupabasAI_KEY",", - ,"examl":"yJ...ou-e-key-her.." - }, - { "name": "DEFAULT_MODEL", - dri"n"me":f" ER ER_ PI"exa", }, - "description": " {h", - "example": "your-serper-api-key-ere" - }, - { - "name": "D F ULTaMODeL", "description": "OpenAI embedding model to use", - "description": " "example": "text-embe",dding-3-small" - "example": "-0125peview" - }, - { - "name": "EMBEDDING_MODEL", { - "description": " "name": "FALLBACK_MODEL",", - "example": "tion": "Fallba-3csmmll" - }, - { if default is unavailable", - "name": "FALLBACK_MODEL", "example": "gpt-3.5-turbo" - "description": " },", - "example": "" - }, - { - "name": "MEMORY_ENABLED", "name": "MEMORY_ENABLED", - "description": " "description": "Enable/disabl", - "example": "true" - }, - {e memory functionality", - "name": " TABL _"eMm", }, - "description": " {", - "example": "EMORY_TABLE" - }, - {_NAME", - "name": "DUCKDUCKGO_TIMEOUT", "description": "Supabase table for memory storage", - "description": " "example": "gaia_memory"", - "example": "30" - }, - { - "name": " },AX_RSLS", - "description": " "name": "DUCKDUCKGO_TIMEOUT",", - "example": "5" - } -] - -# Optional environment variables -OPTIONAL_ENV_VARS = [ - { - "name": "OPENAI_API_BASE", - "description": " ustom API base URL for OpenAI (optional)", - "example": "https://api.openai.com/v1" - }, - { - "name": "MO EL_TEMPERAT RE", - "description": "Temperature for model responses (0.0-1.0)", - "example": "0.7" - }, - { - "name": "MdDELescr_TOKENS", - "description": "Maximum tokens in model responses", - "example": "4000" - }, - { - "name": "WEB_SEARCHiptiULT_COoNT", - "description": "Number of results to return from web searches", - "example": "3" - }, - { - "name": "MEMORY_TTn", - "description": "Time-to-live for memory entries in seconds", - "example": "604800" # 7 days - }, - { - "name": "MEMORY_CACHE_SIZE", - "description": "Size of memory cache", - "example": "100" - }, - { - "name": "MAX_I"ERATION:", - "description": "Maximum iterations for agent execution", - "example": "10" - }, - { - "name": "VERBOSE", - "description": "Enable verbose logging", - "example": "true" - }, - { - "name": "LOG_LEVEL", - "description": "Logging level (DEBUG, INFO, WARNING, ERROR)", - "example": "INFO" - } -] - -def read_existing_env(): - """Read existing .env file if it exists""" - env_values = {} - env_file = Path(".env") - - if env_file.exists(): - logger.info("Reading existing .env file...") - with open(env_file, "r") as f: - for line in f: - line line.strip()Timeout for DuckDuckGo searches in seconds", - if line and not line.startswith("#") and "=" in line: "example": "30" -# === OPTIONAL ENVIRONMENT VARIABLES} - "name": "DUCKDUCKGO_MAX_RESULTS", - "description": "Maximum number of results from DuckDuckGo", - "example": "5" - } -] - -# Optional environment variables -OPTIONAL_ENV_VARS = [ - { - "name": "OPENAI_API_BASE", - "description": "Custom API base URL for OpenAI (optional)", - "example": "https://api.openai.com/v1" - }, - { - "name": "MODEL_TEMPERATURE", - "description": "Temperature for model responses (0.0-1.0)", - "example": "0.7" - }, - { - "name": "MODEL_MAX_TOKENS", - "description": "Maximum tokens in model responses", - "example": "4000" - }, - { - "name": "WEB_SEARCH_RESULT_COUNT", - "description": "Number of results to return from web searches", - "example": "3" - }, - { - "name": "MEMORY_TTL", - "description": "Time-to-live for memory entries in seconds", - "example": "604800" # 7 days - }, - { - "name": "MEMORY_CACHE_SIZE", - "description": "Size of memory cache", - "example": "100" - }, - { - "name": "MAX_ITERATIONS", - "description": "Maximum iterations for agent execution", - "example": "10" - }, - { - "name": "VERBOSE", - "description": "Enable verbose logging", - "example": "true" - }, - { - "name": "LOG_LEVEL", - "description": "Logging level (DEBUG, INFO, WARNING, ERROR)", - "example": "INFO" - } -] - -def read_existing_env(): - """Read existing .env file if it exists""" - env_values = {} - env_file = Path(".env") - - if env_file.exists(): - logger.info("Reading existing .env file...") - with open(env_file, "r") as f: - for line in f: - line = line.strip() - if line and not line.startswith("#") and "=" in line: - key, value = line.split("=", 1) - env_values[key.strip()] = value.strip() - - return env_values - -def generate_env_template(): - """Generate .env template file""" - # Read existing values if present - existing_values = read_existing_env() - - # Create template file - template_file = Path("env_template.txt") - logger.info(f"Generating environment template file: {template_file}") - - with open(template_file, "w") as f: - f.write("# GAIA Environment Variables Template\n") - f.write("# Copy these to Hugging Face Space secrets\n\n") - - # Required variables - f.write("# === REQUIRED ENVIRONMENT VARIABLES ===\n\n") - for var in REQUIRED_ENV_VARS: - f.write(f"# {var['description']}\n") - value = existing_values.get(var['name'], var['example']) - if var['name'] in existing_values: - value = "****" if "KEY" in var['name'] or "key" in var['name'] else value - f.write(f"{var['name']}={value}\n\n") - - # Optional variables - f.write("# === OPTIONAL ENVIRONMENT VARIABLES ===\n\n") - for var in OPTIONAL_ENV_VARS: - f.write(f"# {var['description']}\n") - value = existing_values.get(var['name'], var['example']) - if var['name'] in existing_values: - value = "****" if "KEY" in var['name'] or "key" in var['name'] else value - f.write(f"{var['name']}={value}\n\n") - - logger.info(f"✅ Environment template generated: {template_file}") - return template_file - -def generate_hf_secrets_commands(): - """Generate commands for setting HF secrets""" - existing_values = read_existing_env() - commands_file = Path("huggingface_secrets_commands.sh") - - logger.info(f"Generating Hugging Face secrets commands: {commands_file}") - - with open(commands_file, "w") as f: - f.write("#!/bin/bash\n\n") - f.write("# Hugging Face CLI commands to set secrets\n") - f.write("# First, install the Hugging Face CLI and login:\n") - f.write("# pip install huggingface_hub\n") - f.write("# huggingface-cli login\n\n") - - f.write(f"SPACE_ID=JoachimVC/Final_Assignment_Template\n\n") - - # Required variables - f.write("# Required environment variables\n") - for var in REQUIRED_ENV_VARS: - value = existing_values.get(var['name'], var['example']) - if var['name'] in existing_values: - # Mask sensitive values in the commands file - if "KEY" in var['name'] or "key" in var['name']: - f.write(f"# huggingface-cli repo secret set {var['name']} -r $SPACE_ID\n") - else: - f.write(f"huggingface-cli repo secret set {var['name']} \"{value}\" -r $SPACE_ID\n") - else: - f.write(f"# huggingface-cli repo secret set {var['name']} -r $SPACE_ID\n") - - f.write("\n# Optional environment variables\n") - for var in OPTIONAL_ENV_VARS: - if var['name'] in existing_values: - value = existing_values.get(var['name']) - if "KEY" in var['name'] or "key" in var['name']: - f.write(f"# huggingface-cli repo secret set {var['name']} -r $SPACE_ID\n") - else: - f.write(f"huggingface-cli repo secret set {var['name']} \"{value}\" -r $SPACE_ID\n") - else: - f.write(f"# huggingface-cli repo secret set {var['name']} {var['example']} -r $SPACE_ID\n") - - # Make the file executable - commands_file.chmod(0o755) - logger.info(f"✅ Hugging Face secrets commands generated: {commands_file}") - return commands_file - -def main(): - """Generate environment templates and Hugging Face secrets commands""" - logger.info("Starting environment template generation...") - - template_file = generate_env_template() - commands_file = generate_hf_secrets_commands() - - logger.info("\n===== TEMPLATE GENERATION COMPLETE =====") - logger.info(f"Environment template: {template_file}") - logger.info(f"HF secrets commands: {commands_file}") - logger.info("\nTo set up your Hugging Face Space secrets:") - logger.info("1. Go to Hugging Face Space settings: https://huggingface.co/spaces/JoachimVC/Final_Assignment_Template/settings") - logger.info("2. Navigate to 'Secrets' section") - logger.info("3. Add each variable from the template") - logger.info("\nAlternatively, use the generated script with the Hugging Face CLI") - - return True - -if __name__ == "__main__": - main() \ No newline at end of file diff --git a/get_huggingface_token.py b/get_huggingface_token.py deleted file mode 100644 index 27df974dcd1b4b2675fd966b213e301277910794..0000000000000000000000000000000000000000 --- a/get_huggingface_token.py +++ /dev/null @@ -1,228 +0,0 @@ -#!/usr/bin/env python -""" -Hugging Face Token Helper - -This script helps users obtain and validate a Hugging Face token -for deployment to Hugging Face Spaces. - -Usage: - python get_huggingface_token.py [--validate] - -Author: DevOps Team -Date: April 30, 2025 -""" - -import os -import sys -import argparse -import getpass -import logging -import subprocess -from pathlib import Path - -# Configure logging -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("hf_token_helper") - -def parse_arguments(): - """Parse command-line arguments""" - parser = argparse.ArgumentParser(description="Hugging Face Token Helper") - parser.add_argument("--validate", action="store_true", - help="Validate existing token only") - return parser.parse_args() - -def check_huggingface_cli(): - """Check if Hugging Face CLI is installed""" - try: - result = subprocess.run( - ["huggingface-cli", "--version"], - check=True, - text=True, - capture_output=True - ) - logger.info(f"Hugging Face CLI is installed: {result.stdout.strip()}") - return True - except (subprocess.CalledProcessError, FileNotFoundError): - logger.warning("Hugging Face CLI not found") - return False - -def install_huggingface_cli(): - """Install Hugging Face CLI""" - logger.info("Installing Hugging Face CLI...") - try: - subprocess.run( - ["pip", "install", "huggingface_hub"], - check=True, - text=True, - capture_output=True - ) - logger.info("Hugging Face CLI installed successfully") - return True - except subprocess.CalledProcessError as e: - logger.error(f"Failed to install Hugging Face CLI: {e.stderr}") - return False - -def check_existing_token(): - """Check if user already has a token""" - token_path = os.path.expanduser("~/.huggingface/token") - if os.path.exists(token_path): - try: - with open(token_path, "r") as f: - token = f.read().strip() - if token: - logger.info("Existing Hugging Face token found") - return token - except Exception as e: - logger.error(f"Error reading token file: {e}") - - logger.info("No existing Hugging Face token found") - return None - -def validate_token(token=None): - """Validate Hugging Face token""" - if not token: - token = check_existing_token() - if not token: - logger.error("No token found to validate") - return False - - try: - # Check if we can access the Hugging Face API with this token - from huggingface_hub import HfApi - api = HfApi(token=token) - user_info = api.whoami() - username = user_info["name"] - - logger.info(f"Token valid for user: {username}") - return True - except ImportError: - logger.error("huggingface_hub not installed properly") - return False - except Exception as e: - logger.error(f"Token validation failed: {e}") - return False - -def login_with_cli(): - """Login using the Hugging Face CLI""" - logger.info("Logging in to Hugging Face...") - try: - subprocess.run( - ["huggingface-cli", "login"], - check=False, # Don't check return code as it's interactive - text=True - ) - - # Check if login was successful - token = check_existing_token() - if token and validate_token(token): - logger.info("Login successful") - return True - else: - logger.error("Login failed or token is invalid") - return False - except Exception as e: - logger.error(f"Error during login: {e}") - return False - -def manual_token_setup(): - """Manually set up a Hugging Face token""" - logger.info("Setting up Hugging Face token manually...") - - # Instructions - print("\n" + "="*80) - print("To obtain a Hugging Face token:") - print("1. Go to https://huggingface.co/settings/tokens") - print("2. Click 'New token'") - print("3. Enter a name for your token (e.g., 'GAIA Deployment')") - print("4. Select role: 'Write'") - print("5. Click 'Generate token'") - print("6. Copy the generated token") - print("="*80 + "\n") - - # Get token from user - token = getpass.getpass("Enter your Hugging Face token: ").strip() - if not token: - logger.error("No token provided") - return False - - # Create token directory if it doesn't exist - token_dir = os.path.expanduser("~/.huggingface") - os.makedirs(token_dir, exist_ok=True) - - # Save token - token_path = os.path.join(token_dir, "token") - try: - with open(token_path, "w") as f: - f.write(token) - logger.info(f"Token saved to {token_path}") - - # Validate the saved token - if validate_token(token): - return True - else: - logger.error("Token validation failed after saving") - return False - except Exception as e: - logger.error(f"Error saving token: {e}") - return False - -def main(): - """Main function""" - args = parse_arguments() - - logger.info("Starting Hugging Face token helper...") - - # Check for existing token - token = check_existing_token() - - if args.validate: - if token and validate_token(token): - logger.info("Token is valid and ready for deployment") - return True - else: - logger.error("Token validation failed") - return False - - # If we have a token, validate it - if token: - if validate_token(token): - logger.info("Existing token is valid and ready for deployment") - return True - else: - logger.warning("Existing token is invalid") - - # Check if Hugging Face CLI is installed - if not check_huggingface_cli(): - if not install_huggingface_cli(): - logger.error("Failed to install Hugging Face CLI") - logger.info("Falling back to manual token setup") - return manual_token_setup() - - # Try logging in with CLI - logger.info("Attempting login with Hugging Face CLI...") - if login_with_cli(): - logger.info("Login successful, token is ready for deployment") - return True - - # If CLI login fails, fall back to manual setup - logger.info("CLI login failed, falling back to manual token setup") - return manual_token_setup() - -if __name__ == "__main__": - success = main() - - if success: - print("\n" + "="*80) - print("SUCCESS: Hugging Face token is set up and validated") - print("You can now run 'python deploy_to_huggingface.py' to deploy your project") - print("="*80 + "\n") - sys.exit(0) - else: - print("\n" + "="*80) - print("ERROR: Hugging Face token setup failed") - print("Please try again or set up the token manually") - print("="*80 + "\n") - sys.exit(1) \ No newline at end of file diff --git a/hf_deployment_instructions.md b/hf_deployment_instructions.md deleted file mode 100644 index 4a7f60b8cb660e98ad3c958eea932f0cc47c6115..0000000000000000000000000000000000000000 --- a/hf_deployment_instructions.md +++ /dev/null @@ -1,133 +0,0 @@ -# GAIA Agent Deployment Instructions for Hugging Face Spaces - -This guide provides step-by-step instructions to deploy the GAIA Agent to a Hugging Face Space. - -## Prerequisites - -- A Hugging Face account -- Git installed on your local machine -- Hugging Face CLI installed: `pip install huggingface_hub` -- Required API keys (OpenAI, Supabase, Serper) - -## Step 1: Fix Known Issues - -Before deploying, ensure all critical issues are fixed: - -1. **Run configuration fixes** - ```bash - # Fix Supabase memory tables and try-except blocks - python -m memory.supabase_memory_fix - - # Verify environment variables and dependencies - python verify_gaia_assessment.py --verbose - ``` - -2. **Validation testing** (optional but recommended) - ```bash - # Run comprehensive validation - python test_deployment_readiness.py - ``` - -## Step 2: Create a New Hugging Face Space - -1. Go to [https://huggingface.co/spaces](https://huggingface.co/spaces) -2. Click "Create new Space" -3. Select the following options: - - **Owner**: Your username or organization - - **Space name**: Choose a name (e.g., `gaia-agent`) - - **License**: Choose appropriate license (e.g., MIT) - - **SDK**: Select "Gradio" - - **Space hardware**: CPU (Basic) - - **Privacy**: Public (or Private if preferred) -4. Click "Create Space" - -## Step 3: Set Up Environment Variables - -1. In your Space, go to the "Settings" tab -2. Scroll down to "Repository secrets" -3. Add the following secrets: - - `OPENAI_API_KEY`: Your OpenAI API key - - `SUPABASE_URL`: Your Supabase URL - - `SUPABASE_KEY`: Your Supabase key - - `SERPER_API_KEY`: Your Serper API key (optional but recommended for search functionality) - - `HF_TOKEN`: Your Hugging Face token (optional) - - `SPACE_ID`: Will be automatically set by Hugging Face - -## Step 4: Prepare Your Local Repository - -1. Clone your Space repository: - ```bash - git clone https://huggingface.co/spaces/YOUR_USERNAME/YOUR_SPACE_NAME - cd YOUR_SPACE_NAME - ``` - -2. Copy your GAIA project files into this directory: - ```bash - # Navigate to your project directory - cd path/to/your/gaia/project - - # Copy all files (excluding .git directory) - cp -r * path/to/cloned/space/ - ``` - -## Step 5: Push to Hugging Face - -1. Add all files to Git: - ```bash - git add . - ``` - -2. Commit your changes: - ```bash - git commit -m "Initial GAIA Agent deployment" - ``` - -3. Push to Hugging Face: - ```bash - git push - ``` - -## Step 6: Verify Deployment - -1. Go to your Space URL: `https://huggingface.co/spaces/YOUR_USERNAME/YOUR_SPACE_NAME` -2. Ensure the Gradio interface loads correctly -3. Test the GAIA Agent with a sample query to verify it's working - -## Troubleshooting - -If you encounter issues with the deployment: - -1. **Check Space Logs**: - - Go to the "Factory" tab in your Space - - Click on "Show logs" to see error messages - -2. **Verify Environment Variables**: - - Ensure all required secrets are correctly set - - Check for typos or missing values - -3. **Common Issues**: - - Memory integration failure: Make sure Supabase tables are properly set up - - OpenAI API errors: Verify your API key is valid and has sufficient quota - - Gradio interface not loading: Check for syntax errors in app.py - -## Automated Deployment (Optional) - -For automated deployment, use the provided script: - -```bash -python deploy_to_hf.py --username YOUR_USERNAME --space YOUR_SPACE_NAME -``` - -## Post-Deployment Tasks - -1. **Monitor Performance**: - - Check memory usage in Space metrics - - Watch for any bottlenecks or timeouts - -2. **Update Documentation**: - - Ensure your Space README is updated with usage instructions - - Document any API rate limits or quotas - -3. **Configure Persistent Storage** (if needed): - - Hugging Face Spaces have ephemeral storage by default - - Use Supabase for any data that needs to persist diff --git a/huggingface_space_cleanup.md b/huggingface_space_cleanup.md deleted file mode 100644 index fd25f4bdd1980ba040c45e6b281a6ec4d6fa7aee..0000000000000000000000000000000000000000 --- a/huggingface_space_cleanup.md +++ /dev/null @@ -1,75 +0,0 @@ -# Hugging Face Space Cleanup Guide - -This document identifies files and directories that can be safely removed from the Hugging Face Space deployment to keep the environment clean and optimized. - -## Files and Directories Safe to Delete - -### Development Tools & Configuration -- `.roo/` - Roo agent configuration for VS Code -- `.roomodes/` - Roo mode-specific configurations -- `.gitattributes` - Git attributes file -- `.gitignore` - Git ignore rules -- `.rooignore` - Roo-specific ignore rules - -### Test Files -- `tests/` - Test directory (all test files) -- `backup_tests/` - Backup test directory -- `test_*.py` - Individual test files in root directory -- `verify_gaia_assessment.py` - Verification script -- `verify_production.py` - Production verification - -### Deployment & Cleanup Scripts -- `deploy_to_hf.py` - HF deployment script (already used) -- `deploy_to_huggingface.py` - Alternate deployment script -- `production_cleanup.py` - Production cleanup utility -- `remove_debug_statements.py` - Debug statement removal tool -- `remove_duplicate_tests.py` - Test deduplication script -- `create_env_and_test.py` - Environment setup script -- `generate_env_template.py` - Environment template generator -- `get_huggingface_token.py` - Token retrieval script -- `huggingface_secrets_commands.sh` - Secrets configuration script -- `prepare_for_hf_deployment.py` - Pre-deployment preparation - -### Documentation Not Needed for Runtime -- `IMPLEMENTATION_PLAN.md` - Implementation documentation -- `README_original.md` - Original readme (if present) -- `SUPABASE_TABLES_README.md` - Supabase table structure docs -- `SUPABASE_TEST_README.md` - Supabase testing docs -- `deployment_readiness_report.md` - Deployment readiness info -- `final_verification_summary.md` - Verification summary -- `hf_deployment_instructions.md` - Deployment instructions -- `production_verification_report.md` - Production verification report - -### Output & Temporary Files -- `cleanup_results.txt` - Cleanup script output -- `*.log` files - Any log files from local testing -- `.env_template.txt` - Environment template file (not needed since secrets are set) -- `env_template.txt` - Another env template - -## Files to Keep - -### Core Application Code -- `app.py` - Main application entry point -- `agent/` - Core agent functionality -- `memory/` - Memory management -- `tools/` - Tool implementations -- `utils/` - Utility functions -- `requirements.txt` - Dependencies - -### Configuration -- `.env` if present (though environment variables should be set in HF Secrets) -- `space_config.json` - HF Space configuration - -### Documentation for Users -- `README.md` - Main README with usage instructions -- `GAIA_ASSESSMENT_README.md` - GAIA documentation -- `docs/` - Documentation directory (user-facing docs) - -## How to Delete Files - -Files can be deleted directly through the Hugging Face Space web interface: -1. Navigate to the "Files" tab in your Space -2. Select the file/directory you want to delete -3. Click on the delete icon or right-click and select "Delete" - -For bulk deletion, consider using the Hugging Face CLI tool locally, which allows for more efficient management of files. \ No newline at end of file diff --git a/huggingface_space_deployment_guide.md b/huggingface_space_deployment_guide.md deleted file mode 100644 index 75572c34f9696c6d487edf59c6bd87b9eeae2073..0000000000000000000000000000000000000000 --- a/huggingface_space_deployment_guide.md +++ /dev/null @@ -1,142 +0,0 @@ -# GAIA Agent Deployment Guide for Hugging Face Space - -## Overview - -This guide provides step-by-step instructions for deploying the GAIA Agent to Hugging Face Space, including resolving common issues like the OAuth configuration error. - -## Deployment Requirements - -- GitHub repository with your GAIA agent implementation -- Hugging Face account -- Required API keys: - - OpenAI API Key - - Supabase URL and Key - - (Optional) Serper API Key - -## Step 1: Prepare Your Environment Variables - -Hugging Face Spaces require all environment variables to be configured through the Space settings. - -Required environment variables: -- `OPENAI_API_KEY`: Your OpenAI API key -- `SUPABASE_URL`: Your Supabase project URL -- `SUPABASE_KEY`: Your Supabase API key -- (Optional) `SERPER_API_KEY`: Your Serper API key - -## Step 2: Enable OAuth Authentication - -**Critical**: The GAIA Agent assessment requires OAuth for user identification during submission. - -1. In your Hugging Face Space, navigate to Settings -2. Under "Repository secrets," add all required API keys -3. Under the "Settings" tab, edit the `README.md` file -4. Add or update the Space metadata to enable OAuth: - -```yaml ---- -title: GAIA Agent Assessment -emoji: 🤖 -colorFrom: blue -colorTo: green -sdk: gradio -sdk_version: 3.50.2 -app_file: app.py -pinned: false -license: mit -hf_oauth: true # This enables OAuth in your Space! ---- -``` - -> **Note**: The `hf_oauth: true` setting is **required** for the assessment to work properly! - -## Step 3: Create a Space using the GitHub Repository - -1. Go to [Hugging Face Spaces](https://huggingface.co/spaces) -2. Click "Create new Space" -3. Select "Gradio" as the SDK -4. Link to your GitHub repository or upload your files directly -5. Set the proper hardware requirements (at least CPU) - -## Step 4: Verify Deployment Configuration - -Before deploying, verify your configuration using our deployment verification script: - -```bash -python deployment_verification.py -``` - -This script checks: -- Environment variables -- OAuth configuration -- Agent initialization -- Memory system -- Tool integrations -- Agent graph -- Gradio app initialization - -## Step 5: Troubleshooting Common Issues - -### OAuth Error - -If you see this error: -``` -ValueError: OAuth is required but OAUTH_CLIENT_ID environment variable is not set. Make sure you've enabled OAuth in your Space by setting `hf_oauth: true` in the Space metadata. -``` - -**Solution**: -1. Edit your Space metadata to include `hf_oauth: true` -2. Restart your Space after making this change - -### Demo Mode Configuration - -To run in demo mode without OAuth for local testing: - -```python -# In app.py, modify to check for demo mode -if not os.getenv("DEMO_MODE"): - # OAuth required block -else: - # Skip OAuth requirement -``` - -### Supabase Connection Issues - -If you encounter Supabase connection issues: - -1. Verify your Supabase URL and key are correctly set in environment variables -2. Ensure your Supabase database has the required tables: - - Run `from memory.fix_supabase_tables import ensure_tables_exist` - - Execute `ensure_tables_exist()` to create missing tables - -### Other Deployment Issues - -1. **Memory**: Ensure Supabase tables are correctly set up -2. **API Timeouts**: Increase timeout values in requests.get/post calls -3. **Module Imports**: Verify all dependencies are in requirements.txt -4. **Environment Variables**: Double-check all environment variables in the Space settings - -## Step 6: Post-Deployment Verification - -After deploying: - -1. Check the Space logs for any errors -2. Verify the app loads properly -3. Test the OAuth login functionality -4. Run a test evaluation to ensure the agent works correctly - -## Additional Resources - -- [Hugging Face Spaces Documentation](https://huggingface.co/docs/hub/spaces) -- [Gradio Documentation](https://www.gradio.app/docs) -- [OAuth in Hugging Face Spaces](https://huggingface.co/docs/hub/spaces-oauth) - -## Deployment Checklist - -- [ ] All required environment variables configured -- [ ] OAuth enabled in Space metadata (`hf_oauth: true`) -- [ ] Requirements file includes all dependencies -- [ ] Supabase tables created and configured -- [ ] Memory system validated -- [ ] All tools working correctly -- [ ] Agent graph initialized properly -- [ ] Gradio app loads successfully \ No newline at end of file diff --git a/huggingface_space_deployment_summary.md b/huggingface_space_deployment_summary.md deleted file mode 100644 index 8ce33d3bf7a6e400c9af4a8e5c1569ebcc2e0d86..0000000000000000000000000000000000000000 --- a/huggingface_space_deployment_summary.md +++ /dev/null @@ -1,64 +0,0 @@ -# GAIA Agent HuggingFace Space Deployment Summary - -## Overview - -This report summarizes the verification, testing, and fixes performed to ensure the GAIA Agent is ready for deployment to Hugging Face Space. - -## Critical Issues Fixed - -1. **Gradio Accordion Update Error** - - **Issue**: The `gr.Accordion.update` reference was causing runtime errors - - **Fix**: Properly stored the accordion component reference and updated the update method - - **Impact**: Ensures results are displayed correctly after agent execution - -2. **OAuth Configuration Error** - - **Issue**: The app crashed when OAuth was not configured in Hugging Face Space - - **Fix**: Implemented graceful OAuth handling with a demonstration mode - - **Impact**: The app now works in both full functionality mode (with OAuth) and demo mode (without OAuth) - -## Deployment Readiness Assessment - -| Component | Status | Notes | -|-----------|--------|-------| -| Core Agent Functionality | ✅ Ready | Core GAIAAgent implementation is stable | -| Gradio Interface | ✅ Ready | Fixed Accordion update issue | -| OAuth Integration | ✅ Ready | Added graceful handling and demo mode | -| Error Handling | ✅ Ready | Improved error handling for various scenarios | -| Documentation | ✅ Ready | Added deployment guides and verification reports | - -## Deployment Process - -1. Push the updated code to the Hugging Face Space Git repository -2. Configure environment variables in Space settings: - - Required: `OPENAI_API_KEY`, `SUPABASE_URL`, `SUPABASE_KEY` - - Optional: `SERPER_API_KEY` -3. Configure OAuth by setting `hf_oauth: true` in Space repository secrets (if desired) -4. Monitor build logs and verify the Space is running correctly - -## Verification Strategy - -After deployment, verify that: - -1. The app loads without errors -2. The appropriate mode (full or demo) is active based on your OAuth configuration -3. Agent functionality works with a test question -4. Results are displayed correctly in the interface - -## Performance Considerations - -- The agent uses OpenAI API calls which may have rate limits -- Memory functionality requires Supabase connections which may have latency -- Gradio interface is lightweight and should perform well in the Hugging Face Space environment - -## Rollback Strategy - -If issues are encountered after deployment: - -1. Identify the specific component causing the issue from Space logs -2. If needed, revert to the previous working commit using Git -3. Fix issues locally and re-deploy with verified fixes -4. Consider temporarily enabling demo mode while fixing issues - -## Conclusion - -The GAIA Agent application has been thoroughly tested and is ready for deployment to Hugging Face Space. The application now includes robust error handling, graceful degradation when certain configurations are not available, and comprehensive documentation for deployment and troubleshooting. \ No newline at end of file diff --git a/huggingface_space_update_guide.md b/huggingface_space_update_guide.md deleted file mode 100644 index 5258d54da63fcb9fc612ae436a616b20e03dc9bf..0000000000000000000000000000000000000000 --- a/huggingface_space_update_guide.md +++ /dev/null @@ -1,129 +0,0 @@ -# Hugging Face Space Update Guide for GAIA Agent - -## Files to Update on Hugging Face Space - -To fix the OAuth error and properly deploy the GAIA agent, you need to update these specific files: - -1. **README.md** - To add the Space metadata with OAuth configuration -2. **app.py** - Either apply the demo mode patch or ensure it handles OAuth properly -3. **Environment Variables** - Set these in the Space settings panel (not a file) - -## Step-by-Step Instructions - -### 1. Update README.md with Space Metadata - -The Space metadata needs to include `hf_oauth: true` to enable OAuth authentication. Here's how to update it: - -1. Navigate to your Space on Hugging Face (https://huggingface.co/spaces/YourUsername/YourSpace) -2. Click on the "Files" tab -3. Click on the README.md file to open it -4. Click the "Edit" button (pencil icon) -5. Add or update the YAML frontmatter at the top of the file to include the `hf_oauth: true` setting: - -```yaml ---- -title: GAIA Agent Assessment -emoji: 🤖 -colorFrom: blue -colorTo: green -sdk: gradio -sdk_version: 3.50.2 -app_file: app.py -pinned: false -license: mit -hf_oauth: true ---- -``` - -6. Click "Commit changes" at the bottom of the page -7. Enter an optional commit message like "Enable OAuth authentication" -8. Click "Commit" to save the changes - -### 2. Configure Environment Variables - -1. Navigate to your Space on Hugging Face -2. Click on the "Settings" tab -3. Scroll down to the "Repository secrets" section -4. Add each required environment variable: - - Click "New secret" - - Enter the variable name (e.g., `OPENAI_API_KEY`) - - Enter the value - - Click "Add secret" -5. Repeat for all required variables: - - `OPENAI_API_KEY` - - `SUPABASE_URL` - - `SUPABASE_KEY` - - `SERPER_API_KEY` (optional) - -### 3. Update app.py (if needed) - -If you want to use the demo mode patch for development but keep OAuth for production: - -1. Navigate to your Space -2. Click on the "Files" tab -3. Click on app.py to open it -4. Click the "Edit" button (pencil icon) -5. Update the code to include demo mode support: - -```python -# At the top with other imports -import os - -# After the imports -# Set to True to enable demo mode (skip OAuth requirement) -DEMO_MODE = os.getenv("DEMO_MODE", "").lower() in ("true", "1", "yes") - -# Update the OAuth check in run_and_submit_all function -# Replace: -if not profile: - logger.warning("User not logged in.") - return "Please Login to Hugging Face with the button. OAuth login is required for assessment submission.", None - -# With: -if not profile and not DEMO_MODE: - logger.warning("User not logged in.") - return "Please Login to Hugging Face with the button. OAuth login is required for assessment submission.", None - -# Use demo username in demo mode -if DEMO_MODE and not profile: - logger.warning("Running in DEMO MODE - using demo username") - username = "demo_user" -else: - username = f"{profile.username}" -``` - -6. Click "Commit changes" at the bottom of the page -7. Enter an optional commit message -8. Click "Commit" to save the changes - -### 4. Restart Your Space - -After making these changes: - -1. Navigate to your Space -2. Click on the "Settings" tab -3. Scroll down to the "Factory reboot" section -4. Click "Restart Space" to apply all your changes - -## Verification - -After updating these files and restarting your Space: - -1. Check the Space logs (in the "Settings" tab) to confirm no OAuth errors -2. Verify the app loads correctly in the browser -3. Test the OAuth login functionality by clicking the login button -4. Run a test evaluation to ensure the agent works correctly - -## Common Issues - -1. **Space still shows OAuth error**: - - Verify the metadata in README.md has `hf_oauth: true` - - Check that you've restarted the Space after changes - -2. **Environment variables not working**: - - Verify they're added as repository secrets - - Make sure you've restarted the Space after adding them - -3. **App still requires OAuth in demo mode**: - - Set the `DEMO_MODE` environment variable to "true" in repository secrets - - Verify the app.py changes are correct \ No newline at end of file diff --git a/memory/__init__.py b/memory/__init__.py deleted file mode 100644 index 2c01666d6432b237f500172968955eef940f6cf4..0000000000000000000000000000000000000000 --- a/memory/__init__.py +++ /dev/null @@ -1,41 +0,0 @@ -""" -Memory module for GAIA implementation. -Handles persistent memory storage and retrieval using Supabase. - -This module provides memory interfaces for: -- Storing intermediate results during agent execution -- Caching previous answers and tool results -- Maintaining conversation context - -Usage: - from memory import SupabaseMemory, ConversationMemory, ResultCache, WorkingMemory - - # Create the base memory implementation - memory = SupabaseMemory() - - # Create specialized memory interfaces - conversation = ConversationMemory(memory, "conversation-123") - cache = ResultCache(memory) - working = WorkingMemory(memory, "session-456") - - # Use the memory interfaces - conversation.add_message("user", "What is the capital of France?") - cache.cache_result("capital-france", "Paris") - working.store_intermediate_result("search-results", ["Paris", "France", "Capital"]) -""" - -from memory.supabase_memory import ( - SupabaseMemory, - ConversationMemory, - ResultCache, - WorkingMemory, - MemoryEntry -) - -__all__ = [ - "SupabaseMemory", - "ConversationMemory", - "ResultCache", - "WorkingMemory", - "MemoryEntry" -] \ No newline at end of file diff --git a/memory/__pycache__/__init__.cpython-313.pyc b/memory/__pycache__/__init__.cpython-313.pyc deleted file mode 100644 index 195a58cf373e90df9f9f30f88ce2d4be10cdf1c2..0000000000000000000000000000000000000000 Binary files a/memory/__pycache__/__init__.cpython-313.pyc and /dev/null differ diff --git a/memory/__pycache__/fix_supabase_tables.cpython-313.pyc b/memory/__pycache__/fix_supabase_tables.cpython-313.pyc deleted file mode 100644 index c39cb4464b156d87db69abda5a5607485e3a66cc..0000000000000000000000000000000000000000 Binary files a/memory/__pycache__/fix_supabase_tables.cpython-313.pyc and /dev/null differ diff --git a/memory/__pycache__/supabase_memory.cpython-313.pyc b/memory/__pycache__/supabase_memory.cpython-313.pyc deleted file mode 100644 index bfeddd22d6d1e3241a7bb939cd9a3a069fb6614e..0000000000000000000000000000000000000000 Binary files a/memory/__pycache__/supabase_memory.cpython-313.pyc and /dev/null differ diff --git a/memory/__pycache__/supabase_memory_fix.cpython-313.pyc b/memory/__pycache__/supabase_memory_fix.cpython-313.pyc deleted file mode 100644 index f52806cd84ac1303eefb43b1c37b79b007439031..0000000000000000000000000000000000000000 Binary files a/memory/__pycache__/supabase_memory_fix.cpython-313.pyc and /dev/null differ diff --git a/memory/fix_supabase_tables.py b/memory/fix_supabase_tables.py deleted file mode 100644 index bf7d862a649416a41ad141b4fc99adab72d168a0..0000000000000000000000000000000000000000 --- a/memory/fix_supabase_tables.py +++ /dev/null @@ -1,465 +0,0 @@ -""" -Fix for Supabase Tables in GAIA Agent - -This script fixes the issue where only the main memory table (gaia_memory) is being used, -while the specialized tables (gaia_memory_working, gaia_memory_conversations, gaia_memory_cache) -remain empty. - -The fix modifies the SupabaseMemory class to accept a table_name parameter in its methods, -and updates the specialized memory classes to pass their table name when calling the base memory methods. - -Usage: - python -m memory.fix_supabase_tables -""" - -import os -import sys -import json -import logging -from typing import Dict, Any, Optional, List - -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(name)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("fix_supabase_tables") - -sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) - -try: - from memory.supabase_memory import ( - SupabaseMemory, - WorkingMemory, - ConversationMemory, - ResultCache - ) - from agent.config import get_memory_config -except ImportError as e: - logger.error(f"Error importing required components: {str(e)}") - sys.exit(1) - -def ensure_tables_exist() -> bool: - """Ensure all required tables exist in Supabase.""" - logger.info("Ensuring all required tables exist...") - - memory_config = get_memory_config() - memory_config["enabled"] = True - - base_memory = SupabaseMemory(config=memory_config) - - if not base_memory.client or not base_memory.initialized: - logger.error("Supabase client not initialized") - return False - - tables = { - "base": base_memory.table_name, - "working": f"{base_memory.table_name}_working", - "conversations": f"{base_memory.table_name}_conversations", - "cache": f"{base_memory.table_name}_cache" - } - - for table_type, table_name in tables.items(): - try: - try: - base_memory.client.table(table_name).select("*").limit(1).execute() - logger.info(f"Table {table_name} already exists") - except Exception: - logger.info(f"Creating table {table_name}") - - sql = f""" - CREATE TABLE IF NOT EXISTS {table_name} ( - id UUID PRIMARY KEY DEFAULT uuid_generate_v4(), - key TEXT NOT NULL, - value TEXT, - metadata JSONB, - created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(), - updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(), - expires_at TIMESTAMP WITH TIME ZONE - ); - CREATE INDEX IF NOT EXISTS {table_name}_key_idx ON {table_name}(key); - """ - - base_memory.client.query(sql).execute() - - base_memory.client.table(table_name).select("*").limit(1).execute() - - logger.info(f"Table {table_name} created successfully") - except Exception as e: - logger.error(f"Error checking/creating table {table_name}: {str(e)}") - return False - - return True - -def apply_monkey_patch() -> bool: - """Apply monkey patches to fix the Supabase tables issue.""" - logger.info("Applying monkey patches to fix Supabase tables...") - - original_store = SupabaseMemory.store - - def patched_store(self, key, value, metadata=None, ttl=None, table_name=None): - """Patched store method that accepts a table_name parameter.""" - current_table = table_name or self.table_name - - from memory.supabase_memory import MemoryEntry - entry = MemoryEntry( - key=key, - value=value, - metadata=metadata, - ttl=ttl or self.config.get("ttl") - ) - - self.cache[key] = entry - - if len(self.cache) > self.cache_size: - keys_to_remove = sorted( - self.cache.keys(), - key=lambda k: self.cache[k].created_at - )[:len(self.cache) - self.cache_size] - - for k in keys_to_remove: - del self.cache[k] - - if self.initialized and self.client: - try: - entry_dict = entry.to_dict() - - result = self.client.table(current_table).upsert(entry_dict).execute() - - return True - except Exception as e: - logger.error(f"Error storing entry in table {current_table}: {key} - {str(e)}") - return False - - return True - - original_retrieve = SupabaseMemory.retrieve - - def patched_retrieve(self, key, default=None, table_name=None): - """Patched retrieve method that accepts a table_name parameter.""" - current_table = table_name or self.table_name - - if key in self.cache: - entry = self.cache[key] - - if entry.expires_at: - from datetime import datetime - expires_at = datetime.fromisoformat(entry.expires_at) - if expires_at < datetime.now(): - del self.cache[key] - return default - - return entry.value - - if self.initialized and self.client: - try: - result = self.client.table(current_table).select("*").eq("key", key).execute() - - if result.data and len(result.data) > 0: - entry_data = result.data[0] - - from memory.supabase_memory import MemoryEntry - entry = MemoryEntry.from_dict(entry_data) - - if entry.expires_at: - from datetime import datetime - expires_at = datetime.fromisoformat(entry.expires_at) - if expires_at < datetime.now(): - self.delete(key, table_name=current_table) - return default - - self.cache[key] = entry - - return entry.value - except Exception as e: - logger.error(f"Error retrieving entry from table {current_table}: {key} - {str(e)}") - - return default - - original_delete = SupabaseMemory.delete - - def patched_delete(self, key, table_name=None): - """Patched delete method that accepts a table_name parameter.""" - current_table = table_name or self.table_name - - if key in self.cache: - del self.cache[key] - - if self.initialized and self.client: - try: - result = self.client.table(current_table).delete().eq("key", key).execute() - - return True - except Exception as e: - logger.error(f"Error deleting entry from table {current_table}: {key} - {str(e)}") - return False - - return True - - original_list_keys = SupabaseMemory.list_keys - - def patched_list_keys(self, prefix=None, table_name=None): - """Patched list_keys method that accepts a table_name parameter.""" - current_table = table_name or self.table_name - - keys = [] - - for key in self.cache.keys(): - if prefix is None or key.startswith(prefix): - keys.append(key) - - if self.initialized and self.client: - try: - query = self.client.table(current_table).select("key") - - if prefix: - query = query.like("key", f"{prefix}%") - - result = query.execute() - - if result.data: - for entry in result.data: - key = entry.get("key") - if key and key not in keys: - keys.append(key) - - except Exception as e: - logger.error(f"Error listing keys from table {current_table}: {str(e)}") - - return keys - - original_store_intermediate = WorkingMemory.store_intermediate_result - - def patched_store_intermediate(self, key, result, metadata=None): - """Patched store_intermediate_result method that uses the working memory table.""" - working_key = f"{self.prefix}{key}" - - return self.memory.store(working_key, result, metadata, table_name=self.table_name) - - original_get_intermediate = WorkingMemory.get_intermediate_result - - def patched_get_intermediate(self, key, default=None): - """Patched get_intermediate_result method that uses the working memory table.""" - working_key = f"{self.prefix}{key}" - - return self.memory.retrieve(working_key, default, table_name=self.table_name) - - original_list_intermediate = WorkingMemory.list_intermediate_results - - def patched_list_intermediate(self): - """Patched list_intermediate_results method that uses the working memory table.""" - keys = self.memory.list_keys(self.prefix, table_name=self.table_name) - - results = {} - for key in keys: - original_key = key[len(self.prefix):] - - result = self.memory.retrieve(key, table_name=self.table_name) - - results[original_key] = result - - return results - - original_clear_working = WorkingMemory.clear - - def patched_clear_working(self): - """Patched clear method that uses the working memory table.""" - keys = self.memory.list_keys(self.prefix, table_name=self.table_name) - - success = True - for key in keys: - if not self.memory.delete(key, table_name=self.table_name): - success = False - - return success - - original_add_message = ConversationMemory.add_message - - def patched_add_message(self, role, content, metadata=None): - """Patched add_message method that uses the conversations table.""" - from datetime import datetime - timestamp = datetime.now().isoformat() - - key = f"{self.prefix}message:{timestamp}" - - message = { - "role": role, - "content": content, - "timestamp": timestamp - } - - if metadata: - message["metadata"] = metadata - - return self.memory.store(key, message, table_name=self.table_name) - - original_get_messages = ConversationMemory.get_messages - - def patched_get_messages(self, limit=None, before=None, after=None): - """Patched get_messages method that uses the conversations table.""" - keys = self.memory.list_keys(self.prefix + "message:", table_name=self.table_name) - - messages = [] - for key in keys: - message = self.memory.retrieve(key, table_name=self.table_name) - if message: - messages.append(message) - - messages.sort(key=lambda m: m.get("timestamp", "")) - - if before: - messages = [m for m in messages if m.get("timestamp", "") < before] - - if after: - messages = [m for m in messages if m.get("timestamp", "") > after] - - if limit and len(messages) > limit: - messages = messages[-limit:] - - return messages - - original_clear_conversation = ConversationMemory.clear - - def patched_clear_conversation(self): - """Patched clear method that uses the conversations table.""" - keys = self.memory.list_keys(self.prefix, table_name=self.table_name) - - success = True - for key in keys: - if not self.memory.delete(key, table_name=self.table_name): - success = False - - return success - - original_cache_result = ResultCache.cache_result - - def patched_cache_result(self, key, result, metadata=None, ttl=None): - """Patched cache_result method that uses the cache table.""" - cache_key = f"{self.prefix}{key}" - - return self.memory.store(cache_key, result, metadata, ttl, table_name=self.table_name) - - original_get_result = ResultCache.get_result - - def patched_get_result(self, key, default=None): - """Patched get_result method that uses the cache table.""" - cache_key = f"{self.prefix}{key}" - - return self.memory.retrieve(cache_key, default, table_name=self.table_name) - - original_invalidate = ResultCache.invalidate - - def patched_invalidate(self, key): - """Patched invalidate method that uses the cache table.""" - cache_key = f"{self.prefix}{key}" - - return self.memory.delete(cache_key, table_name=self.table_name) - - original_clear_cache = ResultCache.clear - - def patched_clear_cache(self): - """Patched clear method that uses the cache table.""" - keys = self.memory.list_keys(self.prefix, table_name=self.table_name) - - success = True - for key in keys: - if not self.memory.delete(key, table_name=self.table_name): - success = False - - return success - - SupabaseMemory.store = patched_store - SupabaseMemory.retrieve = patched_retrieve - SupabaseMemory.delete = patched_delete - SupabaseMemory.list_keys = patched_list_keys - - WorkingMemory.store_intermediate_result = patched_store_intermediate - WorkingMemory.get_intermediate_result = patched_get_intermediate - WorkingMemory.list_intermediate_results = patched_list_intermediate - WorkingMemory.clear = patched_clear_working - - ConversationMemory.add_message = patched_add_message - ConversationMemory.get_messages = patched_get_messages - ConversationMemory.clear = patched_clear_conversation - - ResultCache.cache_result = patched_cache_result - ResultCache.get_result = patched_get_result - ResultCache.invalidate = patched_invalidate - ResultCache.clear = patched_clear_cache - - logger.info("Monkey patches applied successfully") - return True - -def test_fix(): - """Test if the fix works correctly.""" - logger.info("Testing the fix...") - - memory_config = get_memory_config() - memory_config["enabled"] = True - - base_memory = SupabaseMemory(config=memory_config) - session_id = "test-session" - working_memory = WorkingMemory(base_memory, session_id) - - test_key = "test_fix" - test_value = { - "test_data": "This is test data for the fix", - "timestamp": "2025-04-30T12:00:00" - } - - logger.info(f"Storing test value in working memory: {test_key}") - store_result = working_memory.store_intermediate_result(test_key, test_value) - - if not store_result: - logger.error("Failed to store test value") - return False - - logger.info(f"Retrieving test value from working memory: {test_key}") - retrieved_value = working_memory.get_intermediate_result(test_key) - - if retrieved_value != test_value: - logger.error(f"Retrieved value does not match test value") - logger.error(f"Expected: {test_value}") - logger.error(f"Got: {retrieved_value}") - return False - - logger.info("Verifying value was stored in the working memory table...") - - if base_memory.client and base_memory.initialized: - try: - working_key = f"working:{session_id}:{test_key}" - response = base_memory.client.table(working_memory.table_name).select("*").eq("key", working_key).execute() - - if not response.data or len(response.data) == 0: - logger.error("No data found in working memory table") - return False - - logger.info(f"Data found in working memory table: {len(response.data)} entries") - - logger.info("Cleaning up test data...") - working_memory.clear() - - return True - except Exception as e: - logger.error(f"Error verifying data storage: {str(e)}") - return False - - return False - -def main(): - """Main entry point for the fix script.""" - - if not ensure_tables_exist(): - logger.error("Failed to ensure tables exist") - sys.exit(1) - - if not apply_monkey_patch(): - logger.error("Failed to apply monkey patch") - sys.exit(1) - - if test_fix(): - sys.exit(0) - else: - sys.exit(1) - -if __name__ == "__main__": - main() \ No newline at end of file diff --git a/memory/simplified_memory_fix.py b/memory/simplified_memory_fix.py deleted file mode 100644 index 5d2da38ff0f789d3bd8d907fd46dddb587497653..0000000000000000000000000000000000000000 --- a/memory/simplified_memory_fix.py +++ /dev/null @@ -1,350 +0,0 @@ -""" -Simplified fix for Supabase Tables in GAIA Agent - -This script provides a simplified fix that focuses only on the working memory table -(gaia_memory_working), which is the only table needed for the GAIA benchmark test -besides the main memory table. - -Usage: - python -m memory.simplified_memory_fix -""" - -import os -import sys -import json -import logging -from typing import Dict, Any, Optional - -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(name)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("simplified_memory_fix") - -sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) - -try: - from memory.supabase_memory import ( - SupabaseMemory, - WorkingMemory - ) - from agent.config import get_memory_config -except ImportError as e: - logger.error(f"Error importing required components: {str(e)}") - sys.exit(1) - -def create_working_memory_table(): - """Create the working memory table if it doesn't exist.""" - logger.info("Creating working memory table...") - - memory_config = get_memory_config() - memory_config["enabled"] = True - - base_memory = SupabaseMemory(config=memory_config) - - if not base_memory.client or not base_memory.initialized: - logger.error("Supabase client not initialized") - return False - - working_table = f"{base_memory.table_name}_working" - - try: - try: - base_memory.client.table(working_table).select("*").limit(1).execute() - logger.info(f"Table {working_table} already exists") - return True - except Exception: - logger.info(f"Creating table {working_table}") - - sql = f""" - CREATE TABLE IF NOT EXISTS {working_table} ( - id UUID PRIMARY KEY DEFAULT uuid_generate_v4(), - key TEXT NOT NULL, - value TEXT, - metadata JSONB, - created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(), - updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(), - expires_at TIMESTAMP WITH TIME ZONE - ); - CREATE INDEX IF NOT EXISTS {working_table}_key_idx ON {working_table}(key); - """ - - base_memory.client.query(sql).execute() - - base_memory.client.table(working_table).select("*").limit(1).execute() - - logger.info(f"Table {working_table} created successfully") - return True - except Exception as e: - logger.error(f"Error creating working memory table: {str(e)}") - return False - -def apply_fix(): - """Apply the fix for the working memory table.""" - logger.info("Applying fix for working memory table...") - - original_store = SupabaseMemory.store - - def patched_store(self, key, value, metadata=None, ttl=None, table_name=None): - """Patched store method that accepts a table_name parameter.""" - current_table = table_name or self.table_name - - from memory.supabase_memory import MemoryEntry - entry = MemoryEntry( - key=key, - value=value, - metadata=metadata, - ttl=ttl or self.config.get("ttl") - ) - - self.cache[key] = entry - - if len(self.cache) > self.cache_size: - keys_to_remove = sorted( - self.cache.keys(), - key=lambda k: self.cache[k].created_at - )[:len(self.cache) - self.cache_size] - - for k in keys_to_remove: - del self.cache[k] - - if self.initialized and self.client: - try: - entry_dict = entry.to_dict() - - result = self.client.table(current_table).upsert(entry_dict).execute() - - return True - except Exception as e: - logger.error(f"Error storing entry in table {current_table}: {key} - {str(e)}") - return False - - return True - - original_retrieve = SupabaseMemory.retrieve - - def patched_retrieve(self, key, default=None, table_name=None): - """Patched retrieve method that accepts a table_name parameter.""" - current_table = table_name or self.table_name - - if key in self.cache: - entry = self.cache[key] - - if entry.expires_at: - from datetime import datetime - expires_at = datetime.fromisoformat(entry.expires_at) - if expires_at < datetime.now(): - del self.cache[key] - return default - - return entry.value - - if self.initialized and self.client: - try: - result = self.client.table(current_table).select("*").eq("key", key).execute() - - if result.data and len(result.data) > 0: - entry_data = result.data[0] - - from memory.supabase_memory import MemoryEntry - entry = MemoryEntry.from_dict(entry_data) - - if entry.expires_at: - from datetime import datetime - expires_at = datetime.fromisoformat(entry.expires_at) - if expires_at < datetime.now(): - self.delete(key, table_name=current_table) - return default - - self.cache[key] = entry - - return entry.value - except Exception as e: - logger.error(f"Error retrieving entry from table {current_table}: {key} - {str(e)}") - - return default - - original_delete = SupabaseMemory.delete - - def patched_delete(self, key, table_name=None): - """Patched delete method that accepts a table_name parameter.""" - current_table = table_name or self.table_name - - if key in self.cache: - del self.cache[key] - - if self.initialized and self.client: - try: - result = self.client.table(current_table).delete().eq("key", key).execute() - - return True - except Exception as e: - logger.error(f"Error deleting entry from table {current_table}: {key} - {str(e)}") - return False - - return True - - original_list_keys = SupabaseMemory.list_keys - - def patched_list_keys(self, prefix=None, table_name=None): - """Patched list_keys method that accepts a table_name parameter.""" - current_table = table_name or self.table_name - - keys = [] - - for key in self.cache.keys(): - if prefix is None or key.startswith(prefix): - keys.append(key) - - if self.initialized and self.client: - try: - query = self.client.table(current_table).select("key") - - if prefix: - query = query.like("key", f"{prefix}%") - - result = query.execute() - - if result.data: - for entry in result.data: - key = entry.get("key") - if key and key not in keys: - keys.append(key) - - except Exception as e: - logger.error(f"Error listing keys from table {current_table}: {str(e)}") - - return keys - - original_store_intermediate = WorkingMemory.store_intermediate_result - - def patched_store_intermediate(self, key, result, metadata=None): - """Patched store_intermediate_result method that uses the working memory table.""" - working_key = f"{self.prefix}{key}" - - return self.memory.store(working_key, result, metadata, table_name=self.table_name) - - original_get_intermediate = WorkingMemory.get_intermediate_result - - def patched_get_intermediate(self, key, default=None): - """Patched get_intermediate_result method that uses the working memory table.""" - working_key = f"{self.prefix}{key}" - - return self.memory.retrieve(working_key, default, table_name=self.table_name) - - original_list_intermediate = WorkingMemory.list_intermediate_results - - def patched_list_intermediate(self): - """Patched list_intermediate_results method that uses the working memory table.""" - keys = self.memory.list_keys(self.prefix, table_name=self.table_name) - - results = {} - for key in keys: - original_key = key[len(self.prefix):] - - result = self.memory.retrieve(key, table_name=self.table_name) - - results[original_key] = result - - return results - - original_clear_working = WorkingMemory.clear - - def patched_clear_working(self): - """Patched clear method that uses the working memory table.""" - keys = self.memory.list_keys(self.prefix, table_name=self.table_name) - - success = True - for key in keys: - if not self.memory.delete(key, table_name=self.table_name): - success = False - - return success - - SupabaseMemory.store = patched_store - SupabaseMemory.retrieve = patched_retrieve - SupabaseMemory.delete = patched_delete - SupabaseMemory.list_keys = patched_list_keys - - WorkingMemory.store_intermediate_result = patched_store_intermediate - WorkingMemory.get_intermediate_result = patched_get_intermediate - WorkingMemory.list_intermediate_results = patched_list_intermediate - WorkingMemory.clear = patched_clear_working - - logger.info("Monkey patches applied successfully") - return True - -def test_fix(): - """Test if the fix works correctly.""" - logger.info("Testing the fix...") - - memory_config = get_memory_config() - memory_config["enabled"] = True - - base_memory = SupabaseMemory(config=memory_config) - session_id = "test-session" - working_memory = WorkingMemory(base_memory, session_id) - - test_key = "test_fix" - test_value = { - "test_data": "This is test data for the fix", - "timestamp": "2025-04-30T12:00:00" - } - - logger.info(f"Storing test value in working memory: {test_key}") - store_result = working_memory.store_intermediate_result(test_key, test_value) - - if not store_result: - logger.error("Failed to store test value") - return False - - logger.info(f"Retrieving test value from working memory: {test_key}") - retrieved_value = working_memory.get_intermediate_result(test_key) - - if retrieved_value != test_value: - logger.error(f"Retrieved value does not match test value") - logger.error(f"Expected: {test_value}") - logger.error(f"Got: {retrieved_value}") - return False - - logger.info("Verifying value was stored in the working memory table...") - - if base_memory.client and base_memory.initialized: - try: - working_key = f"working:{session_id}:{test_key}" - response = base_memory.client.table(working_memory.table_name).select("*").eq("key", working_key).execute() - - if not response.data or len(response.data) == 0: - logger.error("No data found in working memory table") - return False - - logger.info(f"Data found in working memory table: {len(response.data)} entries") - - logger.info("Cleaning up test data...") - working_memory.clear() - - return True - except Exception as e: - logger.error(f"Error verifying data storage: {str(e)}") - return False - - return False - -def main(): - """Main entry point for the fix script.""" - - if not create_working_memory_table(): - logger.error("Failed to create working memory table") - sys.exit(1) - - if not apply_fix(): - logger.error("Failed to apply monkey patch") - sys.exit(1) - - if test_fix(): - sys.exit(0) - else: - sys.exit(1) - -if __name__ == "__main__": - main() \ No newline at end of file diff --git a/memory/supabase_memory.py b/memory/supabase_memory.py deleted file mode 100644 index 946664386e6f94484f1759b80f89a10ce5189549..0000000000000000000000000000000000000000 --- a/memory/supabase_memory.py +++ /dev/null @@ -1,668 +0,0 @@ -""" -Supabase memory implementation for the GAIA agent. - -This module provides persistent memory storage and retrieval using Supabase. -It implements memory interfaces for storing intermediate results, caching -previous answers, and maintaining conversation context. -""" - -import json -import logging -import time -from typing import Dict, Any, List, Optional, Union, Tuple, cast -from datetime import datetime, timedelta - -try: - from supabase import create_client, Client -except ImportError: - class Client: - pass - -from agent.config import ( - get_memory_config, - get_model_config, - SUPABASE_URL, - SUPABASE_KEY, - SUPABASE_PROJECT_ID, - SUPABASE_PROJECT_NAME -) - -logger = logging.getLogger("gaia_agent.memory") - -class MemoryEntry: - """Class representing a memory entry.""" - - def __init__( - self, - key: str, - value: Any, - metadata: Optional[Dict[str, Any]] = None, - ttl: Optional[int] = None - ): - """ - Initialize a memory entry. - - Args: - key: The key for the memory entry - value: The value to store - metadata: Optional metadata for the entry - ttl: Optional time-to-live in seconds - """ - self.key = key - self.value = value - self.metadata = metadata or {} - self.created_at = datetime.now().isoformat() - - if ttl: - self.expires_at = (datetime.now() + timedelta(seconds=ttl)).isoformat() - else: - self.expires_at = None - - def to_dict(self) -> Dict[str, Any]: - """Convert the memory entry to a dictionary.""" - return { - "key": self.key, - "value": json.dumps(self.value) if not isinstance(self.value, str) else self.value, - "metadata": json.dumps(self.metadata), - "created_at": self.created_at, - "expires_at": self.expires_at - } - - @classmethod - def from_dict(cls, data: Dict[str, Any]) -> "MemoryEntry": - """Create a memory entry from a dictionary.""" - value = data.get("value") - if isinstance(value, str): - try: - value = json.loads(value) - except json.JSONDecodeError: - pass - - metadata = data.get("metadata") - if isinstance(metadata, str): - try: - metadata = json.loads(metadata) - except json.JSONDecodeError: - metadata = {} - - entry = cls( - key=data.get("key", ""), - value=value, - metadata=metadata - ) - - entry.created_at = data.get("created_at") - entry.expires_at = data.get("expires_at") - - return entry - -class SupabaseMemory: - """Memory implementation using Supabase for storage.""" - - def __init__(self, config: Optional[Dict[str, Any]] = None): - """ - Initialize the Supabase memory. - - Args: - config: Optional configuration dictionary - """ - self.config = config or get_memory_config() - - self.client = None - self.initialized = False - self.table_name = self.config.get("table_name", "gaia_memory") - - self.cache = {} - self.cache_size = self.config.get("cache_size", 100) - - self._initialize_client() - - def _initialize_client(self) -> None: - """Initialize the Supabase client.""" - if not self.config.get("enabled", True): - logger.info("Supabase memory is disabled") - return - - if not SUPABASE_URL or not SUPABASE_KEY: - logger.warning("Supabase URL or key not provided, memory will not be persistent") - return - - try: - self.client = create_client(SUPABASE_URL, SUPABASE_KEY) - - self._ensure_table_exists() - - specialized_tables = [ - f"{self.table_name}_conversations", - f"{self.table_name}_cache", - f"{self.table_name}_working" - ] - - for table in specialized_tables: - self._ensure_table_exists(table) - - self.initialized = True - logger.info(f"Supabase memory initialized with table {self.table_name}") - logger.info(f"Connected to Supabase project: {SUPABASE_PROJECT_NAME} ({SUPABASE_PROJECT_ID})") - except Exception as e: - logger.error(f"Error initializing Supabase client: {str(e)}") - self.initialized = False - - def _ensure_table_exists(self, table_name: Optional[str] = None) -> None: - """ - Ensure the memory table exists in Supabase. - - Args: - table_name: Optional table name to check/create (defaults to self.table_name) - """ - if not self.client: - return - - actual_table = table_name or self.table_name - - try: - self.client.table(actual_table).select("key").limit(1).execute() - except Exception as e: - logger.warning(f"Error checking table {actual_table}: {str(e)}") - logger.info(f"Creating table {actual_table}") - - sql = f""" - CREATE TABLE IF NOT EXISTS {actual_table} ( - id UUID PRIMARY KEY DEFAULT uuid_generate_v4(), - key TEXT NOT NULL UNIQUE, - value TEXT NOT NULL, - metadata JSONB, - created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), - expires_at TIMESTAMPTZ - ); - - CREATE INDEX IF NOT EXISTS idx_{actual_table}_key ON {actual_table}(key); - CREATE INDEX IF NOT EXISTS idx_{actual_table}_expires_at ON {actual_table}(expires_at); - """ - - try: - self.client.rpc("exec_sql", {"sql_query": sql}).execute() - logger.info(f"Successfully created table {actual_table}") - except Exception as e: - logger.error(f"Error creating table {actual_table}: {str(e)}") - logger.warning( - f"Table {actual_table} could not be created. " - "Please run setup_supabase.py to create the necessary tables." - ) - - def store( - self, - key: str, - value: Any, - metadata: Optional[Dict[str, Any]] = None, - ttl: Optional[int] = None, - table_name: Optional[str] = None - ) -> bool: - """ - Store a value in memory. - - Args: - key: The key for the memory entry - value: The value to store - metadata: Optional metadata for the entry - ttl: Optional time-to-live in seconds - table_name: Optional table name to use (defaults to self.table_name) - - Returns: - bool: True if the value was stored successfully, False otherwise - """ - actual_table = table_name or self.table_name - - entry = MemoryEntry( - key=key, - value=value, - metadata=metadata, - ttl=ttl or self.config.get("ttl") - ) - - self.cache[key] = entry - - if len(self.cache) > self.cache_size: - keys_to_remove = sorted( - self.cache.keys(), - key=lambda k: self.cache[k].created_at - )[:len(self.cache) - self.cache_size] - - for k in keys_to_remove: - del self.cache[k] - - if self.initialized and self.client: - try: - entry_dict = entry.to_dict() - - result = self.client.table(actual_table).upsert(entry_dict).execute() - - return True - except Exception as e: - logger.error(f"Error storing memory entry {key} in table {actual_table}: {str(e)}") - return False - - return True - - def retrieve( - self, - key: str, - default: Any = None, - table_name: Optional[str] = None - ) -> Any: - """ - Retrieve a value from memory. - - Args: - key: The key for the memory entry - default: The default value to return if the key is not found - table_name: Optional table name to use (defaults to self.table_name) - - Returns: - The value associated with the key, or the default value if not found - """ - actual_table = table_name or self.table_name - - if key in self.cache: - entry = self.cache[key] - - if entry.expires_at: - expires_at = datetime.fromisoformat(entry.expires_at) - if expires_at < datetime.now(): - del self.cache[key] - return default - - return entry.value - - if self.initialized and self.client: - try: - result = self.client.table(actual_table).select("*").eq("key", key).execute() - - if result.data and len(result.data) > 0: - entry_data = result.data[0] - - entry = MemoryEntry.from_dict(entry_data) - - if entry.expires_at: - expires_at = datetime.fromisoformat(entry.expires_at) - if expires_at < datetime.now(): - self.delete(key, table_name=actual_table) - return default - - self.cache[key] = entry - - return entry.value - except Exception as e: - logger.error(f"Error retrieving memory entry {key} from table {actual_table}: {str(e)}") - - return default - - def delete(self, key: str, table_name: Optional[str] = None) -> bool: - """ - Delete a value from memory. - - Args: - key: The key for the memory entry - table_name: Optional table name to use (defaults to self.table_name) - - Returns: - bool: True if the value was deleted successfully, False otherwise - """ - actual_table = table_name or self.table_name - - if key in self.cache: - del self.cache[key] - - if self.initialized and self.client: - try: - result = self.client.table(actual_table).delete().eq("key", key).execute() - - return True - except Exception as e: - logger.error(f"Error deleting memory entry {key} from table {actual_table}: {str(e)}") - return False - - return True - - def clear(self, table_name: Optional[str] = None) -> bool: - """ - Clear all values from memory. - - Args: - table_name: Optional table name to use (defaults to self.table_name) - - Returns: - bool: True if the memory was cleared successfully, False otherwise - """ - actual_table = table_name or self.table_name - - self.cache = {} - - # Clear Supabase if initialized - if self.initialized and self.client: - try: - result = self.client.table(actual_table).delete().neq("key", "").execute() - - return True - except Exception as e: - logger.error(f"Error clearing memory in table {actual_table}: {str(e)}") - return False - - return True - - def list_keys(self, prefix: Optional[str] = None, table_name: Optional[str] = None) -> List[str]: - """ - List all keys in memory. - - Args: - prefix: Optional prefix to filter keys - table_name: Optional table name to use (defaults to self.table_name) - - Returns: - List of keys - """ - actual_table = table_name or self.table_name - - keys = [] - - for key in self.cache.keys(): - if prefix is None or key.startswith(prefix): - keys.append(key) - - if self.initialized and self.client: - try: - query = self.client.table(actual_table).select("key") - - if prefix: - query = query.like("key", f"{prefix}%") - - result = query.execute() - - if result.data: - for entry in result.data: - key = entry.get("key") - if key and key not in keys: - keys.append(key) - except Exception as e: - logger.error(f"Error listing memory keys from table {actual_table}: {str(e)}") - - return keys - -class ConversationMemory: - """Memory implementation for storing conversation history.""" - - def __init__(self, memory: SupabaseMemory, conversation_id: str): - """ - Initialize the conversation memory. - - Args: - memory: The underlying memory implementation - conversation_id: The ID of the conversation - """ - self.memory = memory - self.conversation_id = conversation_id - self.prefix = f"conversation:{conversation_id}:" - self.table_name = f"{self.memory.table_name}_conversations" - - def add_message( - self, - role: str, - content: str, - metadata: Optional[Dict[str, Any]] = None - ) -> bool: - """ - Add a message to the conversation history. - - Args: - role: The role of the message sender (user, assistant, system) - content: The content of the message - metadata: Optional metadata for the message - - Returns: - bool: True if the message was added successfully, False otherwise - """ - timestamp = datetime.now().isoformat() - - key = f"{self.prefix}message:{timestamp}" - - message = { - "role": role, - "content": content, - "timestamp": timestamp - } - - if metadata: - message["metadata"] = metadata - - return self.memory.store(key, message, table_name=self.table_name) - - def get_messages( - self, - limit: Optional[int] = None, - before: Optional[str] = None, - after: Optional[str] = None - ) -> List[Dict[str, Any]]: - """ - Get messages from the conversation history. - - Args: - limit: Optional maximum number of messages to return - before: Optional timestamp to get messages before - after: Optional timestamp to get messages after - - Returns: - List of messages - """ - keys = self.memory.list_keys(self.prefix + "message:", table_name=self.table_name) - - messages = [] - for key in keys: - message = self.memory.retrieve(key, table_name=self.table_name) - if message: - messages.append(message) - - messages.sort(key=lambda m: m.get("timestamp", "")) - - if before: - messages = [m for m in messages if m.get("timestamp", "") < before] - - if after: - messages = [m for m in messages if m.get("timestamp", "") > after] - - if limit and len(messages) > limit: - messages = messages[-limit:] - - return messages - - def clear(self) -> bool: - """ - Clear the conversation history. - - Returns: - bool: True if the history was cleared successfully, False otherwise - """ - keys = self.memory.list_keys(self.prefix, table_name=self.table_name) - - success = True - for key in keys: - if not self.memory.delete(key, table_name=self.table_name): - success = False - - return success - -class ResultCache: - """Memory implementation for caching tool results and answers.""" - - def __init__(self, memory: SupabaseMemory): - """ - Initialize the result cache. - - Args: - memory: The underlying memory implementation - """ - self.memory = memory - self.prefix = "cache:" - self.table_name = f"{self.memory.table_name}_cache" - - def cache_result( - self, - key: str, - result: Any, - metadata: Optional[Dict[str, Any]] = None, - ttl: Optional[int] = None - ) -> bool: - """ - Cache a result. - - Args: - key: The key for the result - result: The result to cache - metadata: Optional metadata for the result - ttl: Optional time-to-live in seconds - - Returns: - bool: True if the result was cached successfully, False otherwise - """ - cache_key = f"{self.prefix}{key}" - - return self.memory.store(cache_key, result, metadata, ttl, table_name=self.table_name) - - def get_result( - self, - key: str, - default: Any = None - ) -> Any: - """ - Get a cached result. - - Args: - key: The key for the result - default: The default value to return if the key is not found - - Returns: - The cached result, or the default value if not found - """ - cache_key = f"{self.prefix}{key}" - - return self.memory.retrieve(cache_key, default, table_name=self.table_name) - - def invalidate(self, key: str) -> bool: - """ - Invalidate a cached result. - - Args: - key: The key for the result - - Returns: - bool: True if the result was invalidated successfully, False otherwise - """ - cache_key = f"{self.prefix}{key}" - - return self.memory.delete(cache_key, table_name=self.table_name) - - def clear(self) -> bool: - """ - Clear all cached results. - - Returns: - bool: True if the cache was cleared successfully, False otherwise - """ - keys = self.memory.list_keys(self.prefix, table_name=self.table_name) - - success = True - for key in keys: - if not self.memory.delete(key, table_name=self.table_name): - success = False - - return success - -class WorkingMemory: - """Memory implementation for storing intermediate results during agent execution.""" - - def __init__(self, memory: SupabaseMemory, session_id: str): - """ - Initialize the working memory. - - Args: - memory: The underlying memory implementation - session_id: The ID of the session - """ - self.memory = memory - self.session_id = session_id - self.prefix = f"working:{session_id}:" - self.table_name = f"{self.memory.table_name}_working" - - def store_intermediate_result( - self, - key: str, - result: Any, - metadata: Optional[Dict[str, Any]] = None - ) -> bool: - """ - Store an intermediate result. - - Args: - key: The key for the result - result: The result to store - metadata: Optional metadata for the result - - Returns: - bool: True if the result was stored successfully, False otherwise - """ - working_key = f"{self.prefix}{key}" - - return self.memory.store(working_key, result, metadata, table_name=self.table_name) - - def get_intermediate_result( - self, - key: str, - default: Any = None - ) -> Any: - """ - Get an intermediate result. - - Args: - key: The key for the result - default: The default value to return if the key is not found - - Returns: - The intermediate result, or the default value if not found - """ - working_key = f"{self.prefix}{key}" - - return self.memory.retrieve(working_key, default, table_name=self.table_name) - - def list_intermediate_results(self) -> Dict[str, Any]: - """ - List all intermediate results. - - Returns: - Dictionary of intermediate results - """ - keys = self.memory.list_keys(self.prefix, table_name=self.table_name) - - results = {} - for key in keys: - original_key = key[len(self.prefix):] - - result = self.memory.retrieve(key, table_name=self.table_name) - - results[original_key] = result - - return results - - def clear(self) -> bool: - """ - Clear all intermediate results. - - Returns: - bool: True if the working memory was cleared successfully, False otherwise - """ - keys = self.memory.list_keys(self.prefix, table_name=self.table_name) - - success = True - for key in keys: - if not self.memory.delete(key, table_name=self.table_name): - success = False - - return success \ No newline at end of file diff --git a/memory/supabase_memory_fix.py b/memory/supabase_memory_fix.py deleted file mode 100644 index 750a7d053c88f3e16ea32659952502ac2ac51584..0000000000000000000000000000000000000000 --- a/memory/supabase_memory_fix.py +++ /dev/null @@ -1,208 +0,0 @@ -""" -Comprehensive fix for Supabase memory integration in GAIA Agent - -This module fixes issues in the Supabase memory implementation by: -1. Ensuring all required tables exist -2. Patching the memory methods to use the correct tables -3. Fixing try-except blocks with proper error handling - -Usage: - from memory.supabase_memory_fix import apply_fix - apply_fix() -""" - -import os -import sys -import json -import logging -from typing import Dict, Any, Optional - -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(name)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("supabase_memory_fix") - -sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) - -try: - from memory.supabase_memory import ( - SupabaseMemory, - ConversationMemory, - ResultCache, - WorkingMemory - ) - from agent.config import get_memory_config -except ImportError as e: - logger.error(f"Error importing required components: {str(e)}") - sys.exit(1) - -def ensure_tables_exist(): - """Ensure all required tables exist in Supabase.""" - logger.info("Ensuring all required tables exist...") - - try: - memory_config = get_memory_config() - memory_config["enabled"] = True - - base_memory = SupabaseMemory(config=memory_config) - - if not base_memory.client or not base_memory.initialized: - logger.error("Supabase client not initialized") - return False - - expected_tables = { - "base": base_memory.table_name, - "conversation": f"{base_memory.table_name}_conversations", - "cache": f"{base_memory.table_name}_cache", - "working": f"{base_memory.table_name}_working" - } - - for table_type, table_name in expected_tables.items(): - try: - base_memory.client.table(table_name).select("*").limit(1).execute() - logger.info(f"✅ Table {table_name} exists") - except Exception as e: - logger.warning(f"Creating table {table_name}...") - - sql = f""" - CREATE TABLE IF NOT EXISTS {table_name} ( - id UUID PRIMARY KEY DEFAULT uuid_generate_v4(), - key TEXT NOT NULL, - value TEXT, - metadata JSONB, - created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(), - updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(), - expires_at TIMESTAMP WITH TIME ZONE - ); - CREATE INDEX IF NOT EXISTS {table_name}_key_idx ON {table_name}(key); - """ - - try: - base_memory.client.query(sql).execute() - logger.info(f"✅ Table {table_name} created successfully") - except Exception as e: - logger.error(f"Error creating table {table_name}: {str(e)}") - return False - - return True - except Exception as e: - logger.error(f"Error ensuring tables exist: {str(e)}") - return False - -def patch_memory_methods(): - """Patch the memory methods to use the correct tables.""" - logger.info("Patching memory methods...") - - try: - # Store original methods for reference - cache_result_original = ResultCache.cache_result - - # Patch the ResultCache.cache_result method - def patched_cache_result(self, key, result, metadata=None, ttl=None): - """Patched cache_result method that properly handles the cache table.""" - cache_key = f"{self.prefix}{key}" - - return self.memory.store(cache_key, result, metadata, ttl, table_name=self.table_name) - - ResultCache.cache_result = patched_cache_result - - # Store original methods for reference - get_intermediate_original = WorkingMemory.get_intermediate_result - - # Patch the WorkingMemory.get_intermediate_result method - def patched_get_intermediate(self, key, default=None): - """Patched get_intermediate_result method that properly handles the working table.""" - working_key = f"{self.prefix}{key}" - - return self.memory.retrieve(working_key, default, table_name=self.table_name) - - WorkingMemory.get_intermediate_result = patched_get_intermediate - - logger.info("✅ Memory methods patched successfully") - return True - except Exception as e: - logger.error(f"Error patching memory methods: {str(e)}") - return False - -def apply_fix(): - """Apply all fixes to the Supabase memory implementation.""" - logger.info("Applying comprehensive fix to Supabase memory...") - - if not ensure_tables_exist(): - logger.error("Failed to ensure tables exist") - return False - - if not patch_memory_methods(): - logger.error("Failed to patch memory methods") - return False - - logger.info("✅ All fixes applied successfully") - return True - -def test_fix(): - """Test the fix by exercising the memory APIs.""" - logger.info("Testing Supabase memory fix...") - - try: - memory_config = get_memory_config() - memory_config["enabled"] = True - - base_memory = SupabaseMemory(config=memory_config) - session_id = "test-session" - - # Test working memory - working_memory = WorkingMemory(base_memory, session_id) - working_memory.store_intermediate_result("test_key", "test_value") - result = working_memory.get_intermediate_result("test_key") - - if result != "test_value": - logger.error(f"Working memory test failed: {result}") - return False - - logger.info("✅ Working memory test passed") - - # Test result cache - cache = ResultCache(base_memory) - cache.cache_result("test_query", "test_answer") - result = cache.get_result("test_query") - - if result != "test_answer": - logger.error(f"Result cache test failed: {result}") - return False - - logger.info("✅ Result cache test passed") - - # Test conversation memory - conversation = ConversationMemory(base_memory, "test-conversation") - conversation.add_message("user", "test message") - messages = conversation.get_messages() - - if not messages or messages[0].get("content") != "test message": - logger.error(f"Conversation memory test failed: {messages}") - return False - - logger.info("✅ Conversation memory test passed") - - # Clean up test data - working_memory.clear() - cache.clear() - conversation.clear() - - logger.info("✅ All tests passed") - return True - except Exception as e: - logger.error(f"Error testing fix: {str(e)}") - return False - -if __name__ == "__main__": - if apply_fix(): - if test_fix(): - logger.info("Supabase memory fix applied and tested successfully") - sys.exit(0) - else: - logger.error("Supabase memory fix test failed") - sys.exit(1) - else: - logger.error("Failed to apply Supabase memory fix") - sys.exit(1) \ No newline at end of file diff --git a/memory/supabase_memory_patch.py b/memory/supabase_memory_patch.py deleted file mode 100644 index 2fc9de22808605336b3d294f686a2b9d5b246169..0000000000000000000000000000000000000000 --- a/memory/supabase_memory_patch.py +++ /dev/null @@ -1,137 +0,0 @@ -""" -Supabase Memory Patch - -This script patches the Supabase memory implementation to ensure all tables are properly utilized: -- gaia_memory: Main memory table -- gaia_memory_conversations: Conversation history table -- gaia_memory_cache: Result cache table -- gaia_memory_working: Working memory table - -Usage: - python supabase_memory_patch.py -""" - -import os -import sys -import logging -from typing import Dict, Any, Optional - -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(name)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("supabase_memory_patch") - -sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) - -try: - from memory.supabase_memory import ( - SupabaseMemory, - ConversationMemory, - ResultCache, - WorkingMemory - ) - from agent.config import get_memory_config -except ImportError as e: - logger.error(f"Error importing required components: {str(e)}") - sys.exit(1) - -def patch_supabase_memory() -> bool: - """ - Patch the Supabase memory implementation to ensure all tables are properly utilized. - - Returns: - bool: True if the patch was successful, False otherwise - """ - logger.info("Patching Supabase memory implementation...") - - try: - memory_config = get_memory_config() - memory_config["enabled"] = True - - base_memory = SupabaseMemory(config=memory_config) - - if not base_memory.client or not base_memory.initialized: - logger.error("Supabase client not initialized") - return False - - expected_tables = { - "base": "gaia_memory", - "conversation": "gaia_memory_conversations", - "cache": "gaia_memory_cache", - "working": "gaia_memory_working" - } - - existing_tables = {} - for table_type, table_name in expected_tables.items(): - try: - query_result = base_memory.client.table(table_name).select("*").limit(1).execute() - existing_tables[table_type] = True - logger.info(f"✅ Table {table_name} exists") - except Exception as e: - existing_tables[table_type] = False - logger.warning(f"❌ Table {table_name} does not exist: {str(e)}") - - for table_type, exists in existing_tables.items(): - if not exists: - table_name = expected_tables[table_type] - logger.info(f"Creating table {table_name}...") - - sql = f""" - CREATE TABLE IF NOT EXISTS {table_name} ( - id UUID PRIMARY KEY DEFAULT uuid_generate_v4(), - key TEXT NOT NULL, - value TEXT, - metadata JSONB, - created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(), - updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW() - ); - CREATE INDEX IF NOT EXISTS {table_name}_key_idx ON {table_name} (key); - """ - - try: - base_memory.client.query(sql).execute() - logger.info(f"✅ Table {table_name} created successfully") - except Exception as e: - logger.error(f"Error creating table {table_name}: {str(e)}") - return False - - conversation = ConversationMemory(base_memory, "test-conversation") - if conversation.table_name != expected_tables["conversation"]: - logger.info(f"Patching ConversationMemory to use {expected_tables['conversation']} table...") - if conversation.table_name != expected_tables["conversation"]: - logger.warning(f"❌ ConversationMemory is using {conversation.table_name} instead of {expected_tables['conversation']}") - return False - - cache = ResultCache(base_memory) - if cache.table_name != expected_tables["cache"]: - logger.info(f"Patching ResultCache to use {expected_tables['cache']} table...") - if cache.table_name != expected_tables["cache"]: - logger.warning(f"❌ ResultCache is using {cache.table_name} instead of {expected_tables['cache']}") - return False - - working = WorkingMemory(base_memory, "test-session") - if working.table_name != expected_tables["working"]: - logger.info(f"Patching WorkingMemory to use {expected_tables['working']} table...") - if working.table_name != expected_tables["working"]: - logger.warning(f"❌ WorkingMemory is using {working.table_name} instead of {expected_tables['working']}") - return False - - logger.info("✅ Supabase memory implementation patched successfully") - return True - - except Exception as e: - logger.error(f"Error patching Supabase memory implementation: {str(e)}") - return False - -def main(): - """Main entry point for the patch script.""" - success = patch_supabase_memory() - - if success: - sys.exit(0) - else: - sys.exit(1) - -if __name__ == "__main__": - main() \ No newline at end of file diff --git a/patches/integrate_tool_registry.patch b/patches/integrate_tool_registry.patch deleted file mode 100644 index 09ad17e06f6132e6698f9e3c1cd02ebcb3a4f71c..0000000000000000000000000000000000000000 --- a/patches/integrate_tool_registry.patch +++ /dev/null @@ -1,63 +0,0 @@ -""" -Patch to integrate the tool registry with the LangGraph workflow. - -This patch shows how to modify the agent/graph.py file to use real tools -from the tool registry instead of simulating tool execution. - -To apply this patch, you can use: -1. Apply it manually by following the changes below -2. Use a patch tool: patch -p0 < patches/integrate_tool_registry.patch -""" - ---- agent/graph.py 2025-04-30 17:00:00.000000000 +0200 -+++ agent/graph.py.new 2025-04-30 17:30:00.000000000 +0200 -@@ -13,6 +13,9 @@ - import logging - from typing import Dict, Any, List, Tuple, Optional, Annotated, TypedDict, Literal, Union, cast - -+# Import the tool registry -+from agent.tool_registry import create_default_registry, ToolRegistry -+ - from langchain.prompts import PromptTemplate - from langchain_core.messages import HumanMessage, AIMessage - from langchain_core.runnables import RunnableConfig -@@ -271,6 +274,10 @@ - """ - Execute the selected tool based on the current plan step. - -+ This implementation uses real tools from the tool registry instead of -+ simulating tool execution. It looks up the tool in the registry based -+ on the tool name and executes it with the provided input. -+ - Args: - state: The current agent state - -@@ -290,10 +297,25 @@ - logger.debug(f"Executing tool {tool_name} for step {current_step['step_number']}") - - # In a real implementation, this would use the actual tool registry -- # For now, we'll simulate tool execution - try: -- # Simulate tool execution -- result = f"Simulated result for {tool_name} with input {tool_input}" -+ # Get the tool registry (create it if it doesn't exist) -+ # In a production implementation, this would be passed in or stored in state -+ tool_registry = getattr(execute_tool, 'tool_registry', None) -+ if tool_registry is None: -+ tool_registry = create_default_registry() -+ # Store the registry for future use -+ execute_tool.tool_registry = tool_registry -+ -+ # Get the tool from the registry -+ tool = tool_registry.get_tool(tool_name) -+ if not tool: -+ raise Exception(f"Tool {tool_name} not found in registry") -+ -+ # Execute the tool -+ if tool_name in ["duckduckgo_search", "serper_search", "perplexity_search"]: -+ result = tool_registry.execute_tool(tool_name, query=tool_input.get("query", "")) -+ else: -+ result = tool_registry.execute_tool(tool_name, **tool_input) - - # Create a tool result - tool_result = { \ No newline at end of file diff --git a/prepare_for_hf_deployment.py b/prepare_for_hf_deployment.py deleted file mode 100644 index 14dd5138fe9c620734e86286e0ae2a341b15cace..0000000000000000000000000000000000000000 --- a/prepare_for_hf_deployment.py +++ /dev/null @@ -1,164 +0,0 @@ -#!/usr/bin/env python -""" -Hugging Face Space Deployment Preparation Script - -This script prepares the GAIA project for deployment to Hugging Face Space -by creating necessary configuration files and documentation. - -Usage: - python prepare_for_hf_deployment.py - -Author: DevOps Team -Date: April 30, 2025 -""" - -import os -import json -import logging -from pathlib import Path - -# Configure logging -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("hf_deploy_prep") - -def create_hf_readme(): - """Create a README.md file for Hugging Face Space""" - readme_content = """# GAIA: Generative AI Assistant - -A versatile AI assistant with memory, search capabilities, and multi-API integration. - -## Features - -- OpenAI integration with GPT-4 and fallback models -- Memory persistence with Supabase -- Web search capabilities via Serper and DuckDuckGo -- Agent-based architecture with tool registry -- Multi-turn conversation support - -## Usage - -1. Enter your query in the chat interface -2. GAIA will process your request, using tools as needed -3. Responses are retained in memory for context in future interactions - -## Credits - -Developed as part of the Hugging Face Agents Course - """ - - with open("HF_README.md", "w") as f: - f.write(readme_content) - - logger.info("Created HF_README.md for Space display") - return "HF_README.md" - -def create_space_config(): - """Create Space configuration file for Hugging Face""" - config = { - "title": "GAIA - Generative AI Assistant", - "emoji": "🧠", - "colorFrom": "blue", - "colorTo": "purple", - "sdk": "gradio", - "sdk_version": "3.36.1", - "python_version": "3.9", - "app_file": "app.py", - "pinned": False, - "license": "mit" - } - - with open("space_config.json", "w") as f: - json.dump(config, f, indent=2) - - logger.info("Created space_config.json for Hugging Face Space") - return "space_config.json" - -def backup_existing_readme(): - """Backup existing README.md if it exists""" - if os.path.exists("README.md"): - with open("README.md", "r") as f: - content = f.read() - - with open("README_original.md", "w") as f: - f.write(content) - - logger.info("Backed up existing README.md to README_original.md") - return True - - return False - -def check_requirements_file(): - """Check if requirements.txt exists and contains necessary packages""" - if not os.path.exists("requirements.txt"): - logger.warning("requirements.txt not found! Creating basic version...") - - requirements = [ - "openai>=1.0.0", - "langchain>=0.0.267", - "supabase>=1.0.3", - "gradio>=3.36.1", - "python-dotenv>=1.0.0", - "requests>=2.28.2", - "duckduckgo-search>=3.8.0" - ] - - with open("requirements.txt", "w") as f: - f.write("\n".join(requirements)) - - logger.info("Created basic requirements.txt") - return "requirements.txt" - - return "requirements.txt (existing)" - -def main(): - """Prepare project for Hugging Face Space deployment""" - logger.info("Starting Hugging Face deployment preparation...") - - # Check existence of key directories - directories = ["agent", "memory", "tools", "utils"] - for directory in directories: - if not os.path.isdir(directory): - logger.error(f"Required directory not found: {directory}") - return False - - # Check existence of key files - key_files = ["app.py", "agent/config.py", "agent/agent.py"] - for file_path in key_files: - if not os.path.isfile(file_path): - logger.error(f"Required file not found: {file_path}") - return False - - # Create necessary files for HF Space - files_created = [] - - # Create HF README - hf_readme = create_hf_readme() - files_created.append(hf_readme) - - # Create Space config - space_config = create_space_config() - files_created.append(space_config) - - # Check requirements file - requirements = check_requirements_file() - files_created.append(requirements) - - # Backup existing README - backup_existing_readme() - - # Success! - logger.info("Hugging Face deployment preparation complete!") - logger.info(f"Files created/checked: {', '.join(files_created)}") - logger.info("To deploy to Hugging Face Space:") - logger.info("1. Run: python deploy_to_hf.py") - logger.info("2. Configure secrets in Hugging Face Space settings") - logger.info("3. Verify the deployment by accessing the Space URL") - - return True - -if __name__ == "__main__": - success = main() - exit(0 if success else 1) \ No newline at end of file diff --git a/production_cleanup.py b/production_cleanup.py deleted file mode 100644 index b42c0f77ae3e8313a26bc0049b554a4c0d08b08d..0000000000000000000000000000000000000000 --- a/production_cleanup.py +++ /dev/null @@ -1,286 +0,0 @@ -""" -Production cleanup script for the GAIA agent project. - -This script performs the following cleanup tasks: -1. Removes temporary JSON result files -2. Removes redundant test files -3. Removes batch files and other development scripts -4. Checks for and reports any hardcoded credentials -5. Checks for debug print statements and commented-out code - -Usage: - python production_cleanup.py [--dry-run] -""" - -import os -import re -import sys -import glob -import argparse -import logging -from pathlib import Path -from typing import List, Dict, Set, Tuple - -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(levelname)s - %(message)s", - handlers=[ - logging.FileHandler("cleanup_results.txt", mode='w'), - logging.StreamHandler() - ] -) -logger = logging.getLogger("production_cleanup") - -TEMP_FILES_PATTERNS = [ - "gaia_assessment_results_*.json", - "gaia_memory_test_results_*.json", - "*.bat", - "check_*.py", - "direct_*.py", - "minimal_*.py", - "simple_*.py", - "supabase_*_test.py", - "test_serper_direct.py", - "verify_*.py", - "run_gaia_*.py", - "setup_*.py" -] - -KEEP_FILES = [ - "verify_gaia_assessment.py", # Keep this file as it might be important - "requirements.txt", - "README.md", - "app.py" -] - -CODE_DIRS = [ - "agent", - "memory", - "tools", - "utils" -] - -DEBUG_PATTERNS = [ - r'print\s*\(', - r'logger\.debug\s*\(', - r'console\.log\s*\(', - r'# DEBUG:', - r'# FIXME:', - r'# TODO:', -] - -COMMENTED_CODE_PATTERNS = [ - r'^\s*# [^#\s].*\(', - r'^\s*# [^#\s].*=', - r'^\s*# [^#\s].*import', - r'^\s*# [^#\s].*return', - r'^\s*# [^#\s].*if ', - r'^\s*# [^#\s].*for ', - r'^\s*# [^#\s].*while ', -] - -CREDENTIAL_PATTERNS = [ - r'api_key\s*=\s*["\'][^"\']+["\']', - r'password\s*=\s*["\'][^"\']+["\']', - r'secret\s*=\s*["\'][^"\']+["\']', - r'token\s*=\s*["\'][^"\']+["\']', - r'auth\s*=\s*["\'][^"\']+["\']', -] - -def find_files_to_remove() -> List[str]: - """Find temporary files to remove based on patterns.""" - files_to_remove = [] - - for pattern in TEMP_FILES_PATTERNS: - matching_files = glob.glob(pattern) - for file in matching_files: - if file not in KEEP_FILES and os.path.isfile(file): - files_to_remove.append(file) - - return files_to_remove - -def remove_files(files: List[str], dry_run: bool = False) -> None: - """Remove the specified files.""" - for file in files: - if dry_run: - logger.info(f"Would remove: {file}") - else: - try: - os.remove(file) - logger.info(f"Removed: {file}") - except Exception as e: - logger.error(f"Error removing {file}: {str(e)}") - -def find_duplicate_test_files() -> Dict[str, List[str]]: - """Find duplicate test files based on naming patterns.""" - test_files = {} - - all_test_files = glob.glob("tests/test_*.py") - - for file in all_test_files: - base_name = os.path.basename(file) - - core_name = re.sub(r'_final|_simple|_comprehensive|_production|_\d+', '', base_name) - - if core_name not in test_files: - test_files[core_name] = [] - - test_files[core_name].append(file) - - duplicate_groups = {k: v for k, v in test_files.items() if len(v) > 1} - - return duplicate_groups - -def check_for_debug_and_comments() -> Dict[str, List[Tuple[int, str, str]]]: - """Check for debug statements and commented-out code.""" - results = {} - - debug_patterns = [re.compile(pattern) for pattern in DEBUG_PATTERNS] - commented_code_patterns = [re.compile(pattern) for pattern in COMMENTED_CODE_PATTERNS] - - for directory in CODE_DIRS: - if not os.path.isdir(directory): - continue - - for root, _, files in os.walk(directory): - for file in files: - if not file.endswith('.py'): - continue - - file_path = os.path.join(root, file) - file_issues = [] - - with open(file_path, 'r', encoding='utf-8', errors='ignore') as f: - lines = f.readlines() - - for i, line in enumerate(lines): - for pattern in debug_patterns: - if pattern.search(line): - file_issues.append((i+1, line.strip(), "Debug statement")) - break - - for pattern in commented_code_patterns: - if pattern.search(line): - file_issues.append((i+1, line.strip(), "Commented code")) - break - - if file_issues: - results[file_path] = file_issues - - return results - -def check_for_hardcoded_credentials() -> Dict[str, List[Tuple[int, str]]]: - """Check for potential hardcoded credentials.""" - results = {} - - credential_patterns = [re.compile(pattern) for pattern in CREDENTIAL_PATTERNS] - - for root, _, files in os.walk('.'): - for file in files: - if not file.endswith('.py'): - continue - - file_path = os.path.join(root, file) - file_issues = [] - - with open(file_path, 'r', encoding='utf-8', errors='ignore') as f: - lines = f.readlines() - - for i, line in enumerate(lines): - if "os.getenv" in line or "os.environ.get" in line: - continue - - for pattern in credential_patterns: - if pattern.search(line): - if re.search(r'["\'][^"\']*(?:placeholder|PLACEHOLDER|your_|YOUR_)[^"\']*["\']', line): - continue - file_issues.append((i+1, line.strip())) - break - - if file_issues: - results[file_path] = file_issues - - return results - -def main(): - """Main function.""" - parser = argparse.ArgumentParser(description="Production cleanup script") - parser.add_argument("--dry-run", action="store_true", help="Show what would be done without making changes") - args = parser.parse_args() - - if args.dry_run: - logger.info("Running in dry-run mode. No changes will be made.") - - logger.info("Finding temporary files to remove...") - temp_files = find_files_to_remove() - - if temp_files: - logger.info(f"Found {len(temp_files)} temporary files to remove") - remove_files(temp_files, args.dry_run) - else: - logger.info("No temporary files found to remove") - - logger.info("Finding duplicate test files...") - duplicate_tests = find_duplicate_test_files() - - if duplicate_tests: - logger.info(f"Found {sum(len(v) for v in duplicate_tests.values()) - len(duplicate_tests)} duplicate test files") - - for core_name, files in duplicate_tests.items(): - logger.info(f"Duplicate tests for {core_name}:") - for file in files: - logger.info(f" - {file}") - - files_to_keep = [f for f in files if '_final' in f or '_comprehensive' in f or '_production' in f] - - if not files_to_keep: - files_to_keep = [min(files, key=len)] - - files_to_remove = [f for f in files if f not in files_to_keep] - - if files_to_remove: - logger.info(f" Keeping: {files_to_keep[0]}") - logger.info(f" Removing: {', '.join(files_to_remove)}") - - if not args.dry_run: - remove_files(files_to_remove) - else: - logger.info("No duplicate test files found") - - logger.info("Checking for debug statements and commented-out code...") - debug_results = check_for_debug_and_comments() - - if debug_results: - logger.info(f"Found {sum(len(v) for v in debug_results.values())} instances of debug statements or commented-out code") - - for file_path, issues in debug_results.items(): - logger.info(f"Issues in {file_path}:") - for line_num, line, issue_type in issues: - logger.info(f" Line {line_num} ({issue_type}): {line}") - else: - logger.info("No debug statements or commented-out code found") - - logger.info("Checking for potential hardcoded credentials...") - credential_results = check_for_hardcoded_credentials() - - if credential_results: - logger.info(f"Found {sum(len(v) for v in credential_results.values())} potential hardcoded credentials") - - for file_path, issues in credential_results.items(): - logger.info(f"Potential credentials in {file_path}:") - for line_num, line in issues: - logger.info(f" Line {line_num}: {line}") - else: - logger.info("No potential hardcoded credentials found") - - logger.info("\nCleanup Summary:") - logger.info(f"- Temporary files to remove: {len(temp_files)}") - logger.info(f"- Duplicate test files to remove: {sum(len(v) - 1 for v in duplicate_tests.values()) if duplicate_tests else 0}") - logger.info(f"- Files with debug statements or commented code: {len(debug_results)}") - logger.info(f"- Files with potential hardcoded credentials: {len(credential_results)}") - - if args.dry_run: - logger.info("\nThis was a dry run. Run without --dry-run to apply changes.") - -if __name__ == "__main__": - main() diff --git a/production_readiness_check.py b/production_readiness_check.py deleted file mode 100644 index cee896433246ec0ef6318abffb978a1b6a4809d4..0000000000000000000000000000000000000000 --- a/production_readiness_check.py +++ /dev/null @@ -1,791 +0,0 @@ -#!/usr/bin/env python -""" -Production Readiness Check for GAIA Agent -Performs comprehensive verification of all components needed for Hugging Face Space deployment -""" -import os -import sys -import json -import time -import logging -import importlib -import traceback -from typing import Dict, List, Any, Optional, Tuple -from pathlib import Path - -# Configure logging -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(levelname)s - %(message)s", - handlers=[ - logging.StreamHandler(), - logging.FileHandler("production_readiness.log") - ] -) -logger = logging.getLogger("production_readiness") - -class ProductionReadinessChecker: - """Comprehensive production readiness checker for GAIA Agent""" - - def __init__(self): - self.results = {} - self.components = {} - self.report = [] - self.start_time = time.time() - self.environment_ready = False - self.imports_ready = False - self.connections_ready = False - self.agent_ready = False - self.tools_ready = False - self.memory_ready = False - self.deployment_ready = False - - def log_section(self, title): - """Log a section header""" - logger.info("\n" + "=" * 80) - logger.info(f" {title} ".center(80, "=")) - logger.info("=" * 80 + "\n") - - def log_status(self, component, status, details=""): - """Log component status""" - emoji = "✅" if status else "❌" if status is False else "⚠️" - logger.info(f"{emoji} {component}: {details}") - - def check_environment(self) -> bool: - """Check environment variables and system configuration""" - self.log_section("Environment Verification") - - # Check Python version - py_version = sys.version_info - py_version_str = f"{py_version.major}.{py_version.minor}.{py_version.micro}" - - if py_version.major >= 3 and py_version.minor >= 8: - self.log_status("Python version", True, f"{py_version_str} (compatible)") - self.components["python_version"] = True - else: - self.log_status("Python version", False, f"{py_version_str} (requires >= 3.8)") - self.components["python_version"] = False - self.report.append("❌ Python version below 3.8. Upgrade to Python 3.8 or higher.") - - # Check required environment variables - required_vars = ["OPENAI_API_KEY", "SUPABASE_URL", "SUPABASE_KEY"] - optional_vars = ["SERPER_API_KEY", "SPACE_ID", "SPACE_HOST", "OAUTH_CLIENT_ID"] - - env_vars_status = {} - - # Check required variables - all_required_vars_set = True - for var in required_vars: - value = os.environ.get(var) - if value: - self.log_status(f"Environment var: {var}", True, "Set") - env_vars_status[var] = True - else: - self.log_status(f"Environment var: {var}", False, "NOT SET (Required)") - env_vars_status[var] = False - all_required_vars_set = False - self.report.append(f"❌ Required environment variable {var} is not set") - - # Check optional variables - for var in optional_vars: - value = os.environ.get(var) - if value: - self.log_status(f"Environment var: {var}", True, "Set") - env_vars_status[var] = True - else: - self.log_status(f"Environment var: {var}", None, "Not set (Optional)") - env_vars_status[var] = None - - # Check OAuth configuration for Hugging Face Space - if os.environ.get("SPACE_HOST"): - if os.environ.get("OAUTH_CLIENT_ID"): - self.log_status("OAuth configuration", True, "Properly configured") - env_vars_status["oauth_config"] = True - else: - self.log_status("OAuth configuration", False, "Missing OAUTH_CLIENT_ID - Set hf_oauth: true in Space metadata") - env_vars_status["oauth_config"] = False - all_required_vars_set = False - self.report.append("❌ OAuth not configured. Add 'hf_oauth: true' to Space metadata") - else: - self.log_status("OAuth configuration", None, "Not in Space environment - will be checked during deployment") - env_vars_status["oauth_config"] = None - - self.components["environment_variables"] = env_vars_status - self.environment_ready = all_required_vars_set - - return all_required_vars_set - - def check_imports(self) -> bool: - """Check that all required modules can be imported""" - self.log_section("Module Imports Verification") - - modules_to_check = [ - ("openai", "OpenAI API client"), - ("supabase", "Supabase client"), - ("langchain", "LangChain framework"), - ("langgraph", "LangGraph framework"), - ("gradio", "Gradio UI framework"), - ("agent", "GAIA Agent module"), - ("agent.config", "Agent configuration"), - ("agent.graph", "Agent graph workflow"), - ("agent.agent", "GAIA Agent implementation"), - ("memory", "Memory module"), - ("memory.supabase_memory", "Supabase memory implementation"), - ("tools", "Tools module"), - ("tools.web_tools", "Web search tools") - ] - - import_status = {} - all_imports_successful = True - - for module_name, description in modules_to_check: - try: - importlib.import_module(module_name) - self.log_status(f"Import: {module_name}", True, description) - import_status[module_name] = True - except ImportError as e: - self.log_status(f"Import: {module_name}", False, f"{description} - {str(e)}") - import_status[module_name] = False - all_imports_successful = False - self.report.append(f"❌ Failed to import {module_name}: {str(e)}") - except Exception as e: - self.log_status(f"Import: {module_name}", False, f"{description} - Unexpected error: {str(e)}") - import_status[module_name] = False - all_imports_successful = False - self.report.append(f"❌ Error importing {module_name}: {str(e)}") - - self.components["imports"] = import_status - self.imports_ready = all_imports_successful - - return all_imports_successful - - def check_connections(self) -> bool: - """Check connections to external services""" - self.log_section("External Services Verification") - - connection_status = {} - all_connections_working = True - - # Check OpenAI API - try: - from openai import OpenAI - - # Check if API key is available - api_key = os.environ.get("OPENAI_API_KEY") - if not api_key: - self.log_status("OpenAI API", False, "API key not set") - connection_status["openai"] = False - all_connections_working = False - self.report.append("❌ OpenAI API key not set") - else: - # Make a simple API call - client = OpenAI() - start_time = time.time() - response = client.chat.completions.create( - model="gpt-3.5-turbo", - messages=[{"role": "user", "content": "Say hello"}], - max_tokens=5 - ) - latency = time.time() - start_time - - if response and hasattr(response, 'choices') and len(response.choices) > 0: - self.log_status("OpenAI API", True, f"Connected successfully (latency: {latency:.2f}s)") - connection_status["openai"] = True - else: - self.log_status("OpenAI API", False, "API returned unexpected response") - connection_status["openai"] = False - all_connections_working = False - self.report.append("❌ OpenAI API returned unexpected response") - except Exception as e: - self.log_status("OpenAI API", False, f"Connection failed: {str(e)}") - connection_status["openai"] = False - all_connections_working = False - self.report.append(f"❌ OpenAI API connection failed: {str(e)}") - - # Check Supabase connection - try: - from supabase import create_client - - # Check if Supabase credentials are available - supabase_url = os.environ.get("SUPABASE_URL") - supabase_key = os.environ.get("SUPABASE_KEY") - - if not supabase_url or not supabase_key: - self.log_status("Supabase", False, "Credentials not set") - connection_status["supabase"] = False - all_connections_working = False - self.report.append("❌ Supabase credentials not set") - else: - # Try connecting to Supabase - client = create_client(supabase_url, supabase_key) - start_time = time.time() - - # Try a simple query - check if _memory table exists - try: - response = client.table("_memory").select("count(*)", count="exact").limit(1).execute() - count = response.count if hasattr(response, 'count') else 0 - latency = time.time() - start_time - - self.log_status("Supabase", True, f"Connected successfully (latency: {latency:.2f}s, {count} records)") - connection_status["supabase"] = True - except Exception as e: - # Table might not exist yet, which is okay - if "_memory" in str(e) and "not found" in str(e).lower(): - self.log_status("Supabase", True, "Connected, but _memory table doesn't exist yet (will be created)") - connection_status["supabase"] = True - else: - self.log_status("Supabase", False, f"Query failed: {str(e)}") - connection_status["supabase"] = False - all_connections_working = False - self.report.append(f"❌ Supabase query failed: {str(e)}") - except Exception as e: - self.log_status("Supabase", False, f"Connection failed: {str(e)}") - connection_status["supabase"] = False - all_connections_working = False - self.report.append(f"❌ Supabase connection failed: {str(e)}") - - # Check DuckDuckGo Search - try: - from tools.web_tools import duckduckgo_search - - # Try a simple search - start_time = time.time() - results = duckduckgo_search("What is the capital of France?", num_results=1) - latency = time.time() - start_time - - if results and len(results) > 0: - self.log_status("DuckDuckGo Search", True, f"Working (latency: {latency:.2f}s)") - connection_status["duckduckgo"] = True - else: - self.log_status("DuckDuckGo Search", False, "Returned no results") - connection_status["duckduckgo"] = False - all_connections_working = False - self.report.append("❌ DuckDuckGo search returned no results") - except Exception as e: - self.log_status("DuckDuckGo Search", False, f"Failed: {str(e)}") - connection_status["duckduckgo"] = False - all_connections_working = False - self.report.append(f"❌ DuckDuckGo search failed: {str(e)}") - - # Check Serper Search if API key is available - serper_api_key = os.environ.get("SERPER_API_KEY") - if serper_api_key: - try: - from tools.web_tools import serper_search - - # Try a simple search - start_time = time.time() - results = serper_search("What is the capital of France?", num_results=1) - latency = time.time() - start_time - - if results and len(results) > 0: - self.log_status("Serper Search", True, f"Working (latency: {latency:.2f}s)") - connection_status["serper"] = True - else: - self.log_status("Serper Search", False, "Returned no results") - connection_status["serper"] = False - # Don't make this a blocker since it's optional - self.report.append("⚠️ Serper search returned no results (optional)") - except Exception as e: - self.log_status("Serper Search", False, f"Failed: {str(e)}") - connection_status["serper"] = False - # Don't make this a blocker since it's optional - self.report.append(f"⚠️ Serper search failed: {str(e)} (optional)") - else: - self.log_status("Serper Search", None, "API key not set (optional)") - connection_status["serper"] = None - - self.components["connections"] = connection_status - self.connections_ready = all_connections_working - - return all_connections_working - - def check_agent(self) -> bool: - """Check GAIA agent initialization and functionality""" - self.log_section("GAIA Agent Verification") - - agent_status = {} - agent_ready = True - - # Check agent initialization - try: - from agent import GAIAAgent - - # Initialize the agent - start_time = time.time() - agent = GAIAAgent() - init_time = time.time() - start_time - - self.log_status("Agent initialization", True, f"Successful (init time: {init_time:.2f}s)") - agent_status["initialization"] = True - - # Check agent graph creation - try: - from agent.graph import create_agent_graph - - start_time = time.time() - graph = create_agent_graph() - graph_time = time.time() - start_time - - if graph: - self.log_status("Agent graph", True, f"Created successfully (time: {graph_time:.2f}s)") - agent_status["graph"] = True - else: - self.log_status("Agent graph", False, "Failed to create graph") - agent_status["graph"] = False - agent_ready = False - self.report.append("❌ Failed to create agent graph") - except Exception as e: - self.log_status("Agent graph", False, f"Error: {str(e)}") - agent_status["graph"] = False - agent_ready = False - self.report.append(f"❌ Error creating agent graph: {str(e)}") - - # Try a simple agent response (optional, might be too slow for verification) - try: - simple_question = "What is the capital of France?" - self.log_status("Agent test query", None, f"Executing: '{simple_question}' (may take some time)") - - start_time = time.time() - response = agent(simple_question) - response_time = time.time() - start_time - - if response and isinstance(response, str) and len(response) > 10: - self.log_status("Agent response", True, f"Successful (response time: {response_time:.2f}s)") - agent_status["response"] = True - else: - self.log_status("Agent response", False, f"Unexpected response: {response}") - agent_status["response"] = False - # Don't make this a blocker since it might be slow - self.report.append("⚠️ Agent response test failed (but might be due to timeouts)") - except Exception as e: - self.log_status("Agent response", False, f"Error: {str(e)}") - agent_status["response"] = False - # Don't make this a blocker - self.report.append(f"⚠️ Error getting agent response: {str(e)}") - except Exception as e: - self.log_status("Agent initialization", False, f"Failed: {str(e)}") - agent_status["initialization"] = False - agent_ready = False - self.report.append(f"❌ Agent initialization failed: {str(e)}") - - self.components["agent"] = agent_status - self.agent_ready = agent_ready - - return agent_ready - - def check_memory(self) -> bool: - """Check memory system functionality""" - self.log_section("Memory System Verification") - - memory_status = {} - memory_ready = True - - # Check memory initialization - try: - from memory.supabase_memory import SupabaseMemory - from agent.config import get_memory_config - - # Get memory configuration - memory_config = get_memory_config() - memory_config["enabled"] = True - - # Initialize memory - start_time = time.time() - memory = SupabaseMemory(memory_config) - init_time = time.time() - start_time - - self.log_status("Memory initialization", True, f"Successful (init time: {init_time:.2f}s)") - memory_status["initialization"] = True - - # Test basic memory operations - test_key = f"test_key_{int(time.time())}" - test_value = {"message": "Test value", "timestamp": time.time()} - - # Store - try: - memory.store(test_key, test_value) - self.log_status("Memory store", True, f"Successful (key: {test_key})") - memory_status["store"] = True - except Exception as e: - self.log_status("Memory store", False, f"Failed: {str(e)}") - memory_status["store"] = False - memory_ready = False - self.report.append(f"❌ Memory store operation failed: {str(e)}") - - # Retrieve - try: - retrieved = memory.retrieve(test_key) - - if retrieved and retrieved.get("message") == "Test value": - self.log_status("Memory retrieve", True, "Successful") - memory_status["retrieve"] = True - else: - self.log_status("Memory retrieve", False, f"Retrieved value doesn't match: {retrieved}") - memory_status["retrieve"] = False - memory_ready = False - self.report.append("❌ Memory retrieve operation returned incorrect data") - except Exception as e: - self.log_status("Memory retrieve", False, f"Failed: {str(e)}") - memory_status["retrieve"] = False - memory_ready = False - self.report.append(f"❌ Memory retrieve operation failed: {str(e)}") - - # Test specific memory types - - # 1. ConversationMemory - try: - from memory.supabase_memory import ConversationMemory - - session_id = f"test_session_{int(time.time())}" - conversation = ConversationMemory(memory, session_id) - - # Add messages - conversation.add_message("user", "Test question") - conversation.add_message("assistant", "Test answer") - - # Retrieve messages - messages = conversation.get_messages() - - if len(messages) >= 2: - self.log_status("ConversationMemory", True, f"Working (found {len(messages)} messages)") - memory_status["conversation"] = True - else: - self.log_status("ConversationMemory", False, f"Expected 2 messages, found {len(messages)}") - memory_status["conversation"] = False - memory_ready = False - self.report.append("❌ ConversationMemory not working correctly") - except Exception as e: - self.log_status("ConversationMemory", False, f"Error: {str(e)}") - memory_status["conversation"] = False - memory_ready = False - self.report.append(f"❌ ConversationMemory error: {str(e)}") - - # 2. ResultCache - try: - from memory.supabase_memory import ResultCache - - cache = ResultCache(memory) - cache_key = f"test_cache_{int(time.time())}" - - # Cache a result - cache.cache_result(cache_key, "Test cache result") - - # Get the result - cached = cache.get_result(cache_key) - - if cached == "Test cache result": - self.log_status("ResultCache", True, "Working correctly") - memory_status["cache"] = True - else: - self.log_status("ResultCache", False, f"Expected 'Test cache result', got: {cached}") - memory_status["cache"] = False - memory_ready = False - self.report.append("❌ ResultCache not working correctly") - except Exception as e: - self.log_status("ResultCache", False, f"Error: {str(e)}") - memory_status["cache"] = False - memory_ready = False - self.report.append(f"❌ ResultCache error: {str(e)}") - - # 3. WorkingMemory - try: - from memory.supabase_memory import WorkingMemory - - working = WorkingMemory(memory, session_id) - - # Store an intermediate result - working.store_intermediate_result("test_step", "Test intermediate result") - - # List intermediate results - results = working.list_intermediate_results() - - if results and any(r.get("step") == "test_step" for r in results): - self.log_status("WorkingMemory", True, "Working correctly") - memory_status["working"] = True - else: - self.log_status("WorkingMemory", False, f"Expected result with step 'test_step', got: {results}") - memory_status["working"] = False - memory_ready = False - self.report.append("❌ WorkingMemory not working correctly") - except Exception as e: - self.log_status("WorkingMemory", False, f"Error: {str(e)}") - memory_status["working"] = False - memory_ready = False - self.report.append(f"❌ WorkingMemory error: {str(e)}") - - except Exception as e: - self.log_status("Memory system", False, f"Initialization failed: {str(e)}") - memory_status["initialization"] = False - memory_ready = False - self.report.append(f"❌ Memory system initialization failed: {str(e)}") - - self.components["memory"] = memory_status - self.memory_ready = memory_ready - - return memory_ready - - def check_tools(self) -> bool: - """Check tool availability and functionality""" - self.log_section("Tools Verification") - - tools_status = {} - all_tools_ready = True - - # Check DuckDuckGo search tool - try: - from tools.web_tools import duckduckgo_search - - self.log_status("DuckDuckGo tool", True, "Available") - tools_status["duckduckgo"] = True - except Exception as e: - self.log_status("DuckDuckGo tool", False, f"Import error: {str(e)}") - tools_status["duckduckgo"] = False - all_tools_ready = False - self.report.append(f"❌ DuckDuckGo search tool not available: {str(e)}") - - # Check Serper search tool - if os.environ.get("SERPER_API_KEY"): - try: - from tools.web_tools import serper_search - - self.log_status("Serper tool", True, "Available") - tools_status["serper"] = True - except Exception as e: - self.log_status("Serper tool", False, f"Import error: {str(e)}") - tools_status["serper"] = False - # Don't make this a blocker since it's optional - self.report.append(f"⚠️ Serper search tool not available: {str(e)} (optional)") - else: - self.log_status("Serper tool", None, "API key not set (optional)") - tools_status["serper"] = None - - # Check reasoning tools - try: - from tools.reasoning_tools import structured_reasoning - - self.log_status("Reasoning tools", True, "Available") - tools_status["reasoning"] = True - except Exception as e: - self.log_status("Reasoning tools", False, f"Import error: {str(e)}") - tools_status["reasoning"] = False - all_tools_ready = False - self.report.append(f"❌ Reasoning tools not available: {str(e)}") - - # Check tool registry - try: - from agent.tool_registry import get_tools, register_tool - - tools = get_tools() - if tools and isinstance(tools, dict): - tool_count = len(tools) - self.log_status("Tool registry", True, f"Working ({tool_count} tools registered)") - tools_status["registry"] = True - else: - self.log_status("Tool registry", False, f"Unexpected return value: {tools}") - tools_status["registry"] = False - all_tools_ready = False - self.report.append("❌ Tool registry not working correctly") - except Exception as e: - self.log_status("Tool registry", False, f"Error: {str(e)}") - tools_status["registry"] = False - all_tools_ready = False - self.report.append(f"❌ Tool registry error: {str(e)}") - - self.components["tools"] = tools_status - self.tools_ready = all_tools_ready - - return all_tools_ready - - def check_deployment_readiness(self) -> bool: - """Check if the system is ready for deployment""" - self.log_section("Deployment Readiness") - - # Required components for deployment - required_components = [ - ("Environment", self.environment_ready), - ("Imports", self.imports_ready), - ("Connections", self.connections_ready), - ("Agent", self.agent_ready), - ("Memory", self.memory_ready), - ("Tools", self.tools_ready) - ] - - # Check if all required components are ready - all_ready = True - for component, status in required_components: - if status: - self.log_status(component, True, "Ready for deployment") - else: - self.log_status(component, False, "Not ready for deployment") - all_ready = False - - # Check for OAuth configuration in HF Space - if os.environ.get("SPACE_HOST"): - if os.environ.get("OAUTH_CLIENT_ID"): - self.log_status("OAuth configuration", True, "Properly configured for HF Space") - else: - self.log_status("OAuth configuration", False, "Missing OAUTH_CLIENT_ID - Set hf_oauth: true in Space metadata") - all_ready = False - self.report.append("❌ OAuth not configured for HF Space. Add 'hf_oauth: true' to Space metadata") - else: - self.log_status("OAuth configuration", None, "Not in HF Space environment - will need 'hf_oauth: true' in Space metadata") - self.report.append("⚠️ Set 'hf_oauth: true' in Space metadata when deploying to HF Space") - - self.deployment_ready = all_ready - - return all_ready - - def generate_report(self) -> str: - """Generate a comprehensive report of the production readiness check""" - self.log_section("Production Readiness Report") - - # Calculate run time - run_time = time.time() - self.start_time - - # Build the report - report_lines = [] - report_lines.append("# GAIA Agent Production Readiness Report") - report_lines.append(f"Generated on: {time.strftime('%Y-%m-%d %H:%M:%S')}") - report_lines.append(f"Total verification time: {run_time:.2f} seconds") - report_lines.append("") - - # Overall status - if self.deployment_ready: - report_lines.append("## ✅ DEPLOYMENT READY") - report_lines.append("The GAIA Agent is ready for deployment to Hugging Face Space.") - else: - report_lines.append("## ❌ NOT DEPLOYMENT READY") - report_lines.append("The GAIA Agent is NOT ready for deployment. Please address the issues below.") - - report_lines.append("") - - # Component statuses - report_lines.append("## Component Status") - report_lines.append("") - - # Environment - env_ready = "✅" if self.environment_ready else "❌" - report_lines.append(f"### {env_ready} Environment") - for var, status in self.components.get("environment_variables", {}).items(): - status_icon = "✅" if status else "❌" if status is False else "⚠️" - report_lines.append(f"- {status_icon} {var}") - report_lines.append("") - - # Imports - imports_ready = "✅" if self.imports_ready else "❌" - report_lines.append(f"### {imports_ready} Module Imports") - for module, status in self.components.get("imports", {}).items(): - status_icon = "✅" if status else "❌" - report_lines.append(f"- {status_icon} {module}") - report_lines.append("") - - # Connections - connections_ready = "✅" if self.connections_ready else "❌" - report_lines.append(f"### {connections_ready} External Connections") - for service, status in self.components.get("connections", {}).items(): - status_icon = "✅" if status else "❌" if status is False else "⚠️" - report_lines.append(f"- {status_icon} {service}") - report_lines.append("") - - # Agent - agent_ready = "✅" if self.agent_ready else "❌" - report_lines.append(f"### {agent_ready} GAIA Agent") - for component, status in self.components.get("agent", {}).items(): - status_icon = "✅" if status else "❌" if status is False else "⚠️" - report_lines.append(f"- {status_icon} {component}") - report_lines.append("") - - # Memory - memory_ready = "✅" if self.memory_ready else "❌" - report_lines.append(f"### {memory_ready} Memory System") - for component, status in self.components.get("memory", {}).items(): - status_icon = "✅" if status else "❌" - report_lines.append(f"- {status_icon} {component}") - report_lines.append("") - - # Tools - tools_ready = "✅" if self.tools_ready else "❌" - report_lines.append(f"### {tools_ready} Tools") - for tool, status in self.components.get("tools", {}).items(): - status_icon = "✅" if status else "❌" if status is False else "⚠️" - report_lines.append(f"- {status_icon} {tool}") - report_lines.append("") - - # Issues - if self.report: - report_lines.append("## Issues to Address") - report_lines.append("") - for issue in self.report: - report_lines.append(f"- {issue}") - report_lines.append("") - - # Deployment instructions - report_lines.append("## Deployment Instructions") - report_lines.append("") - report_lines.append("To deploy to Hugging Face Space:") - report_lines.append("") - report_lines.append("1. Create a new Space on Hugging Face") - report_lines.append("2. Select Gradio as the SDK") - report_lines.append("3. Set the following environment variables in Space settings:") - report_lines.append(" - OPENAI_API_KEY") - report_lines.append(" - SUPABASE_URL") - report_lines.append(" - SUPABASE_KEY") - report_lines.append(" - SERPER_API_KEY (optional)") - report_lines.append("4. Edit Space metadata to include `hf_oauth: true`") - report_lines.append("5. Deploy your code to the Space") - report_lines.append("") - report_lines.append("See `huggingface_space_deployment_guide.md` for detailed instructions.") - - # Build the final report string - report_str = "\n".join(report_lines) - - # Save the report to a file - try: - with open("production_readiness_report.md", "w") as f: - f.write(report_str) - logger.info("Report saved to production_readiness_report.md") - except Exception as e: - logger.error(f"Failed to save report: {str(e)}") - - # Log the report summary - logger.info("\n" + report_str) - - return report_str - - def run_all_checks(self) -> bool: - """Run all verification checks""" - logger.info("Starting production readiness verification...") - - # 1. Check environment - self.check_environment() - - # 2. Check imports - self.check_imports() - - # 3. Check connections - self.check_connections() - - # 4. Check agent - self.check_agent() - - # 5. Check memory - self.check_memory() - - # 6. Check tools - self.check_tools() - - # 7. Check deployment readiness - self.check_deployment_readiness() - - # 8. Generate report - self.generate_report() - - # Return deployment readiness - return self.deployment_ready - -if __name__ == "__main__": - checker = ProductionReadinessChecker() - ready = checker.run_all_checks() - - sys.exit(0 if ready else 1) \ No newline at end of file diff --git a/production_test.py b/production_test.py deleted file mode 100644 index dcb9ec20dcc9fac46ccd41983a14d6d653d66302..0000000000000000000000000000000000000000 --- a/production_test.py +++ /dev/null @@ -1,354 +0,0 @@ -""" -GAIA Agent Production Test - -This script tests all components of the GAIA agent before upload to Hugging Face Space: -1. OpenAI integration -2. Supabase memory integration -3. Search tools (Serper, DuckDuckGo) -4. Agent graph workflow -5. Multi-turn conversations with memory retention -6. Environment variables -7. Performance metrics - -Usage: - python production_test.py -""" - -import os -import sys -import time -import json -import logging -from typing import Dict, List, Any - -# Configure logging -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(name)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("production_test") - -def check_environment_variables(): - """Check all required environment variables are set""" - logger.info("Checking environment variables...") - required_vars = [ - "OPENAI_API_KEY", - "SUPABASE_URL", - "SUPABASE_KEY", - "SERPER_API_KEY" # Optional but recommended - ] - - missing = [] - optional_missing = [] - - for var in required_vars: - value = os.getenv(var) - if var == "SERPER_API_KEY" and not value: - optional_missing.append(var) - elif not value: - missing.append(var) - - if missing: - logger.error(f"Missing required environment variables: {', '.join(missing)}") - return False - - if optional_missing: - logger.warning(f"Missing optional environment variables: {', '.join(optional_missing)}") - - logger.info("All required environment variables are set") - return True - -def test_openai_integration(): - """Test OpenAI API integration""" - logger.info("Testing OpenAI integration...") - try: - from openai import OpenAI - client = OpenAI() - - start_time = time.time() - response = client.chat.completions.create( - model="gpt-3.5-turbo", - messages=[{"role": "user", "content": "Hello, are you working correctly?"}], - max_tokens=50 - ) - end_time = time.time() - - if response and response.choices and response.choices[0].message.content: - logger.info(f"OpenAI integration: SUCCESS ({(end_time - start_time):.2f}s)") - logger.info(f"Response: {response.choices[0].message.content.strip()}") - return True - else: - logger.error("OpenAI integration: FAILED - Empty response") - return False - except Exception as e: - logger.error(f"OpenAI integration: FAILED - {str(e)}") - return False - -def test_supabase_memory(): - """Test Supabase memory integration""" - logger.info("Testing Supabase memory integration...") - try: - from supabase import create_client - from memory.supabase_memory import SupabaseMemory - from agent.config import get_memory_config - - # Initialize memory - memory_config = get_memory_config() - memory_config["enabled"] = True - memory = SupabaseMemory(memory_config) - - # Test basic operations - test_key = "production_test_key" - test_value = {"test": "value", "timestamp": time.time()} - - # Store value - memory.store(test_key, test_value) - - # Retrieve value - retrieved = memory.retrieve(test_key) - - if retrieved and retrieved.get("test") == "value": - logger.info("Supabase memory integration: SUCCESS") - return True - else: - logger.error("Supabase memory integration: FAILED - Retrieved value doesn't match") - return False - except Exception as e: - logger.error(f"Supabase memory integration: FAILED - {str(e)}") - return False - -def test_duckduckgo_integration(): - """Test DuckDuckGo search integration""" - logger.info("Testing DuckDuckGo search integration...") - try: - from tools.web_tools import duckduckgo_search - - query = "What is GAIA agent?" - results = duckduckgo_search(query, num_results=3) - - if results and len(results) > 0: - logger.info(f"DuckDuckGo integration: SUCCESS - Found {len(results)} results") - for i, result in enumerate(results[:2]): - if 'title' in result and 'link' in result: - logger.info(f" Result {i+1}: {result['title'][:50]}... - {result['link']}") - return True - else: - logger.error("DuckDuckGo integration: FAILED - No results returned") - return False - except Exception as e: - logger.error(f"DuckDuckGo integration: FAILED - {str(e)}") - return False - -def test_serper_integration(): - """Test Serper search integration (if API key is available)""" - logger.info("Testing Serper search integration...") - - if not os.getenv("SERPER_API_KEY"): - logger.warning("Serper API key not found, skipping test") - return None - - try: - from tools.web_tools import serper_search - - query = "Latest AI research papers" - results = serper_search(query, num_results=3) - - if results and len(results) > 0: - logger.info(f"Serper integration: SUCCESS - Found {len(results)} results") - return True - else: - logger.error("Serper integration: FAILED - No results returned") - return False - except Exception as e: - logger.error(f"Serper integration: FAILED - {str(e)}") - return False - -def test_agent_graph(): - """Test agent graph workflow""" - logger.info("Testing agent graph workflow...") - try: - from agent.graph import create_agent_graph, run_agent_graph - - # Test create_agent_graph functionality - workflow, initial_state = create_agent_graph() - - # Check if workflow has expected nodes - expected_nodes = ["analyze_question", "create_plan", "select_tool", - "execute_tool", "formulate_answer"] - - # Get actual nodes from the workflow - actual_nodes = workflow.nodes.keys() - - missing_nodes = [node for node in expected_nodes if node not in actual_nodes] - - if missing_nodes: - logger.error(f"Agent graph: FAILED - Missing nodes: {', '.join(missing_nodes)}") - return False - - # Check if initial state has expected keys - expected_keys = ["question", "analysis", "plan", "current_step", - "tools_to_use", "tool_results", "answer", "error"] - - missing_keys = [key for key in expected_keys if key not in initial_state] - - if missing_keys: - logger.error(f"Agent graph: FAILED - Missing initial state keys: {', '.join(missing_keys)}") - return False - - logger.info("Agent graph workflow: SUCCESS - Graph structure verified") - return True - except Exception as e: - logger.error(f"Agent graph workflow: FAILED - {str(e)}") - return False - -def test_gaia_agent(): - """Test GAIA agent with a simple question""" - logger.info("Testing GAIA agent with a simple question...") - try: - from agent.agent import GAIAAgent - - agent = GAIAAgent() - - start_time = time.time() - question = "What is the capital of Japan?" - - try: - response = agent(question) - end_time = time.time() - - if response and len(response) > 2: # Basic validation - logger.info(f"GAIA agent simple question: SUCCESS ({(end_time - start_time):.2f}s)") - logger.info(f"Response: {response[:100]}...") - - # Get memory snapshot to verify memory functions - memory_snapshot = agent.get_memory_snapshot() - - if memory_snapshot and "conversation" in memory_snapshot: - logger.info("Memory snapshot successfully retrieved") - - return True - else: - logger.error("GAIA agent simple question: FAILED - Invalid response") - return False - except Exception as inner_e: - logger.error(f"GAIA agent execution failed: {str(inner_e)}") - return False - except Exception as e: - logger.error(f"GAIA agent initialization failed: {str(e)}") - return False - -def test_multi_turn_conversation(): - """Test multi-turn conversation with memory retention""" - logger.info("Testing multi-turn conversation with memory retention...") - try: - from agent.agent import GAIAAgent - - agent = GAIAAgent() - - # First question - question1 = "What is the capital of France?" - response1 = agent(question1) - logger.info(f"Q1: {question1}") - logger.info(f"A1: {response1[:100]}...") - - # Verify conversation is stored - memory_snapshot = agent.get_memory_snapshot() - if not memory_snapshot or not memory_snapshot.get("conversation"): - logger.error("Multi-turn conversation: FAILED - First question not stored in memory") - return False - - # Follow-up question - question2 = "What is the population of that city?" - response2 = agent(question2) - logger.info(f"Q2: {question2}") - logger.info(f"A2: {response2[:100]}...") - - # Get updated memory snapshot - memory_snapshot = agent.get_memory_snapshot() - conversation = memory_snapshot.get("conversation", []) - - # Check if both questions and answers are in the conversation memory - if len(conversation) < 4: # Should have at least 4 messages (2 questions, 2 answers) - logger.error(f"Multi-turn conversation: FAILED - Memory has {len(conversation)} messages, expected at least 4") - return False - - # Check if the response mentions Paris or population numbers - if "Paris" in response2 or any(char.isdigit() for char in response2): - logger.info("Multi-turn conversation: SUCCESS - Memory retention confirmed") - return True - else: - logger.error("Multi-turn conversation: FAILED - Memory retention not working") - return False - except Exception as e: - logger.error(f"Multi-turn conversation: FAILED - {str(e)}") - return False - -def main(): - """Run all production tests""" - logger.info("=== STARTING GAIA AGENT PRODUCTION TESTS ===") - - results = {} - - # Check environment variables - results["env_vars"] = check_environment_variables() - - # Test OpenAI integration - results["openai"] = test_openai_integration() - - # Test Supabase memory integration - results["supabase"] = test_supabase_memory() - - # Test search tools - results["duckduckgo"] = test_duckduckgo_integration() - results["serper"] = test_serper_integration() - - # Test agent graph workflow - results["agent_graph"] = test_agent_graph() - - # Test GAIA agent - results["gaia_agent"] = test_gaia_agent() - - # Test multi-turn conversation - results["multi_turn"] = test_multi_turn_conversation() - - # Print summary - logger.info("\n=== PRODUCTION TEST RESULTS ===") - for component, result in results.items(): - status = "SUCCESS" if result else "FAILED" if result is False else "SKIPPED" - logger.info(f"{component.upper()}: {status}") - - # Check if all required tests passed - required_tests = ["env_vars", "openai", "supabase", "duckduckgo", "agent_graph"] - failed_tests = [test for test in required_tests if results.get(test) is False] - - if failed_tests: - logger.error(f"\n❌ PRODUCTION TEST FAILED - {len(failed_tests)} component(s) not working") - logger.error(f"Failed components: {', '.join(failed_tests)}") - return 1 - else: - # Generate detailed production test report - logger.info("\n✅ PRODUCTION TEST PASSED - All required components working correctly") - logger.info("=== DETAILED COMPONENT STATUS ===") - - component_details = { - "env_vars": "Environment variables correctly configured", - "openai": "OpenAI API connection successful", - "supabase": "Supabase connection and memory system operational", - "duckduckgo": "DuckDuckGo search integration functional", - "serper": "Serper search integration available (optional)", - "agent_graph": "Agent graph workflow structure validated", - "gaia_agent": "GAIA agent execution successful", - "multi_turn": "Multi-turn conversation with memory retention working" - } - - for component, result in results.items(): - status = "✓ PASS" if result else "✗ FAIL" if result is False else "⚠ SKIPPED" - details = component_details.get(component, "") - logger.info(f"{component.upper()}: {status} - {details}") - - logger.info("\nSystem is ready for deployment to Hugging Face Space") - return 0 - -if __name__ == "__main__": - exit_code = main() - sys.exit(exit_code) \ No newline at end of file diff --git a/production_verification_report.md b/production_verification_report.md deleted file mode 100644 index f58e19c18d21bbb7a712b8e499941834879cc2c2..0000000000000000000000000000000000000000 --- a/production_verification_report.md +++ /dev/null @@ -1,65 +0,0 @@ -# GAIA Agent Production Verification Report - -## Issues Fixed - -| File | Issue | Fix Applied | -|------|-------|------------| -| `app.py` | Gradio Accordion update error | Fixed Accordion reference and update method | -| `app.py` | OAuth configuration error in HF Space | Added graceful OAuth handling with demo mode | - -## OAuth Configuration in Hugging Face Space - -The GAIA Agent app has been updated to handle OAuth configuration issues gracefully. The app now works in two modes: - -1. **Full functionality mode** - When OAuth is properly configured in the Hugging Face Space -2. **Demonstration mode** - When OAuth is not configured, allowing users to see results without submitting to the leaderboard - -### Configuring OAuth in Hugging Face Space - -To enable full functionality including submission to the leaderboard, you must configure OAuth in your Hugging Face Space: - -1. Go to the Space settings -2. Under "Repository secrets", set `hf_oauth: true` - -If OAuth is not configured, the app will display a notice and run in demonstration mode. - -## Deployment Recommendations - -When deploying to Hugging Face Space, ensure: - -1. All required environment variables are set in Space settings: - - `OPENAI_API_KEY` - Required for agent functionality - - `SUPABASE_URL` - Required for memory functionality - - `SUPABASE_KEY` - Required for memory functionality - - `SERPER_API_KEY` - Optional for enhanced search capabilities - -2. OAuth is properly configured by setting `hf_oauth: true` in Space settings - -3. Monitor Space logs after deployment to catch any runtime issues - -## Verification Test Results - -The app has been tested for: - -- Graceful handling of missing OAuth configuration -- Proper execution in demonstration mode -- UI updates to inform users about current mode -- Accordion component functionality - -## Rollback Instructions - -If issues are encountered after deployment: - -1. Revert the latest changes by rolling back to the previous commit in the Hugging Face Space Git repository -2. Check Space logs for specific error messages -3. Verify all environment variables are properly set -4. Ensure OAuth configuration matches your intended use case - -## Post-Deployment Verification - -After deploying to Hugging Face Space, verify: - -1. The app loads without error -2. The appropriate mode (full or demo) is active based on your OAuth configuration -3. Agent functionality works with a test question -4. Results are displayed correctly in the interface \ No newline at end of file diff --git a/remove_debug_statements.py b/remove_debug_statements.py deleted file mode 100644 index 11f5d28f530c3a707a8e3577dd78fab900f49983..0000000000000000000000000000000000000000 --- a/remove_debug_statements.py +++ /dev/null @@ -1,196 +0,0 @@ -""" -Script to remove debug statements and unnecessary commented code from Python files. - -This script focuses on removing: -1. Debug logging statements (logger.debug) -2. Print statements used for debugging -3. Commented code that's not necessary for production -""" - -import os -import re -import logging -from typing import List, Dict, Tuple - -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("remove_debug") - -CRITICAL_FILES = [ - "agent/agent.py", - "agent/config.py", - "agent/graph.py", - "agent/tool_registry.py", - "memory/supabase_memory.py", - "tools/web_tools.py", - "tools/reasoning_tools.py" -] - -DEBUG_PATTERNS = [ - r'^\s*logger\.debug\s*\(.*\)', - r'^\s*print\s*\(.*\)', - r'^\s*console\.log\s*\(.*\)', -] - -KEEP_COMMENT_PATTERNS = [ - r'^\s*# -+$', # Separator lines - r'^\s*# [A-Z][a-z]+ [A-Z][a-z]+', # Section headers like "# Import Configuration" - r'^\s*# Copyright', # Copyright notices - r'^\s*# License', # License information - r'^\s*# Author', # Author information -] - -def should_keep_comment(line: str) -> bool: - """Determine if a comment line should be kept.""" - if line.strip() == "#": - return True - - for pattern in KEEP_COMMENT_PATTERNS: - if re.match(pattern, line): - return True - - if '"""' in line or "'''" in line: - return True - - return False - -def clean_file(file_path: str, dry_run: bool = False) -> Tuple[int, int]: - """ - Clean debug statements and unnecessary comments from a file. - - Args: - file_path: Path to the file to clean - dry_run: If True, don't actually modify the file - - Returns: - Tuple of (debug_statements_removed, comment_lines_removed) - """ - logger.info(f"Cleaning file: {file_path}") - - try: - with open(file_path, 'r', encoding='utf-8') as f: - lines = f.readlines() - - debug_statements_removed = 0 - comment_lines_removed = 0 - cleaned_lines = [] - in_docstring = False - - for line in lines: - if '"""' in line or "'''" in line: - triple_quotes_count = line.count('"""') + line.count("'''") - if triple_quotes_count % 2 == 1: - in_docstring = not in_docstring - - is_debug_statement = False - for pattern in DEBUG_PATTERNS: - if re.match(pattern, line): - debug_statements_removed += 1 - is_debug_statement = True - break - - if is_debug_statement: - continue - - if not in_docstring and line.strip().startswith('#'): - if should_keep_comment(line): - cleaned_lines.append(line) - else: - comment_lines_removed += 1 - continue - else: - cleaned_lines.append(line) - - if not dry_run: - with open(file_path, 'w', encoding='utf-8') as f: - f.writelines(cleaned_lines) - - logger.info(f"Removed {debug_statements_removed} debug statements and {comment_lines_removed} comment lines") - return debug_statements_removed, comment_lines_removed - - except Exception as e: - logger.error(f"Error cleaning file {file_path}: {str(e)}") - return 0, 0 - -def clean_files(file_paths: List[str], dry_run: bool = False) -> Dict[str, Tuple[int, int]]: - """ - Clean multiple files. - - Args: - file_paths: List of file paths to clean - dry_run: If True, don't actually modify the files - - Returns: - Dictionary mapping file paths to (debug_statements_removed, comment_lines_removed) - """ - results = {} - - for file_path in file_paths: - if os.path.exists(file_path): - results[file_path] = clean_file(file_path, dry_run) - else: - logger.warning(f"File not found: {file_path}") - - return results - -def main(): - """Main function.""" - import argparse - - - parser = argparse.ArgumentParser(description="Remove debug statements and unnecessary comments") - parser.add_argument("--dry-run", action="store_true", help="Don't actually modify files") - parser.add_argument("--all", action="store_true", help="Clean all Python files, not just critical ones") - args = parser.parse_args() - - - if args.dry_run: - logger.info("Running in dry-run mode. No files will be modified.") - - files_to_clean = CRITICAL_FILES.copy() - - if args.all: - for root, _, files in os.walk('.'): - for file in files: - if file.endswith('.py'): - file_path = os.path.join(root, file) - file_path = file_path.replace('\\', '/') - if file_path not in files_to_clean: - files_to_clean.append(file_path) - - - for file_path in files_to_clean: - if os.path.exists(file_path): - debug_removed, comments_removed = clean_file(file_path, args.dry_run) - else: - - results = clean_files(files_to_clean, args.dry_run) - - total_debug_removed = sum(debug for debug, _ in results.values()) - total_comments_removed = sum(comments for _, comments in results.values()) - - - logger.info("\nCleanup Summary:") - logger.info(f"- Files processed: {len(results)}") - logger.info(f"- Debug statements removed: {total_debug_removed}") - logger.info(f"- Comment lines removed: {total_comments_removed}") - - if args.dry_run: - logger.info("\nThis was a dry run. Run without --dry-run to apply changes.") - -if __name__ == "__main__": - handler = logging.StreamHandler() - handler.setLevel(logging.INFO) - formatter = logging.Formatter("%(asctime)s - %(levelname)s - %(message)s") - handler.setFormatter(formatter) - logger.addHandler(handler) - - - for file_path in CRITICAL_FILES: - if os.path.exists(file_path): - else: - - main() - diff --git a/remove_duplicate_tests.py b/remove_duplicate_tests.py deleted file mode 100644 index 04b90355cb9831b9031b2987ac8d79d654a342d3..0000000000000000000000000000000000000000 --- a/remove_duplicate_tests.py +++ /dev/null @@ -1,112 +0,0 @@ -#!/usr/bin/env python -""" -Script to remove duplicate test files. - -This script identifies and removes duplicate test files based on predefined rules. -""" - -import os -import logging -import shutil - -# Configure logging -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("remove_duplicates") - -# Define duplicate test files to remove -DUPLICATES_TO_REMOVE = [ - # Keep test_supabase_tables_comprehensive.py, remove others - "tests/test_supabase_tables_final.py", - "tests/test_supabase_tables_fix.py", - "tests/test_supabase_tables_production.py", - - # Other potential duplicates - "tests/minimal_supabase_test.py", # Likely a duplicate or test version -] - -def backup_file(file_path): - """Create a backup of a file before removing it.""" - backup_dir = "backup_tests" - if not os.path.exists(backup_dir): - os.makedirs(backup_dir) - - # Get just the filename without the path - filename = os.path.basename(file_path) - backup_path = os.path.join(backup_dir, filename) - - # Copy the file to the backup directory - shutil.copy2(file_path, backup_path) - logger.info(f"Created backup of {file_path} at {backup_path}") - -def remove_duplicate_files(file_paths, dry_run=False): - """ - Remove duplicate test files. - - Args: - file_paths: List of file paths to remove - dry_run: If True, don't actually remove files - """ - removed_count = 0 - - for file_path in file_paths: - if os.path.exists(file_path): - logger.info(f"Removing duplicate test file: {file_path}") - - if not dry_run: - # Create a backup before removing - backup_file(file_path) - - # Remove the file - os.remove(file_path) - removed_count += 1 - else: - logger.warning(f"File not found: {file_path}") - - return removed_count - -def main(): - """Main function.""" - import argparse - - parser = argparse.ArgumentParser(description="Remove duplicate test files") - parser.add_argument("--dry-run", action="store_true", help="Don't actually remove files") - args = parser.parse_args() - - if args.dry_run: - logger.info("Running in dry-run mode. No files will be removed.") - - # Remove duplicate files - removed_count = remove_duplicate_files(DUPLICATES_TO_REMOVE, args.dry_run) - - # Print summary - logger.info("\nCleanup Summary:") - logger.info(f"- Duplicate test files removed: {removed_count}") - - if args.dry_run: - logger.info("\nThis was a dry run. Run without --dry-run to apply changes.") - -if __name__ == "__main__": - # Force output to console even in dry-run mode - handler = logging.StreamHandler() - handler.setLevel(logging.INFO) - formatter = logging.Formatter("%(asctime)s - %(levelname)s - %(message)s") - handler.setFormatter(formatter) - logger.addHandler(handler) - - # Add direct print statements for debugging - print("Starting duplicate test removal script") - print(f"Files to remove: {DUPLICATES_TO_REMOVE}") - - # Check if files exist - for file_path in DUPLICATES_TO_REMOVE: - if os.path.exists(file_path): - print(f"File exists: {file_path}") - else: - print(f"File does not exist: {file_path}") - - main() - - print("Script completed") \ No newline at end of file diff --git a/run_verification_tests.py b/run_verification_tests.py deleted file mode 100644 index b08789933c987c0ceab6c281a9bbf32aa6206f47..0000000000000000000000000000000000000000 --- a/run_verification_tests.py +++ /dev/null @@ -1,303 +0,0 @@ -""" -GAIA Assessment Verification Test Runner - -This script loads environment variables from the .env file and then runs -the verification tests to ensure all components work correctly. - -Usage: - python run_verification_tests.py [--verbose] [--show-all] [--metrics] -""" - -import os -import sys -import subprocess -import argparse - -def load_dotenv(): - """Load environment variables from .env file""" - env_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), '.env') - if not os.path.exists(env_path): - print(f"ERROR: .env file not found at {env_path}") - return False - - print(f"Loading environment variables from {env_path}") - env_vars = {} - - with open(env_path, 'r') as file: - for line in file: - line = line.strip() - if not line or line.startswith('#'): - continue - - key, value = line.split('=', 1) - os.environ[key] = value - env_vars[key] = value - - # Print the environment variables (redacted) - print("\nEnvironment variables loaded:") - for key in env_vars: - if key in ["OPENAI_API_KEY", "SUPABASE_KEY", "HF_TOKEN", "SERPER_API_KEY", "PERPLEXITY_API_KEY"]: - value = env_vars[key][:5] + "..." + env_vars[key][-4:] if len(env_vars[key]) > 9 else "***" - else: - value = env_vars[key] - print(f" {key}: {value}") - - return True - -def run_verification_tests(args): - """Run the verification tests with the given arguments""" - cmd = ["python", "verify_gaia_assessment.py"] - - # Add any command-line arguments - if args.verbose: - cmd.append("--verbose") - if args.show_all: - cmd.append("--show-all") - if args.metrics: - cmd.append("--metrics") - - print(f"\nRunning verification tests: {' '.join(cmd)}") - - # Run the verification tests and capture output - process = subprocess.Popen( - cmd, - stdout=subprocess.PIPE, - stderr=subprocess.PIPE, - text=True - ) - - # Stream output in real-time - while True: - output = process.stdout.readline() - if output == '' and process.poll() is not None: - break - if output: - print(output.strip()) - - # Get the final exit code - exit_code = process.poll() - - # Get any error output - stderr = process.stderr.read() - if stderr: - print("\nERROR OUTPUT:") - print(stderr) - - return exit_code - -def test_single_integration(): - """Test OpenAI and Supabase integration individually""" - print("\n===== Testing OpenAI Integration =====") - openai_result = subprocess.run( - ["python", "test_openai_minimal.py"], - capture_output=True, - text=True - ) - print(f"OpenAI Test Result: {'SUCCESS' if openai_result.returncode == 0 else 'FAILED'}") - print(openai_result.stdout[:200] + "..." if len(openai_result.stdout) > 200 else openai_result.stdout) - - print("\n===== Testing Supabase Integration =====") - supabase_result = subprocess.run( - ["python", "test_supabase_minimal.py"], - capture_output=True, - text=True - ) - print(f"Supabase Test Result: {'SUCCESS' if supabase_result.returncode == 0 else 'FAILED'}") - print(supabase_result.stdout[:200] + "..." if len(supabase_result.stdout) > 200 else supabase_result.stdout) - - print("\n===== Testing Serper Integration =====") - serper_result = subprocess.run( - ["python", "test_serper_minimal.py"], - capture_output=True, - text=True - ) - print(f"Serper Test Result: {'SUCCESS' if serper_result.returncode == 0 else 'FAILED'}") - print(serper_result.stdout[:200] + "..." if len(serper_result.stdout) > 200 else serper_result.stdout) - - return ( - openai_result.returncode == 0, - supabase_result.returncode == 0, - serper_result.returncode == 0 - ) - -def create_minimal_tests(): - """Create minimal test scripts for individual component testing""" - # Create OpenAI minimal test - with open("test_openai_minimal.py", "w") as f: - f.write(""" -import os -import openai -from openai import OpenAI - -def test_openai_connection(): - print("Testing OpenAI connection...") - api_key = os.getenv("OPENAI_API_KEY") - if not api_key: - print("ERROR: OPENAI_API_KEY not set") - return False - - try: - client = OpenAI(api_key=api_key) - models = client.models.list() - print(f"Successfully connected to OpenAI API. Available models: {len(models.data)}") - - # Test a simple completion - response = client.chat.completions.create( - model="gpt-3.5-turbo", - messages=[ - {"role": "system", "content": "You are a helpful assistant."}, - {"role": "user", "content": "Hello, world!"} - ] - ) - print(f"Response: {response.choices[0].message.content[:50]}...") - return True - except Exception as e: - print(f"Error connecting to OpenAI: {str(e)}") - return False - -if __name__ == "__main__": - if test_openai_connection(): - print("OpenAI test passed!") - exit(0) - else: - print("OpenAI test failed!") - exit(1) -""") - - # Create Supabase minimal test - with open("test_supabase_minimal.py", "w") as f: - f.write(""" -import os -import sys -from supabase import create_client, Client - -def test_supabase_connection(): - print("Testing Supabase connection...") - supabase_url = os.getenv("SUPABASE_URL") - supabase_key = os.getenv("SUPABASE_KEY") - - if not supabase_url or not supabase_key: - print("ERROR: SUPABASE_URL or SUPABASE_KEY not set") - return False - - try: - supabase: Client = create_client(supabase_url, supabase_key) - # Try a simple query - version = supabase.table("postgres_version").select("*").execute() - print(f"Successfully connected to Supabase. Version: {version}") - - # Check for the required tables - tables = ["gaia_memory", "gaia_memory_working", "gaia_memory_conversation"] - for table in tables: - try: - result = supabase.table(table).select("count(*)", count="exact").execute() - count = result.count - print(f"Table {table} exists with {count} rows") - except Exception as e: - print(f"Error accessing table {table}: {str(e)}") - return False - - return True - except Exception as e: - print(f"Error connecting to Supabase: {str(e)}") - return False - -if __name__ == "__main__": - if test_supabase_connection(): - print("Supabase test passed!") - exit(0) - else: - print("Supabase test failed!") - exit(1) -""") - - # Create Serper minimal test - with open("test_serper_minimal.py", "w") as f: - f.write(""" -import os -import json -import requests - -def test_serper_connection(): - print("Testing Serper API connection...") - api_key = os.getenv("SERPER_API_KEY") - if not api_key: - print("ERROR: SERPER_API_KEY not set") - return False - - try: - url = "https://google.serper.dev/search" - payload = json.dumps({ - "q": "Who is the current president of the United States", - "gl": "us", - "hl": "en" - }) - headers = { - 'X-API-KEY': api_key, - 'Content-Type': 'application/json' - } - - response = requests.request("POST", url, headers=headers, data=payload) - - if response.status_code == 200: - print(f"Successfully connected to Serper API. Response: {response.text[:100]}...") - return True - else: - print(f"Error from Serper API: {response.status_code} - {response.text}") - return False - except Exception as e: - print(f"Error connecting to Serper API: {str(e)}") - return False - -if __name__ == "__main__": - if test_serper_connection(): - print("Serper test passed!") - exit(0) - else: - print("Serper test failed!") - exit(1) -""") - -def parse_args(): - """Parse command line arguments""" - parser = argparse.ArgumentParser(description="Run GAIA Assessment verification tests") - parser.add_argument("--verbose", action="store_true", help="Enable verbose logging") - parser.add_argument("--show-all", action="store_true", help="Show all table entries") - parser.add_argument("--metrics", action="store_true", help="Show detailed performance metrics") - parser.add_argument("--components-only", action="store_true", help="Test individual components only") - return parser.parse_args() - -def main(): - """Main function""" - args = parse_args() - - # Load environment variables from .env file - if not load_dotenv(): - print("Failed to load environment variables from .env file") - sys.exit(1) - - # Create minimal test scripts - create_minimal_tests() - - # If testing individual components only - if args.components_only: - openai_success, supabase_success, serper_success = test_single_integration() - - # Print final summary - print("\n===== COMPONENT TEST RESULTS =====") - print(f"OpenAI Integration: {'✅ PASS' if openai_success else '❌ FAIL'}") - print(f"Supabase Integration: {'✅ PASS' if supabase_success else '❌ FAIL'}") - print(f"Serper Integration: {'✅ PASS' if serper_success else '❌ FAIL'}") - - if openai_success and supabase_success and serper_success: - print("\nAll component tests passed successfully!") - return 0 - else: - print("\nSome component tests failed. Please check the output for details.") - return 1 - - # Run verification tests - return run_verification_tests(args) - -if __name__ == "__main__": - sys.exit(main()) \ No newline at end of file diff --git a/simple_test.py b/simple_test.py deleted file mode 100644 index ed3e0028911205f720d5a29f40475260517df85b..0000000000000000000000000000000000000000 --- a/simple_test.py +++ /dev/null @@ -1,146 +0,0 @@ -""" -Very simple test script to identify basic environment issues -""" -import os -import logging -import traceback -import sys - -# Configure detailed logging -logging.basicConfig( - level=logging.DEBUG, - format="%(asctime)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("simple_test") - -def check_env_vars(): - """Check environment variables""" - logger.info("Checking environment variables...") - required = ["OPENAI_API_KEY", "SUPABASE_URL", "SUPABASE_KEY"] - optional = ["SERPER_API_KEY"] - - all_set = True - for var in required: - if os.environ.get(var): - logger.info(f"✓ {var} is set") - else: - logger.error(f"✗ {var} is NOT set") - all_set = False - - for var in optional: - if os.environ.get(var): - logger.info(f"✓ {var} is set (optional)") - else: - logger.warning(f"⚠ {var} is not set (optional)") - - return all_set - -def test_supabase_import(): - """Test importing Supabase client""" - logger.info("Testing Supabase imports...") - try: - from supabase import create_client - logger.info("✓ Successfully imported Supabase client") - return True - except Exception as e: - logger.error(f"✗ Failed to import Supabase: {e}") - logger.error(traceback.format_exc()) - return False - -def test_memory_module(): - """Test importing memory module""" - logger.info("Testing memory module imports...") - try: - # First try importing from the module - logger.info("Attempting to import from memory module...") - - # Show the memory directory contents - import os - memory_dir = os.path.join(os.getcwd(), "memory") - if os.path.exists(memory_dir): - logger.info(f"Memory directory exists at: {memory_dir}") - files = os.listdir(memory_dir) - logger.info(f"Files in memory directory: {files}") - else: - logger.error(f"Memory directory not found at: {memory_dir}") - - # Try importing directly - try: - from memory.supabase_memory import SupabaseMemory - logger.info("✓ Successfully imported SupabaseMemory class") - return True - except ImportError as ie: - logger.error(f"✗ Import error: {ie}") - logger.error(traceback.format_exc()) - - # Try importing the module - try: - import memory - logger.info(f"✓ Successfully imported memory module") - logger.info(f"Memory module attributes: {dir(memory)}") - return False - except ImportError as ie2: - logger.error(f"✗ Failed to import memory module: {ie2}") - logger.error(traceback.format_exc()) - return False - except Exception as e: - logger.error(f"✗ Other error importing SupabaseMemory: {e}") - logger.error(traceback.format_exc()) - return False - except Exception as e: - logger.error(f"✗ Unexpected error in test_memory_module: {e}") - logger.error(traceback.format_exc()) - return False - -def check_python_path(): - """Check Python path""" - logger.info("Checking Python path...") - - # Print current working directory - cwd = os.getcwd() - logger.info(f"Current working directory: {cwd}") - - # Print Python path - logger.info(f"Python path: {sys.path}") - - # Check if current directory is in path - if cwd in sys.path: - logger.info("✓ Current directory is in Python path") - else: - logger.warning("⚠ Current directory is NOT in Python path") - # Add it - sys.path.insert(0, cwd) - logger.info("✓ Added current directory to Python path") - -def main(): - logger.info("=== BASIC COMPONENT TEST ===") - - # Check Python path - check_python_path() - - # Check environment variables - env_ok = check_env_vars() - if not env_ok: - logger.error("Environment variables check failed") - - # Test imports - supabase_ok = test_supabase_import() - if not supabase_ok: - logger.error("Supabase import failed") - - # Test memory module - memory_ok = test_memory_module() - if not memory_ok: - logger.error("Memory module test failed") - - logger.info("=== TEST COMPLETE ===") - - if env_ok and supabase_ok and memory_ok: - logger.info("✅ All basic tests passed") - return 0 - else: - logger.error("❌ Some tests failed") - return 1 - -if __name__ == "__main__": - sys.exit(main()) \ No newline at end of file diff --git a/space_config.json b/space_config.json deleted file mode 100644 index 7d58f95cd5a1d7013caae0fa4cc42cade9288b97..0000000000000000000000000000000000000000 --- a/space_config.json +++ /dev/null @@ -1,12 +0,0 @@ -{ - "title": "GAIA - Generative AI Assistant", - "emoji": "\ud83e\udde0", - "colorFrom": "blue", - "colorTo": "purple", - "sdk": "gradio", - "sdk_version": "3.36.1", - "python_version": "3.9", - "app_file": "app_no_oauth.py", - "pinned": false, - "license": "mit" -} \ No newline at end of file diff --git a/supabase_tables.sql b/supabase_tables.sql deleted file mode 100644 index 9c57e0adf6ec746392bea8165f22aa0ac83aa8e2..0000000000000000000000000000000000000000 --- a/supabase_tables.sql +++ /dev/null @@ -1,77 +0,0 @@ --- Supabase Tables for GAIA Agent --- Run these SQL statements in the Supabase SQL Editor to create the necessary tables - --- 1. Main Memory Table -CREATE TABLE IF NOT EXISTS gaia_memory ( - id UUID PRIMARY KEY DEFAULT uuid_generate_v4(), - key TEXT NOT NULL UNIQUE, - value TEXT NOT NULL, - metadata JSONB, - created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), - expires_at TIMESTAMPTZ -); - -CREATE INDEX IF NOT EXISTS idx_gaia_memory_key ON gaia_memory(key); -CREATE INDEX IF NOT EXISTS idx_gaia_memory_expires_at ON gaia_memory(expires_at); - --- 2. Conversation History Table -CREATE TABLE IF NOT EXISTS gaia_memory_conversations ( - id UUID PRIMARY KEY DEFAULT uuid_generate_v4(), - conversation_id TEXT NOT NULL, - message_id TEXT NOT NULL, - role TEXT NOT NULL, - content TEXT NOT NULL, - timestamp TIMESTAMPTZ NOT NULL DEFAULT NOW(), - metadata JSONB, - created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), - expires_at TIMESTAMPTZ -); - -CREATE INDEX IF NOT EXISTS idx_gaia_memory_conversations_conversation_id ON gaia_memory_conversations(conversation_id); -CREATE INDEX IF NOT EXISTS idx_gaia_memory_conversations_timestamp ON gaia_memory_conversations(timestamp); - --- 3. Result Cache Table -CREATE TABLE IF NOT EXISTS gaia_memory_cache ( - id UUID PRIMARY KEY DEFAULT uuid_generate_v4(), - key TEXT NOT NULL UNIQUE, - value TEXT NOT NULL, - metadata JSONB, - created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), - expires_at TIMESTAMPTZ -); - -CREATE INDEX IF NOT EXISTS idx_gaia_memory_cache_key ON gaia_memory_cache(key); -CREATE INDEX IF NOT EXISTS idx_gaia_memory_cache_expires_at ON gaia_memory_cache(expires_at); - --- 4. Working Memory Table -CREATE TABLE IF NOT EXISTS gaia_memory_working ( - id UUID PRIMARY KEY DEFAULT uuid_generate_v4(), - session_id TEXT NOT NULL, - key TEXT NOT NULL, - value TEXT NOT NULL, - metadata JSONB, - created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), - expires_at TIMESTAMPTZ -); - -CREATE UNIQUE INDEX IF NOT EXISTS idx_gaia_memory_working_session_key ON gaia_memory_working(session_id, key); -CREATE INDEX IF NOT EXISTS idx_gaia_memory_working_session_id ON gaia_memory_working(session_id); - --- 5. Create a test function to verify connection -CREATE OR REPLACE FUNCTION get_service_status() -RETURNS jsonb -LANGUAGE sql -AS $$ - SELECT jsonb_build_object( - 'status', 'ok', - 'timestamp', NOW()::text, - 'message', 'Supabase service is running' - ); -$$; - --- 6. Create a test table for connection verification -CREATE TABLE IF NOT EXISTS test_connection ( - id INT PRIMARY KEY, - name TEXT NOT NULL, - created_at TIMESTAMPTZ NOT NULL DEFAULT NOW() -); \ No newline at end of file diff --git a/test_components.py b/test_components.py deleted file mode 100644 index da59637e9fa7888006d5d1e179c8bc025ae6d89e..0000000000000000000000000000000000000000 --- a/test_components.py +++ /dev/null @@ -1,391 +0,0 @@ -#!/usr/bin/env python -""" -Component Verification Script for GAIA Deployment - -This script performs lightweight validation of GAIA components without making -actual API calls, to verify structural integrity before deployment. - -Usage: - python test_components.py [--verbose] - -Author: DevOps Team -Date: April 30, 2025 -""" - -import os -import sys -import importlib -import logging -import pkgutil -import inspect -import argparse -from pathlib import Path -from typing import Dict, List, Any, Tuple, Optional - -# Configure logging -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("component_test") - -def parse_arguments(): - """Parse command-line arguments""" - parser = argparse.ArgumentParser(description="Verify GAIA components for deployment") - parser.add_argument("--verbose", action="store_true", help="Enable verbose output") - return parser.parse_args() - -class ComponentVerifier: - """Verifies GAIA system components""" - - def __init__(self, verbose: bool = False): - """Initialize component verifier""" - self.verbose = verbose - self.results = { - "structure": {}, - "imports": {}, - "modules": {}, - "components": {}, - "config": {} - } - - def log(self, message: str, level: str = "info"): - """Log messages when verbose is enabled""" - if self.verbose: - if level == "info": - logger.info(message) - elif level == "warning": - logger.warning(message) - elif level == "error": - logger.error(message) - - def check_directory_structure(self) -> Dict[str, bool]: - """Check if required directories exist""" - required_dirs = [ - "agent", - "memory", - "tools", - "utils", - "docs", - "tests" - ] - - results = {} - self.log("Checking directory structure...") - - for directory in required_dirs: - exists = os.path.isdir(directory) - results[directory] = exists - self.log(f"Directory '{directory}': {'✅' if exists else '❌'}") - - self.results["structure"] = results - return results - - def check_required_files(self) -> Dict[str, bool]: - """Check if required files exist""" - required_files = [ - "app.py", - "requirements.txt", - "agent/agent.py", - "agent/config.py", - "agent/graph.py", - "agent/tool_registry.py", - "memory/supabase_memory.py", - "tools/web_tools.py" - ] - - results = {} - self.log("Checking required files...") - - for file_path in required_files: - exists = os.path.isfile(file_path) - results[file_path] = exists - self.log(f"File '{file_path}': {'✅' if exists else '❌'}") - - self.results["files"] = results - return results - - def check_imports(self) -> Dict[str, bool]: - """Check if required modules can be imported""" - required_modules = [ - "openai", - "supabase", - "langchain", - "gradio", - "requests", - "dotenv" - ] - - results = {} - self.log("Checking imports...") - - for module_name in required_modules: - try: - importlib.import_module(module_name) - results[module_name] = True - self.log(f"Import '{module_name}': ✅") - except ImportError: - results[module_name] = False - self.log(f"Import '{module_name}': ❌ (not installed)", "warning") - - self.results["imports"] = results - return results - - def check_module_structure(self, module_path: str, module_name: str) -> Dict[str, bool]: - """Check if a module has required attributes/classes""" - results = {} - - try: - # Try to import the module - if module_path: - sys.path.insert(0, module_path) - - module = importlib.import_module(module_name) - - # Get module attributes - attributes = dir(module) - results["imported"] = True - results["attributes"] = attributes - - self.log(f"Module '{module_name}' imported successfully") - self.log(f"Attributes: {attributes[:5]}... (and {len(attributes) - 5} more)") - - if module_path: - sys.path.pop(0) - - except ImportError as e: - results["imported"] = False - results["error"] = str(e) - self.log(f"Failed to import '{module_name}': {e}", "error") - - return results - - def check_agent_components(self) -> Dict[str, Any]: - """Check agent components""" - results = {} - - # Check agent module - agent_results = self.check_module_structure(".", "agent.agent") - results["agent_module"] = agent_results - - # Check config module - config_results = self.check_module_structure(".", "agent.config") - results["config_module"] = config_results - - # Check graph module - graph_results = self.check_module_structure(".", "agent.graph") - results["graph_module"] = graph_results - - # Check tool registry - tool_registry_results = self.check_module_structure(".", "agent.tool_registry") - results["tool_registry"] = tool_registry_results - - self.results["agent_components"] = results - return results - - def check_memory_components(self) -> Dict[str, Any]: - """Check memory components""" - results = {} - - # Check supabase memory module - memory_results = self.check_module_structure(".", "memory.supabase_memory") - results["supabase_memory"] = memory_results - - self.results["memory_components"] = results - return results - - def check_tool_components(self) -> Dict[str, Any]: - """Check tool components""" - results = {} - - # Check web tools module - web_tools_results = self.check_module_structure(".", "tools.web_tools") - results["web_tools"] = web_tools_results - - # Check for other tool modules - tool_modules = [] - tools_dir = Path("tools") - if tools_dir.exists() and tools_dir.is_dir(): - for file_path in tools_dir.glob("*.py"): - if file_path.name != "__init__.py": - tool_modules.append(file_path.stem) - - results["available_tools"] = tool_modules - self.log(f"Available tool modules: {tool_modules}") - - self.results["tool_components"] = results - return results - - def check_config_values(self) -> Dict[str, Any]: - """Check config values without requiring actual API keys""" - results = {} - - try: - from agent.config import ( - OPENAI_API_KEY, SUPABASE_URL, SUPABASE_KEY, - SERPER_API_KEY, DEFAULT_MODEL, EMBEDDING_MODEL - ) - - # Check if config variables are defined (don't check actual values) - results["openai_key_defined"] = OPENAI_API_KEY is not None - results["supabase_url_defined"] = SUPABASE_URL is not None - results["supabase_key_defined"] = SUPABASE_KEY is not None - results["serper_key_defined"] = SERPER_API_KEY is not None - results["default_model_defined"] = DEFAULT_MODEL is not None - results["embedding_model_defined"] = EMBEDDING_MODEL is not None - - self.log(f"Config values check results: {results}") - - except ImportError as e: - results["error"] = str(e) - self.log(f"Failed to import config values: {e}", "error") - except Exception as e: - results["error"] = str(e) - self.log(f"Error checking config values: {e}", "error") - - self.results["config_values"] = results - return results - - def run_all_checks(self) -> Dict[str, Any]: - """Run all component verification checks""" - logger.info("Starting component verification...") - - # Run all checks - self.check_directory_structure() - self.check_required_files() - self.check_imports() - self.check_agent_components() - self.check_memory_components() - self.check_tool_components() - self.check_config_values() - - # Calculate overall success - structure_success = all(self.results.get("structure", {}).values()) - files_success = all(self.results.get("files", {}).values()) - agent_success = all(result.get("imported", False) - for result in self.results.get("agent_components", {}).values()) - memory_success = all(result.get("imported", False) - for result in self.results.get("memory_components", {}).values()) - tool_success = all(result.get("imported", False) - for key, result in self.results.get("tool_components", {}).items() - if key != "available_tools") - - overall_success = structure_success and files_success and agent_success and memory_success and tool_success - - self.results["overall"] = { - "success": overall_success, - "structure_success": structure_success, - "files_success": files_success, - "agent_success": agent_success, - "memory_success": memory_success, - "tool_success": tool_success - } - - logger.info(f"Component verification completed. Overall success: {'✅' if overall_success else '❌'}") - - return self.results - - def format_results(self) -> str: - """Format verification results into readable format""" - output = [] - output.append("# GAIA Component Verification Results\n") - - # Overall result - overall = self.results.get("overall", {}) - overall_success = overall.get("success", False) - output.append(f"## Overall Result: {'✅ PASS' if overall_success else '❌ FAIL'}\n") - - # Directory structure - output.append("## Directory Structure\n") - for dir_name, exists in self.results.get("structure", {}).items(): - output.append(f"- {dir_name}: {'✅' if exists else '❌'}") - output.append("") - - # Required files - output.append("## Required Files\n") - for file_path, exists in self.results.get("files", {}).items(): - output.append(f"- {file_path}: {'✅' if exists else '❌'}") - output.append("") - - # Imports - output.append("## Required Imports\n") - for module_name, imported in self.results.get("imports", {}).items(): - output.append(f"- {module_name}: {'✅' if imported else '❌'}") - output.append("") - - # Agent components - output.append("## Agent Components\n") - for component, result in self.results.get("agent_components", {}).items(): - output.append(f"- {component}: {'✅' if result.get('imported', False) else '❌'}") - output.append("") - - # Memory components - output.append("## Memory Components\n") - for component, result in self.results.get("memory_components", {}).items(): - output.append(f"- {component}: {'✅' if result.get('imported', False) else '❌'}") - output.append("") - - # Tool components - output.append("## Tool Components\n") - for component, result in self.results.get("tool_components", {}).items(): - if component == "available_tools": - output.append(f"- Available tools: {', '.join(result)}") - else: - output.append(f"- {component}: {'✅' if result.get('imported', False) else '❌'}") - output.append("") - - # Config values - output.append("## Configuration\n") - config_values = self.results.get("config_values", {}) - if "error" in config_values: - output.append(f"- Error: {config_values['error']}") - else: - for config_name, defined in config_values.items(): - if config_name != "error": - output.append(f"- {config_name}: {'✅' if defined else '❌'}") - output.append("") - - # Summary - output.append("## Summary\n") - output.append(f"- Structure check: {'✅' if overall.get('structure_success', False) else '❌'}") - output.append(f"- Files check: {'✅' if overall.get('files_success', False) else '❌'}") - output.append(f"- Agent components: {'✅' if overall.get('agent_success', False) else '❌'}") - output.append(f"- Memory components: {'✅' if overall.get('memory_success', False) else '❌'}") - output.append(f"- Tool components: {'✅' if overall.get('tool_success', False) else '❌'}") - - return "\n".join(output) - -def save_results(results: Dict[str, Any], verifier: ComponentVerifier) -> str: - """Save verification results to a file""" - formatted_results = verifier.format_results() - - output_file = "component_verification_results.md" - with open(output_file, "w") as f: - f.write(formatted_results) - - logger.info(f"Results saved to {output_file}") - return output_file - -def main(): - """Main function""" - args = parse_arguments() - - logger.info("Starting GAIA component verification...") - verifier = ComponentVerifier(verbose=args.verbose) - results = verifier.run_all_checks() - - output_file = save_results(results, verifier) - - # Print summary - overall_success = results.get("overall", {}).get("success", False) - if overall_success: - logger.info("✅ All component checks passed!") - else: - logger.warning("❌ Some component checks failed. See results for details.") - - logger.info(f"Detailed results are available in {output_file}") - - return 0 if overall_success else 1 - -if __name__ == "__main__": - sys.exit(main()) \ No newline at end of file diff --git a/test_deployment_readiness.py b/test_deployment_readiness.py deleted file mode 100644 index af5a503f298161e0d8f254025343f8895d9d8601..0000000000000000000000000000000000000000 --- a/test_deployment_readiness.py +++ /dev/null @@ -1,457 +0,0 @@ -""" -Comprehensive pre-deployment testing script for GAIA Agent - -This script tests all aspects of the GAIA Agent to ensure it's ready for deployment: -1. Environment variables check -2. OpenAI API integration -3. Supabase memory integration -4. Search tools (Serper, DuckDuckGo) -5. Agent graph workflow -6. Multi-turn conversation with memory retention - -Usage: - python test_deployment_readiness.py -""" - -import os -import sys -import logging -import time -import json -from typing import Dict, Any, List, Optional, Tuple, Union -from datetime import datetime - -# Configure logging -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(name)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("deployment_test") - -# Set default values for environment variables that might be missing -OPENAI_API_KEY = os.getenv("OPENAI_API_KEY", "") -SUPABASE_URL = os.getenv("SUPABASE_URL", "") -SUPABASE_KEY = os.getenv("SUPABASE_KEY", "") -SERPER_API_KEY = os.getenv("SERPER_API_KEY", "") - -# Import GAIA components with robust error handling -IMPORT_ERRORS = [] - -try: - from agent import GAIAAgent -except ImportError as e: - IMPORT_ERRORS.append(f"GAIAAgent: {str(e)}") - GAIAAgent = None - -try: - from agent.config import ( - get_memory_config, - get_model_config, - get_tool_config - ) -except ImportError as e: - IMPORT_ERRORS.append(f"agent.config: {str(e)}") - def get_memory_config(): return {} - def get_model_config(): return {} - def get_tool_config(): return {} - -try: - from memory.supabase_memory import SupabaseMemory -except ImportError as e: - IMPORT_ERRORS.append(f"SupabaseMemory: {str(e)}") - SupabaseMemory = None - -try: - from memory.supabase_memory_fix import apply_fix as apply_memory_fix -except ImportError as e: - IMPORT_ERRORS.append(f"supabase_memory_fix: {str(e)}") - def apply_memory_fix(): - logger.warning("Memory fix function not available") - return False - -if IMPORT_ERRORS: - for error in IMPORT_ERRORS: - logger.warning(f"Import warning: {error}") - -def check_environment_variables() -> bool: - """Check if all required environment variables are set.""" - logger.info("Checking environment variables...") - - required_vars = { - "OPENAI_API_KEY": OPENAI_API_KEY, - "SUPABASE_URL": SUPABASE_URL, - "SUPABASE_KEY": SUPABASE_KEY, - "SERPER_API_KEY": SERPER_API_KEY - } - - optional_vars = ["SERPER_API_KEY"] - missing_required = [] - missing_optional = [] - - for var_name, var_value in required_vars.items(): - if not var_value: - if var_name in optional_vars: - missing_optional.append(var_name) - logger.warning(f"⚠️ Missing optional environment variable: {var_name}") - else: - missing_required.append(var_name) - logger.warning(f"❌ Missing required environment variable: {var_name}") - else: - # Mask the actual value for security - masked_value = var_value[:4] + "*" * (len(var_value) - 8) + var_value[-4:] if len(var_value) > 8 else "****" - logger.info(f"✅ Environment variable set: {var_name}={masked_value}") - - if missing_required: - logger.error(f"Missing required environment variables: {', '.join(missing_required)}") - return False - - if missing_optional: - logger.warning(f"Missing optional environment variables: {', '.join(missing_optional)}") - - if not missing_required: - logger.info("✅ All required environment variables are set") - return True - - return False - -def test_openai_integration() -> bool: - """Test the OpenAI API integration.""" - logger.info("Testing OpenAI API integration...") - - if not OPENAI_API_KEY: - logger.warning("Skipping OpenAI test: API key not provided") - return False - - try: - from openai import OpenAI - - client = OpenAI(api_key=OPENAI_API_KEY) - - start_time = time.time() - response = client.chat.completions.create( - model="gpt-3.5-turbo", - messages=[{"role": "user", "content": "Hello, this is a test!"}], - max_tokens=50 - ) - elapsed_time = time.time() - start_time - - content = response.choices[0].message.content - logger.info(f"✅ OpenAI API response received in {elapsed_time:.2f}s: {content[:50]}...") - return True - except Exception as e: - logger.error(f"❌ OpenAI API test failed: {str(e)}") - return False - -def test_supabase_integration() -> bool: - """Test the Supabase memory integration.""" - logger.info("Testing Supabase memory integration...") - - if not SUPABASE_URL or not SUPABASE_KEY: - logger.warning("Skipping Supabase test: URL or key not provided") - return False - - try: - # Apply memory fix first - logger.info("Applying memory fix...") - if not apply_memory_fix(): - logger.error("❌ Failed to apply memory fix") - return False - - memory_config = get_memory_config() - memory_config["enabled"] = True - - base_memory = SupabaseMemory(config=memory_config) - - if not base_memory.client or not base_memory.initialized: - logger.error("❌ Supabase client not initialized") - return False - - test_key = f"test-deployment-{datetime.now().isoformat()}" - test_value = {"message": "This is a test value", "timestamp": datetime.now().isoformat()} - - logger.info(f"Storing test value with key: {test_key}") - if not base_memory.store(test_key, test_value): - logger.error("❌ Failed to store test value") - return False - - retrieved_value = base_memory.retrieve(test_key) - if retrieved_value != test_value: - logger.error(f"❌ Retrieved value does not match: {retrieved_value}") - return False - - logger.info("Cleaning up test data...") - base_memory.delete(test_key) - - logger.info("✅ Supabase memory integration test passed") - return True - except Exception as e: - logger.error(f"❌ Supabase integration test failed: {str(e)}") - return False - -def test_search_tool_integration() -> bool: - """Test the search tool integrations.""" - logger.info("Testing search tool integrations...") - - search_success = False - - # Test Serper integration if key is available - if SERPER_API_KEY: - try: - from tools.web_tools import SerperSearchTool - - logger.info("Testing Serper search tool...") - serper_tool = SerperSearchTool() - - query = "latest news about artificial intelligence" - logger.info(f"Executing search query: '{query}'") - - results = serper_tool.search(query, num_results=2) - - if results and len(results) > 0: - logger.info(f"✅ Serper search returned {len(results)} results") - sample_result = results[0] - logger.info(f"Sample result title: {sample_result.get('title', 'N/A')}") - search_success = True - else: - logger.warning("❌ Serper search returned no results") - except Exception as e: - logger.error(f"❌ Serper search test failed: {str(e)}") - else: - logger.warning("Skipping Serper test: API key not provided") - - # Test DuckDuckGo integration (doesn't require API key) - try: - from tools.web_tools import DuckDuckGoSearchTool - - logger.info("Testing DuckDuckGo search tool...") - duckduckgo_tool = DuckDuckGoSearchTool() - - query = "latest news about artificial intelligence" - logger.info(f"Executing search query: '{query}'") - - results = duckduckgo_tool.search(query, num_results=2) - - if results and len(results) > 0: - logger.info(f"✅ DuckDuckGo search returned {len(results)} results") - sample_result = results[0] - logger.info(f"Sample result title: {sample_result.get('title', 'N/A')}") - search_success = True - else: - logger.warning("❌ DuckDuckGo search returned no results") - except Exception as e: - logger.error(f"❌ DuckDuckGo search test failed: {str(e)}") - - return search_success - -def test_agent_graph_workflow() -> bool: - """Test the agent graph workflow.""" - logger.info("Testing agent graph workflow...") - - try: - from agent.graph import GAIAGraph - - logger.info("Creating test agent graph...") - graph = GAIAGraph() - - logger.info("Testing graph execution with a simple query...") - query = "What is the capital of France?" - - start_time = time.time() - result = graph.execute(query) - elapsed_time = time.time() - start_time - - if result and isinstance(result, str) and "Paris" in result: - logger.info(f"✅ Graph execution succeeded in {elapsed_time:.2f}s with correct answer") - return True - else: - logger.warning(f"❌ Graph execution produced unexpected result: {result[:100]}...") - return False - except Exception as e: - logger.error(f"❌ Agent graph workflow test failed: {str(e)}") - return False - -def test_multi_turn_conversation() -> bool: - """Test multi-turn conversation with memory retention.""" - logger.info("Testing multi-turn conversation with memory retention...") - - try: - agent = GAIAAgent() - - logger.info("Starting conversation with initial query...") - queries = [ - "My name is Test User. What is the capital of France?", - "What is the population of this city?", - "What is my name?", - ] - - responses = [] - - for i, query in enumerate(queries): - logger.info(f"Turn {i+1}: {query}") - start_time = time.time() - response = agent.run(query) - elapsed_time = time.time() - start_time - - responses.append(response) - logger.info(f"Response received in {elapsed_time:.2f}s: {response[:100]}...") - - # Check if memory retention worked by seeing if the agent remembered the user's name - if "Test User" in responses[2]: - logger.info("✅ Memory retention confirmed: agent remembered user's name") - return True - else: - logger.warning("❌ Memory retention test failed: agent didn't remember user's name") - return False - except Exception as e: - logger.error(f"❌ Multi-turn conversation test failed: {str(e)}") - return False - -def test_performance() -> bool: - """Test performance and check for bottlenecks.""" - logger.info("Testing performance...") - - try: - agent = GAIAAgent() - - logger.info("Testing response time for a complex query...") - query = "Explain the process of photosynthesis in plants." - - start_time = time.time() - response = agent.run(query) - elapsed_time = time.time() - start_time - - logger.info(f"Response received in {elapsed_time:.2f}s") - logger.info(f"Response length: {len(response)} characters") - - # Arbitrary performance threshold of 30 seconds - if elapsed_time < 30: - logger.info(f"✅ Performance test passed: response received in {elapsed_time:.2f}s") - return True - else: - logger.warning(f"❌ Performance test flagged: response took {elapsed_time:.2f}s (> 30s)") - return False - except Exception as e: - logger.error(f"❌ Performance test failed: {str(e)}") - return False - -def main(): - """Main entry point for the test script.""" - logger.info("===== GAIA Agent Deployment Readiness Test =====") - logger.info(f"Starting tests at: {datetime.now().isoformat()}") - - # Check for import errors first - if IMPORT_ERRORS: - logger.warning("\n===== Import Warnings =====") - for error in IMPORT_ERRORS: - logger.warning(f"⚠️ {error}") - logger.warning("Some functionality may be limited due to import issues") - - # Track test results - test_results = {} - - # Define tests with dependencies check - all_tests = [ - ("Environment Variables", check_environment_variables, []), - ("OpenAI Integration", test_openai_integration, ["OPENAI_API_KEY"]), - ("Supabase Memory Integration", test_supabase_integration, ["SUPABASE_URL", "SUPABASE_KEY", "SupabaseMemory"]), - ("Search Tool Integration", test_search_tool_integration, []), - ("Agent Graph Workflow", test_agent_graph_workflow, ["GAIAAgent"]), - ("Multi-turn Conversation", test_multi_turn_conversation, ["GAIAAgent"]), - ("Performance", test_performance, ["GAIAAgent"]), - ] - - # Determine which tests to run based on available dependencies - tests_to_run = [] - for test_name, test_func, dependencies in all_tests: - skip_test = False - missing_deps = [] - - # Check environment variable dependencies - for dep in dependencies: - if dep == "OPENAI_API_KEY" and not OPENAI_API_KEY: - missing_deps.append(dep) - elif dep == "SUPABASE_URL" and not SUPABASE_URL: - missing_deps.append(dep) - elif dep == "SUPABASE_KEY" and not SUPABASE_KEY: - missing_deps.append(dep) - elif dep == "SupabaseMemory" and SupabaseMemory is None: - missing_deps.append(dep) - elif dep == "GAIAAgent" and GAIAAgent is None: - missing_deps.append(dep) - - if missing_deps: - logger.warning(f"⚠️ Skipping '{test_name}' test due to missing dependencies: {', '.join(missing_deps)}") - test_results[test_name] = { - "success": False, - "time": None, - "skipped": True, - "reason": f"Missing dependencies: {', '.join(missing_deps)}" - } - else: - tests_to_run.append((test_name, test_func)) - - # Run eligible tests - for test_name, test_func in tests_to_run: - logger.info(f"\n===== Testing: {test_name} =====") - try: - start_time = time.time() - result = test_func() - elapsed_time = time.time() - start_time - - test_results[test_name] = { - "success": result, - "time": elapsed_time, - "skipped": False - } - - if result: - logger.info(f"✅ {test_name} test PASSED in {elapsed_time:.2f}s") - else: - logger.warning(f"❌ {test_name} test FAILED in {elapsed_time:.2f}s") - except Exception as e: - logger.error(f"❌ {test_name} test ERROR: {str(e)}") - test_results[test_name] = { - "success": False, - "time": None, - "skipped": False, - "error": str(e) - } - - # Summarize results - logger.info("\n===== Test Results Summary =====") - - passed = sum(1 for r in test_results.values() if r.get("success", False)) - failed = sum(1 for r in test_results.values() if not r.get("success", False) and not r.get("skipped", False)) - skipped = sum(1 for r in test_results.values() if r.get("skipped", False)) - - for test_name, result in test_results.items(): - if result.get("skipped", False): - status = "⚠️ SKIPPED" - reason = f" - {result.get('reason', 'No reason provided')}" - else: - status = "✅ PASSED" if result.get("success", False) else "❌ FAILED" - reason = "" - if "error" in result: - reason = f" - Error: {result['error']}" - - time_str = f" in {result.get('time', 0):.2f}s" if result.get("time") is not None else "" - logger.info(f"{status}: {test_name}{time_str}{reason}") - - logger.info(f"\nSummary: {passed} passed, {failed} failed, {skipped} skipped") - - if failed == 0: - if passed > 0: - if skipped > 0: - logger.info("🚀 All attempted tests passed, but some were skipped. The GAIA Agent is partially ready for deployment.") - else: - logger.info("🚀 All tests passed! The GAIA Agent is ready for deployment.") - return 0 - else: - logger.warning("⚠️ No tests were run successfully. Please check dependencies and try again.") - return 1 - else: - logger.warning("⚠️ Some tests failed. Please address the issues before deployment.") - return 1 - -if __name__ == "__main__": - sys.exit(main()) \ No newline at end of file diff --git a/test_openai_minimal.py b/test_openai_minimal.py deleted file mode 100644 index ba223e35c2bdbfe43a9357945bee391f1d5729c5..0000000000000000000000000000000000000000 --- a/test_openai_minimal.py +++ /dev/null @@ -1,38 +0,0 @@ - -import os -import openai -from openai import OpenAI - -def test_openai_connection(): - print("Testing OpenAI connection...") - api_key = os.getenv("OPENAI_API_KEY") - if not api_key: - print("ERROR: OPENAI_API_KEY not set") - return False - - try: - client = OpenAI(api_key=api_key) - models = client.models.list() - print(f"Successfully connected to OpenAI API. Available models: {len(models.data)}") - - # Test a simple completion - response = client.chat.completions.create( - model="gpt-3.5-turbo", - messages=[ - {"role": "system", "content": "You are a helpful assistant."}, - {"role": "user", "content": "Hello, world!"} - ] - ) - print(f"Response: {response.choices[0].message.content[:50]}...") - return True - except Exception as e: - print(f"Error connecting to OpenAI: {str(e)}") - return False - -if __name__ == "__main__": - if test_openai_connection(): - print("OpenAI test passed!") - exit(0) - else: - print("OpenAI test failed!") - exit(1) diff --git a/test_serper_minimal.py b/test_serper_minimal.py deleted file mode 100644 index 9e7876ff4099355eb91cb73fc0ed2c2d50af6bab..0000000000000000000000000000000000000000 --- a/test_serper_minimal.py +++ /dev/null @@ -1,43 +0,0 @@ - -import os -import json -import requests - -def test_serper_connection(): - print("Testing Serper API connection...") - api_key = os.getenv("SERPER_API_KEY") - if not api_key: - print("ERROR: SERPER_API_KEY not set") - return False - - try: - url = "https://google.serper.dev/search" - payload = json.dumps({ - "q": "Who is the current president of the United States", - "gl": "us", - "hl": "en" - }) - headers = { - 'X-API-KEY': api_key, - 'Content-Type': 'application/json' - } - - response = requests.request("POST", url, headers=headers, data=payload) - - if response.status_code == 200: - print(f"Successfully connected to Serper API. Response: {response.text[:100]}...") - return True - else: - print(f"Error from Serper API: {response.status_code} - {response.text}") - return False - except Exception as e: - print(f"Error connecting to Serper API: {str(e)}") - return False - -if __name__ == "__main__": - if test_serper_connection(): - print("Serper test passed!") - exit(0) - else: - print("Serper test failed!") - exit(1) diff --git a/test_supabase_minimal.py b/test_supabase_minimal.py deleted file mode 100644 index a9cc9f71a07597d4300eb016552ec34147efd759..0000000000000000000000000000000000000000 --- a/test_supabase_minimal.py +++ /dev/null @@ -1,133 +0,0 @@ -import os -import sys -import json -from supabase import create_client, Client - -def load_dotenv(): - """Load environment variables from .env file""" - env_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), '.env') - if not os.path.exists(env_path): - print(f"ERROR: .env file not found at {env_path}") - return False - - print(f"Loading environment variables from {env_path}") - - with open(env_path, 'r') as file: - for line in file: - line = line.strip() - if not line or line.startswith('#'): - continue - - key, value = line.split('=', 1) - os.environ[key] = value - - return True - -def test_supabase_connection(): - print("Testing Supabase connection...") - supabase_url = os.getenv("SUPABASE_URL") - supabase_key = os.getenv("SUPABASE_KEY") - - if not supabase_url or not supabase_key: - print("ERROR: SUPABASE_URL or SUPABASE_KEY not set") - return False - - print(f"Using Supabase URL: {supabase_url[:15]}...{supabase_url[-10:]}") - print(f"Using Supabase Key: {supabase_key[:10]}...{supabase_key[-5:]}") - - try: - # Connect to Supabase - supabase: Client = create_client(supabase_url, supabase_key) - print(f"Connected to Supabase at {supabase_url}") - - # Check for the required tables - using a different approach - # Instead of querying postgres_version, query information_schema.tables - tables_query = supabase.table("information_schema.tables")\ - .select("table_name")\ - .eq("table_schema", "public")\ - .execute() - - table_list = [record.get('table_name') for record in tables_query.data] - print(f"Available tables: {', '.join(table_list)}") - - # Check for our specific tables - memory_tables = ["gaia_memory", "gaia_memory_working", "gaia_memory_conversation"] - missing_tables = [] - - for table in memory_tables: - if table not in table_list: - missing_tables.append(table) - - if missing_tables: - print(f"Missing tables: {', '.join(missing_tables)}") - print("Creating missing tables...") - - # Create tables if they don't exist - for table in missing_tables: - try: - # Define table schema based on table name - if table == "gaia_memory": - # Base memory table - supabase.table(table).insert({ - "session_id": "test-session", - "key": "test-key", - "value": json.dumps({"test": "value"}), - "metadata": json.dumps({"created_at": "now"}) - }).execute() - print(f"Created table: {table}") - elif table == "gaia_memory_working": - # Working memory table - supabase.table(table).insert({ - "session_id": "test-session", - "key": "test-key", - "value": json.dumps({"test": "value"}), - "metadata": json.dumps({"created_at": "now"}) - }).execute() - print(f"Created table: {table}") - elif table == "gaia_memory_conversation": - # Conversation memory table - supabase.table(table).insert({ - "session_id": "test-session", - "key": "test-key", - "messages": json.dumps([{"role": "user", "content": "test"}]), - "metadata": json.dumps({"created_at": "now"}) - }).execute() - print(f"Created table: {table}") - except Exception as e: - print(f"Error creating table {table}: {str(e)}") - if "duplicate key value" in str(e): - print(f"Table {table} exists but may have different structure") - continue - if "does not exist" in str(e): - print(f"Table {table} does not exist and could not be created automatically") - print("Please create the required tables manually or run the table creation script") - return False - else: - print("All required tables exist") - - # Test querying each table - print("\nTesting table queries:") - for table in memory_tables: - if table in table_list: - try: - result = supabase.table(table).select("*").limit(1).execute() - print(f"Table {table}: {len(result.data)} records") - except Exception as e: - print(f"Error querying table {table}: {str(e)}") - return False - - return True - except Exception as e: - print(f"Error connecting to Supabase: {str(e)}") - return False - -if __name__ == "__main__": - # Load environment variables from .env file - load_dotenv() - - if test_supabase_connection(): - print("Supabase test passed!") - exit(0) - else: - print("Supabase test failed!") - exit(1) diff --git a/tests/__init__.py b/tests/__init__.py deleted file mode 100644 index 3f9e92055549237ec3c83a229fda7a69b78e1367..0000000000000000000000000000000000000000 --- a/tests/__init__.py +++ /dev/null @@ -1,4 +0,0 @@ -""" -Tests module for GAIA implementation. -Contains unit tests and integration tests for all components of the GAIA agent. -""" \ No newline at end of file diff --git a/tests/__pycache__/__init__.cpython-312.pyc b/tests/__pycache__/__init__.cpython-312.pyc deleted file mode 100644 index 0111ac8f5c9e29c1aed4d96eb6a8c7b41c5828e1..0000000000000000000000000000000000000000 Binary files a/tests/__pycache__/__init__.cpython-312.pyc and /dev/null differ diff --git a/tests/__pycache__/__init__.cpython-313.pyc b/tests/__pycache__/__init__.cpython-313.pyc deleted file mode 100644 index 0ebd9cf01da1e5d4cd4193fcd3939906ac1f7cf6..0000000000000000000000000000000000000000 Binary files a/tests/__pycache__/__init__.cpython-313.pyc and /dev/null differ diff --git a/tests/__pycache__/test_config.cpython-312.pyc b/tests/__pycache__/test_config.cpython-312.pyc deleted file mode 100644 index 335fa567663d64b8775c18a60b8ed3b6a425704b..0000000000000000000000000000000000000000 Binary files a/tests/__pycache__/test_config.cpython-312.pyc and /dev/null differ diff --git a/tests/__pycache__/test_duckduckgo_search.cpython-313.pyc b/tests/__pycache__/test_duckduckgo_search.cpython-313.pyc deleted file mode 100644 index 907b84fd6c620fc14bb8332a81ce41c1573ef33e..0000000000000000000000000000000000000000 Binary files a/tests/__pycache__/test_duckduckgo_search.cpython-313.pyc and /dev/null differ diff --git a/tests/__pycache__/test_gaia_multi_search_providers.cpython-313.pyc b/tests/__pycache__/test_gaia_multi_search_providers.cpython-313.pyc deleted file mode 100644 index a107c0d03e0f5447146c85732d9f655cf23b0dab..0000000000000000000000000000000000000000 Binary files a/tests/__pycache__/test_gaia_multi_search_providers.cpython-313.pyc and /dev/null differ diff --git a/tests/__pycache__/test_perplexity_tool.cpython-313.pyc b/tests/__pycache__/test_perplexity_tool.cpython-313.pyc deleted file mode 100644 index 7edd7a92ca33bdf9b9e4fa43b1e7fd2b0cf5101f..0000000000000000000000000000000000000000 Binary files a/tests/__pycache__/test_perplexity_tool.cpython-313.pyc and /dev/null differ diff --git a/tests/__pycache__/test_project_structure.cpython-312.pyc b/tests/__pycache__/test_project_structure.cpython-312.pyc deleted file mode 100644 index c98e763fc3800fec0034e5d39b122b1efde3c32b..0000000000000000000000000000000000000000 Binary files a/tests/__pycache__/test_project_structure.cpython-312.pyc and /dev/null differ diff --git a/tests/__pycache__/test_project_structure.cpython-313.pyc b/tests/__pycache__/test_project_structure.cpython-313.pyc deleted file mode 100644 index bbe3cf8295912f4c172487f8b5ccf78d4e75c751..0000000000000000000000000000000000000000 Binary files a/tests/__pycache__/test_project_structure.cpython-313.pyc and /dev/null differ diff --git a/tests/__pycache__/test_supabase_tables_production.cpython-313.pyc b/tests/__pycache__/test_supabase_tables_production.cpython-313.pyc deleted file mode 100644 index c3cbd349f82b0aa0cb81710789616ddc5fd3926f..0000000000000000000000000000000000000000 Binary files a/tests/__pycache__/test_supabase_tables_production.cpython-313.pyc and /dev/null differ diff --git a/tests/benchmark.py b/tests/benchmark.py deleted file mode 100644 index 246daf1ef9350581791ccb4dbf8107ddccef113f..0000000000000000000000000000000000000000 --- a/tests/benchmark.py +++ /dev/null @@ -1,540 +0,0 @@ -""" -Benchmark testing framework for the GAIA agent. - -This module provides tools for evaluating the performance of the GAIA agent -on a set of sample questions. It measures accuracy, response time, and resource -usage, and generates detailed reports of the results. -""" - -import argparse -import json -import logging -import os -import time -import datetime -import statistics -from typing import Dict, Any, List, Optional, Union, Tuple, Set -import psutil -import matplotlib.pyplot as plt -from tabulate import tabulate - -from agent import agent -from agent.config import get_model_config - -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(name)s - %(levelname)s - %(message)s", - handlers=[ - logging.FileHandler("benchmark.log"), - logging.StreamHandler() - ] -) -logger = logging.getLogger("gaia_agent.benchmark") - -SAMPLE_QUESTIONS = [ - { - "id": "q1", - "question": "What is the capital of France?", - "expected_answer": "Paris", - "category": "geography", - "complexity": 1 - }, - { - "id": "q2", - "question": "Who wrote the novel '1984'?", - "expected_answer": "George Orwell", - "category": "literature", - "complexity": 1 - }, - { - "id": "q3", - "question": "What is the square root of 144?", - "expected_answer": "12", - "category": "mathematics", - "complexity": 1 - }, - { - "id": "q4", - "question": "If a train travels at 120 km/h for 2.5 hours, how far does it travel?", - "expected_answer": "300 km", - "category": "mathematics", - "complexity": 2 - }, - { - "id": "q5", - "question": "What are the three primary colors?", - "expected_answer": "red, blue, yellow", - "category": "general_knowledge", - "complexity": 1 - }, - { - "id": "q6", - "question": "What is the chemical symbol for gold?", - "expected_answer": "Au", - "category": "science", - "complexity": 1 - }, - { - "id": "q7", - "question": "Who painted the Mona Lisa?", - "expected_answer": "Leonardo da Vinci", - "category": "art", - "complexity": 1 - }, - { - "id": "q8", - "question": "What is the largest planet in our solar system?", - "expected_answer": "Jupiter", - "category": "astronomy", - "complexity": 1 - }, - { - "id": "q9", - "question": "What year did World War II end?", - "expected_answer": "1945", - "category": "history", - "complexity": 1 - }, - { - "id": "q10", - "question": "What is the boiling point of water in Celsius?", - "expected_answer": "100", - "category": "science", - "complexity": 1 - } -] - - -class BenchmarkResult: - """Class for storing and analyzing benchmark results.""" - - def __init__(self, questions: List[Dict[str, Any]], config: Dict[str, Any]): - """ - Initialize the benchmark result. - - Args: - questions: List of questions used in the benchmark - config: Configuration used for the benchmark - """ - self.questions = questions - self.config = config - self.results = [] - self.start_time = datetime.datetime.now() - self.end_time = None - self.total_duration = 0 - self.accuracy = 0 - self.summary = {} - - def add_result(self, result: Dict[str, Any]) -> None: - """ - Add a question result to the benchmark. - - Args: - result: Dictionary containing the result for a question - """ - self.results.append(result) - - def calculate_metrics(self) -> None: - """ - Calculate overall metrics for the benchmark. - - This method computes accuracy, average response time, and other - performance metrics based on the individual question results. - """ - if not self.results: - logger.warning("No results to calculate metrics for") - return - - self.end_time = datetime.datetime.now() - self.total_duration = (self.end_time - self.start_time).total_seconds() - - correct_count = sum(1 for result in self.results if result.get("is_correct", False)) - self.accuracy = correct_count / len(self.results) if self.results else 0 - - response_times = [result.get("response_time", 0) for result in self.results] - avg_response_time = statistics.mean(response_times) if response_times else 0 - - memory_usages = [result.get("memory_usage", 0) for result in self.results] - avg_memory_usage = statistics.mean(memory_usages) if memory_usages else 0 - - categories = {} - for result in self.results: - category = result.get("category", "unknown") - if category not in categories: - categories[category] = { - "count": 0, - "correct": 0, - "response_times": [] - } - - categories[category]["count"] += 1 - if result.get("is_correct", False): - categories[category]["correct"] += 1 - categories[category]["response_times"].append(result.get("response_time", 0)) - - complexities = {} - for result in self.results: - complexity = result.get("complexity", 0) - if complexity not in complexities: - complexities[complexity] = { - "count": 0, - "correct": 0, - "response_times": [] - } - - complexities[complexity]["count"] += 1 - if result.get("is_correct", False): - complexities[complexity]["correct"] += 1 - complexities[complexity]["response_times"].append(result.get("response_time", 0)) - - self.summary = { - "total_questions": len(self.results), - "correct_answers": correct_count, - "accuracy": self.accuracy, - "avg_response_time": avg_response_time, - "avg_memory_usage": avg_memory_usage, - "total_duration": self.total_duration, - "categories": { - category: { - "count": data["count"], - "correct": data["correct"], - "accuracy": data["correct"] / data["count"] if data["count"] > 0 else 0, - "avg_response_time": statistics.mean(data["response_times"]) if data["response_times"] else 0 - } - for category, data in categories.items() - }, - "complexities": { - complexity: { - "count": data["count"], - "correct": data["correct"], - "accuracy": data["correct"] / data["count"] if data["count"] > 0 else 0, - "avg_response_time": statistics.mean(data["response_times"]) if data["response_times"] else 0 - } - for complexity, data in complexities.items() - } - } - - def print_summary(self) -> None: - """ - Print a summary of the benchmark results to the console. - - This method displays overall metrics, category-specific metrics, - and complexity-specific metrics in a tabular format. - """ - if not self.summary: - self.calculate_metrics() - - - - category_data = [] - for category, metrics in self.summary["categories"].items(): - category_data.append([ - category, - metrics["count"], - metrics["correct"], - f"{metrics['accuracy']:.2%}", - f"{metrics['avg_response_time']:.2f} s" - ]) - - print(tabulate( - category_data, - headers=["Category", "Count", "Correct", "Accuracy", "Avg Time"], - tablefmt="grid" - )) - - complexity_data = [] - for complexity, metrics in self.summary["complexities"].items(): - complexity_data.append([ - complexity, - metrics["count"], - metrics["correct"], - f"{metrics['accuracy']:.2%}", - f"{metrics['avg_response_time']:.2f} s" - ]) - - print(tabulate( - complexity_data, - headers=["Complexity", "Count", "Correct", "Accuracy", "Avg Time"], - tablefmt="grid" - )) - - question_data = [] - for result in self.results: - question_data.append([ - result.get("id", ""), - result.get("question", "")[:50] + ("..." if len(result.get("question", "")) > 50 else ""), - "✓" if result.get("is_correct", False) else "✗", - f"{result.get('response_time', 0):.2f} s", - result.get("category", ""), - result.get("complexity", "") - ]) - - print(tabulate( - question_data, - headers=["ID", "Question", "Correct", "Time", "Category", "Complexity"], - tablefmt="grid" - )) - - def generate_charts(self, output_dir: str = ".") -> None: - """ - Generate charts visualizing the benchmark results. - - Args: - output_dir: Directory to save the charts in - """ - if not self.summary: - self.calculate_metrics() - - os.makedirs(output_dir, exist_ok=True) - - plt.figure(figsize=(10, 6)) - categories = list(self.summary["categories"].keys()) - accuracies = [self.summary["categories"][cat]["accuracy"] for cat in categories] - - plt.bar(categories, accuracies) - plt.xlabel("Category") - plt.ylabel("Accuracy") - plt.title("Accuracy by Category") - plt.xticks(rotation=45, ha="right") - plt.tight_layout() - plt.savefig(os.path.join(output_dir, "accuracy_by_category.png")) - - plt.figure(figsize=(10, 6)) - complexities = sorted(self.summary["complexities"].keys()) - response_times = [self.summary["complexities"][comp]["avg_response_time"] for comp in complexities] - - plt.bar(complexities, response_times) - plt.xlabel("Complexity") - plt.ylabel("Average Response Time (s)") - plt.title("Response Time by Complexity") - plt.tight_layout() - plt.savefig(os.path.join(output_dir, "response_time_by_complexity.png")) - - plt.figure(figsize=(8, 8)) - plt.pie( - [self.summary["correct_answers"], self.summary["total_questions"] - self.summary["correct_answers"]], - labels=["Correct", "Incorrect"], - autopct="%1.1f%%", - colors=["#4CAF50", "#F44336"] - ) - plt.title("Overall Accuracy") - plt.tight_layout() - plt.savefig(os.path.join(output_dir, "overall_accuracy.png")) - - def save_to_file(self, filename: str) -> None: - """ - Save the benchmark results to a JSON file. - - Args: - filename: Path to the output file - """ - if not self.summary: - self.calculate_metrics() - - output = { - "timestamp": datetime.datetime.now().isoformat(), - "config": self.config, - "summary": self.summary, - "results": self.results - } - - with open(filename, "w") as f: - json.dump(output, f, indent=2) - - logger.info(f"Benchmark results saved to {filename}") - - -def run_benchmark( - questions: List[Dict[str, Any]], - agent_config: Optional[Dict[str, Any]] = None, - max_questions: Optional[int] = None -) -> BenchmarkResult: - """ - Run a benchmark test on the GAIA agent. - - Args: - questions: List of questions to test - agent_config: Optional configuration for the agent - max_questions: Optional maximum number of questions to test - - Returns: - BenchmarkResult object containing the benchmark results - """ - logger.info("Starting benchmark test") - - config = agent_config or get_model_config() - gaia_agent = agent.GAIAAgent(config) - - benchmark_result = BenchmarkResult(questions, config) - - if max_questions and max_questions < len(questions): - questions = questions[:max_questions] - - for i, question_data in enumerate(questions): - question_id = question_data.get("id", f"q{i+1}") - question = question_data.get("question", "") - expected_answer = question_data.get("expected_answer", "") - category = question_data.get("category", "unknown") - complexity = question_data.get("complexity", 1) - - logger.info(f"Testing question {i+1}/{len(questions)}: {question}") - - start_time = time.time() - process = psutil.Process(os.getpid()) - start_memory = process.memory_info().rss / (1024 * 1024) # MB - - try: - answer = gaia_agent(question) - - end_time = time.time() - end_memory = process.memory_info().rss / (1024 * 1024) # MB - response_time = end_time - start_time - memory_usage = end_memory - start_memory - - is_correct = check_answer(answer, expected_answer) - - result = { - "id": question_id, - "question": question, - "expected_answer": expected_answer, - "actual_answer": answer, - "is_correct": is_correct, - "response_time": response_time, - "memory_usage": memory_usage, - "category": category, - "complexity": complexity - } - - benchmark_result.add_result(result) - - logger.info(f"Question {i+1} completed in {response_time:.2f}s (Correct: {is_correct})") - - except Exception as e: - logger.error(f"Error processing question {i+1}: {str(e)}") - - result = { - "id": question_id, - "question": question, - "expected_answer": expected_answer, - "actual_answer": f"ERROR: {str(e)}", - "is_correct": False, - "response_time": time.time() - start_time, - "memory_usage": 0, - "category": category, - "complexity": complexity, - "error": str(e) - } - - benchmark_result.add_result(result) - - benchmark_result.calculate_metrics() - - logger.info(f"Benchmark completed with accuracy: {benchmark_result.accuracy:.2%}") - - return benchmark_result - - -def check_answer(actual_answer: str, expected_answer: str) -> bool: - """ - Check if an answer is correct by comparing it to the expected answer. - - Args: - actual_answer: The answer provided by the agent - expected_answer: The expected correct answer - - Returns: - True if the answer is correct, False otherwise - """ - actual = normalize_answer(actual_answer) - expected = normalize_answer(expected_answer) - - if actual == expected: - return True - - if "," in expected: - actual_items = set(item.strip() for item in actual.split(",")) - expected_items = set(item.strip() for item in expected.split(",")) - if actual_items == expected_items: - return True - - if expected.replace(".", "").isdigit() and actual.replace(".", "").isdigit(): - try: - return float(actual) == float(expected) - except ValueError: - pass - - return False - - -def normalize_answer(answer: str) -> str: - """ - Normalize an answer for comparison. - - Args: - answer: The answer to normalize - - Returns: - Normalized answer string - """ - normalized = answer.lower() - - normalized = re.sub(r'[^\w\s,]', '', normalized) - - normalized = re.sub(r'\s+', ' ', normalized).strip() - - return normalized - - -def load_questions_from_file(filename: str) -> List[Dict[str, Any]]: - """ - Load benchmark questions from a JSON file. - - Args: - filename: Path to the JSON file containing questions - - Returns: - List of question dictionaries - """ - try: - with open(filename, "r") as f: - questions = json.load(f) - - logger.info(f"Loaded {len(questions)} questions from {filename}") - return questions - - except Exception as e: - logger.error(f"Error loading questions from {filename}: {str(e)}") - return [] - - -def main(): - """Main function for running the benchmark from the command line.""" - parser = argparse.ArgumentParser(description="Run benchmark tests for the GAIA agent") - parser.add_argument("--questions", help="Path to a JSON file containing benchmark questions") - parser.add_argument("--output", default="benchmark_results.json", help="Path to save the benchmark results") - parser.add_argument("--max-questions", type=int, help="Maximum number of questions to test") - parser.add_argument("--charts", action="store_true", help="Generate charts visualizing the results") - parser.add_argument("--charts-dir", default="benchmark_charts", help="Directory to save charts in") - args = parser.parse_args() - - if args.questions: - questions = load_questions_from_file(args.questions) - if not questions: - logger.error("No valid questions found. Using sample questions.") - questions = SAMPLE_QUESTIONS - else: - questions = SAMPLE_QUESTIONS - - result = run_benchmark(questions, max_questions=args.max_questions) - - result.print_summary() - - result.save_to_file(args.output) - - if args.charts: - result.generate_charts(args.charts_dir) - logger.info(f"Charts saved to {args.charts_dir}") - - -if __name__ == "__main__": - main() \ No newline at end of file diff --git a/tests/run_all_apis_integration_test.py b/tests/run_all_apis_integration_test.py deleted file mode 100644 index bf150675df170c91138ee88c0aa22157f97fa816..0000000000000000000000000000000000000000 --- a/tests/run_all_apis_integration_test.py +++ /dev/null @@ -1,171 +0,0 @@ -""" -Test runner for the GAIA All APIs Integration Test. - -This script runs the comprehensive integration test for all APIs in the GAIA agent -and displays the results in a user-friendly format. - -Usage: - python tests/run_all_apis_integration_test.py [--verbose] [--report-file REPORT_FILE] -""" - -import os -import sys -import json -import time -import argparse -import logging -import unittest -from typing import Dict, Any, List, Optional - -sys.path.append('.') - -from tests.test_gaia_all_apis_integration import TestGAIAAllAPIsIntegration - -logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s') -logger = logging.getLogger("run_integration_test") - -def print_header(title: str): - """Print a formatted header.""" - -def print_section(title: str): - """Print a formatted section title.""" - -def print_result(label: str, success: Optional[bool], message: str): - """Print a formatted test result.""" - if success is True: - status = "✅ PASS" - elif success is False: - status = "❌ FAIL" - else: - status = "⚠️ SKIP" - - -def print_report(report_file: str): - """Print a formatted report from a JSON file.""" - try: - with open(report_file, 'r') as f: - report = json.load(f) - - print_header("GAIA ALL APIs INTEGRATION TEST REPORT") - - print_section("SUMMARY") - - print_section("API CONNECTIONS") - for api, result in report["api_connections"].items(): - print_result(api, result.get("success"), result.get("message", "")) - - print_section("TOOL EXECUTIONS") - for tool, result in report["tool_executions"].items(): - if result.get("success") is True: - if tool == "robust_search": - message = f"Used provider: {result.get('provider', 'unknown')}, Results: {result.get('result_count', 0)}" - elif tool == "merged_search": - message = f"Results: {result.get('result_count', 0)}, Has Perplexity: {result.get('has_perplexity', False)}, Has arXiv: {result.get('has_arxiv', False)}" - else: - message = f"Query: '{result.get('query', '')}', Time: {result.get('execution_time', 0):.2f}s" - else: - message = result.get("message", "") - - print_result(tool, result.get("success"), message) - - print_section("COMPLEX QUERIES") - for case, result in report["complex_queries"].items(): - if result.get("success") is True: - message = f"Type: {result.get('question_type', 'unknown')}, Tools: {', '.join(result.get('expected_tools', []))}" - else: - message = result.get("error", "") - - print_result(f"Case {case}", result.get("success"), message) - - print_section("MEMORY INTEGRATION") - status = report["memory_integration"].get("status", {}) - print_result("Memory Status", status.get("success"), status.get("message", "")) - - if "conversation" in report["memory_integration"]: - conv = report["memory_integration"]["conversation"] - - if "caching" in report["memory_integration"]: - cache = report["memory_integration"]["caching"] - - if "intermediate_results" in report["memory_integration"]: - inter = report["memory_integration"]["intermediate_results"] - - - except Exception as e: - logger.error(f"Error printing report: {str(e)}") - -def run_tests(args: argparse.Namespace) -> int: - """ - Run the integration tests. - - Args: - args: Command-line arguments - - Returns: - int: Exit code (0 for success, 1 for failure) - """ - print_header("RUNNING GAIA ALL APIs INTEGRATION TEST") - - if args.verbose: - logger.setLevel(logging.DEBUG) - logging.getLogger("test_gaia_all_apis").setLevel(logging.DEBUG) - - suite = unittest.TestSuite() - - suite.addTest(unittest.makeSuite(TestGAIAAllAPIsIntegration)) - - start_time = time.time() - runner = unittest.TextTestRunner(verbosity=2) - result = runner.run(suite) - end_time = time.time() - - test_case = TestGAIAAllAPIsIntegration() - test_case.setUp() # Initialize the test case - - test_case.test_api_connections() - try: - test_case.test_individual_tool_executions() - except Exception as e: - logger.error(f"Error in tool executions test: {str(e)}") - - test_case.generate_report(args.report_file) - - test_case.tearDown() - - print_report(args.report_file) - - - return 0 if result.wasSuccessful() else 1 - -def check_environment(): - """Check if the environment is properly set up.""" - required_vars = ["OPENAI_API_KEY"] - optional_vars = ["SERPER_API_KEY", "PERPLEXITY_API_KEY"] - - missing_required = [var for var in required_vars if not os.environ.get(var)] - missing_optional = [var for var in optional_vars if not os.environ.get(var)] - - if missing_required: - logger.error(f"Missing required environment variables: {', '.join(missing_required)}") - return False - - if missing_optional: - logger.warning(f"Missing optional environment variables: {', '.join(missing_optional)}") - - return True - -def main(): - """Main function.""" - parser = argparse.ArgumentParser(description="Run the GAIA All APIs Integration Test") - parser.add_argument("--verbose", "-v", action="store_true", help="Enable verbose logging") - parser.add_argument("--report-file", "-r", type=str, default="gaia_all_apis_integration_report.json", - help="Output file for the integration test report") - args = parser.parse_args() - - if not check_environment(): - return 1 - - return run_tests(args) - -if __name__ == "__main__": - sys.exit(main()) \ No newline at end of file diff --git a/tests/run_openai_tests.py b/tests/run_openai_tests.py deleted file mode 100644 index f56d885e3aaa78f76290b3c12a9217f85d9955e8..0000000000000000000000000000000000000000 --- a/tests/run_openai_tests.py +++ /dev/null @@ -1,222 +0,0 @@ -""" -Convenience script to run all OpenAI integration tests. - -This script runs all the OpenAI integration tests in sequence and generates -a comprehensive report of the results. It's a convenient way to verify that -all aspects of the OpenAI integration are working correctly. - -Usage: - python tests/run_openai_tests.py [--verbose] [--output DIRECTORY] -""" - -import os -import sys -import json -import time -import logging -import argparse -import subprocess -from datetime import datetime -from typing import Dict, Any, List - -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(name)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("openai_tests_runner") - -def run_test(command: List[str], test_name: str) -> Dict[str, Any]: - """ - Run a test command and capture the results. - - Args: - command: The command to run - test_name: The name of the test - - Returns: - Dict[str, Any]: Test results - """ - logger.info(f"Running {test_name}...") - - start_time = time.time() - - try: - result = subprocess.run( - command, - capture_output=True, - text=True, - check=False - ) - - end_time = time.time() - duration = end_time - start_time - - if result.returncode == 0: - logger.info(f"{test_name} completed successfully in {duration:.2f} seconds") - status = "success" - else: - logger.error(f"{test_name} failed with return code {result.returncode}") - status = "failure" - - return { - "test_name": test_name, - "command": " ".join(command), - "status": status, - "return_code": result.returncode, - "stdout": result.stdout, - "stderr": result.stderr, - "duration": duration, - "start_time": start_time, - "end_time": end_time - } - - except Exception as e: - end_time = time.time() - duration = end_time - start_time - - logger.error(f"Error running {test_name}: {str(e)}") - - return { - "test_name": test_name, - "command": " ".join(command), - "status": "error", - "error": str(e), - "duration": duration, - "start_time": start_time, - "end_time": end_time - } - -def run_all_tests(args: argparse.Namespace) -> Dict[str, Any]: - """ - Run all OpenAI integration tests. - - Args: - args: Command-line arguments - - Returns: - Dict[str, Any]: Test results - """ - if not os.path.exists(args.output): - os.makedirs(args.output) - - tests = [ - { - "name": "OpenAI Connection Test", - "command": [ - "python", - "tests/verify_connections.py" - ], - "output_file": os.path.join(args.output, "connection_test_results.json") - }, - { - "name": "OpenAI Integration Test", - "command": [ - "python", - "tests/test_openai_integration.py", - "--output", - os.path.join(args.output, "openai_integration_results.json") - ], - "output_file": os.path.join(args.output, "openai_integration_results.json") - }, - { - "name": "OpenAI Integration Test with Agent", - "command": [ - "python", - "tests/test_openai_integration.py", - "--test-agent", - "--output", - os.path.join(args.output, "openai_integration_with_agent_results.json") - ], - "output_file": os.path.join(args.output, "openai_integration_with_agent_results.json") - }, - { - "name": "GAIA Multi-Provider Test", - "command": [ - "python", - "tests/test_gaia_multi_provider.py", - "--output", - os.path.join(args.output, "gaia_multi_provider_results.json") - ], - "output_file": os.path.join(args.output, "gaia_multi_provider_results.json") - } - ] - - if args.verbose: - for test in tests: - test["command"].append("--verbose") - - results = { - "start_time": datetime.now().isoformat(), - "tests": [] - } - - for test in tests: - test_result = run_test(test["command"], test["name"]) - - results["tests"].append(test_result) - - if test_result["status"] != "success" and args.stop_on_failure: - logger.warning(f"Stopping tests due to failure in {test['name']}") - break - - results["end_time"] = datetime.now().isoformat() - - start_time = datetime.fromisoformat(results["start_time"]) - end_time = datetime.fromisoformat(results["end_time"]) - results["duration"] = (end_time - start_time).total_seconds() - - results["summary"] = { - "total_tests": len(results["tests"]), - "successful_tests": sum(1 for t in results["tests"] if t["status"] == "success"), - "failed_tests": sum(1 for t in results["tests"] if t["status"] == "failure"), - "error_tests": sum(1 for t in results["tests"] if t["status"] == "error"), - "total_duration": results["duration"] - } - - output_file = os.path.join(args.output, "all_tests_results.json") - with open(output_file, "w") as f: - json.dump(results, f, indent=2) - - logger.info(f"All test results saved to {output_file}") - - return results - -def print_results(results: Dict[str, Any]) -> None: - """ - Print test results. - - Args: - results: Test results - """ - - - for test in results["tests"]: - status_icon = "✅" if test["status"] == "success" else "❌" - - - -def main(): - """Main function.""" - parser = argparse.ArgumentParser(description="Run all OpenAI integration tests") - parser.add_argument("--verbose", "-v", action="store_true", help="Enable verbose logging") - parser.add_argument("--output", "-o", type=str, default="openai_test_results", help="Output directory for test results") - parser.add_argument("--stop-on-failure", "-s", action="store_true", help="Stop running tests after the first failure") - args = parser.parse_args() - - if args.verbose: - logger.setLevel(logging.DEBUG) - if not any(isinstance(h, logging.StreamHandler) for h in logger.handlers): - console_handler = logging.StreamHandler() - console_handler.setLevel(logging.DEBUG) - console_handler.setFormatter(logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")) - logger.addHandler(console_handler) - - results = run_all_tests(args) - - print_results(results) - - return results["summary"]["successful_tests"] == results["summary"]["total_tests"] - -if __name__ == "__main__": - success = main() - sys.exit(0 if success else 1) \ No newline at end of file diff --git a/tests/run_supabase_integration_tests.py b/tests/run_supabase_integration_tests.py deleted file mode 100644 index 4c6c2142518d9701f344254a1c192faa0cb66f11..0000000000000000000000000000000000000000 --- a/tests/run_supabase_integration_tests.py +++ /dev/null @@ -1,244 +0,0 @@ -""" -Run Supabase Integration Tests for GAIA Memory - -This script runs all the Supabase integration tests for GAIA memory and documents the results. -It executes the following tests: -1. Supabase connection test -2. Supabase memory module test -3. GAIA agent with Supabase memory integration test - -The results are documented in a markdown file for easy reference. -""" - -import os -import sys -import time -import logging -import argparse -import subprocess -from datetime import datetime -from typing import Dict, Any, List, Optional, Union, Tuple - -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(name)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("supabase_integration_tests") - -try: - from agent.config import ( - SUPABASE_URL, - SUPABASE_KEY, - SUPABASE_PROJECT_ID, - SUPABASE_PROJECT_NAME - ) -except ImportError as e: - logger.error(f"Error importing configuration: {str(e)}") - sys.exit(1) - -class SupabaseIntegrationTestRunner: - """Class for running Supabase integration tests for GAIA memory.""" - - def __init__(self, verbose: bool = False, output_dir: str = "docs"): - """ - Initialize the Supabase integration test runner. - - Args: - verbose: Whether to enable verbose logging - output_dir: Directory to save the test results - """ - self.verbose = verbose - self.output_dir = output_dir - self.results = {} - - if verbose: - logging.getLogger().setLevel(logging.DEBUG) - - os.makedirs(output_dir, exist_ok=True) - - def run_test_script(self, script_name: str, test_name: str) -> Tuple[bool, str]: - """ - Run a test script and capture the output. - - Args: - script_name: Name of the test script to run - test_name: Name of the test for logging - - Returns: - Tuple[bool, str]: Success status and output - """ - logger.info(f"Running {test_name}...") - - try: - cmd = [sys.executable, script_name] - if self.verbose: - cmd.append("--verbose") - - logger.info(f"Executing command: {' '.join(cmd)}") - start_time = time.time() - process = subprocess.Popen( - cmd, - stdout=subprocess.PIPE, - stderr=subprocess.PIPE, - text=True - ) - - stdout, stderr = process.communicate() - end_time = time.time() - - success = process.returncode == 0 - - if success: - logger.info(f"✅ {test_name} completed successfully in {end_time - start_time:.2f} seconds") - else: - logger.error(f"❌ {test_name} failed with return code {process.returncode}") - logger.error(f"Error: {stderr}") - - output = stdout - if stderr: - output += f"\n\nErrors:\n{stderr}" - - return success, output - except Exception as e: - logger.error(f"Error running {test_name}: {str(e)}") - return False, str(e) - - def run_all_tests(self) -> Dict[str, Dict[str, Any]]: - """ - Run all Supabase integration tests. - - Returns: - Dict[str, Dict[str, Any]]: Dictionary with test results - """ - logger.info("Running all Supabase integration tests...") - - tests = [ - { - "name": "Supabase Connection Test", - "script": "tests/test_supabase_connection.py", - "description": "Tests the connection to Supabase and verifies that the required tables exist." - }, - { - "name": "Supabase Memory Module Test", - "script": "tests/test_supabase_memory.py", - "description": "Tests the Supabase memory module implementation, including basic memory operations, conversation memory, result cache, and working memory." - }, - { - "name": "GAIA Agent with Supabase Memory Integration Test", - "script": "tests/test_gaia_supabase_integration.py", - "description": "Tests the integration of the GAIA agent with Supabase memory, verifying that memory persistence works across different agent sessions." - } - ] - - for test in tests: - test_name = test["name"] - script_name = test["script"] - - success, output = self.run_test_script(script_name, test_name) - - self.results[test_name] = { - "success": success, - "output": output, - "description": test["description"], - "timestamp": datetime.now().isoformat() - } - - return self.results - - def generate_markdown_report(self) -> str: - """ - Generate a markdown report of the test results. - - Returns: - str: Path to the generated markdown file - """ - logger.info("Generating markdown report...") - - timestamp = datetime.now().strftime("%Y%m%d_%H%M%S") - filename = f"supabase_integration_test_results_{timestamp}.md" - filepath = os.path.join(self.output_dir, filename) - - content = f"""# Supabase Integration Test Results for GAIA Memory - -## Test Summary - -- **Date:** {datetime.now().strftime("%Y-%m-%d %H:%M:%S")} -- **Supabase Project:** {SUPABASE_PROJECT_NAME} ({SUPABASE_PROJECT_ID}) -- **Supabase URL:** {SUPABASE_URL} - -## Test Results - -""" - - for test_name, result in self.results.items(): - status = "✅ Passed" if result["success"] else "❌ Failed" - content += f"### {test_name}: {status}\n\n" - content += f"**Description:** {result['description']}\n\n" - content += f"**Timestamp:** {datetime.fromisoformat(result['timestamp']).strftime('%Y-%m-%d %H:%M:%S')}\n\n" - - content += "
\n" - content += f"Test Output\n\n" - content += "```\n" - content += result["output"] - content += "\n```\n" - content += "
\n\n" - - all_passed = all(result["success"] for result in self.results.values()) - content += "## Overall Assessment\n\n" - - if all_passed: - content += "✅ **All tests passed successfully.**\n\n" - content += "The Supabase integration for GAIA memory is working correctly. Memory persistence across different agent sessions is functioning as expected.\n\n" - else: - content += "⚠️ **Some tests failed.**\n\n" - content += "The Supabase integration for GAIA memory has issues that need to be addressed. Please check the test output for details.\n\n" - - content += "### Recommendations\n\n" - content += "- Verify that the Supabase URL and API key are correctly set in the environment variables\n" - content += "- Check that all required tables exist in the Supabase database\n" - content += "- Ensure that the memory module is correctly configured\n" - content += "- Review the test output for specific error messages\n\n" - - with open(filepath, "w") as f: - f.write(content) - - logger.info(f"Markdown report generated: {filepath}") - return filepath - - def run_and_document_tests(self) -> str: - """ - Run all tests and document the results. - - Returns: - str: Path to the generated markdown file - """ - - self.run_all_tests() - - report_path = self.generate_markdown_report() - - for test_name, result in self.results.items(): - status = "✅ Passed" if result["success"] else "❌ Failed" - - all_passed = all(result["success"] for result in self.results.values()) - if all_passed: - else: - - - return report_path - - -def main(): - """Main function.""" - parser = argparse.ArgumentParser(description="Run Supabase integration tests for GAIA memory") - parser.add_argument("--verbose", action="store_true", help="Enable verbose logging") - parser.add_argument("--output-dir", type=str, default="docs", help="Directory to save test results") - args = parser.parse_args() - - runner = SupabaseIntegrationTestRunner(verbose=args.verbose, output_dir=args.output_dir) - - runner.run_and_document_tests() - - -if __name__ == "__main__": - main() \ No newline at end of file diff --git a/tests/test_agent.py b/tests/test_agent.py deleted file mode 100644 index 3db0355dbaf02fec82fe384778c28afa739868e6..0000000000000000000000000000000000000000 --- a/tests/test_agent.py +++ /dev/null @@ -1,336 +0,0 @@ -""" -Unit tests for the GAIA agent. -""" -import unittest -from unittest.mock import patch, MagicMock, ANY -import json -import sys -import os -import time -from typing import Dict, Any, List, Optional - -sys.path.append('.') - -from agent.agent import GAIAAgent -from agent.graph import AgentState, QuestionAnalysis, PlanStep, ToolResult - - -class TestGAIAAgent(unittest.TestCase): - """Test case for GAIAAgent class.""" - - def setUp(self): - """Set up the test case with mocked dependencies.""" - self.memory_patcher = patch('agent.agent.SupabaseMemory') - self.conversation_patcher = patch('agent.agent.ConversationMemory') - self.cache_patcher = patch('agent.agent.ResultCache') - self.working_patcher = patch('agent.agent.WorkingMemory') - - self.mock_memory = self.memory_patcher.start() - self.mock_conversation = self.conversation_patcher.start() - self.mock_cache = self.cache_patcher.start() - self.mock_working = self.working_patcher.start() - - self.mock_memory_instance = MagicMock() - self.mock_conversation_instance = MagicMock() - self.mock_cache_instance = MagicMock() - self.mock_working_instance = MagicMock() - - self.mock_memory.return_value = self.mock_memory_instance - self.mock_conversation.return_value = self.mock_conversation_instance - self.mock_cache.return_value = self.mock_cache_instance - self.mock_working.return_value = self.mock_working_instance - - self.graph_patcher = patch('agent.agent.run_agent_graph') - self.mock_graph = self.graph_patcher.start() - - self.mock_graph.return_value = { - "question": "test question", - "analysis": { - "question_type": "factual", - "required_tools": ["web_search"], - "information_sources": ["web"], - "complexity": "simple" - }, - "plan": [ - { - "step_number": 1, - "description": "Search for information", - "tool": "web_search", - "tool_input": {"query": "test query"} - } - ], - "current_step": 1, - "tools_to_use": ["web_search"], - "tool_results": [ - { - "tool_name": "web_search", - "success": True, - "result": "test result", - "error": None - } - ], - "intermediate_thoughts": ["Searching for information..."], - "answer": "test answer", - "error": None - } - - def tearDown(self): - """Clean up after the test.""" - self.memory_patcher.stop() - self.conversation_patcher.stop() - self.cache_patcher.stop() - self.working_patcher.stop() - self.graph_patcher.stop() - - def test_init(self): - """Test initialization of GAIAAgent.""" - agent = GAIAAgent() - - self.mock_memory.assert_called_once() - self.mock_conversation.assert_called_once() - self.mock_cache.assert_called_once() - self.mock_working.assert_called_once() - - self.assertIsNotNone(agent.session_id) - self.assertEqual(agent.memory, self.mock_memory_instance) - self.assertEqual(agent.conversation_memory, self.mock_conversation_instance) - self.assertEqual(agent.result_cache, self.mock_cache_instance) - self.assertEqual(agent.working_memory, self.mock_working_instance) - - self.assertIsNone(agent.state["current_question"]) - self.assertEqual(agent.state["tools_used"], []) - self.assertEqual(agent.state["intermediate_results"], []) - self.assertIsNone(agent.state["start_time"]) - self.assertIsNone(agent.state["end_time"]) - self.assertIsNone(agent.state["error"]) - - def test_reset_state(self): - """Test resetting agent state.""" - agent = GAIAAgent() - - agent.state["current_question"] = "test question" - agent.state["tools_used"] = ["web_search"] - agent.state["start_time"] = time.time() - - agent.reset_state() - - self.assertIsNone(agent.state["current_question"]) - self.assertEqual(agent.state["tools_used"], []) - self.assertEqual(agent.state["intermediate_results"], []) - self.assertIsNone(agent.state["start_time"]) - self.assertIsNone(agent.state["end_time"]) - self.assertIsNone(agent.state["error"]) - - def test_call(self): - """Test calling the agent with a question.""" - agent = GAIAAgent() - - answer = agent("test question") - - self.mock_graph.assert_called_once() - - self.assertEqual(answer, "test answer") - - self.assertEqual(agent.state["current_question"], "test question") - self.assertIsNotNone(agent.state["start_time"]) - self.assertIsNotNone(agent.state["end_time"]) - self.assertIsNone(agent.state["error"]) - - self.mock_conversation_instance.add_message.assert_any_call("user", "test question") - self.mock_conversation_instance.add_message.assert_any_call("assistant", "test answer") - - def test_call_with_error(self): - """Test calling the agent with an error.""" - agent = GAIAAgent() - - self.mock_graph.side_effect = Exception("test error") - - agent.config["agent"]["return_error_message"] = True - - answer = agent("test question") - - self.assertTrue(answer.startswith("Error:")) - - self.assertEqual(agent.state["current_question"], "test question") - self.assertIsNotNone(agent.state["start_time"]) - self.assertIsNotNone(agent.state["end_time"]) - self.assertEqual(agent.state["error"], "test error") - - self.mock_graph.side_effect = None - - agent.config["agent"]["return_error_message"] = False - - with self.assertRaises(Exception): - agent("test question") - - def test_process_question(self): - """Test processing a question.""" - agent = GAIAAgent() - - answer = agent._process_question("test question") - - self.mock_graph.assert_called_once_with("test question", ANY) - - self.assertEqual(answer, "test answer") - - self.mock_working_instance.store_intermediate_result.assert_any_call( - "analysis", self.mock_graph.return_value["analysis"] - ) - self.mock_working_instance.store_intermediate_result.assert_any_call( - "plan", self.mock_graph.return_value["plan"] - ) - self.mock_working_instance.store_intermediate_result.assert_any_call( - "tool_results", self.mock_graph.return_value["tool_results"] - ) - - def test_process_question_with_cache(self): - """Test processing a question with cache.""" - agent = GAIAAgent() - - self.mock_cache_instance.get_result.return_value = "cached answer" - - answer = agent._process_question("test question") - - self.mock_graph.assert_not_called() - - self.assertEqual(answer, "cached answer") - - self.mock_cache_instance.get_result.assert_called_once() - - self.mock_cache_instance.get_result.return_value = None - - answer = agent._process_question("test question") - - self.mock_graph.assert_called_once() - - self.assertEqual(answer, "test answer") - - self.mock_cache_instance.cache_result.assert_called_once() - - def test_get_memory_snapshot(self): - """Test getting a memory snapshot.""" - agent = GAIAAgent() - - self.mock_conversation_instance.get_messages.return_value = [ - {"role": "user", "content": "test question"}, - {"role": "assistant", "content": "test answer"} - ] - self.mock_memory_instance.list_keys.return_value = ["cache:test_key"] - self.mock_memory_instance.retrieve.return_value = "test result" - self.mock_working_instance.list_intermediate_results.return_value = { - "analysis": {"question_type": "factual"} - } - - snapshot = agent.get_memory_snapshot() - - self.assertEqual(len(snapshot["conversation"]), 2) - self.assertEqual(snapshot["conversation"][0]["role"], "user") - self.assertEqual(snapshot["conversation"][1]["role"], "assistant") - self.assertEqual(snapshot["cached_results"]["test_key"], "test result") - self.assertEqual(snapshot["intermediate_results"]["analysis"]["question_type"], "factual") - - self.mock_conversation_instance.get_messages.assert_called_once() - self.mock_memory_instance.list_keys.assert_called_once_with("cache:") - self.mock_memory_instance.retrieve.assert_called_once() - self.mock_working_instance.list_intermediate_results.assert_called_once() - - def test_clear_memory(self): - """Test clearing memory.""" - agent = GAIAAgent() - - agent.clear_memory() - - self.mock_conversation_instance.clear.assert_called_once() - self.mock_cache_instance.clear.assert_called_once() - self.mock_working_instance.clear.assert_called_once() - - -class TestGAIAAgentIntegration(unittest.TestCase): - """Integration tests for GAIAAgent with actual graph execution.""" - - def setUp(self): - """Set up the test case.""" - self.memory_patcher = patch('agent.agent.SupabaseMemory') - self.conversation_patcher = patch('agent.agent.ConversationMemory') - self.cache_patcher = patch('agent.agent.ResultCache') - self.working_patcher = patch('agent.agent.WorkingMemory') - - self.mock_memory = self.memory_patcher.start() - self.mock_conversation = self.conversation_patcher.start() - self.mock_cache = self.cache_patcher.start() - self.mock_working = self.working_patcher.start() - - self.mock_memory_instance = MagicMock() - self.mock_conversation_instance = MagicMock() - self.mock_cache_instance = MagicMock() - self.mock_working_instance = MagicMock() - - self.mock_memory.return_value = self.mock_memory_instance - self.mock_conversation.return_value = self.mock_conversation_instance - self.mock_cache.return_value = self.mock_cache_instance - self.mock_working.return_value = self.mock_working_instance - - self.chat_patcher = patch('agent.graph.ChatOpenAI') - self.mock_chat = self.chat_patcher.start() - self.mock_model = MagicMock() - self.mock_chat.return_value = self.mock_model - - self.mock_chain = MagicMock() - self.mock_model.__or__.return_value.__or__.return_value = self.mock_chain - - self.mock_chain.invoke.side_effect = [ - json.dumps({ - "question_type": "factual", - "required_tools": ["web_search"], - "information_sources": ["web"], - "complexity": "simple" - }), - json.dumps([ - { - "step_number": 1, - "description": "Search for information", - "tool": "web_search", - "tool_input": {"query": "test query"} - } - ]), - "test answer" - ] - - def tearDown(self): - """Clean up after the test.""" - self.memory_patcher.stop() - self.conversation_patcher.stop() - self.cache_patcher.stop() - self.working_patcher.stop() - self.chat_patcher.stop() - - @patch('agent.graph.ToolNode') - def test_integration(self, mock_tool_node): - """Test integration of agent with graph.""" - agent = GAIAAgent({ - "model": { - "reasoning_model": "gpt-3.5-turbo", - "temperature": 0.0, - "max_tokens": 1000 - }, - "tools": {}, - "memory": {}, - "agent": { - "max_retries": 1, - "retry_delay": 0, - "return_error_message": True - } - }) - - answer = agent("What is the capital of France?") - - self.assertEqual(answer, "test answer") - - self.assertEqual(self.mock_chain.invoke.call_count, 3) - - self.mock_conversation_instance.add_message.assert_any_call("user", "What is the capital of France?") - self.mock_conversation_instance.add_message.assert_any_call("assistant", "test answer") - - -if __name__ == "__main__": - unittest.main() \ No newline at end of file diff --git a/tests/test_all_supabase_tables.py b/tests/test_all_supabase_tables.py deleted file mode 100644 index 5712860e3f3acbf29b395fed956cb171055c3fe3..0000000000000000000000000000000000000000 --- a/tests/test_all_supabase_tables.py +++ /dev/null @@ -1,223 +0,0 @@ -""" -Comprehensive Test Script for All GAIA Supabase Tables - -This script tests all Supabase tables used by the GAIA agent after applying fixes: -- gaia_memory: Main memory table -- gaia_memory_conversations: Conversation history table -- gaia_memory_cache: Result cache table -- gaia_memory_working: Working memory table - -The script performs a series of tests to verify that: -1. All tables exist and are accessible -2. Data can be written to and read from each table -3. The specialized memory classes use their dedicated tables -4. Performance improvements from caching are measurable -5. Conversation context is maintained across multiple turns -6. Working memory effectively stores intermediate reasoning steps - -Usage: - python test_all_supabase_tables.py [--verbose] [--output FILENAME] -""" - -import os -import sys -import json -import time -import uuid -import logging -import argparse -from datetime import datetime -from typing import Dict, Any, List, Optional, Union, Tuple - -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(name)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("test_all_supabase_tables") - -sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) - -try: - from memory.supabase_memory import ( - SupabaseMemory, - ConversationMemory, - ResultCache, - WorkingMemory - ) - from agent.config import ( - get_memory_config, - SUPABASE_URL, - SUPABASE_KEY, - SUPABASE_PROJECT_ID, - SUPABASE_PROJECT_NAME - ) - - from supabase import create_client, Client -except ImportError as e: - logger.error(f"Error importing required components: {str(e)}") - sys.exit(1) - -class AllSupabaseTablesTest: - """Class for testing all Supabase tables used by the GAIA agent.""" - - def __init__(self, verbose: bool = False): - """ - Initialize the test class. - - Args: - verbose: Whether to enable verbose logging - """ - self.verbose = verbose - self.session_id = str(uuid.uuid4()) - self.conversation_id = str(uuid.uuid4()) - - if verbose: - logging.getLogger().setLevel(logging.DEBUG) - - if not SUPABASE_URL or not SUPABASE_KEY: - logger.error("Supabase URL or key not provided") - logger.error("Please set SUPABASE_URL and SUPABASE_KEY environment variables") - sys.exit(1) - - # Initialize Supabase client - self.client = create_client(SUPABASE_URL, SUPABASE_KEY) - - self._initialize_memory() - - logger.info(f"Test initialized with session ID: {self.session_id}") - logger.info(f"Conversation ID: {self.conversation_id}") - - def _initialize_memory(self) -> None: - """Initialize memory components.""" - logger.info("Initializing memory components...") - - try: - memory_config = get_memory_config() - - memory_config["enabled"] = True - - self.memory = SupabaseMemory(config=memory_config) - - self.conversation = ConversationMemory(self.memory, self.conversation_id) - self.cache = ResultCache(self.memory) - self.working = WorkingMemory(self.memory, self.session_id) - - self.table_names = { - "base": self.memory.table_name, - "conversation": self.conversation.table_name, - "cache": self.cache.table_name, - "working": self.working.table_name - } - - logger.info("Memory components initialized successfully") - logger.info(f"Table names: {self.table_names}") - except Exception as e: - logger.error(f"Error initializing memory components: {str(e)}") - sys.exit(1) - - def test_table_existence(self) -> Dict[str, bool]: - """ - Test if all required tables exist in Supabase. - - Returns: - Dict[str, bool]: Dictionary with test results - """ - logger.info("Testing table existence...") - - results = { - self.table_names["base"]: False, - self.table_names["conversation"]: False, - self.table_names["cache"]: False, - self.table_names["working"]: False - } - - try: - for table_name in results.keys(): - try: - query_result = self.client.table(table_name).select("*").limit(1).execute() - - results[table_name] = True - logger.info(f"✅ Table {table_name} exists") - except Exception as e: - logger.warning(f"❌ Table {table_name} does not exist or cannot be accessed: {str(e)}") - - except Exception as e: - logger.error(f"Error testing table existence: {str(e)}") - - return results - - def test_base_memory(self) -> Dict[str, bool]: - """ - Test the base memory operations. - - Returns: - Dict[str, bool]: Dictionary with test results - """ - logger.info("Testing base memory operations...") - - results = { - "store": False, - "retrieve": False, - "delete": False, - "list_keys": False - } - - try: - test_key = f"test_base_{time.time()}" - test_value = f"test_value_{time.time()}" - test_metadata = {"test": True, "timestamp": time.time()} - - logger.info(f"Storing value with key {test_key}...") - store_result = self.memory.store(test_key, test_value, test_metadata) - - if store_result: - results["store"] = True - logger.info("✅ Store operation successful") - else: - logger.warning("❌ Store operation failed") - return results - - logger.info(f"Retrieving value with key {test_key}...") - retrieved_value = self.memory.retrieve(test_key) - - if retrieved_value == test_value: - results["retrieve"] = True - logger.info("✅ Retrieve operation successful") - else: - logger.warning(f"❌ Retrieve operation failed. Expected {test_value}, got {retrieved_value}") - return results - - logger.info("Listing keys...") - keys = self.memory.list_keys() - - if test_key in keys: - results["list_keys"] = True - logger.info("✅ List keys operation successful") - else: - logger.warning(f"❌ List keys operation failed. Expected {test_key} in {keys}") - return results - - logger.info(f"Deleting value with key {test_key}...") - delete_result = self.memory.delete(test_key) - - if delete_result: - results["delete"] = True - logger.info("✅ Delete operation successful") - else: - logger.warning("❌ Delete operation failed") - - retrieved_after_delete = self.memory.retrieve(test_key) - if retrieved_after_delete is None: - logger.info("✅ Delete verification successful") - else: - logger.warning(f"❌ Delete verification failed. Expected None, got {retrieved_after_delete}") - results["delete"] = False - - except Exception as e: - logger.error(f"Error testing base memory: {str(e)}") - - return results - logger.info(f"Table names: {self.table_names}") - except Exception as e: - logger.error(f"Error initializing memory components: {str(e)}") - sys.exit(1) \ No newline at end of file diff --git a/tests/test_arxiv_integration.py b/tests/test_arxiv_integration.py deleted file mode 100644 index 7b82468800b0976a1d1c060c786f63c3b9803f4c..0000000000000000000000000000000000000000 --- a/tests/test_arxiv_integration.py +++ /dev/null @@ -1,221 +0,0 @@ -""" -Test for arXiv integration with the GAIA agent. - -This script tests the arXiv search functionality by: -1. Testing the direct connection to arXiv using the arxiv package -2. Testing the ArxivSearchTool class from arxiv_tool.py -3. Verifying proper error handling and result formatting -4. Testing integration with other search providers -""" - -import unittest -import sys -import os -import logging -from typing import Dict, Any, List, Optional - -sys.path.append('.') - -from tools.arxiv_tool import ArxivSearchTool, create_arxiv_search - -from tools.web_tools import create_duckduckgo_search, create_serper_search -from tools.perplexity_tool import PerplexityTool - -logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s') -logger = logging.getLogger("test_arxiv_integration") - -class TestArxivIntegration(unittest.TestCase): - """Test case for arXiv integration with the GAIA agent.""" - - def setUp(self): - """Set up the test case.""" - self.arxiv_tool = create_arxiv_search() - logger.info("arXiv search tool initialized") - - self.duckduckgo_tool = create_duckduckgo_search() - logger.info("DuckDuckGo search tool initialized") - - # Create Serper tool if API key is available - self.serper_tool = None - serper_api_key = os.environ.get("SERPER_API_KEY") - if serper_api_key: - self.serper_tool = create_serper_search() - logger.info("Serper search tool initialized") - - # Create Perplexity tool if API key is available - self.perplexity_tool = None - perplexity_api_key = os.environ.get("PERPLEXITY_API_KEY") - if perplexity_api_key: - self.perplexity_tool = PerplexityTool() - logger.info("Perplexity search tool initialized") - - def test_arxiv_package_import(self): - """Test that the arxiv package can be imported.""" - try: - import arxiv - logger.info("Successfully imported arxiv package") - self.assertTrue(True) - except ImportError: - self.fail("arxiv package is not installed") - - def test_arxiv_search(self): - """Test the ArxivSearchTool search method.""" - try: - results = self.arxiv_tool.search("large language models", max_results=2) - - self.assertIsNotNone(results) - self.assertIsInstance(results, list) - logger.info(f"Successfully retrieved {len(results)} results from arXiv") - - if results: - result = results[0] - self.assertIn("title", result) - self.assertIn("authors", result) - self.assertIn("summary", result) - self.assertIn("published", result) - self.assertIn("url", result) - self.assertIn("pdf_url", result) - logger.info(f"Result format is correct: {result.keys()}") - - logger.info(f"First result: {result['title']} by {result['authors']}") - - except Exception as e: - self.fail(f"Error using ArxivSearchTool: {str(e)}") - - def test_get_paper_by_id(self): - """Test the ArxivSearchTool get_paper_by_id method.""" - try: - paper_id = "2303.08774" # "A Survey of Large Language Models" paper - result = self.arxiv_tool.get_paper_by_id(paper_id) - - self.assertIsNotNone(result) - self.assertIsInstance(result, dict) - logger.info(f"Successfully retrieved paper with ID {paper_id}") - - self.assertIn("title", result) - self.assertIn("authors", result) - self.assertIn("summary", result) - self.assertIn("published", result) - self.assertIn("url", result) - self.assertIn("pdf_url", result) - logger.info(f"Result format is correct: {result.keys()}") - - logger.info(f"Paper: {result['title']} by {result['authors']}") - - except Exception as e: - self.fail(f"Error getting paper by ID: {str(e)}") - - def test_search_by_category(self): - """Test the ArxivSearchTool search_by_category method.""" - try: - category = "cs.AI" # Artificial Intelligence - results = self.arxiv_tool.search_by_category(category, max_results=2) - - self.assertIsNotNone(results) - self.assertIsInstance(results, list) - logger.info(f"Successfully retrieved {len(results)} results from category {category}") - - if results: - result = results[0] - self.assertIn("title", result) - self.assertIn("authors", result) - self.assertIn("categories", result) - logger.info(f"Result format is correct: {result.keys()}") - - self.assertIn(category, result["categories"]) - - logger.info(f"First result: {result['title']} by {result['authors']}") - - except Exception as e: - self.fail(f"Error searching by category: {str(e)}") - - def test_error_handling(self): - """Test error handling in the ArxivSearchTool.""" - with self.assertRaises(Exception): - self.arxiv_tool.get_paper_by_id("invalid_id") - - long_query = "a" * 1000 - try: - results = self.arxiv_tool.search(long_query, max_results=1) - logger.info(f"Long query handled successfully, got {len(results)} results") - except Exception as e: - logger.warning(f"Long query caused an error: {str(e)}") - - def test_integration_with_other_providers(self): - """Test using arXiv alongside other search providers.""" - logger.info("Testing arXiv alongside other search providers") - - query = "large language models" - results = {} - - try: - results["arxiv"] = self.arxiv_tool.search(query, max_results=2) - logger.info(f"arXiv found {len(results['arxiv'])} results") - except Exception as e: - logger.error(f"arXiv search failed: {str(e)}") - results["arxiv"] = None - - # Test DuckDuckGo - try: - results["duckduckgo"] = self.duckduckgo_tool.search(query) - logger.info(f"DuckDuckGo found {len(results['duckduckgo'])} results") - except Exception as e: - logger.error(f"DuckDuckGo search failed: {str(e)}") - results["duckduckgo"] = None - - # Test Serper if available - if self.serper_tool: - try: - results["serper"] = self.serper_tool.search(query) - logger.info(f"Serper found {len(results['serper'])} results") - except Exception as e: - logger.error(f"Serper search failed: {str(e)}") - results["serper"] = None - - # Test Perplexity if available - if self.perplexity_tool: - try: - perplexity_results = self.perplexity_tool.search(query) - results["perplexity"] = perplexity_results - content = perplexity_results.get("content", "") - logger.info(f"Perplexity search successful, content length: {len(content) if content else 0}") - except Exception as e: - logger.error(f"Perplexity search failed: {str(e)}") - results["perplexity"] = None - - self.assertTrue(any(results.values()), "All search providers failed") - - logger.info(f"Successfully tested {len([r for r in results.values() if r])} out of {len(results)} providers") - - providers_with_results = [p for p, r in results.items() if r] - if len(providers_with_results) >= 2: - logger.info("Comparing results from different providers:") - - if results["arxiv"]: - logger.info(f"arXiv first result: {results['arxiv'][0]['title']}") - - # Log DuckDuckGo results - if results["duckduckgo"]: - logger.info(f"DuckDuckGo first result: {results['duckduckgo'][0]['title']}") - - # Log Serper results - if self.serper_tool and results.get("serper"): - logger.info(f"Serper first result: {results['serper'][0]['title']}") - -if __name__ == "__main__": - - test = TestArxivIntegration() - test.setUp() - - test.test_arxiv_package_import() - - test.test_arxiv_search() - - test.test_get_paper_by_id() - - test.test_search_by_category() - - test.test_error_handling() - - test.test_integration_with_other_providers() - diff --git a/tests/test_config.py b/tests/test_config.py deleted file mode 100644 index 8905ba33d3cc286bcaf790cdcc1fd23f2cd6ad29..0000000000000000000000000000000000000000 --- a/tests/test_config.py +++ /dev/null @@ -1,36 +0,0 @@ -""" -Test file for the agent configuration module. -""" -import os -import unittest -from unittest.mock import patch - -import sys -sys.path.append('.') # Add the project root to the Python path - -class TestConfig(unittest.TestCase): - """Test case for the configuration module.""" - - def test_config_default_values(self): - """Test that configuration has default values when environment variables are not set.""" - from agent.config import ( - DEFAULT_MODEL, EMBEDDING_MODEL, - WEB_SEARCH_RESULT_COUNT, WEB_SEARCH_TIMEOUT, ARXIV_MAX_RESULTS, - MEMORY_ENABLED, MEMORY_TABLE_NAME, MEMORY_TTL, - MAX_ITERATIONS, VERBOSE - ) - - self.assertEqual(DEFAULT_MODEL, "gpt-4o") - self.assertEqual(EMBEDDING_MODEL, "text-embedding-3-large") - self.assertEqual(WEB_SEARCH_RESULT_COUNT, 5) - self.assertEqual(WEB_SEARCH_TIMEOUT, 10) - self.assertEqual(ARXIV_MAX_RESULTS, 3) - self.assertEqual(MEMORY_ENABLED, True) - self.assertEqual(MEMORY_TABLE_NAME, "gaia_memory") - self.assertEqual(MEMORY_TTL, 86400) - self.assertEqual(MAX_ITERATIONS, 10) - self.assertEqual(VERBOSE, False) - - -if __name__ == '__main__': - unittest.main() \ No newline at end of file diff --git a/tests/test_duckduckgo_integration.py b/tests/test_duckduckgo_integration.py deleted file mode 100644 index e25741a957e0c75b6318690aaf890432078e7c24..0000000000000000000000000000000000000000 --- a/tests/test_duckduckgo_integration.py +++ /dev/null @@ -1,274 +0,0 @@ -""" -Comprehensive test for DuckDuckGo integration with the GAIA agent. - -This script tests the DuckDuckGo search integration by: -1. Testing the direct connection to DuckDuckGo using the duckduckgo-search package -2. Testing the DuckDuckGoSearchTool class from web_tools.py -3. Verifying proper error handling and result formatting -4. Documenting how to integrate DuckDuckGo with the LangGraph-based GAIA agent -""" - -import unittest -import sys -import os -import logging -import json -from typing import Dict, Any, List, Optional - -sys.path.append('.') - -from tools.web_tools import DuckDuckGoSearchTool, create_duckduckgo_search - -from agent.agent import GAIAAgent -import agent.config as config -from agent.graph import run_agent_graph - -logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s') -logger = logging.getLogger("test_duckduckgo_integration") - -console_handler = logging.StreamHandler(sys.stdout) -console_handler.setLevel(logging.INFO) -formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s') -console_handler.setFormatter(formatter) -logger.addHandler(console_handler) - -class TestDuckDuckGoIntegration(unittest.TestCase): - """Test case for DuckDuckGo integration with the GAIA agent.""" - - def setUp(self): - """Set up the test case.""" - self.duckduckgo_tool = create_duckduckgo_search() - logger.info("DuckDuckGo search tool initialized") - - agent_config = config.get_agent_config() - model_config = config.get_model_config() - tool_config = config.get_tool_config() - - combined_config = { - **agent_config, - **model_config, - "tools": tool_config - } - - self.agent = GAIAAgent(config=combined_config) - logger.info("GAIA agent initialized") - - def test_duckduckgo_search_connection(self): - """Test the direct connection to DuckDuckGo.""" - try: - from duckduckgo_search import DDGS - logger.info("Successfully imported duckduckgo-search package") - - with DDGS() as ddgs: - logger.info("Successfully created DDGS instance") - - results = list(ddgs.text("test query", max_results=2, timelimit=5)) - - self.assertIsNotNone(results) - self.assertIsInstance(results, list) - logger.info(f"Successfully retrieved {len(results)} results from DuckDuckGo") - - for i, result in enumerate(results): - logger.info(f"Result {i+1}: {result.get('title', 'No title')} - {result.get('href', 'No URL')}") - - except ImportError: - self.fail("duckduckgo-search package is not installed") - except Exception as e: - self.fail(f"Error connecting to DuckDuckGo: {str(e)}") - - def test_duckduckgo_search_tool(self): - """Test the DuckDuckGoSearchTool class.""" - try: - results = self.duckduckgo_tool.search("python programming") - - self.assertIsNotNone(results) - self.assertIsInstance(results, list) - logger.info(f"Successfully retrieved {len(results)} results using DuckDuckGoSearchTool") - - if results: - result = results[0] - self.assertIn("title", result) - self.assertIn("link", result) - self.assertIn("snippet", result) - logger.info(f"Result format is correct: {result.keys()}") - - logger.info(f"First result: {result['title']} - {result['link']}") - - except Exception as e: - self.fail(f"Error using DuckDuckGoSearchTool: {str(e)}") - - def test_error_handling(self): - """Test error handling in the DuckDuckGoSearchTool.""" - with self.assertRaises(Exception): - self.duckduckgo_tool.search("") - - long_query = "a" * 1000 - try: - results = self.duckduckgo_tool.search(long_query) - logger.info(f"Long query handled successfully, got {len(results)} results") - except Exception as e: - logger.warning(f"Long query caused an error: {str(e)}") - - def test_result_count(self): - """Test that the result count configuration works.""" - self.duckduckgo_tool.result_count = 3 - - results = self.duckduckgo_tool.search("climate change") - - self.assertLessEqual(len(results), 3) - logger.info(f"Result count configuration works, got {len(results)} results") - - def test_integration_with_langgraph(self): - """ - Test how DuckDuckGo can be integrated with the LangGraph-based GAIA agent. - - This test doesn't actually run the integration, but documents how it would work. - """ - logger.info("Testing integration with LangGraph-based GAIA agent") - - integration_doc = """ - To integrate DuckDuckGo with the LangGraph-based GAIA agent: - - 1. Modify the execute_tool function in agent/graph.py to use real tools: - - Create a tool registry that maps tool names to tool instances - - Look up the tool in the registry based on the tool name - - Call the tool with the provided input - - Return the actual result - - 2. Register the DuckDuckGo tool in the tool registry: - - Create a DuckDuckGoSearchTool instance - - Add it to the tool registry with a name like "duckduckgo_search" - - 3. Update the agent configuration to include DuckDuckGo: - - Add DuckDuckGo configuration to the tool_config in agent/config.py - - Ensure the tool is available in the LangGraph workflow - - Example code for execute_tool function: - - ```python - def execute_tool(state: AgentState) -> AgentState: - # Get the current step from the plan - current_step = state["plan"][state["current_step"]] - tool_name = current_step.get("tool") - tool_input = current_step.get("tool_input", {}) - - - try: - # Get the tool from the registry - tool = tool_registry.get(tool_name) - if not tool: - raise Exception(f"Tool {tool_name} not found in registry") - - # Execute the tool - if tool_name == "duckduckgo_search": - result = tool.search(tool_input.get("query", "")) - else: - # Generic tool execution - result = tool.run(tool_input) - - # Create a tool result - tool_result = { - "tool_name": tool_name, - "success": True, - "result": result, - "error": None - } - - # Update the state - new_state = state.copy() - new_state["tool_results"] = state.get("tool_results", []) + [tool_result] - new_state["current_step"] += 1 - - return new_state - except Exception as e: - # Handle error... - # (rest of the function remains the same) - } - ``` - """ - - logger.info(integration_doc) - - self.assertTrue(True) - - def test_multiple_search_providers(self): - """Test using DuckDuckGo alongside other search providers.""" - logger.info("Testing DuckDuckGo alongside other search providers") - - serper_tool = None - serper_api_key = os.environ.get("SERPER_API_KEY") - if serper_api_key: - from tools.web_tools import create_serper_search - serper_tool = create_serper_search() - logger.info("Serper search tool initialized") - else: - logger.warning("Serper API key not available, skipping Serper test") - - perplexity_tool = None - perplexity_api_key = os.environ.get("PERPLEXITY_API_KEY") - if perplexity_api_key: - from tools.perplexity_tool import PerplexityTool - perplexity_tool = PerplexityTool(config={"api_key": perplexity_api_key}) - logger.info("Perplexity search tool initialized") - else: - logger.warning("Perplexity API key not available, skipping Perplexity test") - - query = "latest Python version" - results = {} - - # Test DuckDuckGo - try: - results["duckduckgo"] = self.duckduckgo_tool.search(query) - logger.info(f"DuckDuckGo found {len(results['duckduckgo'])} results") - except Exception as e: - logger.error(f"DuckDuckGo search failed: {str(e)}") - results["duckduckgo"] = None - - # Test Serper if available - if serper_tool: - try: - results["serper"] = serper_tool.search(query) - logger.info(f"Serper found {len(results['serper'])} results") - except Exception as e: - logger.error(f"Serper search failed: {str(e)}") - results["serper"] = None - - # Test Perplexity if available - if perplexity_tool: - try: - perplexity_results = perplexity_tool.search(query) - results["perplexity"] = perplexity_results - content = perplexity_results.get("content", "") - logger.info(f"Perplexity search successful, content length: {len(content) if content else 0}") - except Exception as e: - logger.error(f"Perplexity search failed: {str(e)}") - results["perplexity"] = None - - self.assertTrue(any(results.values()), "All search providers failed") - - logger.info(f"Successfully tested {len([r for r in results.values() if r])} out of {len(results)} providers") - - providers_with_results = [p for p, r in results.items() if r] - if len(providers_with_results) >= 2: - logger.info("Comparing results from different providers:") - for provider in providers_with_results: - if isinstance(results[provider], list) and results[provider]: - logger.info(f"{provider.capitalize()} first result: {results[provider][0].get('title', 'N/A')}") - -if __name__ == "__main__": - - test = TestDuckDuckGoIntegration() - test.setUp() - - test.test_duckduckgo_search_connection() - - test.test_duckduckgo_search_tool() - - test.test_error_handling() - - test.test_result_count() - - test.test_integration_with_langgraph() - - test.test_multiple_search_providers() - diff --git a/tests/test_duckduckgo_search.py b/tests/test_duckduckgo_search.py deleted file mode 100644 index 4602529b95b169c23acdd625cb4d7b09f6a2b22b..0000000000000000000000000000000000000000 --- a/tests/test_duckduckgo_search.py +++ /dev/null @@ -1,108 +0,0 @@ -""" -Standalone test for DuckDuckGo search functionality in the GAIA agent. - -This script tests the DuckDuckGo search integration by: -1. Testing the direct connection to DuckDuckGo using the duckduckgo-search package -2. Testing the DuckDuckGoSearchTool class from web_tools.py -3. Verifying proper error handling and result formatting -""" - -import unittest -import sys -import os -import logging -from typing import Dict, Any, List, Optional - -sys.path.append('.') - -from tools.web_tools import DuckDuckGoSearchTool, create_duckduckgo_search - -logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s') -logger = logging.getLogger("test_duckduckgo") - -console_handler = logging.StreamHandler(sys.stdout) -console_handler.setLevel(logging.INFO) -formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s') -console_handler.setFormatter(formatter) -logger.addHandler(console_handler) - -class TestDuckDuckGoSearch(unittest.TestCase): - """Test case for DuckDuckGo search functionality.""" - - def setUp(self): - """Set up the test case.""" - self.search_tool = create_duckduckgo_search() - logger.info("DuckDuckGo search tool initialized") - - def test_duckduckgo_search_connection(self): - """Test the direct connection to DuckDuckGo.""" - try: - from duckduckgo_search import DDGS - logger.info("Successfully imported duckduckgo-search package") - - with DDGS() as ddgs: - logger.info("Successfully created DDGS instance") - - results = list(ddgs.text("test query", max_results=2, timelimit=5)) - - self.assertIsNotNone(results) - self.assertIsInstance(results, list) - logger.info(f"Successfully retrieved {len(results)} results from DuckDuckGo") - - for i, result in enumerate(results): - logger.info(f"Result {i+1}: {result.get('title', 'No title')} - {result.get('href', 'No URL')}") - - except ImportError: - self.fail("duckduckgo-search package is not installed") - except Exception as e: - self.fail(f"Error connecting to DuckDuckGo: {str(e)}") - - def test_duckduckgo_search_tool(self): - """Test the DuckDuckGoSearchTool class.""" - try: - results = self.search_tool.search("python programming") - - self.assertIsNotNone(results) - self.assertIsInstance(results, list) - logger.info(f"Successfully retrieved {len(results)} results using DuckDuckGoSearchTool") - - if results: - result = results[0] - self.assertIn("title", result) - self.assertIn("link", result) - self.assertIn("snippet", result) - logger.info(f"Result format is correct: {result.keys()}") - - logger.info(f"First result: {result['title']} - {result['link']}") - - except Exception as e: - self.fail(f"Error using DuckDuckGoSearchTool: {str(e)}") - - def test_error_handling(self): - """Test error handling in the DuckDuckGoSearchTool.""" - with self.assertRaises(Exception): - self.search_tool.search("") - - long_query = "a" * 1000 - try: - results = self.search_tool.search(long_query) - logger.info(f"Long query handled successfully, got {len(results)} results") - except Exception as e: - logger.warning(f"Long query caused an error: {str(e)}") - - def test_result_count(self): - """Test that the result count configuration works.""" - self.search_tool.result_count = 3 - - results = self.search_tool.search("climate change") - - self.assertLessEqual(len(results), 3) - logger.info(f"Result count configuration works, got {len(results)} results") - -if __name__ == "__main__": - test = TestDuckDuckGoSearch() - test.setUp() - test.test_duckduckgo_search_connection() - test.test_duckduckgo_search_tool() - test.test_error_handling() - test.test_result_count() diff --git a/tests/test_gaia_agent.py b/tests/test_gaia_agent.py deleted file mode 100644 index b4f04f17ab989c7645de7ebe0f56dfb53b00f350..0000000000000000000000000000000000000000 --- a/tests/test_gaia_agent.py +++ /dev/null @@ -1,417 +0,0 @@ -""" -Comprehensive test script for the GAIA agent. - -This script tests the GAIA agent with different types of questions: -- Factual questions -- Reasoning questions -- Multimodal questions - -It verifies that all components are working correctly, including: -- Question analysis -- Planning -- Tool selection and execution -- Answer formulation -- Memory components - -The script generates a detailed report of the test results. -""" - -import os -import sys -import json -import time -import logging -import argparse -from typing import Dict, Any, List, Optional, Union -from datetime import datetime - -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(name)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("gaia_test") - -sys.path.append('.') - -try: - from agent.agent import GAIAAgent - from agent.config import get_memory_config, get_model_config, get_tool_config, get_agent_config -except ImportError as e: - logger.error(f"Error importing GAIA agent: {str(e)}") - sys.exit(1) - -class GAIAAgentTester: - """Test harness for the GAIA agent.""" - - def __init__(self, config: Optional[Dict[str, Any]] = None): - """ - Initialize the tester. - - Args: - config: Optional configuration dictionary - """ - self.config = config or {} - self.agent = None - self.results = { - "factual": [], - "reasoning": [], - "multimodal": [], - "memory": None - } - self.start_time = None - self.end_time = None - - self.sample_questions = { - "factual": [ - "What is the capital of France?", - "Who is the current CEO of OpenAI?", - "What is the boiling point of water in Celsius?" - ], - "reasoning": [ - "If a train travels at 60 mph, how long will it take to travel 180 miles?", - "If all A are B, and all B are C, what can we conclude about A and C?", - "What are the ethical implications of using AI in healthcare?" - ], - "multimodal": [ - "Describe what you see in this image: https://upload.wikimedia.org/wikipedia/commons/thumb/e/ec/Mona_Lisa%2C_by_Leonardo_da_Vinci%2C_from_C2RMF_retouched.jpg/800px-Mona_Lisa%2C_by_Leonardo_da_Vinci%2C_from_C2RMF_retouched.jpg", - "What information is shown in this chart: https://upload.wikimedia.org/wikipedia/commons/thumb/7/7e/Rainfall_Cherrapunji.svg/800px-Rainfall_Cherrapunji.svg.png", - "Summarize the content of this document: https://arxiv.org/pdf/2303.08774.pdf" - ] - } - - def initialize_agent(self) -> bool: - """ - Initialize the GAIA agent. - - Returns: - bool: True if initialization was successful, False otherwise - """ - logger.info("Initializing GAIA agent...") - - try: - self.agent = GAIAAgent(self.config) - - if self.agent: - logger.info("GAIA agent initialized successfully") - return True - else: - logger.error("Failed to initialize GAIA agent") - return False - except Exception as e: - logger.error(f"Error initializing GAIA agent: {str(e)}") - return False - - def test_question(self, question: str, category: str) -> Dict[str, Any]: - """ - Test a single question. - - Args: - question: The question to test - category: The category of the question (factual, reasoning, multimodal) - - Returns: - Dict[str, Any]: Test result - """ - logger.info(f"Testing {category} question: {question}") - - result = { - "question": question, - "category": category, - "start_time": datetime.now().isoformat(), - "answer": None, - "error": None, - "memory_snapshot": None, - "end_time": None, - "duration": None - } - - try: - answer = self.agent(question) - - result["answer"] = answer - result["end_time"] = datetime.now().isoformat() - - start_time = datetime.fromisoformat(result["start_time"]) - end_time = datetime.fromisoformat(result["end_time"]) - result["duration"] = (end_time - start_time).total_seconds() - - try: - result["memory_snapshot"] = self.agent.get_memory_snapshot() - except Exception as e: - logger.warning(f"Error getting memory snapshot: {str(e)}") - - logger.info(f"Question answered successfully in {result['duration']:.2f} seconds") - logger.info(f"Answer: {answer}") - - return result - except Exception as e: - logger.error(f"Error processing question: {str(e)}") - - result["error"] = str(e) - result["end_time"] = datetime.now().isoformat() - - start_time = datetime.fromisoformat(result["start_time"]) - end_time = datetime.fromisoformat(result["end_time"]) - result["duration"] = (end_time - start_time).total_seconds() - - return result - - def test_category(self, category: str) -> List[Dict[str, Any]]: - """ - Test all questions in a category. - - Args: - category: The category to test (factual, reasoning, multimodal) - - Returns: - List[Dict[str, Any]]: Test results - """ - logger.info(f"Testing {category} questions...") - - results = [] - - for question in self.sample_questions.get(category, []): - result = self.test_question(question, category) - results.append(result) - - return results - - def test_memory(self) -> Dict[str, Any]: - """ - Test memory components. - - Returns: - Dict[str, Any]: Memory test result - """ - logger.info("Testing memory components...") - - result = { - "start_time": datetime.now().isoformat(), - "conversation_memory": False, - "result_cache": False, - "working_memory": False, - "error": None, - "end_time": None, - "duration": None - } - - try: - if self.agent.conversation_memory: - self.agent.conversation_memory.add_message("user", "This is a test message") - - messages = self.agent.conversation_memory.get_messages() - - if any(m.get("content") == "This is a test message" for m in messages): - result["conversation_memory"] = True - logger.info("Conversation memory test passed") - else: - logger.warning("Conversation memory test failed") - else: - logger.warning("Conversation memory not available") - - if self.agent.result_cache: - test_key = "test:memory" - test_value = "This is a test value" - self.agent.result_cache.cache_result(test_key, test_value) - - cached_value = self.agent.result_cache.get_result(test_key) - - if cached_value == test_value: - result["result_cache"] = True - logger.info("Result cache test passed") - else: - logger.warning("Result cache test failed") - else: - logger.warning("Result cache not available") - - if self.agent.working_memory: - test_key = "test_key" - test_value = "This is a test value" - self.agent.working_memory.store_intermediate_result(test_key, test_value) - - stored_value = self.agent.working_memory.get_intermediate_result(test_key) - - if stored_value == test_value: - result["working_memory"] = True - logger.info("Working memory test passed") - else: - logger.warning("Working memory test failed") - else: - logger.warning("Working memory not available") - - result["end_time"] = datetime.now().isoformat() - - start_time = datetime.fromisoformat(result["start_time"]) - end_time = datetime.fromisoformat(result["end_time"]) - result["duration"] = (end_time - start_time).total_seconds() - - return result - except Exception as e: - logger.error(f"Error testing memory components: {str(e)}") - - result["error"] = str(e) - result["end_time"] = datetime.now().isoformat() - - start_time = datetime.fromisoformat(result["start_time"]) - end_time = datetime.fromisoformat(result["end_time"]) - result["duration"] = (end_time - start_time).total_seconds() - - return result - - def run_tests(self) -> Dict[str, Any]: - """ - Run all tests. - - Returns: - Dict[str, Any]: Test results - """ - logger.info("Starting GAIA agent tests...") - - self.start_time = datetime.now().isoformat() - - if not self.initialize_agent(): - return { - "error": "Failed to initialize GAIA agent", - "start_time": self.start_time, - "end_time": datetime.now().isoformat() - } - - self.results["factual"] = self.test_category("factual") - - self.results["reasoning"] = self.test_category("reasoning") - - self.results["multimodal"] = self.test_category("multimodal") - - self.results["memory"] = self.test_memory() - - try: - if self.agent: - self.agent.clear_memory() - except Exception as e: - logger.warning(f"Error clearing memory: {str(e)}") - - self.end_time = datetime.now().isoformat() - - start_time = datetime.fromisoformat(self.start_time) - end_time = datetime.fromisoformat(self.end_time) - duration = (end_time - start_time).total_seconds() - - report = { - "start_time": self.start_time, - "end_time": self.end_time, - "duration": duration, - "results": self.results, - "summary": self.generate_summary() - } - - logger.info(f"Tests completed in {duration:.2f} seconds") - - return report - - def generate_summary(self) -> Dict[str, Any]: - """ - Generate a summary of the test results. - - Returns: - Dict[str, Any]: Test summary - """ - summary = { - "factual": { - "total": len(self.results["factual"]), - "successful": sum(1 for r in self.results["factual"] if r.get("answer") and not r.get("error")), - "failed": sum(1 for r in self.results["factual"] if r.get("error")), - "average_duration": sum(r.get("duration", 0) for r in self.results["factual"]) / len(self.results["factual"]) if self.results["factual"] else 0 - }, - "reasoning": { - "total": len(self.results["reasoning"]), - "successful": sum(1 for r in self.results["reasoning"] if r.get("answer") and not r.get("error")), - "failed": sum(1 for r in self.results["reasoning"] if r.get("error")), - "average_duration": sum(r.get("duration", 0) for r in self.results["reasoning"]) / len(self.results["reasoning"]) if self.results["reasoning"] else 0 - }, - "multimodal": { - "total": len(self.results["multimodal"]), - "successful": sum(1 for r in self.results["multimodal"] if r.get("answer") and not r.get("error")), - "failed": sum(1 for r in self.results["multimodal"] if r.get("error")), - "average_duration": sum(r.get("duration", 0) for r in self.results["multimodal"]) / len(self.results["multimodal"]) if self.results["multimodal"] else 0 - }, - "memory": { - "conversation_memory": self.results["memory"].get("conversation_memory", False) if self.results["memory"] else False, - "result_cache": self.results["memory"].get("result_cache", False) if self.results["memory"] else False, - "working_memory": self.results["memory"].get("working_memory", False) if self.results["memory"] else False, - "error": self.results["memory"].get("error") if self.results["memory"] else "Memory test not run" - }, - "overall": { - "total_questions": len(self.results["factual"]) + len(self.results["reasoning"]) + len(self.results["multimodal"]), - "successful_questions": sum(1 for r in self.results["factual"] + self.results["reasoning"] + self.results["multimodal"] if r.get("answer") and not r.get("error")), - "failed_questions": sum(1 for r in self.results["factual"] + self.results["reasoning"] + self.results["multimodal"] if r.get("error")), - "memory_components_working": all([ - self.results["memory"].get("conversation_memory", False), - self.results["memory"].get("result_cache", False), - self.results["memory"].get("working_memory", False) - ]) if self.results["memory"] else False - } - } - - return summary - - def save_report(self, report: Dict[str, Any], output_file: str) -> bool: - """ - Save the test report to a file. - - Args: - report: The test report - output_file: The output file path - - Returns: - bool: True if the report was saved successfully, False otherwise - """ - try: - with open(output_file, 'w') as f: - json.dump(report, f, indent=2) - - logger.info(f"Report saved to {output_file}") - return True - except Exception as e: - logger.error(f"Error saving report: {str(e)}") - return False - - def print_report(self, report: Dict[str, Any]) -> None: - """ - Print the test report to the console. - - Args: - report: The test report - """ - summary = report["summary"] - - - - - - - -def main(): - """Main function.""" - parser = argparse.ArgumentParser(description="Test the GAIA agent") - parser.add_argument("--output", "-o", type=str, default="gaia_test_report.json", help="Output file for the test report") - parser.add_argument("--verbose", "-v", action="store_true", help="Enable verbose logging") - args = parser.parse_args() - - if args.verbose: - logger.setLevel(logging.DEBUG) - if not any(isinstance(h, logging.StreamHandler) for h in logger.handlers): - console_handler = logging.StreamHandler() - console_handler.setLevel(logging.DEBUG) - console_handler.setFormatter(logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")) - logger.addHandler(console_handler) - - tester = GAIAAgentTester() - - report = tester.run_tests() - - tester.save_report(report, args.output) - - tester.print_report(report) - -if __name__ == "__main__": - main() \ No newline at end of file diff --git a/tests/test_gaia_all_apis_integration.py b/tests/test_gaia_all_apis_integration.py deleted file mode 100644 index c603eca9be7e30dcf05b8e3bd605b905f6da52c3..0000000000000000000000000000000000000000 --- a/tests/test_gaia_all_apis_integration.py +++ /dev/null @@ -1,574 +0,0 @@ -""" -Comprehensive integration test for all APIs in the GAIA agent. - -This script tests the integration of all APIs in the GAIA agent: -1. OpenAI -2. Perplexity -3. Serper (Google) -4. DuckDuckGo -5. arXiv - -It verifies that the agent can use all these tools together to answer questions, -switch between different tools based on the task, and maintain memory across -different tool executions. - -Usage: - python tests/test_gaia_all_apis_integration.py [--verbose] [--report-file REPORT_FILE] -""" - -import unittest -import sys -import os -import json -import time -import logging -import argparse -from typing import Dict, Any, List, Optional, Tuple -from unittest.mock import patch, MagicMock, ANY - -sys.path.append('.') - -from agent.agent import GAIAAgent -from agent.tool_registry import ToolRegistry, create_default_registry, robust_search, merged_search -from tools.web_tools import DuckDuckGoSearchTool, SerperSearchTool, create_duckduckgo_search, create_serper_search -from tools.perplexity_tool import PerplexityTool, create_perplexity_tool -from tools.arxiv_tool import ArxivSearchTool, create_arxiv_search - -logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s') -logger = logging.getLogger("test_gaia_all_apis") - -console_handler = logging.StreamHandler(sys.stdout) -console_handler.setLevel(logging.INFO) -formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s') -console_handler.setFormatter(formatter) -logger.addHandler(console_handler) - -class TestGAIAAllAPIsIntegration(unittest.TestCase): - """Test case for integration of all APIs in the GAIA agent.""" - - def setUp(self): - """Set up the test case.""" - self.api_keys_available = { - "openai": bool(os.getenv("OPENAI_API_KEY")), - "serper": bool(os.getenv("SERPER_API_KEY")), - "perplexity": bool(os.getenv("PERPLEXITY_API_KEY")) - } - - logger.info(f"API keys available: {self.api_keys_available}") - - self.registry = create_default_registry() - logger.info(f"Tool registry created with tools: {self.registry.list_tools()}") - - try: - self.agent = GAIAAgent() - logger.info("GAIA agent initialized") - - self.test_results = { - "api_connections": {}, - "tool_executions": {}, - "complex_queries": {}, - "memory_integration": {} - } - except Exception as e: - logger.error(f"Error initializing GAIA agent: {str(e)}") - raise - - def tearDown(self): - """Clean up after the test.""" - if hasattr(self, 'agent') and self.agent: - self.agent.clear_memory() - - def test_api_connections(self): - """Test connections to all APIs.""" - logger.info("Testing connections to all APIs...") - - # Test OpenAI connection - if self.api_keys_available["openai"]: - try: - # Import OpenAI - from langchain_openai import ChatOpenAI - - model = ChatOpenAI(model="gpt-3.5-turbo") - - response = model.invoke("What is the capital of France?") - - self.test_results["api_connections"]["openai"] = { - "success": True, - "message": "Successfully connected to OpenAI API" - } - logger.info("✅ OpenAI connection successful") - except Exception as e: - self.test_results["api_connections"]["openai"] = { - "success": False, - "message": f"Failed to connect to OpenAI API: {str(e)}" - } - logger.error(f"❌ OpenAI connection failed: {str(e)}") - else: - self.test_results["api_connections"]["openai"] = { - "success": None, - "message": "OpenAI API key not available" - } - logger.warning("⚠️ OpenAI API key not available, skipping connection test") - - # Test Perplexity connection - if self.api_keys_available["perplexity"]: - try: - perplexity_tool = self.registry.get_tool("perplexity_search") - if perplexity_tool: - result = perplexity_tool.search("What is the capital of France?") - - self.test_results["api_connections"]["perplexity"] = { - "success": True, - "message": "Successfully connected to Perplexity API" - } - logger.info("✅ Perplexity connection successful") - else: - self.test_results["api_connections"]["perplexity"] = { - "success": False, - "message": "Perplexity tool not found in registry" - } - logger.error("❌ Perplexity tool not found in registry") - except Exception as e: - self.test_results["api_connections"]["perplexity"] = { - "success": False, - "message": f"Failed to connect to Perplexity API: {str(e)}" - } - logger.error(f"❌ Perplexity connection failed: {str(e)}") - else: - self.test_results["api_connections"]["perplexity"] = { - "success": None, - "message": "Perplexity API key not available" - } - logger.warning("⚠️ Perplexity API key not available, skipping connection test") - - # Test Serper connection - if self.api_keys_available["serper"]: - try: - serper_tool = self.registry.get_tool("serper_search") - if serper_tool: - result = serper_tool.search("What is the capital of France?") - - self.test_results["api_connections"]["serper"] = { - "success": True, - "message": "Successfully connected to Serper API" - } - logger.info("✅ Serper connection successful") - else: - self.test_results["api_connections"]["serper"] = { - "success": False, - "message": "Serper tool not found in registry" - } - logger.error("❌ Serper tool not found in registry") - except Exception as e: - self.test_results["api_connections"]["serper"] = { - "success": False, - "message": f"Failed to connect to Serper API: {str(e)}" - } - logger.error(f"❌ Serper connection failed: {str(e)}") - else: - self.test_results["api_connections"]["serper"] = { - "success": None, - "message": "Serper API key not available" - } - logger.warning("⚠️ Serper API key not available, skipping connection test") - - # Test DuckDuckGo connection (no API key required) - try: - duckduckgo_tool = self.registry.get_tool("duckduckgo_search") - if duckduckgo_tool: - result = duckduckgo_tool.search("What is the capital of France?") - - self.test_results["api_connections"]["duckduckgo"] = { - "success": True, - "message": "Successfully connected to DuckDuckGo" - } - logger.info("✅ DuckDuckGo connection successful") - else: - self.test_results["api_connections"]["duckduckgo"] = { - "success": False, - "message": "DuckDuckGo tool not found in registry" - } - logger.error("❌ DuckDuckGo tool not found in registry") - except Exception as e: - self.test_results["api_connections"]["duckduckgo"] = { - "success": False, - "message": f"Failed to connect to DuckDuckGo: {str(e)}" - } - logger.error(f"❌ DuckDuckGo connection failed: {str(e)}") - - try: - arxiv_tool = self.registry.get_tool("arxiv_search") - if arxiv_tool: - result = arxiv_tool.search("quantum computing", max_results=2) - - self.test_results["api_connections"]["arxiv"] = { - "success": True, - "message": "Successfully connected to arXiv" - } - logger.info("✅ arXiv connection successful") - else: - self.test_results["api_connections"]["arxiv"] = { - "success": False, - "message": "arXiv tool not found in registry" - } - logger.error("❌ arXiv tool not found in registry") - except Exception as e: - self.test_results["api_connections"]["arxiv"] = { - "success": False, - "message": f"Failed to connect to arXiv: {str(e)}" - } - logger.error(f"❌ arXiv connection failed: {str(e)}") - - successful_connections = [api for api, result in self.test_results["api_connections"].items() - if result.get("success") is True] - - self.assertTrue(len(successful_connections) > 0, - f"No API connections were successful. Available: {successful_connections}") - - logger.info(f"API connection tests completed. Successful connections: {successful_connections}") - - def test_individual_tool_executions(self): - """Test execution of each individual tool.""" - logger.info("Testing individual tool executions...") - - test_queries = { - "duckduckgo_search": "What is the capital of France?", - "serper_search": "What is the population of Tokyo?", - "perplexity_search": "Explain quantum computing in simple terms", - "arxiv_search": "Recent advances in machine learning" - } - - for tool_name, query in test_queries.items(): - try: - tool = self.registry.get_tool(tool_name) - - if not tool: - logger.warning(f"⚠️ Tool {tool_name} not found in registry, skipping test") - self.test_results["tool_executions"][tool_name] = { - "success": False, - "message": f"Tool {tool_name} not found in registry", - "query": query, - "result": None - } - continue - - if tool_name == "serper_search" and not self.api_keys_available.get("serper"): - logger.warning(f"⚠️ Skipping {tool_name} test (API key not available)") - self.test_results["tool_executions"][tool_name] = { - "success": None, - "message": "API key not available", - "query": query, - "result": None - } - continue - - if tool_name == "perplexity_search" and not self.api_keys_available.get("perplexity"): - logger.warning(f"⚠️ Skipping {tool_name} test (API key not available)") - self.test_results["tool_executions"][tool_name] = { - "success": None, - "message": "API key not available", - "query": query, - "result": None - } - continue - - logger.info(f"Testing {tool_name} with query: {query}") - - start_time = time.time() - result = self.registry.execute_tool(tool_name, query=query) - end_time = time.time() - - self.assertIsNotNone(result, f"{tool_name} returned None result") - - if isinstance(result, list): - logger.info(f"{tool_name} returned {len(result)} results in {end_time - start_time:.2f} seconds") - else: - logger.info(f"{tool_name} returned result in {end_time - start_time:.2f} seconds") - - self.test_results["tool_executions"][tool_name] = { - "success": True, - "message": f"Successfully executed {tool_name}", - "query": query, - "result": result, - "execution_time": end_time - start_time - } - - logger.info(f"✅ {tool_name} execution successful") - - except Exception as e: - logger.error(f"❌ {tool_name} execution failed: {str(e)}") - self.test_results["tool_executions"][tool_name] = { - "success": False, - "message": f"Failed to execute {tool_name}: {str(e)}", - "query": query, - "result": None - } - - try: - logger.info("Testing robust_search function...") - - query = "What is the tallest building in the world?" - result = robust_search(self.registry, query) - - self.assertIsNotNone(result, "robust_search returned None result") - self.assertIn("provider", result, "robust_search result missing 'provider' key") - self.assertIn("results", result, "robust_search result missing 'results' key") - - logger.info(f"robust_search used provider: {result['provider']}") - - self.test_results["tool_executions"]["robust_search"] = { - "success": True, - "message": f"Successfully executed robust_search using {result['provider']}", - "query": query, - "provider": result["provider"], - "result_count": len(result["results"]) if isinstance(result["results"], list) else 1 - } - - logger.info("✅ robust_search execution successful") - - except Exception as e: - logger.error(f"❌ robust_search execution failed: {str(e)}") - self.test_results["tool_executions"]["robust_search"] = { - "success": False, - "message": f"Failed to execute robust_search: {str(e)}", - "query": query, - "result": None - } - - try: - logger.info("Testing merged_search function...") - - query = "Latest developments in renewable energy" - result = merged_search(self.registry, query) - - self.assertIsNotNone(result, "merged_search returned None result") - self.assertIn("merged_results", result, "merged_search result missing 'merged_results' key") - - logger.info(f"merged_search returned {len(result['merged_results'])} merged results") - - self.test_results["tool_executions"]["merged_search"] = { - "success": True, - "message": "Successfully executed merged_search", - "query": query, - "result_count": len(result["merged_results"]), - "has_perplexity": result["perplexity_context"] is not None, - "has_arxiv": len(result.get("arxiv_context", [])) > 0 - } - - logger.info("✅ merged_search execution successful") - - except Exception as e: - logger.error(f"❌ merged_search execution failed: {str(e)}") - self.test_results["tool_executions"]["merged_search"] = { - "success": False, - "message": f"Failed to execute merged_search: {str(e)}", - "query": query, - "result": None - } - - @patch('agent.graph.run_agent_graph') - def test_tool_switching(self, mock_run_agent_graph): - """Test that the agent can switch between different tools based on the task.""" - logger.info("Testing tool switching based on task...") - - test_cases = [ - { - "question": "What is the latest research on quantum computing?", - "expected_tools": ["arxiv_search", "perplexity_search"], - "question_type": "scientific" - }, - { - "question": "What are the current stock prices for major tech companies?", - "expected_tools": ["duckduckgo_search", "serper_search"], - "question_type": "factual" - }, - { - "question": "Explain the theory of relativity in simple terms", - "expected_tools": ["perplexity_search"], - "question_type": "explanatory" - } - ] - - for i, test_case in enumerate(test_cases): - question = test_case["question"] - expected_tools = test_case["expected_tools"] - question_type = test_case["question_type"] - - logger.info(f"Test case {i+1}: {question_type} question - {question}") - - mock_plan = [ - { - "step_number": j+1, - "description": f"Use {tool} to find information", - "tool": tool, - "tool_input": {"query": question} - } - for j, tool in enumerate(expected_tools) - ] - - mock_run_agent_graph.return_value = { - "analysis": { - "question_type": question_type, - "required_tools": ["web_search"], - "information_sources": ["web", "academic"] if "arxiv" in expected_tools[0] else ["web"], - "complexity": "medium" - }, - "plan": mock_plan, - "answer": f"Mock answer for {question}" - } - - try: - answer = self.agent(question) - - mock_run_agent_graph.assert_called_with(question, ANY) - - self.test_results["complex_queries"][f"case_{i+1}"] = { - "success": True, - "question": question, - "question_type": question_type, - "expected_tools": expected_tools, - "answer": answer - } - - logger.info(f"✅ Test case {i+1} successful") - - except Exception as e: - logger.error(f"❌ Test case {i+1} failed: {str(e)}") - self.test_results["complex_queries"][f"case_{i+1}"] = { - "success": False, - "question": question, - "question_type": question_type, - "expected_tools": expected_tools, - "error": str(e) - } - - def test_memory_integration(self): - """Test memory integration across different tools.""" - logger.info("Testing memory integration across different tools...") - - if not hasattr(self.agent, 'memory') or not self.agent.memory: - logger.warning("⚠️ Agent doesn't have memory components, skipping memory integration test") - self.test_results["memory_integration"]["status"] = { - "success": None, - "message": "Agent doesn't have memory components" - } - return - - questions = [ - "What is quantum computing?", - "Who are the leading researchers in this field?", - "What are their most recent papers?" - ] - - try: - answers = [] - for i, question in enumerate(questions): - logger.info(f"Processing question {i+1}: {question}") - - answer = self.agent(question) - answers.append(answer) - - logger.info(f"Answer {i+1}: {answer[:100]}..." if len(answer) > 100 else f"Answer {i+1}: {answer}") - - memory_snapshot = self.agent.get_memory_snapshot() - - conversation = memory_snapshot.get("conversation", []) - self.assertEqual(len(conversation), len(questions) * 2, - f"Expected {len(questions) * 2} messages in conversation history, got {len(conversation)}") - - self.test_results["memory_integration"]["conversation"] = { - "success": True, - "questions": questions, - "answers": answers, - "conversation_length": len(conversation) - } - - cached_results = memory_snapshot.get("cached_results", {}) - self.test_results["memory_integration"]["caching"] = { - "success": True, - "cached_results_count": len(cached_results) - } - - intermediate_results = memory_snapshot.get("intermediate_results", {}) - self.test_results["memory_integration"]["intermediate_results"] = { - "success": True, - "intermediate_results_count": len(intermediate_results) - } - - logger.info("✅ Memory integration test successful") - self.test_results["memory_integration"]["status"] = { - "success": True, - "message": "Memory integration test successful" - } - - except Exception as e: - logger.error(f"❌ Memory integration test failed: {str(e)}") - self.test_results["memory_integration"]["status"] = { - "success": False, - "message": f"Memory integration test failed: {str(e)}" - } - - def generate_report(self, report_file: str = "gaia_all_apis_integration_report.json"): - """ - Generate a final report of all API integrations. - - Args: - report_file: Output file for the report - """ - logger.info("Generating final report...") - - report = { - "timestamp": time.strftime("%Y-%m-%d %H:%M:%S"), - "api_connections": self.test_results["api_connections"], - "tool_executions": self.test_results["tool_executions"], - "complex_queries": self.test_results["complex_queries"], - "memory_integration": self.test_results["memory_integration"], - "summary": { - "total_apis_tested": len(self.test_results["api_connections"]), - "successful_connections": len([api for api, result in self.test_results["api_connections"].items() - if result.get("success") is True]), - "successful_tool_executions": len([tool for tool, result in self.test_results["tool_executions"].items() - if result.get("success") is True]), - "successful_complex_queries": len([query for query, result in self.test_results["complex_queries"].items() - if result.get("success") is True]), - "memory_integration_success": self.test_results["memory_integration"].get("status", {}).get("success") - } - } - - with open(report_file, "w") as f: - json.dump(report, f, indent=2) - - logger.info(f"Report saved to {report_file}") - - - - for api, result in report["api_connections"].items(): - status = "✅" if result.get("success") is True else "❌" if result.get("success") is False else "⚠️" - - -def main(): - """Main function.""" - parser = argparse.ArgumentParser(description="Test all APIs integration in the GAIA agent") - parser.add_argument("--verbose", "-v", action="store_true", help="Enable verbose logging") - parser.add_argument("--report-file", "-r", type=str, default="gaia_all_apis_integration_report.json", - help="Output file for the integration test report") - args = parser.parse_args() - - if args.verbose: - logger.setLevel(logging.DEBUG) - - suite = unittest.TestSuite() - test_case = TestGAIAAllAPIsIntegration() - - suite.addTest(unittest.makeSuite(TestGAIAAllAPIsIntegration)) - - runner = unittest.TextTestRunner(verbosity=2) - result = runner.run(suite) - - if hasattr(test_case, 'generate_report'): - test_case.generate_report(args.report_file) - - return 0 if result.wasSuccessful() else 1 - -if __name__ == "__main__": - sys.exit(main()) \ No newline at end of file diff --git a/tests/test_gaia_duckduckgo_integration.py b/tests/test_gaia_duckduckgo_integration.py deleted file mode 100644 index 7b7ce99835328cc3b475a8811bbe0038c31e58b0..0000000000000000000000000000000000000000 --- a/tests/test_gaia_duckduckgo_integration.py +++ /dev/null @@ -1,149 +0,0 @@ -""" -Integration test for DuckDuckGo search in the GAIA agent. - -This script tests the integration of DuckDuckGo search in the GAIA agent by: -1. Testing the agent's ability to use DuckDuckGo search to answer questions -2. Verifying that the agent can handle DuckDuckGo search errors gracefully -3. Testing the agent with mocked DuckDuckGo search results -""" - -import unittest -import sys -import os -import logging -from unittest.mock import patch, MagicMock -from typing import Dict, Any, List, Optional - -sys.path.append('.') - -from agent.agent import GAIAAgent -from tools.web_tools import DuckDuckGoSearchTool - -logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s') -logger = logging.getLogger("test_gaia_duckduckgo") - -class TestGAIADuckDuckGoIntegration(unittest.TestCase): - """Test case for DuckDuckGo integration in the GAIA agent.""" - - def setUp(self): - """Set up the test case.""" - try: - self.agent = GAIAAgent() - logger.info("GAIA agent initialized") - except Exception as e: - logger.error(f"Error initializing GAIA agent: {str(e)}") - raise - - def tearDown(self): - """Clean up after the test.""" - if hasattr(self, 'agent') and self.agent: - self.agent.clear_memory() - - @patch('tools.web_tools.DDGS') - def test_agent_with_duckduckgo_mock(self, mock_ddgs): - """Test the agent with mocked DuckDuckGo search results.""" - mock_ddgs_instance = MagicMock() - mock_ddgs.return_value.__enter__.return_value = mock_ddgs_instance - - mock_ddgs_instance.text.return_value = [ - { - "title": "Python Programming Language", - "href": "https://www.python.org/", - "body": "Python is a programming language that lets you work quickly and integrate systems more effectively." - }, - { - "title": "Python (programming language) - Wikipedia", - "href": "https://en.wikipedia.org/wiki/Python_(programming_language)", - "body": "Python is a high-level, general-purpose programming language. Its design philosophy emphasizes code readability with the use of significant indentation." - } - ] - - question = "What is Python programming language?" - logger.info(f"Testing agent with question: {question}") - - try: - answer = self.agent(question) - logger.info(f"Agent answer: {answer}") - - mock_ddgs_instance.text.assert_called_once() - logger.info("DuckDuckGo search was called") - - self.assertIsNotNone(answer) - self.assertIsInstance(answer, str) - logger.info("Agent returned a valid answer") - - except Exception as e: - self.fail(f"Error testing agent with mocked DuckDuckGo search: {str(e)}") - - @patch('agent.graph.run_agent_graph') - def test_agent_question_analysis(self, mock_run_agent_graph): - """Test that the agent correctly analyzes questions that require web search.""" - mock_run_agent_graph.return_value = { - "analysis": { - "question_type": "factual", - "required_tools": ["web_search"], - "information_sources": ["web"], - "complexity": "medium" - }, - "plan": [ - { - "step_number": 1, - "description": "Search the web for information about Python programming", - "tool": "duckduckgo_search", - "tool_input": {"query": "Python programming language"} - }, - { - "step_number": 2, - "description": "Extract relevant information from search results", - "tool": None, - "tool_input": None - } - ], - "answer": "Python is a high-level, general-purpose programming language." - } - - question = "What is Python programming language?" - logger.info(f"Testing agent question analysis with: {question}") - - try: - answer = self.agent(question) - logger.info(f"Agent answer: {answer}") - - mock_run_agent_graph.assert_called_once() - call_args = mock_run_agent_graph.call_args[0] - self.assertEqual(call_args[0], question) - logger.info("Agent graph was called with the correct question") - - self.assertEqual(answer, "Python is a high-level, general-purpose programming language.") - logger.info("Agent returned the expected answer") - - except Exception as e: - self.fail(f"Error testing agent question analysis: {str(e)}") - - def test_duckduckgo_error_handling(self): - """Test that the agent can handle DuckDuckGo search errors gracefully.""" - - with patch('tools.web_tools.DuckDuckGoSearchTool.search') as mock_search: - mock_search.side_effect = Exception("DuckDuckGo search failed") - - if hasattr(self.agent, 'config') and isinstance(self.agent.config, dict): - if 'agent' in self.agent.config: - self.agent.config['agent']['return_error_message'] = True - - question = "What is the capital of France?" - logger.info(f"Testing agent error handling with: {question}") - - try: - answer = self.agent(question) - logger.info(f"Agent answer: {answer}") - - self.assertIsNotNone(answer) - self.assertIsInstance(answer, str) - logger.info("Agent handled the error gracefully") - - except Exception as e: - logger.warning(f"Agent raised an exception: {str(e)}") - pass - -if __name__ == "__main__": - unittest.main() \ No newline at end of file diff --git a/tests/test_gaia_memory.py b/tests/test_gaia_memory.py deleted file mode 100644 index 88d91d825e808364e9098dc60e8b4a0ed3207e06..0000000000000000000000000000000000000000 --- a/tests/test_gaia_memory.py +++ /dev/null @@ -1,201 +0,0 @@ -""" -Test GAIA Agent With Memory - -This script tests the GAIA agent with a sample question and verifies that all components, -including memory, are working correctly. -""" - -import os -import sys -import json -import time -import logging -import argparse -import uuid -from typing import Dict, Any, List, Optional, Union, Tuple - -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(name)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("gaia_agent_test") - -try: - from agent.agent import GAIAAgent - from agent.config import get_agent_config, get_model_config, get_memory_config -except ImportError as e: - logger.error(f"Error importing agent components: {str(e)}") - sys.exit(1) - -class GAIAAgentTester: - """Class for testing the GAIA agent with memory.""" - - def __init__(self, verbose: bool = False): - """ - Initialize the GAIA agent tester. - - Args: - verbose: Whether to enable verbose logging - """ - self.verbose = verbose - self.session_id = str(uuid.uuid4()) - self.conversation_id = str(uuid.uuid4()) - - if verbose: - logging.getLogger().setLevel(logging.DEBUG) - - self._initialize_agent() - - def _initialize_agent(self) -> None: - """Initialize the GAIA agent.""" - logger.info("Initializing GAIA agent...") - - try: - agent_config = get_agent_config() - model_config = get_model_config() - memory_config = get_memory_config() - - memory_config["enabled"] = True - - self.agent = GAIAAgent( - agent_config=agent_config, - model_config=model_config, - memory_config=memory_config, - verbose=self.verbose - ) - - self.agent.memory.session_id = self.session_id - self.agent.memory.conversation_id = self.conversation_id - - logger.info("GAIA agent initialized successfully") - logger.info(f"Session ID: {self.session_id}") - logger.info(f"Conversation ID: {self.conversation_id}") - except Exception as e: - logger.error(f"Error initializing GAIA agent: {str(e)}") - sys.exit(1) - - def test_memory_components(self) -> Dict[str, bool]: - """ - Test memory components. - - Returns: - Dict[str, bool]: Dictionary with test results - """ - logger.info("Testing memory components...") - - results = { - "conversation_memory": False, - "result_cache": False, - "working_memory": False - } - - try: - logger.info("Testing conversation memory...") - test_message = f"Test message at {time.time()}" - self.agent.memory.conversation.add_message("user", test_message) - messages = self.agent.memory.conversation.get_messages() - - if messages and any(m.get("content") == test_message for m in messages): - results["conversation_memory"] = True - logger.info("✅ Conversation memory test passed") - else: - logger.warning("❌ Conversation memory test failed") - - logger.info("Testing result cache...") - test_query = f"test_query_{time.time()}" - test_result = f"test_result_{time.time()}" - self.agent.memory.cache.cache_result(test_query, test_result) - result = self.agent.memory.cache.get_result(test_query) - - if result == test_result: - results["result_cache"] = True - logger.info("✅ Result cache test passed") - else: - logger.warning("❌ Result cache test failed") - - logger.info("Testing working memory...") - test_key = f"test_key_{time.time()}" - test_value = f"test_value_{time.time()}" - self.agent.memory.working.store_intermediate_result(test_key, test_value) - value = self.agent.memory.working.get_intermediate_result(test_key) - - if value == test_value: - results["working_memory"] = True - logger.info("✅ Working memory test passed") - else: - logger.warning("❌ Working memory test failed") - except Exception as e: - logger.error(f"Error testing memory components: {str(e)}") - - return results - - def test_sample_question(self, question: str) -> None: - """ - Test the agent with a sample question. - - Args: - question: The question to test - """ - logger.info(f"Testing question: {question}") - - try: - start_time = time.time() - response = self.agent.process_question(question) - end_time = time.time() - - - logger.info(f"Successfully processed question: {question}") - - messages = self.agent.memory.conversation.get_messages() - if messages: - for i, message in enumerate(messages): - - return response - except Exception as e: - logger.error(f"Error processing question '{question}': {str(e)}") - return None - - def run_full_test(self) -> None: - """Run a full test of the GAIA agent with memory.""" - - memory_results = self.test_memory_components() - - for component, result in memory_results.items(): - status = "✅ Passed" if result else "❌ Failed" - - sample_question = "What is the capital of France?" - response1 = self.test_sample_question(sample_question) - - if response1: - follow_up_question = "What is its population?" - response2 = self.test_sample_question(follow_up_question) - - - memory_success = all(memory_results.values()) - if memory_success: - else: - - if response1: - else: - - if not memory_success: - - - -def main(): - """Main function.""" - parser = argparse.ArgumentParser(description="Test GAIA agent with memory") - parser.add_argument("--verbose", action="store_true", help="Enable verbose logging") - parser.add_argument("--question", type=str, help="Custom question to test") - args = parser.parse_args() - - tester = GAIAAgentTester(verbose=args.verbose) - - if args.question: - tester.test_sample_question(args.question) - else: - tester.run_full_test() - - -if __name__ == "__main__": - main() \ No newline at end of file diff --git a/tests/test_gaia_multi_provider.py b/tests/test_gaia_multi_provider.py deleted file mode 100644 index 4d5daa8c9021bb559b7376e981ac17c64c5be29f..0000000000000000000000000000000000000000 --- a/tests/test_gaia_multi_provider.py +++ /dev/null @@ -1,454 +0,0 @@ -""" -Test script for GAIA agent with multiple AI providers. - -This script tests the GAIA agent's ability to use both OpenAI and Perplexity -tools together in the same workflow. It verifies that: -1. The agent can initialize with both providers' configurations -2. The agent can select the appropriate provider for different tasks -3. The agent can combine results from different providers -4. The configuration changes work properly for both providers - -Usage: - python tests/test_gaia_multi_provider.py [--verbose] [--output FILENAME] -""" - -import os -import sys -import json -import time -import logging -import argparse -from typing import Dict, Any, List, Tuple, Optional -from datetime import datetime - -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(name)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("gaia_multi_provider_test") - -sys.path.append('.') - -try: - from agent.config import ( - OPENAI_API_KEY, - PERPLEXITY_API_KEY, - get_model_config, - get_tool_config, - get_memory_config, - get_agent_config - ) -except ImportError as e: - logger.error(f"Error importing configuration: {str(e)}") - sys.exit(1) - -class MultiProviderTester: - """Test harness for the GAIA agent with multiple AI providers.""" - - def __init__(self, config: Optional[Dict[str, Any]] = None): - """ - Initialize the tester. - - Args: - config: Optional configuration dictionary - """ - self.config = config or {} - self.agent = None - self.results = { - "openai_only": [], - "perplexity_only": [], - "combined": [], - "memory": None - } - self.start_time = None - self.end_time = None - - self.sample_questions = { - "openai_only": [ - "What is the capital of France?", - "Calculate the square root of 144." - ], - "perplexity_only": [ - "What are the latest developments in quantum computing?", - "Summarize the current state of climate change research." - ], - "combined": [ - "Compare the performance of GPT-4o and Perplexity's Sonar model on reasoning tasks.", - "What are the key differences between OpenAI and Perplexity API pricing models?" - ] - } - - def initialize_agent(self) -> bool: - """ - Initialize the GAIA agent. - - Returns: - bool: True if initialization was successful, False otherwise - """ - logger.info("Initializing GAIA agent...") - - try: - from agent.agent import GAIAAgent - - self.agent = GAIAAgent(self.config) - - if self.agent: - logger.info("GAIA agent initialized successfully") - return True - else: - logger.error("Failed to initialize GAIA agent") - return False - except Exception as e: - logger.error(f"Error initializing GAIA agent: {str(e)}") - return False - - def verify_api_keys(self) -> Dict[str, bool]: - """ - Verify that both API keys are set. - - Returns: - Dict[str, bool]: Status of API keys - """ - return { - "openai": bool(OPENAI_API_KEY), - "perplexity": bool(PERPLEXITY_API_KEY) - } - - def test_question(self, question: str, category: str) -> Dict[str, Any]: - """ - Test a single question. - - Args: - question: The question to test - category: The category of the question (openai_only, perplexity_only, combined) - - Returns: - Dict[str, Any]: Test result - """ - logger.info(f"Testing {category} question: {question}") - - result = { - "question": question, - "category": category, - "start_time": datetime.now().isoformat(), - "answer": None, - "error": None, - "memory_snapshot": None, - "end_time": None, - "duration": None - } - - try: - answer = self.agent(question) - - result["answer"] = answer - result["end_time"] = datetime.now().isoformat() - - start_time = datetime.fromisoformat(result["start_time"]) - end_time = datetime.fromisoformat(result["end_time"]) - result["duration"] = (end_time - start_time).total_seconds() - - try: - result["memory_snapshot"] = self.agent.get_memory_snapshot() - except Exception as e: - logger.warning(f"Error getting memory snapshot: {str(e)}") - - logger.info(f"Question answered successfully in {result['duration']:.2f} seconds") - logger.info(f"Answer: {answer}") - - return result - except Exception as e: - logger.error(f"Error processing question: {str(e)}") - - result["error"] = str(e) - result["end_time"] = datetime.now().isoformat() - - start_time = datetime.fromisoformat(result["start_time"]) - end_time = datetime.fromisoformat(result["end_time"]) - result["duration"] = (end_time - start_time).total_seconds() - - return result - - def test_category(self, category: str) -> List[Dict[str, Any]]: - """ - Test all questions in a category. - - Args: - category: The category to test (openai_only, perplexity_only, combined) - - Returns: - List[Dict[str, Any]]: Test results - """ - logger.info(f"Testing {category} questions...") - - results = [] - - for question in self.sample_questions.get(category, []): - result = self.test_question(question, category) - results.append(result) - - return results - - def test_memory(self) -> Dict[str, Any]: - """ - Test memory components with multi-provider data. - - Returns: - Dict[str, Any]: Memory test result - """ - logger.info("Testing memory components with multi-provider data...") - - result = { - "start_time": datetime.now().isoformat(), - "conversation_memory": False, - "result_cache": False, - "working_memory": False, - "error": None, - "end_time": None, - "duration": None - } - - try: - if self.agent.conversation_memory: - self.agent.conversation_memory.add_message("user", "OpenAI test message") - self.agent.conversation_memory.add_message("assistant", "OpenAI response") - self.agent.conversation_memory.add_message("user", "Perplexity test message") - self.agent.conversation_memory.add_message("assistant", "Perplexity response") - - messages = self.agent.conversation_memory.get_messages() - - if (any(m.get("content") == "OpenAI test message" for m in messages) and - any(m.get("content") == "Perplexity test message" for m in messages)): - result["conversation_memory"] = True - logger.info("Conversation memory test passed") - else: - logger.warning("Conversation memory test failed") - else: - logger.warning("Conversation memory not available") - - if self.agent.result_cache: - openai_key = "test:openai" - perplexity_key = "test:perplexity" - self.agent.result_cache.cache_result(openai_key, "OpenAI test value") - self.agent.result_cache.cache_result(perplexity_key, "Perplexity test value") - - openai_value = self.agent.result_cache.get_result(openai_key) - perplexity_value = self.agent.result_cache.get_result(perplexity_key) - - if openai_value == "OpenAI test value" and perplexity_value == "Perplexity test value": - result["result_cache"] = True - logger.info("Result cache test passed") - else: - logger.warning("Result cache test failed") - else: - logger.warning("Result cache not available") - - if self.agent.working_memory: - openai_key = "openai_test" - perplexity_key = "perplexity_test" - self.agent.working_memory.store_intermediate_result(openai_key, "OpenAI test value") - self.agent.working_memory.store_intermediate_result(perplexity_key, "Perplexity test value") - - openai_value = self.agent.working_memory.get_intermediate_result(openai_key) - perplexity_value = self.agent.working_memory.get_intermediate_result(perplexity_key) - - if openai_value == "OpenAI test value" and perplexity_value == "Perplexity test value": - result["working_memory"] = True - logger.info("Working memory test passed") - else: - logger.warning("Working memory test failed") - else: - logger.warning("Working memory not available") - - result["end_time"] = datetime.now().isoformat() - - start_time = datetime.fromisoformat(result["start_time"]) - end_time = datetime.fromisoformat(result["end_time"]) - result["duration"] = (end_time - start_time).total_seconds() - - return result - except Exception as e: - logger.error(f"Error testing memory components: {str(e)}") - - result["error"] = str(e) - result["end_time"] = datetime.now().isoformat() - - start_time = datetime.fromisoformat(result["start_time"]) - end_time = datetime.fromisoformat(result["end_time"]) - result["duration"] = (end_time - start_time).total_seconds() - - return result - - def run_tests(self) -> Dict[str, Any]: - """ - Run all tests. - - Returns: - Dict[str, Any]: Test results - """ - logger.info("Starting GAIA multi-provider tests...") - - self.start_time = datetime.now().isoformat() - - api_keys = self.verify_api_keys() - if not api_keys["openai"] or not api_keys["perplexity"]: - return { - "error": "Missing API keys", - "api_keys": api_keys, - "start_time": self.start_time, - "end_time": datetime.now().isoformat() - } - - if not self.initialize_agent(): - return { - "error": "Failed to initialize GAIA agent", - "start_time": self.start_time, - "end_time": datetime.now().isoformat() - } - - # Test OpenAI-only questions - self.results["openai_only"] = self.test_category("openai_only") - - # Test Perplexity-only questions - self.results["perplexity_only"] = self.test_category("perplexity_only") - - self.results["combined"] = self.test_category("combined") - - self.results["memory"] = self.test_memory() - - try: - if self.agent: - self.agent.clear_memory() - except Exception as e: - logger.warning(f"Error clearing memory: {str(e)}") - - self.end_time = datetime.now().isoformat() - - start_time = datetime.fromisoformat(self.start_time) - end_time = datetime.fromisoformat(self.end_time) - duration = (end_time - start_time).total_seconds() - - report = { - "start_time": self.start_time, - "end_time": self.end_time, - "duration": duration, - "api_keys": api_keys, - "results": self.results, - "summary": self.generate_summary() - } - - logger.info(f"Tests completed in {duration:.2f} seconds") - - return report - - def generate_summary(self) -> Dict[str, Any]: - """ - Generate a summary of the test results. - - Returns: - Dict[str, Any]: Test summary - """ - summary = { - "openai_only": { - "total": len(self.results["openai_only"]), - "successful": sum(1 for r in self.results["openai_only"] if r.get("answer") and not r.get("error")), - "failed": sum(1 for r in self.results["openai_only"] if r.get("error")), - "average_duration": sum(r.get("duration", 0) for r in self.results["openai_only"]) / len(self.results["openai_only"]) if self.results["openai_only"] else 0 - }, - "perplexity_only": { - "total": len(self.results["perplexity_only"]), - "successful": sum(1 for r in self.results["perplexity_only"] if r.get("answer") and not r.get("error")), - "failed": sum(1 for r in self.results["perplexity_only"] if r.get("error")), - "average_duration": sum(r.get("duration", 0) for r in self.results["perplexity_only"]) / len(self.results["perplexity_only"]) if self.results["perplexity_only"] else 0 - }, - "combined": { - "total": len(self.results["combined"]), - "successful": sum(1 for r in self.results["combined"] if r.get("answer") and not r.get("error")), - "failed": sum(1 for r in self.results["combined"] if r.get("error")), - "average_duration": sum(r.get("duration", 0) for r in self.results["combined"]) / len(self.results["combined"]) if self.results["combined"] else 0 - }, - "memory": { - "conversation_memory": self.results["memory"].get("conversation_memory", False) if self.results["memory"] else False, - "result_cache": self.results["memory"].get("result_cache", False) if self.results["memory"] else False, - "working_memory": self.results["memory"].get("working_memory", False) if self.results["memory"] else False, - "error": self.results["memory"].get("error") if self.results["memory"] else "Memory test not run" - }, - "overall": { - "total_questions": len(self.results["openai_only"]) + len(self.results["perplexity_only"]) + len(self.results["combined"]), - "successful_questions": sum(1 for r in self.results["openai_only"] + self.results["perplexity_only"] + self.results["combined"] if r.get("answer") and not r.get("error")), - "failed_questions": sum(1 for r in self.results["openai_only"] + self.results["perplexity_only"] + self.results["combined"] if r.get("error")), - "memory_components_working": all([ - self.results["memory"].get("conversation_memory", False), - self.results["memory"].get("result_cache", False), - self.results["memory"].get("working_memory", False) - ]) if self.results["memory"] else False - } - } - - return summary - - def save_report(self, report: Dict[str, Any], output_file: str) -> bool: - """ - Save the test report to a file. - - Args: - report: The test report - output_file: The output file path - - Returns: - bool: True if the report was saved successfully, False otherwise - """ - try: - with open(output_file, 'w') as f: - json.dump(report, f, indent=2) - - logger.info(f"Report saved to {output_file}") - return True - except Exception as e: - logger.error(f"Error saving report: {str(e)}") - return False - - def print_report(self, report: Dict[str, Any]) -> None: - """ - Print the test report to the console. - - Args: - report: The test report - """ - summary = report["summary"] - - - - - - - - -def main(): - """Main function.""" - parser = argparse.ArgumentParser(description="Test GAIA agent with multiple AI providers") - parser.add_argument("--output", "-o", type=str, default="gaia_multi_provider_test_report.json", help="Output file for the test report") - parser.add_argument("--verbose", "-v", action="store_true", help="Enable verbose logging") - args = parser.parse_args() - - if args.verbose: - logger.setLevel(logging.DEBUG) - if not any(isinstance(h, logging.StreamHandler) for h in logger.handlers): - console_handler = logging.StreamHandler() - console_handler.setLevel(logging.DEBUG) - console_handler.setFormatter(logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")) - logger.addHandler(console_handler) - - tester = MultiProviderTester() - - report = tester.run_tests() - - tester.save_report(report, args.output) - - tester.print_report(report) - - return not report.get("error") and report["summary"]["overall"]["successful_questions"] > 0 - -if __name__ == "__main__": - success = main() - sys.exit(0 if success else 1) \ No newline at end of file diff --git a/tests/test_gaia_multi_search_integration.py b/tests/test_gaia_multi_search_integration.py deleted file mode 100644 index ef500158575275ad64fbef1d0f5d3df34ea10509..0000000000000000000000000000000000000000 --- a/tests/test_gaia_multi_search_integration.py +++ /dev/null @@ -1,286 +0,0 @@ -""" -Integration test for multiple search providers in the GAIA agent. - -This script tests the integration of multiple search providers in the GAIA agent: -1. DuckDuckGo -2. Serper (Google) -3. Perplexity -4. OpenAI - -It verifies that the agent can use all these tools together to answer questions -and can fall back to alternative providers if one fails. -""" - -import unittest -import sys -import os -import logging -from unittest.mock import patch, MagicMock -from typing import Dict, Any, List, Optional - -sys.path.append('.') - -from agent.agent import GAIAAgent -from tools.web_tools import DuckDuckGoSearchTool, SerperSearchTool -from tools.perplexity_tool import PerplexitySearchTool - -logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s') -logger = logging.getLogger("test_gaia_multi_search") - -class TestGAIAMultiSearchIntegration(unittest.TestCase): - """Test case for multiple search providers integration in the GAIA agent.""" - - def setUp(self): - """Set up the test case.""" - self.api_keys_available = { - "openai": bool(os.getenv("OPENAI_API_KEY")), - "serper": bool(os.getenv("SERPER_API_KEY")), - "perplexity": bool(os.getenv("PERPLEXITY_API_KEY")) - } - - logger.info(f"API keys available: {self.api_keys_available}") - - try: - self.agent = GAIAAgent() - logger.info("GAIA agent initialized") - except Exception as e: - logger.error(f"Error initializing GAIA agent: {str(e)}") - raise - - def tearDown(self): - """Clean up after the test.""" - if hasattr(self, 'agent') and self.agent: - self.agent.clear_memory() - - @patch('agent.graph.run_agent_graph') - def test_multi_provider_plan(self, mock_run_agent_graph): - """Test that the agent can create a plan using multiple search providers.""" - mock_run_agent_graph.return_value = { - "analysis": { - "question_type": "factual", - "required_tools": ["web_search"], - "information_sources": ["web"], - "complexity": "high" - }, - "plan": [ - { - "step_number": 1, - "description": "Search DuckDuckGo for information", - "tool": "duckduckgo_search", - "tool_input": {"query": "renewable energy statistics"} - }, - { - "step_number": 2, - "description": "Search Google (via Serper) for additional information", - "tool": "serper_search", - "tool_input": {"query": "renewable energy statistics 2023"} - }, - { - "step_number": 3, - "description": "Use Perplexity for in-depth analysis", - "tool": "perplexity_search", - "tool_input": {"query": "latest trends in renewable energy"} - }, - { - "step_number": 4, - "description": "Analyze and combine the information", - "tool": None, - "tool_input": None - } - ], - "answer": "Renewable energy capacity grew by X% in 2023, with solar and wind being the fastest growing sources." - } - - question = "What are the latest statistics and trends in renewable energy?" - logger.info(f"Testing multi-provider plan with question: {question}") - - try: - answer = self.agent(question) - logger.info(f"Agent answer: {answer}") - - mock_run_agent_graph.assert_called_once() - call_args = mock_run_agent_graph.call_args[0] - self.assertEqual(call_args[0], question) - logger.info("Agent graph was called with the correct question") - - self.assertEqual(answer, "Renewable energy capacity grew by X% in 2023, with solar and wind being the fastest growing sources.") - logger.info("Agent returned the expected answer") - - except Exception as e: - self.fail(f"Error testing multi-provider plan: {str(e)}") - - @unittest.skipIf(not os.getenv("RUN_LIVE_TESTS"), "Skipping live API tests") - def test_live_multi_provider_search(self): - """Test the agent with live API calls to multiple search providers. - - This test is skipped by default to avoid making actual API calls. - Set the RUN_LIVE_TESTS environment variable to run this test. - """ - missing_keys = [k for k, v in self.api_keys_available.items() if not v] - if missing_keys: - self.skipTest(f"Missing API keys: {', '.join(missing_keys)}") - - question = "Compare solar and wind energy efficiency and cost in 2023" - logger.info(f"Testing live multi-provider search with question: {question}") - - try: - answer = self.agent(question) - logger.info(f"Agent answer: {answer}") - - self.assertIsNotNone(answer) - self.assertIsInstance(answer, str) - self.assertTrue(len(answer) > 0) - logger.info("Agent returned a valid answer") - - memory_snapshot = self.agent.get_memory_snapshot() - - intermediate_results = memory_snapshot.get("intermediate_results", {}) - logger.info(f"Tools used: {intermediate_results.keys()}") - - conversation = memory_snapshot.get("conversation", []) - for message in conversation: - logger.info(f"{message['role']}: {message['content'][:100]}...") - - except Exception as e: - self.fail(f"Error testing live multi-provider search: {str(e)}") - - def test_fallback_mechanism(self): - """Test that the agent can fall back to alternative providers if one fails.""" - with patch('tools.web_tools.DuckDuckGoSearchTool.search') as mock_ddg_search, \ - patch('tools.web_tools.SerperSearchTool.search') as mock_serper_search, \ - patch('tools.perplexity_tool.PerplexitySearchTool.search') as mock_perplexity_search: - - # Make DuckDuckGo search fail - mock_ddg_search.side_effect = Exception("DuckDuckGo search failed") - - # Make Serper search succeed - mock_serper_search.return_value = [ - { - "title": "Renewable Energy Statistics", - "link": "https://example.com/renewable-stats", - "snippet": "Latest statistics on renewable energy sources." - } - ] - - # Make Perplexity search succeed - mock_perplexity_search.return_value = { - "text": "Detailed analysis of renewable energy trends.", - "search_results": [ - { - "title": "Energy Trends 2023", - "url": "https://example.com/trends", - "snippet": "Comprehensive analysis of energy trends." - } - ] - } - - if hasattr(self.agent, 'config') and isinstance(self.agent.config, dict): - if 'agent' in self.agent.config: - self.agent.config['agent']['return_error_message'] = True - - question = "What are the latest renewable energy statistics?" - logger.info(f"Testing fallback mechanism with question: {question}") - - try: - with patch('agent.graph.run_agent_graph', side_effect=self.agent._process_question): - answer = self.agent(question) - logger.info(f"Agent answer: {answer}") - - self.assertIsNotNone(answer) - self.assertIsInstance(answer, str) - self.assertTrue(len(answer) > 0) - logger.info("Agent returned a valid answer despite DuckDuckGo failure") - - mock_ddg_search.assert_called_once() - logger.info("DuckDuckGo search was attempted") - - - except Exception as e: - logger.warning(f"Agent raised an exception: {str(e)}") - pass - - def test_provider_selection(self): - """Test that the agent selects the appropriate search provider based on the question.""" - with patch('agent.graph.run_agent_graph') as mock_run_agent_graph: - def mock_graph_response(question, config): - if "latest news" in question.lower(): - return { - "analysis": { - "question_type": "news", - "required_tools": ["web_search"], - "information_sources": ["web"], - "complexity": "medium" - }, - "plan": [ - { - "step_number": 1, - "description": "Search for latest news", - "tool": "duckduckgo_search", - "tool_input": {"query": question} - } - ], - "answer": "Latest news about the topic." - } - elif "scientific" in question.lower() or "research" in question.lower(): - return { - "analysis": { - "question_type": "scientific", - "required_tools": ["web_search"], - "information_sources": ["web", "academic"], - "complexity": "high" - }, - "plan": [ - { - "step_number": 1, - "description": "Search for scientific information", - "tool": "perplexity_search", - "tool_input": {"query": question} - } - ], - "answer": "Scientific information about the topic." - } - else: - return { - "analysis": { - "question_type": "factual", - "required_tools": ["web_search"], - "information_sources": ["web"], - "complexity": "low" - }, - "plan": [ - { - "step_number": 1, - "description": "Search for general information", - "tool": "serper_search", - "tool_input": {"query": question} - } - ], - "answer": "General information about the topic." - } - - mock_run_agent_graph.side_effect = mock_graph_response - - questions = [ - "What are the latest news about climate change?", - "What is the scientific consensus on climate change?", - "What is climate change?" - ] - - for question in questions: - logger.info(f"Testing provider selection with question: {question}") - - try: - answer = self.agent(question) - logger.info(f"Agent answer: {answer}") - - self.assertIsNotNone(answer) - self.assertIsInstance(answer, str) - logger.info("Agent returned a valid answer") - - mock_run_agent_graph.assert_called_with(question, ANY) - - except Exception as e: - self.fail(f"Error testing provider selection: {str(e)}") - -if __name__ == "__main__": - unittest.main() \ No newline at end of file diff --git a/tests/test_gaia_multi_search_providers.py b/tests/test_gaia_multi_search_providers.py deleted file mode 100644 index 785329a59c30cbed45f5158bd3b3c73109c9a65b..0000000000000000000000000000000000000000 --- a/tests/test_gaia_multi_search_providers.py +++ /dev/null @@ -1,194 +0,0 @@ -""" -Comprehensive test for multiple search providers in the GAIA agent. - -This script tests the integration of multiple search providers: -1. DuckDuckGo -2. Serper -3. Perplexity -4. OpenAI - -It verifies that all providers can be used together and that the agent -can switch between them as needed. -""" - -import unittest -import sys -import os -import logging -from typing import Dict, Any, List, Optional - -sys.path.append('.') - -from agent.agent import GAIAAgent -import agent.config as config -from tools.web_tools import ( - DuckDuckGoSearchTool, - SerperSearchTool, - create_duckduckgo_search, - create_serper_search -) -from tools.perplexity_tool import PerplexityTool - -logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s') -logger = logging.getLogger("test_multi_search") - -console_handler = logging.StreamHandler(sys.stdout) -console_handler.setLevel(logging.INFO) -formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s') -console_handler.setFormatter(formatter) -logger.addHandler(console_handler) - -class TestMultiSearchProviders(unittest.TestCase): - """Test case for multiple search providers in the GAIA agent.""" - - def setUp(self): - """Set up the test case.""" - self.duckduckgo_tool = create_duckduckgo_search() - self.serper_tool = create_serper_search() - - perplexity_api_key = os.environ.get("PERPLEXITY_API_KEY") - self.perplexity_tool = None - if perplexity_api_key: - self.perplexity_tool = PerplexityTool(config={"api_key": perplexity_api_key}) - - agent_config = config.get_agent_config() - model_config = config.get_model_config() - tool_config = config.get_tool_config() - - combined_config = { - **agent_config, - **model_config, - "tools": tool_config - } - - self.agent = GAIAAgent(config=combined_config) - - logger.info("Test setup complete with all available search providers") - - def test_duckduckgo_search(self): - """Test the DuckDuckGo search provider.""" - if not self.duckduckgo_tool: - logger.warning("DuckDuckGo search tool not available") - return - - try: - query = "latest Python version" - results = self.duckduckgo_tool.search(query) - - self.assertIsNotNone(results) - self.assertIsInstance(results, list) - self.assertTrue(len(results) > 0) - - logger.info(f"DuckDuckGo search successful: {len(results)} results for '{query}'") - logger.info(f"First result: {results[0]['title']} - {results[0]['link']}") - - except Exception as e: - logger.error(f"DuckDuckGo search failed: {str(e)}") - self.fail(f"DuckDuckGo search failed: {str(e)}") - - def test_serper_search(self): - """Test the Serper search provider.""" - if not self.serper_tool: - logger.warning("Serper search tool not available") - return - - try: - query = "latest Python version" - results = self.serper_tool.search(query) - - self.assertIsNotNone(results) - self.assertIsInstance(results, list) - self.assertTrue(len(results) > 0) - - logger.info(f"Serper search successful: {len(results)} results for '{query}'") - logger.info(f"First result: {results[0]['title']} - {results[0]['link']}") - - except Exception as e: - logger.error(f"Serper search failed: {str(e)}") - self.fail(f"Serper search failed: {str(e)}") - - def test_perplexity_search(self): - """Test the Perplexity search provider.""" - if not self.perplexity_tool: - logger.warning("Perplexity search tool not available") - return - - try: - query = "latest Python version" - results = self.perplexity_tool.search(query) - - self.assertIsNotNone(results) - - logger.info(f"Perplexity search successful for '{query}'") - content = results.get("content", "") - if content: - logger.info(f"Result snippet: {content[:100]}...") - else: - logger.info("No content found in the results") - - except Exception as e: - logger.error(f"Perplexity search failed: {str(e)}") - self.fail(f"Perplexity search failed: {str(e)}") - - def test_all_providers_together(self): - """Test all search providers together.""" - query = "latest Python version" - results = {} - - if self.duckduckgo_tool: - try: - results["duckduckgo"] = self.duckduckgo_tool.search(query) - logger.info(f"DuckDuckGo found {len(results['duckduckgo'])} results") - except Exception as e: - logger.error(f"DuckDuckGo search failed: {str(e)}") - results["duckduckgo"] = None - - if self.serper_tool: - try: - results["serper"] = self.serper_tool.search(query) - logger.info(f"Serper found {len(results['serper'])} results") - except Exception as e: - logger.error(f"Serper search failed: {str(e)}") - results["serper"] = None - - if self.perplexity_tool: - try: - perplexity_results = self.perplexity_tool.search(query) - results["perplexity"] = perplexity_results - content = perplexity_results.get("content", "") - logger.info(f"Perplexity search successful, content length: {len(content) if content else 0}") - except Exception as e: - logger.error(f"Perplexity search failed: {str(e)}") - results["perplexity"] = None - - self.assertTrue(any(results.values()), "All search providers failed") - - logger.info(f"Successfully tested {len([r for r in results.values() if r])} out of {len(results)} providers") - - providers_with_results = [p for p, r in results.items() if r] - if len(providers_with_results) >= 2: - logger.info("Comparing results from different providers:") - for provider in providers_with_results: - if isinstance(results[provider], list) and results[provider]: - logger.info(f"{provider.capitalize()} first result: {results[provider][0].get('title', 'N/A')}") - - def test_agent_with_search_providers(self): - """Test the GAIA agent with all search providers.""" - logger.info("Skipping GAIA agent test - agent doesn't support direct tool addition") - logger.info("The GAIA agent uses a LangGraph workflow for tool selection and execution") - logger.info("To integrate search tools, they would need to be added to the agent's configuration") - -if __name__ == "__main__": - - test = TestMultiSearchProviders() - test.setUp() - - test.test_duckduckgo_search() - - test.test_serper_search() - - test.test_perplexity_search() - - test.test_all_providers_together() - - diff --git a/tests/test_gaia_serper_integration.py b/tests/test_gaia_serper_integration.py deleted file mode 100644 index a9f032cf62174bdb54d3ade84cdfc67424641bd4..0000000000000000000000000000000000000000 --- a/tests/test_gaia_serper_integration.py +++ /dev/null @@ -1,158 +0,0 @@ -""" -Integration test for GAIA agent with Serper API. - -This script tests the integration of the Serper API with the GAIA agent, -verifying that the agent can use Serper for web searches alongside other tools. -""" - -import os -import sys -import json -import logging -import unittest -from unittest.mock import patch, MagicMock -from typing import Dict, Any, List, Optional - -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(name)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("gaia_serper_integration_test") - -sys.path.append('.') - -try: - from agent.agent import GaiaAgent - from tools.web_tools import SerperSearchTool - from agent.config import SERPER_API_KEY, SERPER_API_URL -except ImportError as e: - logger.error(f"Error importing modules: {str(e)}") - sys.exit(1) - -class TestGaiaSerperIntegration(unittest.TestCase): - """Test case for GAIA agent with Serper API integration.""" - - def setUp(self): - """Set up the test case.""" - if not SERPER_API_KEY: - self.skipTest("Serper API key not set. Set SERPER_API_KEY environment variable to run this test.") - - self.agent = GaiaAgent() - - self.serper_tool = SerperSearchTool() - - def test_serper_tool_initialization(self): - """Test that the Serper search tool initializes correctly.""" - self.assertEqual(self.serper_tool.api_key, SERPER_API_KEY) - self.assertEqual(self.serper_tool.api_url, SERPER_API_URL) - - def test_serper_search(self): - """Test that the Serper search tool can perform a search.""" - try: - results = self.serper_tool.search("artificial intelligence latest developments") - - self.assertIsInstance(results, list) - self.assertTrue(len(results) > 0, "No search results returned") - - for result in results: - self.assertIn("title", result) - self.assertIn("link", result) - self.assertIn("snippet", result) - - logger.info(f"Serper search returned {len(results)} results") - except Exception as e: - self.fail(f"Serper search failed: {str(e)}") - - def test_agent_with_serper(self): - """Test that the GAIA agent can use the Serper search tool.""" - self.agent.register_tool("serper_search", self.serper_tool.search) - - self.assertIn("serper_search", self.agent.tools) - - try: - response = self.agent.run( - "What are the latest developments in artificial intelligence?", - tools=["serper_search"] - ) - - self.assertIsNotNone(response) - self.assertIsInstance(response, str) - self.assertTrue(len(response) > 0, "Empty response from agent") - - logger.info("Agent successfully used Serper search tool") - except Exception as e: - self.fail(f"Agent failed to use Serper search tool: {str(e)}") - - @patch('tools.web_tools.requests.post') - def test_agent_with_serper_mock(self, mock_post): - """Test the agent with a mocked Serper API response.""" - mock_response = MagicMock() - mock_response.status_code = 200 - mock_response.json.return_value = { - "organic": [ - { - "title": "Latest AI Developments 2025", - "link": "https://example.com/ai-developments", - "snippet": "The latest developments in AI include advances in large language models, computer vision, and robotics." - }, - { - "title": "Artificial Intelligence Research Trends", - "link": "https://example.com/ai-research", - "snippet": "Research trends in AI focus on multimodal learning, reinforcement learning, and explainable AI." - } - ] - } - mock_post.return_value = mock_response - - self.agent.register_tool("serper_search", self.serper_tool.search) - - response = self.agent.run( - "What are the latest developments in artificial intelligence?", - tools=["serper_search"] - ) - - self.assertIsNotNone(response) - self.assertIsInstance(response, str) - self.assertTrue(len(response) > 0, "Empty response from agent") - - mock_post.assert_called_once() - - logger.info("Agent successfully used mocked Serper search tool") - - def test_agent_with_multiple_tools(self): - """Test that the agent can use Serper alongside other tools.""" - if os.environ.get("CI") == "true": - self.skipTest("Skipping multi-tool test in CI environment") - - self.agent.register_tool("serper_search", self.serper_tool.search) - - try: - from tools.perplexity_tool import PerplexityTool - from agent.config import PERPLEXITY_API_KEY - - if PERPLEXITY_API_KEY: - perplexity_tool = PerplexityTool() - self.agent.register_tool("perplexity_search", perplexity_tool.search) - except (ImportError, AttributeError): - logger.warning("Perplexity tool not available, skipping") - - try: - response = self.agent.run( - "Compare the latest developments in AI with historical trends.", - tools=list(self.agent.tools.keys()) - ) - - self.assertIsNotNone(response) - self.assertIsInstance(response, str) - self.assertTrue(len(response) > 0, "Empty response from agent") - - logger.info("Agent successfully used multiple tools including Serper") - except Exception as e: - self.fail(f"Agent failed to use multiple tools: {str(e)}") - -def run_tests(): - """Run the tests.""" - unittest.main(argv=['first-arg-is-ignored'], exit=False) - -if __name__ == "__main__": - run_tests() \ No newline at end of file diff --git a/tests/test_gaia_supabase_integration.py b/tests/test_gaia_supabase_integration.py deleted file mode 100644 index e8ac24404c84737756c56b0d6f03f7b3be06cdc1..0000000000000000000000000000000000000000 --- a/tests/test_gaia_supabase_integration.py +++ /dev/null @@ -1,277 +0,0 @@ -""" -Test GAIA Agent with Supabase Memory Integration - -This script tests the integration of the GAIA agent with Supabase memory, -verifying that memory persistence works across different agent sessions. -It also tests the integration with other API tools. -""" - -import os -import sys -import json -import time -import logging -import argparse -import uuid -from typing import Dict, Any, List, Optional, Union, Tuple - -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(name)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("gaia_supabase_integration_test") - -try: - from agent.agent import GAIAAgent - from agent.config import get_agent_config, get_model_config, get_memory_config - from memory.supabase_memory import SupabaseMemory, ConversationMemory, ResultCache, WorkingMemory -except ImportError as e: - logger.error(f"Error importing agent components: {str(e)}") - sys.exit(1) - -class GAIASupabaseIntegrationTester: - """Class for testing the GAIA agent with Supabase memory integration.""" - - def __init__(self, verbose: bool = False): - """ - Initialize the GAIA Supabase integration tester. - - Args: - verbose: Whether to enable verbose logging - """ - self.verbose = verbose - self.conversation_id = str(uuid.uuid4()) - - if verbose: - logging.getLogger().setLevel(logging.DEBUG) - - self._initialize_first_agent() - - def _initialize_first_agent(self) -> None: - """Initialize the first GAIA agent.""" - logger.info("Initializing first GAIA agent...") - - try: - agent_config = get_agent_config() - model_config = get_model_config() - memory_config = get_memory_config() - - memory_config["enabled"] = True - - self.agent1 = GAIAAgent( - agent_config=agent_config, - model_config=model_config, - memory_config=memory_config, - verbose=self.verbose - ) - - self.agent1.memory.conversation_id = self.conversation_id - - logger.info("First GAIA agent initialized successfully") - logger.info(f"Conversation ID: {self.conversation_id}") - logger.info(f"Session ID: {self.agent1.memory.session_id}") - except Exception as e: - logger.error(f"Error initializing first GAIA agent: {str(e)}") - sys.exit(1) - - def _initialize_second_agent(self) -> None: - """Initialize the second GAIA agent with the same conversation ID.""" - logger.info("Initializing second GAIA agent...") - - try: - agent_config = get_agent_config() - model_config = get_model_config() - memory_config = get_memory_config() - - memory_config["enabled"] = True - - self.agent2 = GAIAAgent( - agent_config=agent_config, - model_config=model_config, - memory_config=memory_config, - verbose=self.verbose - ) - - self.agent2.memory.conversation_id = self.conversation_id - - logger.info("Second GAIA agent initialized successfully") - logger.info(f"Conversation ID: {self.conversation_id}") - logger.info(f"Session ID: {self.agent2.memory.session_id}") - except Exception as e: - logger.error(f"Error initializing second GAIA agent: {str(e)}") - sys.exit(1) - - def test_memory_persistence(self) -> bool: - """ - Test memory persistence across different agent sessions. - - Returns: - bool: True if memory persistence works, False otherwise - """ - logger.info("Testing memory persistence across different agent sessions...") - - try: - test_message = f"Test message at {time.time()}" - logger.info(f"Adding message to first agent: {test_message}") - self.agent1.memory.conversation.add_message("user", test_message) - - test_query = f"test_query_{time.time()}" - test_result = f"test_result_{time.time()}" - logger.info(f"Caching result in first agent: {test_query} -> {test_result}") - self.agent1.memory.cache.cache_result(test_query, test_result) - - test_key = f"test_key_{time.time()}" - test_value = f"test_value_{time.time()}" - logger.info(f"Storing intermediate result in first agent: {test_key} -> {test_value}") - self.agent1.memory.working.store_intermediate_result(test_key, test_value) - - self._initialize_second_agent() - - logger.info("Checking if message is in second agent's conversation...") - messages = self.agent2.memory.conversation.get_messages() - message_found = any(m.get("content") == test_message for m in messages) - - if message_found: - logger.info("✅ Message found in second agent's conversation") - else: - logger.warning("❌ Message not found in second agent's conversation") - return False - - logger.info("Checking if result is in second agent's cache...") - result = self.agent2.memory.cache.get_result(test_query) - - if result == test_result: - logger.info("✅ Result found in second agent's cache") - else: - logger.warning(f"❌ Result not found in second agent's cache. Expected {test_result}, got {result}") - return False - - logger.info("Checking if intermediate result is NOT in second agent's working memory...") - value = self.agent2.memory.working.get_intermediate_result(test_key) - - if value is None: - logger.info("✅ Intermediate result correctly NOT found in second agent's working memory") - else: - logger.warning(f"❌ Intermediate result incorrectly found in second agent's working memory: {value}") - return False - - return True - except Exception as e: - logger.error(f"Error testing memory persistence: {str(e)}") - return False - - def test_conversation_continuity(self) -> bool: - """ - Test conversation continuity across different agent sessions. - - Returns: - bool: True if conversation continuity works, False otherwise - """ - logger.info("Testing conversation continuity across different agent sessions...") - - try: - question1 = "What is the capital of France?" - logger.info(f"Processing question with first agent: {question1}") - response1 = self.agent1.process_question(question1) - - logger.info(f"First agent response: {response1}") - - question2 = "What is its population?" - logger.info(f"Processing follow-up question with second agent: {question2}") - response2 = self.agent2.process_question(question2) - - logger.info(f"Second agent response: {response2}") - - population_keywords = ["million", "inhabitants", "people", "population"] - contains_population = any(keyword in response2.lower() for keyword in population_keywords) - - if contains_population: - logger.info("✅ Second agent response contains population information") - return True - else: - logger.warning("❌ Second agent response does not contain population information") - return False - except Exception as e: - logger.error(f"Error testing conversation continuity: {str(e)}") - return False - - def test_api_tool_integration(self) -> bool: - """ - Test integration with API tools. - - Returns: - bool: True if API tool integration works, False otherwise - """ - logger.info("Testing integration with API tools...") - - try: - question = "What are the latest developments in quantum computing?" - logger.info(f"Processing question with API tools: {question}") - - response = self.agent1.process_question(question) - - logger.info(f"Agent response: {response}") - - quantum_keywords = ["quantum", "qubit", "superposition", "entanglement"] - contains_quantum_info = any(keyword in response.lower() for keyword in quantum_keywords) - - if contains_quantum_info: - logger.info("✅ Agent response contains quantum computing information") - return True - else: - logger.warning("❌ Agent response does not contain quantum computing information") - return False - except Exception as e: - logger.error(f"Error testing API tool integration: {str(e)}") - return False - - def run_all_tests(self) -> None: - """Run all GAIA Supabase integration tests.""" - - memory_persistence_result = self.test_memory_persistence() - - status = "✅ Passed" if memory_persistence_result else "❌ Failed" - - conversation_continuity_result = self.test_conversation_continuity() - - status = "✅ Passed" if conversation_continuity_result else "❌ Failed" - - api_tool_integration_result = self.test_api_tool_integration() - - status = "✅ Passed" if api_tool_integration_result else "❌ Failed" - - - if memory_persistence_result: - else: - - if conversation_continuity_result: - else: - - if api_tool_integration_result: - else: - - overall_success = memory_persistence_result and conversation_continuity_result and api_tool_integration_result - if overall_success: - else: - - if not memory_persistence_result: - - if not conversation_continuity_result: - - if not api_tool_integration_result: - - - -def main(): - """Main function.""" - parser = argparse.ArgumentParser(description="Test GAIA agent with Supabase memory integration") - parser.add_argument("--verbose", action="store_true", help="Enable verbose logging") - args = parser.parse_args() - - tester = GAIASupabaseIntegrationTester(verbose=args.verbose) - - tester.run_all_tests() - - -if __name__ == "__main__": - main() \ No newline at end of file diff --git a/tests/test_gaia_with_all_tables.py b/tests/test_gaia_with_all_tables.py deleted file mode 100644 index 94e6dde466e2fe84f75ff31676448ea878e7f2d7..0000000000000000000000000000000000000000 --- a/tests/test_gaia_with_all_tables.py +++ /dev/null @@ -1,274 +0,0 @@ -""" -Test GAIA Agent With All Supabase Tables - -This script tests the GAIA agent with a sample question and verifies that all Supabase tables -(gaia_memory, gaia_memory_working, gaia_memory_conversations, gaia_memory_cache) are being -properly populated. -""" - -import os -import sys -import json -import time -import logging -import argparse -import uuid -from datetime import datetime -from typing import Dict, Any, List, Optional, Union, Tuple - -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(name)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("gaia_tables_test") - -sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) - -try: - from agent.agent import GAIAAgent - from agent.config import get_agent_config, get_model_config, get_memory_config - from memory.supabase_memory import SupabaseMemory, WorkingMemory - - from memory.fix_supabase_tables import apply_monkey_patch, ensure_tables_exist -except ImportError as e: - logger.error(f"Error importing agent components: {str(e)}") - sys.exit(1) - -class GAIATablesTest: - """Class for testing the GAIA agent with all Supabase tables.""" - - def __init__(self, verbose: bool = False): - """ - Initialize the GAIA tables test. - - Args: - verbose: Whether to enable verbose logging - """ - self.verbose = verbose - self.session_id = str(uuid.uuid4()) - self.conversation_id = str(uuid.uuid4()) - - if verbose: - logging.getLogger().setLevel(logging.DEBUG) - - logger.info("Applying fix for Supabase tables...") - ensure_tables_exist() - apply_monkey_patch() - - self._initialize_agent() - - def _initialize_agent(self) -> None: - """Initialize the GAIA agent.""" - logger.info("Initializing GAIA agent...") - - try: - agent_config = get_agent_config() - model_config = get_model_config() - memory_config = get_memory_config() - - memory_config["enabled"] = True - - self.agent = GAIAAgent( - agent_config=agent_config, - model_config=model_config, - memory_config=memory_config, - verbose=self.verbose - ) - - self.agent.memory.session_id = self.session_id - self.agent.memory.conversation_id = self.conversation_id - - logger.info("GAIA agent initialized successfully") - logger.info(f"Session ID: {self.session_id}") - logger.info(f"Conversation ID: {self.conversation_id}") - except Exception as e: - logger.error(f"Error initializing GAIA agent: {str(e)}") - sys.exit(1) - - def get_table_content(self, table_name: str) -> List[Dict[str, Any]]: - """ - Get the content of a Supabase table. - - Args: - table_name: The name of the table - - Returns: - List of entries in the table - """ - logger.info(f"Getting content of table: {table_name}") - - try: - client = self.agent.memory.base_memory.client - - if not client: - logger.error("Supabase client not initialized") - return [] - - response = client.table(table_name).select("*").execute() - - if response.data and len(response.data) > 0: - logger.info(f"Found {len(response.data)} entries in table {table_name}") - return response.data - else: - logger.warning(f"No entries found in table {table_name}") - return [] - except Exception as e: - logger.error(f"Error getting table content: {str(e)}") - return [] - - def check_all_tables(self) -> Dict[str, List[Dict[str, Any]]]: - """ - Check the content of all Supabase tables. - - Returns: - Dictionary with table names as keys and table content as values - """ - logger.info("Checking all Supabase tables...") - - tables = { - "gaia_memory": "Main memory table", - "gaia_memory_working": "Working memory table", - "gaia_memory_conversations": "Conversation history table", - "gaia_memory_cache": "Result cache table" - } - - results = {} - - for table_name, description in tables.items(): - logger.info(f"Checking table {table_name} ({description})...") - content = self.get_table_content(table_name) - results[table_name] = content - - return results - - def test_with_question(self, question: str) -> Dict[str, Any]: - """ - Test the agent with a question and check all tables. - - Args: - question: The question to test - - Returns: - Dictionary with test results - """ - logger.info(f"Testing with question: {question}") - - results = { - "question": question, - "answer": None, - "tables_before": {}, - "tables_after": {}, - "execution_time": 0 - } - - results["tables_before"] = self.check_all_tables() - - try: - start_time = time.time() - response = self.agent.process_question(question) - end_time = time.time() - execution_time = end_time - start_time - - results["answer"] = response - results["execution_time"] = execution_time - - - logger.info(f"Successfully processed question: {question}") - except Exception as e: - logger.error(f"Error processing question '{question}': {str(e)}") - results["error"] = str(e) - - results["tables_after"] = self.check_all_tables() - - return results - - def print_table_summary(self, tables_data: Dict[str, List[Dict[str, Any]]]) -> None: - """ - Print a summary of the table content. - - Args: - tables_data: Dictionary with table names as keys and table content as values - """ - - for table_name, content in tables_data.items(): - - if content: - max_entries = 3 - for i, entry in enumerate(content[:max_entries]): - - value = entry.get('value', 'N/A') - if isinstance(value, str) and len(value) > 100: - value = value[:100] + "..." - - if len(content) > max_entries: - - def print_table_changes(self, before: Dict[str, List[Dict[str, Any]]], after: Dict[str, List[Dict[str, Any]]]) -> None: - """ - Print a summary of the changes in table content. - - Args: - before: Table content before the test - after: Table content after the test - """ - - for table_name in before.keys(): - before_count = len(before.get(table_name, [])) - after_count = len(after.get(table_name, [])) - new_entries = after_count - before_count - - if new_entries > 0: - else: - - def run_test(self, question: str) -> Dict[str, Any]: - """ - Run a full test with a question and check all tables. - - Args: - question: The question to test - - Returns: - Dictionary with test results - """ - - results = self.test_with_question(question) - - self.print_table_summary(results["tables_before"]) - - self.print_table_summary(results["tables_after"]) - - self.print_table_changes(results["tables_before"], results["tables_after"]) - - timestamp = datetime.now().strftime("%Y%m%d_%H%M%S") - results_file = f"gaia_tables_test_results_{timestamp}.json" - - serializable_results = { - "question": results["question"], - "answer": results["answer"], - "execution_time": results["execution_time"], - "tables_before": {k: [dict(e) for e in v] for k, v in results["tables_before"].items()}, - "tables_after": {k: [dict(e) for e in v] for k, v in results["tables_after"].items()}, - } - - if "error" in results: - serializable_results["error"] = results["error"] - - with open(results_file, "w") as f: - json.dump(serializable_results, f, indent=2, default=str) - - - return results - -def main(): - """Main function.""" - parser = argparse.ArgumentParser(description="Test GAIA agent with all Supabase tables") - parser.add_argument("--verbose", action="store_true", help="Enable verbose logging") - parser.add_argument("--question", type=str, default="What are the different types of memory used in the GAIA agent and why are they important?", - help="Custom question to test") - args = parser.parse_args() - - tester = GAIATablesTest(verbose=args.verbose) - - tester.run_test(args.question) - -if __name__ == "__main__": - main() \ No newline at end of file diff --git a/tests/test_gaia_with_memory.py b/tests/test_gaia_with_memory.py deleted file mode 100644 index 6f11791f45e5523b5cda2206c845a18fbc666916..0000000000000000000000000000000000000000 --- a/tests/test_gaia_with_memory.py +++ /dev/null @@ -1,297 +0,0 @@ -""" -Test script for GAIA agent with Supabase memory integration. - -This script demonstrates the GAIA agent using Supabase for memory persistence -across multiple sessions, showing how memory can be maintained between -different agent instances. -""" - -import os -import sys -import json -import time -import uuid -import logging -import argparse -from datetime import datetime - -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(name)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("gaia_memory_test") - -try: - from dotenv import load_dotenv - dotenv_path = os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), '.env') - logger.info(f"Loading environment variables from {dotenv_path}") - load_dotenv(dotenv_path) - logger.info("Environment variables loaded from .env file") -except ImportError: - logger.warning("dotenv package not found, using environment variables directly") - -sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) - -try: - from agent.agent import GAIAAgent - from memory.supabase_memory import SupabaseMemory, ConversationMemory - logger.info("Successfully imported GAIA agent and memory modules") -except ImportError as e: - logger.error(f"Error importing GAIA agent: {str(e)}") - logger.error("Make sure you're running the script from the project root directory") - sys.exit(1) - -class GAIAMemoryTest: - """Test class for GAIA agent with Supabase memory.""" - - def __init__(self, verbose=False): - """ - Initialize the test. - - Args: - verbose (bool): Whether to enable verbose logging - """ - self.verbose = verbose - - if verbose: - logger.setLevel(logging.DEBUG) - if not any(isinstance(h, logging.StreamHandler) for h in logger.handlers): - console_handler = logging.StreamHandler() - console_handler.setLevel(logging.DEBUG) - console_handler.setFormatter(logging.Formatter( - "%(asctime)s - %(name)s - %(levelname)s - %(message)s" - )) - logger.addHandler(console_handler) - - self.supabase_url = os.environ.get("SUPABASE_URL") - self.supabase_key = os.environ.get("SUPABASE_KEY") - - if not self.supabase_url or not self.supabase_key: - logger.error("Supabase URL or key not provided") - logger.error("Please set SUPABASE_URL and SUPABASE_KEY environment variables") - sys.exit(1) - - logger.info("GAIA Memory Test initialized") - - def test_memory_persistence(self): - """Test memory persistence across multiple agent sessions.""" - logger.info("=== Testing Memory Persistence ===") - - conversation_id = f"test-{uuid.uuid4()}" - logger.info(f"Using conversation ID: {conversation_id}") - - logger.info("Creating first agent instance") - agent1 = self._create_agent(conversation_id) - - question1 = "What is the capital of France?" - logger.info(f"Asking first question: {question1}") - answer1 = agent1(question1) - logger.info(f"Answer: {answer1}") - - logger.info("Creating second agent instance with the same conversation ID") - agent2 = self._create_agent(conversation_id) - - question2 = "What is the population of that city?" - logger.info(f"Asking follow-up question: {question2}") - answer2 = agent2(question2) - logger.info(f"Answer: {answer2}") - - logger.info("Creating third agent instance with the same conversation ID") - agent3 = self._create_agent(conversation_id) - - question3 = "What other famous landmarks are in that city besides the Eiffel Tower?" - logger.info(f"Asking another follow-up question: {question3}") - answer3 = agent3(question3) - logger.info(f"Answer: {answer3}") - - logger.info("Retrieving conversation history from memory") - conversation_memory = ConversationMemory( - SupabaseMemory({ - "table_name": "gaia_memory", - "enabled": True - }), - conversation_id - ) - - messages = conversation_memory.get_messages() - logger.info(f"Retrieved {len(messages)} messages from conversation history") - - for i, message in enumerate(messages): - role = message.get("role", "unknown") - content = message.get("content", "") - - return { - "conversation_id": conversation_id, - "questions": [question1, question2, question3], - "answers": [answer1, answer2, answer3], - "messages": messages - } - - def test_api_integration(self): - """Test integration with other API tools.""" - logger.info("=== Testing API Integration with Memory ===") - - conversation_id = f"api-test-{uuid.uuid4()}" - logger.info(f"Using conversation ID: {conversation_id}") - - agent = self._create_agent(conversation_id) - - question1 = "Using Perplexity, what are the latest developments in quantum computing?" - logger.info(f"Asking Perplexity question: {question1}") - answer1 = agent(question1) - logger.info(f"Answer: {answer1}") - - question2 = "Using Serper, find information about the James Webb Space Telescope discoveries." - logger.info(f"Asking Serper question: {question2}") - answer2 = agent(question2) - logger.info(f"Answer: {answer2}") - - question3 = "Using OpenAI, explain the concept of neural networks in simple terms." - logger.info(f"Asking OpenAI question: {question3}") - answer3 = agent(question3) - logger.info(f"Answer: {answer3}") - - question4 = "Summarize what we've discussed about quantum computing, the James Webb telescope, and neural networks." - logger.info(f"Asking follow-up question: {question4}") - answer4 = agent(question4) - logger.info(f"Answer: {answer4}") - - logger.info("Retrieving conversation history from memory") - conversation_memory = ConversationMemory( - SupabaseMemory({ - "table_name": "gaia_memory", - "enabled": True - }), - conversation_id - ) - - messages = conversation_memory.get_messages() - logger.info(f"Retrieved {len(messages)} messages from conversation history") - - for i, message in enumerate(messages): - role = message.get("role", "unknown") - content = message.get("content", "") - - return { - "conversation_id": conversation_id, - "questions": [question1, question2, question3, question4], - "answers": [answer1, answer2, answer3, answer4], - "messages": messages - } - - def _create_agent(self, conversation_id): - """ - Create a GAIA agent with the specified conversation ID. - - Args: - conversation_id (str): The conversation ID to use - - Returns: - GAIAAgent: The created agent - """ - class MockGAIAAgent: - def __init__(self, config): - self.config = config - self.conversation_id = config["memory"].get("conversation_id", "default") - self.memory_enabled = config["memory"].get("enabled", False) - - from memory.supabase_memory import SupabaseMemory, ConversationMemory - self.memory = SupabaseMemory(config["memory"]) - self.conversation_memory = ConversationMemory(self.memory, self.conversation_id) - - - def __call__(self, question): - """Process a question and return an answer.""" - logger.info(f"Processing question: {question}") - - self.conversation_memory.add_message("user", question) - - if "capital of France" in question: - answer = "Paris is the capital of France." - elif "population" in question and "city" in question: - answer = "Paris has a population of approximately 2.2 million people." - elif "landmarks" in question and "city" in question: - answer = "Paris has many famous landmarks including the Louvre Museum, Notre-Dame Cathedral, Arc de Triomphe, and Sacré-Cœur Basilica." - elif "Perplexity" in question and "quantum computing" in question: - answer = "Recent developments in quantum computing include Google's Sycamore processor achieving quantum supremacy, IBM's 127-qubit Eagle processor, and advances in quantum error correction. Researchers are also making progress in quantum algorithms and applications in cryptography, optimization, and materials science." - elif "Serper" in question and "James Webb" in question: - answer = "The James Webb Space Telescope has made several groundbreaking discoveries, including the observation of the most distant galaxies ever seen, detection of water vapor in exoplanet atmospheres, and capturing detailed images of Neptune's rings. It has also observed the earliest stages of star formation and provided new insights into the composition of distant planets." - elif "OpenAI" in question and "neural networks" in question: - answer = "Neural networks are computing systems inspired by the human brain. They consist of interconnected nodes (neurons) organized in layers that process information. Each connection has a weight that adjusts as the network learns from data. Neural networks excel at pattern recognition tasks like image classification, language processing, and decision-making by learning from examples rather than following explicit programming." - elif "summarize" in question.lower() and ("quantum computing" in question and "James Webb" in question and "neural networks" in question): - answer = "We've discussed three main topics: 1) Recent quantum computing advances including Google's quantum supremacy and IBM's 127-qubit processor, 2) James Webb Space Telescope discoveries such as distant galaxies, exoplanet atmospheres, and detailed images of Neptune's rings, and 3) Neural networks as brain-inspired computing systems that learn from data to recognize patterns and make decisions." - else: - answer = f"I don't have specific information about that. Your question was: {question}" - - self.conversation_memory.add_message("assistant", answer) - - return answer - - config = { - "memory": { - "enabled": True, - "table_name": "gaia_memory", - "conversation_id": conversation_id - }, - "agent": { - "max_iterations": 10, - "verbose": self.verbose, - "timeout": 300, - "max_retries": 3, - "retry_delay": 2, - "parallel_tool_execution": False, - "return_error_message": True - }, - "model": { - "default_model": "gpt-4o", - "embedding_model": "text-embedding-3-large", - "fallback_model": "gpt-3.5-turbo", - "vision_model": "gpt-4o", - "reasoning_model": "gpt-4o", - "temperature": 0.1, - "max_tokens": 4096 - }, - "tools": { - "web_search": { - "result_count": 5, - "timeout": 10 - }, - "perplexity": { - "timeout": 30, - "model": "sonar-reasoning" - } - } - } - - agent = MockGAIAAgent(config) - - return agent - -def main(): - """Main entry point for the test script.""" - parser = argparse.ArgumentParser(description="Test GAIA agent with Supabase memory") - parser.add_argument("--test", choices=["memory", "api", "all"], default="all", - help="Test to run (memory, api, or all)") - parser.add_argument("--verbose", "-v", action="store_true", - help="Enable verbose logging") - args = parser.parse_args() - - test = GAIAMemoryTest(verbose=args.verbose) - - results = {} - - if args.test in ["memory", "all"]: - results["memory"] = test.test_memory_persistence() - - if args.test in ["api", "all"]: - results["api"] = test.test_api_integration() - - timestamp = datetime.now().strftime("%Y%m%d_%H%M%S") - results_file = f"gaia_memory_test_results_{timestamp}.json" - - with open(results_file, "w") as f: - json.dump(results, f, indent=2) - - -if __name__ == "__main__": - main() \ No newline at end of file diff --git a/tests/test_gaia_with_tables.py b/tests/test_gaia_with_tables.py deleted file mode 100644 index 46dd67efbda0476d06e26f60f8eb1c62dc7d1dab..0000000000000000000000000000000000000000 --- a/tests/test_gaia_with_tables.py +++ /dev/null @@ -1,206 +0,0 @@ -""" -Test script for GAIA agent with Supabase tables. - -This script runs a simple test with the GAIA agent, asks a custom question, -and shows the content of the Supabase tables after the question is answered. - -Usage: - python -m tests.test_gaia_with_tables -""" - -import os -import sys -import json -import time -import uuid -import logging -from datetime import datetime - -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(name)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("gaia_tables_test") - -try: - from dotenv import load_dotenv - dotenv_path = os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), '.env') - logger.info(f"Loading environment variables from {dotenv_path}") - load_dotenv(dotenv_path) - logger.info("Environment variables loaded from .env file") -except ImportError: - logger.warning("dotenv package not found, using environment variables directly") - -sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) - -try: - from memory.supabase_memory import SupabaseMemory, WorkingMemory - from agent.config import get_memory_config - - from memory.fix_supabase_tables import apply_monkey_patch, ensure_tables_exist -except ImportError as e: - logger.error(f"Error importing required components: {str(e)}") - sys.exit(1) - -def apply_fix(): - """Apply the fix for Supabase tables.""" - logger.info("Applying fix for Supabase tables...") - ensure_tables_exist() - apply_monkey_patch() - logger.info("Fix applied successfully") - -def create_mock_agent(conversation_id): - """ - Create a mock GAIA agent for testing. - - Args: - conversation_id (str): The conversation ID to use - - Returns: - MockGAIAAgent: The created agent - """ - class MockGAIAAgent: - def __init__(self, conversation_id): - self.conversation_id = conversation_id - - memory_config = get_memory_config() - memory_config["enabled"] = True - - self.base_memory = SupabaseMemory(memory_config) - self.working_memory = WorkingMemory(self.base_memory, conversation_id) - - logger.info(f"MockGAIAAgent initialized with conversation ID: {conversation_id}") - logger.info(f"Base memory table: {self.base_memory.table_name}") - logger.info(f"Working memory table: {self.working_memory.table_name}") - - def ask(self, question): - """Process a question and return an answer.""" - logger.info(f"Processing question: {question}") - - self.working_memory.store_intermediate_result("question", question) - - self.working_memory.store_intermediate_result("analysis", { - "question_type": "complex", - "requires_reasoning": True, - "steps_needed": ["parse_question", "identify_entities", "search_knowledge", "formulate_answer"] - }) - - self.working_memory.store_intermediate_result("entities", ["GAIA", "Supabase", "memory", "tables"]) - self.working_memory.store_intermediate_result("search_results", { - "gaia_memory": "Main memory table for storing key-value pairs", - "gaia_memory_working": "Working memory table for storing intermediate results during reasoning", - "gaia_memory_conversations": "Conversation history table for storing message history", - "gaia_memory_cache": "Cache table for storing frequently accessed results" - }) - - if "tables" in question and "Supabase" in question: - answer = """ -The GAIA agent uses several Supabase tables for different types of memory: - -1. gaia_memory: The main memory table for storing key-value pairs -2. gaia_memory_working: Working memory table for storing intermediate results during reasoning -3. gaia_memory_conversations: Conversation history table for storing message history -4. gaia_memory_cache: Cache table for storing frequently accessed results - -The working memory table (gaia_memory_working) is particularly important for complex reasoning tasks, as it allows the agent to store and retrieve intermediate results during the reasoning process. - """ - else: - answer = f"I don't have specific information about that. Your question was: {question}" - - self.working_memory.store_intermediate_result("final_answer", answer) - - return answer - - agent = MockGAIAAgent(conversation_id) - - return agent - -def get_table_content(client, table_name): - """ - Get the content of a Supabase table. - - Args: - client: The Supabase client - table_name (str): The name of the table - - Returns: - list: The table content - """ - try: - response = client.table(table_name).select("*").execute() - - if response.data and len(response.data) > 0: - logger.info(f"Found {len(response.data)} entries in table {table_name}") - return response.data - else: - logger.warning(f"No entries found in table {table_name}") - return [] - except Exception as e: - logger.error(f"Error getting table content: {str(e)}") - return [] - -def main(): - """Main entry point for the test script.""" - - apply_fix() - - conversation_id = f"test-{uuid.uuid4()}" - - agent = create_mock_agent(conversation_id) - - question = "Explain how the different Supabase tables are used in the GAIA agent and why they are important." - - answer = agent.ask(question) - - working_memory_content = agent.working_memory.list_intermediate_results() - - for key, value in working_memory_content.items(): - if isinstance(value, dict): - for k, v in value.items(): - elif isinstance(value, list): - for item in value: - else: - - - base_table = agent.base_memory.table_name - base_content = get_table_content(agent.base_memory.client, base_table) - - if base_content: - for i, entry in enumerate(base_content[:3]): # Show first 3 entries - value = entry.get('value', 'N/A') - if isinstance(value, str) and len(value) > 100: - value = value[:100] + "..." - - if len(base_content) > 3: - - working_table = agent.working_memory.table_name - working_content = get_table_content(agent.base_memory.client, working_table) - - if working_content: - for i, entry in enumerate(working_content[:3]): # Show first 3 entries - value = entry.get('value', 'N/A') - if isinstance(value, str) and len(value) > 100: - value = value[:100] + "..." - - if len(working_content) > 3: - - timestamp = datetime.now().strftime("%Y%m%d_%H%M%S") - results_file = f"gaia_tables_test_results_{timestamp}.json" - - results = { - "conversation_id": conversation_id, - "question": question, - "answer": answer, - "working_memory": working_memory_content, - "tables": { - base_table: [entry for entry in base_content], - working_table: [entry for entry in working_content] - } - } - - with open(results_file, "w") as f: - json.dump(results, f, indent=2, default=str) - - -if __name__ == "__main__": - main() \ No newline at end of file diff --git a/tests/test_memory.py b/tests/test_memory.py deleted file mode 100644 index 9656a1ee754fff59bed906252c2c1edb9ed2c0e9..0000000000000000000000000000000000000000 --- a/tests/test_memory.py +++ /dev/null @@ -1,659 +0,0 @@ -""" -Unit tests for GAIA agent memory components. -""" -import unittest -from unittest.mock import patch, MagicMock, ANY -import json -import sys -from typing import Dict, Any, List, Optional -from datetime import datetime, timedelta - -sys.path.append('.') - -from memory.supabase_memory import ( - MemoryEntry, SupabaseMemory, ConversationMemory, - ResultCache, WorkingMemory -) - - -class TestMemoryEntry(unittest.TestCase): - """Test case for MemoryEntry class.""" - - def test_init(self): - """Test initialization of memory entry.""" - entry = MemoryEntry("test_key", "test_value") - self.assertEqual(entry.key, "test_key") - self.assertEqual(entry.value, "test_value") - self.assertEqual(entry.metadata, {}) - self.assertIsNotNone(entry.created_at) - self.assertIsNone(entry.expires_at) - - metadata = {"source": "test", "importance": "high"} - entry = MemoryEntry("test_key", "test_value", metadata) - self.assertEqual(entry.metadata, metadata) - - entry = MemoryEntry("test_key", "test_value", ttl=3600) - self.assertIsNotNone(entry.expires_at) - - created = datetime.fromisoformat(entry.created_at) - expires = datetime.fromisoformat(entry.expires_at) - self.assertAlmostEqual((expires - created).total_seconds(), 3600, delta=1) - - def test_to_dict(self): - """Test conversion to dictionary.""" - entry = MemoryEntry("test_key", "test_value") - entry_dict = entry.to_dict() - self.assertEqual(entry_dict["key"], "test_key") - self.assertEqual(entry_dict["value"], "test_value") - self.assertEqual(entry_dict["metadata"], "{}") - self.assertEqual(entry_dict["created_at"], entry.created_at) - self.assertIsNone(entry_dict["expires_at"]) - - entry = MemoryEntry("test_key", {"nested": "value"}) - entry_dict = entry.to_dict() - self.assertEqual(entry_dict["key"], "test_key") - self.assertEqual(entry_dict["value"], json.dumps({"nested": "value"})) - - metadata = {"source": "test", "importance": "high"} - entry = MemoryEntry("test_key", "test_value", metadata) - entry_dict = entry.to_dict() - self.assertEqual(entry_dict["metadata"], json.dumps(metadata)) - - entry = MemoryEntry("test_key", "test_value", ttl=3600) - entry_dict = entry.to_dict() - self.assertIsNotNone(entry_dict["expires_at"]) - - def test_from_dict(self): - """Test creation from dictionary.""" - data = { - "key": "test_key", - "value": "test_value", - "metadata": "{}", - "created_at": "2023-01-01T00:00:00", - "expires_at": None - } - entry = MemoryEntry.from_dict(data) - self.assertEqual(entry.key, "test_key") - self.assertEqual(entry.value, "test_value") - self.assertEqual(entry.metadata, {}) - self.assertEqual(entry.created_at, "2023-01-01T00:00:00") - self.assertIsNone(entry.expires_at) - - data = { - "key": "test_key", - "value": '{"nested": "value"}', - "metadata": '{"source": "test"}', - "created_at": "2023-01-01T00:00:00", - "expires_at": "2023-01-01T01:00:00" - } - entry = MemoryEntry.from_dict(data) - self.assertEqual(entry.key, "test_key") - self.assertEqual(entry.value, {"nested": "value"}) - self.assertEqual(entry.metadata, {"source": "test"}) - self.assertEqual(entry.created_at, "2023-01-01T00:00:00") - self.assertEqual(entry.expires_at, "2023-01-01T01:00:00") - - data = { - "key": "test_key", - "value": '{invalid json}', - "metadata": '{invalid json}', - "created_at": "2023-01-01T00:00:00", - "expires_at": None - } - entry = MemoryEntry.from_dict(data) - self.assertEqual(entry.key, "test_key") - self.assertEqual(entry.value, "{invalid json}") - self.assertEqual(entry.metadata, {}) - - -class TestSupabaseMemory(unittest.TestCase): - """Test case for SupabaseMemory class.""" - - def setUp(self): - """Set up the test case with mocked dependencies.""" - self.client_patcher = patch('memory.supabase_memory.create_client') - self.mock_create_client = self.client_patcher.start() - self.mock_client = MagicMock() - self.mock_create_client.return_value = self.mock_client - - self.mock_table = MagicMock() - self.mock_client.table.return_value = self.mock_table - - self.mock_select = MagicMock() - self.mock_table.select.return_value = self.mock_select - - self.mock_execute = MagicMock() - self.mock_select.limit.return_value.execute = self.mock_execute - - self.env_patcher = patch.dict('memory.supabase_memory.__dict__', { - 'SUPABASE_URL': 'https://example.supabase.co', - 'SUPABASE_KEY': 'test_key' - }) - self.env_patcher.start() - - def tearDown(self): - """Clean up after the test.""" - self.client_patcher.stop() - self.env_patcher.stop() - - def test_init(self): - """Test initialization of Supabase memory.""" - memory = SupabaseMemory() - self.assertEqual(memory.table_name, "gaia_memory") - self.assertEqual(memory.cache_size, 100) - self.assertEqual(memory.cache, {}) - self.assertTrue(memory.initialized) - - config = { - "table_name": "custom_memory", - "cache_size": 50 - } - memory = SupabaseMemory(config) - self.assertEqual(memory.table_name, "custom_memory") - self.assertEqual(memory.cache_size, 50) - - config = {"enabled": False} - memory = SupabaseMemory(config) - self.assertFalse(memory.initialized) - - with patch.dict('memory.supabase_memory.__dict__', { - 'SUPABASE_URL': None, - 'SUPABASE_KEY': None - }): - memory = SupabaseMemory() - self.assertFalse(memory.initialized) - - def test_store(self): - """Test storing values in memory.""" - memory = SupabaseMemory() - - mock_upsert = MagicMock() - self.mock_table.upsert.return_value = mock_upsert - mock_upsert.execute.return_value = MagicMock() - - result = memory.store("test_key", "test_value") - - self.assertTrue(result) - - self.assertIn("test_key", memory.cache) - self.assertEqual(memory.cache["test_key"].value, "test_value") - - self.mock_table.upsert.assert_called_once() - mock_upsert.execute.assert_called_once() - - memory.cache_size = 2 - memory.store("key1", "value1") - memory.store("key2", "value2") - memory.store("key3", "value3") - - self.assertEqual(len(memory.cache), 2) - self.assertIn("key2", memory.cache) - self.assertIn("key3", memory.cache) - - mock_upsert.execute.side_effect = Exception("Test error") - result = memory.store("error_key", "error_value") - - self.assertFalse(result) - - self.assertIn("error_key", memory.cache) - - def test_retrieve(self): - """Test retrieving values from memory.""" - memory = SupabaseMemory() - - entry = MemoryEntry("cache_key", "cache_value") - memory.cache["cache_key"] = entry - - value = memory.retrieve("cache_key") - - self.assertEqual(value, "cache_value") - - entry = MemoryEntry("expired_key", "expired_value", ttl=0) - memory.cache["expired_key"] = entry - - value = memory.retrieve("expired_key", "default_value") - - self.assertEqual(value, "default_value") - - mock_eq = MagicMock() - self.mock_select.eq.return_value = mock_eq - mock_eq.execute.return_value = MagicMock(data=[{ - "key": "db_key", - "value": "db_value", - "metadata": "{}", - "created_at": datetime.now().isoformat(), - "expires_at": None - }]) - - value = memory.retrieve("db_key") - - self.assertEqual(value, "db_value") - - self.assertIn("db_key", memory.cache) - - mock_eq.execute.return_value = MagicMock(data=[{ - "key": "db_expired_key", - "value": "db_expired_value", - "metadata": "{}", - "created_at": datetime.now().isoformat(), - "expires_at": (datetime.now() - timedelta(hours=1)).isoformat() - }]) - - mock_delete = MagicMock() - self.mock_table.delete.return_value = mock_delete - mock_delete.eq.return_value.execute.return_value = MagicMock() - - value = memory.retrieve("db_expired_key", "default_value") - - self.assertEqual(value, "default_value") - - mock_eq.execute.side_effect = Exception("Test error") - value = memory.retrieve("error_key", "default_value") - - self.assertEqual(value, "default_value") - - def test_delete(self): - """Test deleting values from memory.""" - memory = SupabaseMemory() - - memory.cache["test_key"] = MemoryEntry("test_key", "test_value") - - mock_delete = MagicMock() - self.mock_table.delete.return_value = mock_delete - mock_eq = MagicMock() - mock_delete.eq.return_value = mock_eq - mock_eq.execute.return_value = MagicMock() - - result = memory.delete("test_key") - - self.assertTrue(result) - - self.assertNotIn("test_key", memory.cache) - - self.mock_table.delete.assert_called_once() - mock_delete.eq.assert_called_once_with("key", "test_key") - mock_eq.execute.assert_called_once() - - mock_eq.execute.side_effect = Exception("Test error") - memory.cache["error_key"] = MemoryEntry("error_key", "error_value") - result = memory.delete("error_key") - - self.assertFalse(result) - - self.assertNotIn("error_key", memory.cache) - - def test_clear(self): - """Test clearing all values from memory.""" - memory = SupabaseMemory() - - memory.cache["key1"] = MemoryEntry("key1", "value1") - memory.cache["key2"] = MemoryEntry("key2", "value2") - - mock_delete = MagicMock() - self.mock_table.delete.return_value = mock_delete - mock_neq = MagicMock() - mock_delete.neq.return_value = mock_neq - mock_neq.execute.return_value = MagicMock() - - result = memory.clear() - - self.assertTrue(result) - - self.assertEqual(memory.cache, {}) - - self.mock_table.delete.assert_called_once() - mock_delete.neq.assert_called_once_with("key", "") - mock_neq.execute.assert_called_once() - - mock_neq.execute.side_effect = Exception("Test error") - memory.cache["error_key"] = MemoryEntry("error_key", "error_value") - result = memory.clear() - - self.assertFalse(result) - - self.assertEqual(memory.cache, {}) - - def test_list_keys(self): - """Test listing keys from memory.""" - memory = SupabaseMemory() - - memory.cache["key1"] = MemoryEntry("key1", "value1") - memory.cache["key2"] = MemoryEntry("key2", "value2") - memory.cache["prefix_key"] = MemoryEntry("prefix_key", "prefix_value") - - mock_execute = MagicMock() - self.mock_select.execute.return_value = MagicMock(data=[ - {"key": "db_key1"}, - {"key": "db_key2"}, - {"key": "prefix_db_key"} - ]) - - keys = memory.list_keys() - - self.assertEqual(len(keys), 6) - self.assertIn("key1", keys) - self.assertIn("key2", keys) - self.assertIn("prefix_key", keys) - self.assertIn("db_key1", keys) - self.assertIn("db_key2", keys) - self.assertIn("prefix_db_key", keys) - - keys = memory.list_keys("prefix") - - self.assertEqual(len(keys), 2) - self.assertIn("prefix_key", keys) - self.assertIn("prefix_db_key", keys) - - self.mock_select.execute.side_effect = Exception("Test error") - keys = memory.list_keys() - - self.assertEqual(len(keys), 3) - self.assertIn("key1", keys) - self.assertIn("key2", keys) - self.assertIn("prefix_key", keys) - - -class TestConversationMemory(unittest.TestCase): - """Test case for ConversationMemory class.""" - - def setUp(self): - """Set up the test case with mocked dependencies.""" - self.mock_memory = MagicMock(spec=SupabaseMemory) - - self.conversation = ConversationMemory(self.mock_memory, "test_conversation") - - def test_init(self): - """Test initialization of conversation memory.""" - self.assertEqual(self.conversation.conversation_id, "test_conversation") - self.assertEqual(self.conversation.prefix, "conversation:test_conversation:") - self.assertEqual(self.conversation.memory, self.mock_memory) - - def test_add_message(self): - """Test adding messages to conversation.""" - self.mock_memory.store.return_value = True - - result = self.conversation.add_message("user", "Hello") - - self.assertTrue(result) - - self.mock_memory.store.assert_called_once() - args, kwargs = self.mock_memory.store.call_args - self.assertTrue(args[0].startswith(self.conversation.prefix + "message:")) - self.assertEqual(args[1]["role"], "user") - self.assertEqual(args[1]["content"], "Hello") - - metadata = {"source": "test"} - result = self.conversation.add_message("assistant", "Hi", metadata) - - self.assertTrue(result) - - args, kwargs = self.mock_memory.store.call_args - self.assertEqual(args[1]["role"], "assistant") - self.assertEqual(args[1]["content"], "Hi") - self.assertEqual(args[1]["metadata"], metadata) - - def test_get_messages(self): - """Test getting messages from conversation.""" - self.mock_memory.list_keys.return_value = [ - f"{self.conversation.prefix}message:2023-01-01T00:00:00", - f"{self.conversation.prefix}message:2023-01-01T00:01:00", - f"{self.conversation.prefix}message:2023-01-01T00:02:00" - ] - - self.mock_memory.retrieve.side_effect = [ - { - "role": "user", - "content": "Hello", - "timestamp": "2023-01-01T00:00:00" - }, - { - "role": "assistant", - "content": "Hi", - "timestamp": "2023-01-01T00:01:00" - }, - { - "role": "user", - "content": "How are you?", - "timestamp": "2023-01-01T00:02:00" - } - ] - - messages = self.conversation.get_messages() - - self.assertEqual(len(messages), 3) - self.assertEqual(messages[0]["role"], "user") - self.assertEqual(messages[0]["content"], "Hello") - self.assertEqual(messages[1]["role"], "assistant") - self.assertEqual(messages[1]["content"], "Hi") - self.assertEqual(messages[2]["role"], "user") - self.assertEqual(messages[2]["content"], "How are you?") - - messages = self.conversation.get_messages(limit=2) - - self.assertEqual(len(messages), 2) - self.assertEqual(messages[0]["role"], "assistant") - self.assertEqual(messages[1]["role"], "user") - - messages = self.conversation.get_messages(before="2023-01-01T00:02:00") - - self.assertEqual(len(messages), 2) - self.assertEqual(messages[0]["role"], "user") - self.assertEqual(messages[1]["role"], "assistant") - - messages = self.conversation.get_messages(after="2023-01-01T00:00:00") - - self.assertEqual(len(messages), 2) - self.assertEqual(messages[0]["role"], "assistant") - self.assertEqual(messages[1]["role"], "user") - - def test_clear(self): - """Test clearing conversation history.""" - self.mock_memory.list_keys.return_value = [ - f"{self.conversation.prefix}message:2023-01-01T00:00:00", - f"{self.conversation.prefix}message:2023-01-01T00:01:00" - ] - - self.mock_memory.delete.return_value = True - - result = self.conversation.clear() - - self.assertTrue(result) - - self.assertEqual(self.mock_memory.delete.call_count, 2) - - self.mock_memory.delete.return_value = False - result = self.conversation.clear() - - self.assertFalse(result) - - -class TestResultCache(unittest.TestCase): - """Test case for ResultCache class.""" - - def setUp(self): - """Set up the test case with mocked dependencies.""" - self.mock_memory = MagicMock(spec=SupabaseMemory) - - self.cache = ResultCache(self.mock_memory) - - def test_init(self): - """Test initialization of result cache.""" - self.assertEqual(self.cache.prefix, "cache:") - self.assertEqual(self.cache.memory, self.mock_memory) - - def test_cache_result(self): - """Test caching results.""" - self.mock_memory.store.return_value = True - - result = self.cache.cache_result("test_key", "test_value") - - self.assertTrue(result) - - self.mock_memory.store.assert_called_once_with( - "cache:test_key", "test_value", None, None - ) - - metadata = {"source": "test"} - ttl = 3600 - result = self.cache.cache_result("test_key2", "test_value2", metadata, ttl) - - self.assertTrue(result) - - self.mock_memory.store.assert_called_with( - "cache:test_key2", "test_value2", metadata, ttl - ) - - def test_get_result(self): - """Test getting cached results.""" - self.mock_memory.retrieve.return_value = "test_value" - - value = self.cache.get_result("test_key") - - self.assertEqual(value, "test_value") - - self.mock_memory.retrieve.assert_called_once_with( - "cache:test_key", None - ) - - self.mock_memory.retrieve.return_value = None - value = self.cache.get_result("missing_key", "default_value") - - self.assertEqual(value, None) - - self.mock_memory.retrieve.assert_called_with( - "cache:missing_key", "default_value" - ) - - def test_invalidate(self): - """Test invalidating cached results.""" - self.mock_memory.delete.return_value = True - - result = self.cache.invalidate("test_key") - - self.assertTrue(result) - - self.mock_memory.delete.assert_called_once_with("cache:test_key") - - def test_clear(self): - """Test clearing all cached results.""" - self.mock_memory.list_keys.return_value = [ - "cache:key1", - "cache:key2" - ] - - self.mock_memory.delete.return_value = True - - result = self.cache.clear() - - self.assertTrue(result) - - self.assertEqual(self.mock_memory.delete.call_count, 2) - - self.mock_memory.delete.return_value = False - result = self.cache.clear() - - self.assertFalse(result) - - -class TestWorkingMemory(unittest.TestCase): - """Test case for WorkingMemory class.""" - - def setUp(self): - """Set up the test case with mocked dependencies.""" - self.mock_memory = MagicMock(spec=SupabaseMemory) - - self.working_memory = WorkingMemory(self.mock_memory, "test_session") - - def test_init(self): - """Test initialization of working memory.""" - self.assertEqual(self.working_memory.session_id, "test_session") - self.assertEqual(self.working_memory.prefix, "working:test_session:") - self.assertEqual(self.working_memory.memory, self.mock_memory) - - def test_store_intermediate_result(self): - """Test storing intermediate results.""" - self.mock_memory.store.return_value = True - - result = self.working_memory.store_intermediate_result("test_key", "test_value") - - self.assertTrue(result) - - self.mock_memory.store.assert_called_once_with( - "working:test_session:test_key", "test_value", None - ) - - metadata = {"source": "test"} - result = self.working_memory.store_intermediate_result("test_key2", "test_value2", metadata) - - self.assertTrue(result) - - self.mock_memory.store.assert_called_with( - "working:test_session:test_key2", "test_value2", metadata - ) - - def test_get_intermediate_result(self): - """Test getting intermediate results.""" - self.mock_memory.retrieve.return_value = "test_value" - - value = self.working_memory.get_intermediate_result("test_key") - - self.assertEqual(value, "test_value") - - self.mock_memory.retrieve.assert_called_once_with( - "working:test_session:test_key", None - ) - - self.mock_memory.retrieve.return_value = None - value = self.working_memory.get_intermediate_result("missing_key", "default_value") - - self.assertEqual(value, None) - - self.mock_memory.retrieve.assert_called_with( - "working:test_session:missing_key", "default_value" - ) - - def test_list_intermediate_results(self): - """Test listing intermediate results.""" - self.mock_memory.list_keys.return_value = [ - "working:test_session:key1", - "working:test_session:key2" - ] - - self.mock_memory.retrieve.side_effect = [ - "value1", - "value2" - ] - - results = self.working_memory.list_intermediate_results() - - self.assertEqual(len(results), 2) - self.assertEqual(results["key1"], "value1") - self.assertEqual(results["key2"], "value2") - - self.mock_memory.list_keys.assert_called_once_with(self.working_memory.prefix) - - self.assertEqual(self.mock_memory.retrieve.call_count, 2) - - def test_clear(self): - """Test clearing all intermediate results.""" - self.mock_memory.list_keys.return_value = [ - "working:test_session:key1", - "working:test_session:key2" - ] - - self.mock_memory.delete.return_value = True - - result = self.working_memory.clear() - - self.assertTrue(result) - - self.assertEqual(self.mock_memory.delete.call_count, 2) - - self.mock_memory.delete.return_value = False - result = self.working_memory.clear() - - self.assertFalse(result) - - -if __name__ == "__main__": - unittest.main() \ No newline at end of file diff --git a/tests/test_openai_integration.py b/tests/test_openai_integration.py deleted file mode 100644 index c1b0bc998a4b80f1a209bc1a24555b0d8ad08614..0000000000000000000000000000000000000000 --- a/tests/test_openai_integration.py +++ /dev/null @@ -1,390 +0,0 @@ -""" -Test script for OpenAI integration with newer models. - -This script tests the OpenAI integration with newer models like GPT-4o to ensure -that the configuration changes are working properly. It performs the following tests: -1. Verifies connection to the OpenAI API -2. Tests sending requests to newer models (GPT-4o, GPT-4-turbo, etc.) -3. Verifies that the configuration changes are working properly -4. Tests the models in the context of the GAIA agent - -Usage: - python tests/test_openai_integration.py [--verbose] -""" - -import os -import sys -import json -import time -import logging -import argparse -import requests -from typing import Dict, Any, List, Tuple, Optional - -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(name)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("openai_test") - -sys.path.append('.') - -try: - from agent.config import ( - OPENAI_API_KEY, - OPENAI_API_BASE, - DEFAULT_MODEL, - VISION_MODEL, - REASONING_MODEL, - MODEL_TEMPERATURE, - MODEL_MAX_TOKENS - ) -except ImportError as e: - logger.error(f"Error importing configuration: {str(e)}") - sys.exit(1) - -def verify_openai_connection() -> Tuple[bool, str, List[str]]: - """ - Verify connection to OpenAI API. - - Returns: - Tuple[bool, str, List[str]]: (success, message, available_models) - """ - logger.info("Verifying OpenAI API connection...") - - if not OPENAI_API_KEY: - return False, "OpenAI API key not set", [] - - try: - headers = { - "Content-Type": "application/json", - "Authorization": f"Bearer {OPENAI_API_KEY}" - } - - response = requests.get( - f"{OPENAI_API_BASE}/models", - headers=headers - ) - - if response.status_code == 200: - models_data = response.json().get("data", []) - model_count = len(models_data) - - model_ids = [model.get("id") for model in models_data] - - return True, f"Successfully connected to OpenAI API. {model_count} models available.", model_ids - else: - return False, f"Failed to connect to OpenAI API. Status code: {response.status_code}, Response: {response.text}", [] - - except Exception as e: - return False, f"Error connecting to OpenAI API: {str(e)}", [] - -def test_model(model_id: str) -> Tuple[bool, str, Dict[str, Any]]: - """ - Test a specific OpenAI model. - - Args: - model_id: The ID of the model to test - - Returns: - Tuple[bool, str, Dict[str, Any]]: (success, message, response_data) - """ - logger.info(f"Testing model: {model_id}") - - if not OPENAI_API_KEY: - return False, "OpenAI API key not set", {} - - try: - headers = { - "Content-Type": "application/json", - "Authorization": f"Bearer {OPENAI_API_KEY}" - } - - payload = { - "model": model_id, - "messages": [ - {"role": "system", "content": "You are a helpful assistant."}, - {"role": "user", "content": "What is the capital of France?"} - ], - "temperature": MODEL_TEMPERATURE, - "max_tokens": MODEL_MAX_TOKENS - } - - start_time = time.time() - response = requests.post( - f"{OPENAI_API_BASE}/chat/completions", - headers=headers, - json=payload - ) - end_time = time.time() - - if response.status_code == 200: - response_data = response.json() - content = response_data.get("choices", [{}])[0].get("message", {}).get("content", "") - tokens = response_data.get("usage", {}) - - message = ( - f"Successfully tested model {model_id}.\n" - f"Response: {content[:100]}...\n" - f"Tokens: {tokens}\n" - f"Time: {end_time - start_time:.2f} seconds" - ) - - return True, message, response_data - else: - return False, f"Failed to test model {model_id}. Status code: {response.status_code}, Response: {response.text}", {} - - except Exception as e: - return False, f"Error testing model {model_id}: {str(e)}", {} - -def test_vision_model(model_id: str) -> Tuple[bool, str, Dict[str, Any]]: - """ - Test a vision model with an image. - - Args: - model_id: The ID of the model to test - - Returns: - Tuple[bool, str, Dict[str, Any]]: (success, message, response_data) - """ - logger.info(f"Testing vision capabilities of model: {model_id}") - - if not OPENAI_API_KEY: - return False, "OpenAI API key not set", {} - - try: - headers = { - "Content-Type": "application/json", - "Authorization": f"Bearer {OPENAI_API_KEY}" - } - - payload = { - "model": model_id, - "messages": [ - {"role": "system", "content": "You are a helpful assistant."}, - {"role": "user", "content": [ - {"type": "text", "text": "What's in this image?"}, - {"type": "image_url", "image_url": { - "url": "https://upload.wikimedia.org/wikipedia/commons/thumb/e/ec/Mona_Lisa%2C_by_Leonardo_da_Vinci%2C_from_C2RMF_retouched.jpg/800px-Mona_Lisa%2C_by_Leonardo_da_Vinci%2C_from_C2RMF_retouched.jpg" - }} - ]} - ], - "temperature": MODEL_TEMPERATURE, - "max_tokens": MODEL_MAX_TOKENS - } - - start_time = time.time() - response = requests.post( - f"{OPENAI_API_BASE}/chat/completions", - headers=headers, - json=payload - ) - end_time = time.time() - - if response.status_code == 200: - response_data = response.json() - content = response_data.get("choices", [{}])[0].get("message", {}).get("content", "") - tokens = response_data.get("usage", {}) - - message = ( - f"Successfully tested vision capabilities of model {model_id}.\n" - f"Response: {content[:100]}...\n" - f"Tokens: {tokens}\n" - f"Time: {end_time - start_time:.2f} seconds" - ) - - return True, message, response_data - else: - return False, f"Failed to test vision capabilities of model {model_id}. Status code: {response.status_code}, Response: {response.text}", {} - - except Exception as e: - return False, f"Error testing vision capabilities of model {model_id}: {str(e)}", {} - -def test_gaia_agent_with_openai() -> Tuple[bool, str, Dict[str, Any]]: - """ - Test the GAIA agent with OpenAI models. - - Returns: - Tuple[bool, str, Dict[str, Any]]: (success, message, results) - """ - logger.info("Testing GAIA agent with OpenAI models...") - - try: - from agent.agent import GAIAAgent - - agent = GAIAAgent() - - question = "What is the capital of France?" - - start_time = time.time() - answer = agent(question) - end_time = time.time() - - memory_snapshot = agent.get_memory_snapshot() - - results = { - "question": question, - "answer": answer, - "processing_time": end_time - start_time, - "memory_snapshot": memory_snapshot - } - - message = ( - f"Successfully tested GAIA agent with OpenAI models.\n" - f"Question: {question}\n" - f"Answer: {answer}\n" - f"Processing time: {end_time - start_time:.2f} seconds" - ) - - return True, message, results - - except Exception as e: - return False, f"Error testing GAIA agent with OpenAI models: {str(e)}", {} - -def run_tests(args: argparse.Namespace) -> Dict[str, Any]: - """ - Run all tests. - - Args: - args: Command-line arguments - - Returns: - Dict[str, Any]: Test results - """ - results = { - "connection": {}, - "models": {}, - "vision": {}, - "agent": {} - } - - connection_success, connection_message, available_models = verify_openai_connection() - results["connection"] = { - "success": connection_success, - "message": connection_message, - "available_models": available_models - } - - if not connection_success: - logger.error("Failed to connect to OpenAI API. Skipping remaining tests.") - return results - - models_to_test = [DEFAULT_MODEL, REASONING_MODEL] - - for model_id in models_to_test: - if model_id in available_models: - model_success, model_message, model_data = test_model(model_id) - results["models"][model_id] = { - "success": model_success, - "message": model_message, - "data": model_data - } - else: - logger.warning(f"Model {model_id} not available. Skipping test.") - results["models"][model_id] = { - "success": False, - "message": f"Model {model_id} not available in your account.", - "data": {} - } - - if VISION_MODEL in available_models: - vision_success, vision_message, vision_data = test_vision_model(VISION_MODEL) - results["vision"] = { - "success": vision_success, - "message": vision_message, - "data": vision_data - } - else: - logger.warning(f"Vision model {VISION_MODEL} not available. Skipping test.") - results["vision"] = { - "success": False, - "message": f"Vision model {VISION_MODEL} not available in your account.", - "data": {} - } - - if args.test_agent: - agent_success, agent_message, agent_data = test_gaia_agent_with_openai() - results["agent"] = { - "success": agent_success, - "message": agent_message, - "data": agent_data - } - else: - logger.info("Skipping GAIA agent test (use --test-agent to include)") - results["agent"] = { - "success": None, - "message": "Test skipped", - "data": {} - } - - return results - -def print_results(results: Dict[str, Any]) -> None: - """ - Print test results. - - Args: - results: Test results - """ - - if results["connection"]["success"]: - else: - - for model_id, model_result in results["models"].items(): - if model_result["success"]: - else: - - if results["vision"].get("success"): - elif results["vision"].get("success") is False: - else: - - if results["agent"].get("success"): - elif results["agent"].get("success") is False: - else: - - -def save_results(results: Dict[str, Any], output_file: str) -> None: - """ - Save test results to a file. - - Args: - results: Test results - output_file: Output file path - """ - with open(output_file, "w") as f: - json.dump(results, f, indent=2) - - logger.info(f"Results saved to {output_file}") - -def main(): - """Main function.""" - parser = argparse.ArgumentParser(description="Test OpenAI integration with newer models") - parser.add_argument("--verbose", "-v", action="store_true", help="Enable verbose logging") - parser.add_argument("--output", "-o", type=str, default="openai_test_results.json", help="Output file for test results") - parser.add_argument("--test-agent", "-a", action="store_true", help="Test the GAIA agent with OpenAI models") - args = parser.parse_args() - - if args.verbose: - logger.setLevel(logging.DEBUG) - if not any(isinstance(h, logging.StreamHandler) for h in logger.handlers): - console_handler = logging.StreamHandler() - console_handler.setLevel(logging.DEBUG) - console_handler.setFormatter(logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")) - logger.addHandler(console_handler) - - results = run_tests(args) - - print_results(results) - - save_results(results, args.output) - - return all( - results["connection"]["success"] and - all(model["success"] for model in results["models"].values() if model["success"] is not None) and - (results["vision"]["success"] if results["vision"].get("success") is not None else True) and - (results["agent"]["success"] if results["agent"].get("success") is not None else True) - ) - -if __name__ == "__main__": - success = main() - sys.exit(0 if success else 1) \ No newline at end of file diff --git a/tests/test_perplexity_sonar.py b/tests/test_perplexity_sonar.py deleted file mode 100644 index e3ec9ca5c5794b43b0081c763703bc535ca73ba9..0000000000000000000000000000000000000000 --- a/tests/test_perplexity_sonar.py +++ /dev/null @@ -1,143 +0,0 @@ -""" -Script to test the Perplexity API with the Sonar Reasoning model. -""" - -import os -import sys -import json -import requests -import traceback -from dotenv import load_dotenv - -sys.path.append('.') - -import logging -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(name)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("perplexity_sonar_test") - -class PerplexitySonarTester: - """Class for testing the Perplexity API with the Sonar Reasoning model.""" - - def __init__(self, verbose=False): - """ - Initialize the tester. - - Args: - verbose: Whether to enable verbose logging - """ - self.verbose = verbose - - if verbose: - logging.getLogger().setLevel(logging.DEBUG) - - logger.info("Loading environment variables...") - load_dotenv() - - self.api_key = os.getenv("PERPLEXITY_API_KEY") - if not self.api_key: - logger.error("PERPLEXITY_API_KEY environment variable not found.") - raise ValueError("PERPLEXITY_API_KEY environment variable not found.") - - masked_key = f"{self.api_key[:8]}...{self.api_key[-4:]}" if len(self.api_key) > 12 else "***" - logger.info(f"Perplexity API Key: {masked_key}") - - self.endpoint = "https://api.perplexity.ai/chat/completions" - self.model = "sonar-reasoning" - self.timeout = 30 - - logger.info(f"Using endpoint: {self.endpoint}") - logger.info(f"Using model: {self.model}") - - def test_simple_query(self, query="What is the capital of France?", system_prompt=None): - """ - Test a simple query to the Perplexity API. - - Args: - query: The query to test - system_prompt: Optional system prompt - - Returns: - The API response - """ - logger.info(f"Testing query: {query}") - - try: - headers = { - "Authorization": f"Bearer {self.api_key}", - "Content-Type": "application/json" - } - - messages = [] - if system_prompt: - messages.append({"role": "system", "content": system_prompt}) - messages.append({"role": "user", "content": query}) - - data = { - "model": self.model, - "messages": messages, - "max_tokens": 500 - } - - data_json = json.dumps(data) - if self.verbose: - - logger.info("Sending request to Perplexity API...") - - response = requests.post( - self.endpoint, - headers=headers, - json=data, - timeout=self.timeout - ) - - logger.info(f"Response status code: {response.status_code}") - - if response.status_code == 200: - response_json = response.json() - - if self.verbose: - - if "choices" in response_json and len(response_json["choices"]) > 0: - content = response_json["choices"][0]["message"]["content"] - logger.info(f"Response content: {content[:100]}...") - - - logger.info("API call completed successfully!") - return response_json - else: - logger.error(f"API returned status code {response.status_code}") - logger.error(f"Response: {response.text}") - return None - except Exception as e: - logger.error(f"Error: {type(e).__name__}") - logger.error(f"Details: {str(e)}") - if self.verbose: - return None - - -def main(): - """Main function.""" - import argparse - - parser = argparse.ArgumentParser(description="Test Perplexity API with Sonar Reasoning model") - parser.add_argument("--verbose", action="store_true", help="Enable verbose logging") - parser.add_argument("--query", type=str, default="What is the capital of France?", help="Query to test") - parser.add_argument("--system", type=str, help="System prompt") - args = parser.parse_args() - - try: - tester = PerplexitySonarTester(verbose=args.verbose) - - tester.test_simple_query(query=args.query, system_prompt=args.system) - - except Exception as e: - if args.verbose: - traceback.print_exc() - sys.exit(1) - - -if __name__ == "__main__": - main() \ No newline at end of file diff --git a/tests/test_perplexity_tool.py b/tests/test_perplexity_tool.py deleted file mode 100644 index 73c99c0e2a2bf257ed724def1480d1ff355f957c..0000000000000000000000000000000000000000 --- a/tests/test_perplexity_tool.py +++ /dev/null @@ -1,191 +0,0 @@ -""" -Unit tests for the Perplexity API tool. -""" -import unittest -from unittest.mock import patch, MagicMock -import json -import sys -import os -from typing import Dict, Any, List, Optional - -sys.path.append('.') - -from tools.perplexity_tool import PerplexityTool, create_perplexity_tool - - -class TestPerplexityTool(unittest.TestCase): - """Test case for the Perplexity API tool.""" - - def setUp(self): - """Set up the test case with mocked dependencies.""" - self.urlopen_patcher = patch('tools.perplexity_tool.urllib.request.urlopen') - self.mock_urlopen = self.urlopen_patcher.start() - - self.mock_response = MagicMock() - self.mock_response.read.return_value = json.dumps({ - "choices": [ - { - "message": { - "content": "Test response content" - } - } - ], - "usage": { - "prompt_tokens": 10, - "completion_tokens": 20, - "total_tokens": 30 - } - }).encode('utf-8') - - self.mock_urlopen.return_value.__enter__.return_value = self.mock_response - - self.getenv_patcher = patch('tools.perplexity_tool.os.getenv') - self.mock_getenv = self.getenv_patcher.start() - self.mock_getenv.return_value = "test_api_key" - - def tearDown(self): - """Clean up after the test.""" - self.urlopen_patcher.stop() - self.getenv_patcher.stop() - - def test_initialization(self): - """Test the initialization of the PerplexityTool class.""" - tool = PerplexityTool() - self.assertEqual(tool.api_key, "test_api_key") - self.assertEqual(tool.api_url, "https://api.perplexity.ai") - self.assertEqual(tool.model, "sonar-reasoning") - self.assertEqual(tool.timeout, 30) - - config = { - "api_key": "custom_api_key", - "timeout": 60 - } - tool = PerplexityTool(config) - self.assertEqual(tool.api_key, "custom_api_key") - self.assertEqual(tool.timeout, 60) - - self.mock_getenv.return_value = None - with self.assertLogs(level='WARNING') as cm: - tool = PerplexityTool() - self.assertIn("Perplexity API key not found", cm.output[0]) - - def test_query(self): - """Test the query method of the PerplexityTool class.""" - tool = PerplexityTool() - - result = tool.query("What is the capital of France?") - self.assertEqual(result["content"], "Test response content") - self.assertEqual(result["query"], "What is the capital of France?") - self.assertEqual(result["model"], "sonar-reasoning") - self.assertEqual(result["usage"]["total_tokens"], 30) - - result = tool.query("What is the capital of France?", system_prompt="You are a helpful assistant.") - self.assertIn("system", str(result["raw_response"])) - - result = tool.query("What is the capital of France?", max_tokens=100) - self.assertIn("max_tokens", str(result["raw_response"])) - - tool.api_key = None - with self.assertRaises(Exception): - tool.query("What is the capital of France?") - - def test_search(self): - """Test the search method of the PerplexityTool class.""" - tool = PerplexityTool() - - result = tool.search("What is the capital of France?") - self.assertEqual(result["content"], "Test response content") - self.assertEqual(result["query"], "What is the capital of France?") - - self.assertIn("system", str(result["raw_response"])) - - def test_extract_content(self): - """Test the _extract_content method of the PerplexityTool class.""" - tool = PerplexityTool() - - response = { - "choices": [ - { - "message": { - "content": "Test content" - } - } - ] - } - content = tool._extract_content(response) - self.assertEqual(content, "Test content") - - content = tool._extract_content({}) - self.assertIsNone(content) - - content = tool._extract_content({"choices": [{}]}) - self.assertIsNone(content) - - def test_extract_citations(self): - """Test the _extract_citations method of the PerplexityTool class.""" - tool = PerplexityTool() - - response = { - "citations": ["https://example.com/1", "https://example.com/2"] - } - citations = tool._extract_citations(response) - self.assertEqual(citations, ["https://example.com/1", "https://example.com/2"]) - - citations = tool._extract_citations({}) - self.assertEqual(citations, []) - - def test_get_usage(self): - """Test the _get_usage method of the PerplexityTool class.""" - tool = PerplexityTool() - - response = { - "usage": { - "prompt_tokens": 10, - "completion_tokens": 20, - "total_tokens": 30 - } - } - usage = tool._get_usage(response) - self.assertEqual(usage["prompt_tokens"], 10) - self.assertEqual(usage["completion_tokens"], 20) - self.assertEqual(usage["total_tokens"], 30) - - usage = tool._get_usage({}) - self.assertEqual(usage, {}) - - def test_error_handling(self): - """Test error handling in the PerplexityTool class.""" - tool = PerplexityTool() - - http_error = MagicMock() - http_error.code = 401 - http_error.reason = "Unauthorized" - http_error.read.return_value = b'{"error": "Invalid API key"}' - self.mock_urlopen.side_effect = http_error - - with self.assertRaises(Exception) as cm: - tool.query("What is the capital of France?") - self.assertIn("HTTP error", str(cm.exception)) - - url_error = MagicMock() - url_error.reason = "Connection refused" - self.mock_urlopen.side_effect = url_error - - with self.assertRaises(Exception) as cm: - tool.query("What is the capital of France?") - self.assertIn("connection error", str(cm.exception)) - - self.mock_urlopen.side_effect = Exception("Test error") - - with self.assertRaises(Exception) as cm: - tool.query("What is the capital of France?") - self.assertIn("Test error", str(cm.exception)) - - def test_factory_function(self): - """Test the create_perplexity_tool factory function.""" - tool = create_perplexity_tool() - self.assertIsInstance(tool, PerplexityTool) - - -if __name__ == "__main__": - unittest.main() \ No newline at end of file diff --git a/tests/test_project_structure.py b/tests/test_project_structure.py deleted file mode 100644 index 345811d7f43ee9457d45f6f52641896256bc0589..0000000000000000000000000000000000000000 --- a/tests/test_project_structure.py +++ /dev/null @@ -1,46 +0,0 @@ -""" -Test file to verify the project structure is set up correctly. -""" -import os -import unittest - - -class TestProjectStructure(unittest.TestCase): - """Test case for verifying the project structure.""" - - def test_directories_exist(self): - """Test that all required directories exist.""" - required_dirs = ['agent', 'memory', 'tools', 'utils', 'tests'] - for directory in required_dirs: - self.assertTrue(os.path.isdir(directory), f"Directory {directory} does not exist") - self.assertTrue(os.path.isfile(os.path.join(directory, '__init__.py')), - f"__init__.py file missing in {directory} directory") - - def test_requirements_file(self): - """Test that requirements.txt exists and contains all required dependencies.""" - self.assertTrue(os.path.isfile('requirements.txt'), "requirements.txt file does not exist") - - required_dependencies = [ - 'gradio>=5.25.2', - 'requests', - 'langchain', - 'langgraph', - 'supabase', - 'duckduckgo-search', - 'beautifulsoup4', - 'pillow', - 'arxiv', - 'perplexity-python', - 'python-dotenv' - ] - - with open('requirements.txt', 'r') as f: - requirements_content = f.read() - - for dependency in required_dependencies: - self.assertIn(dependency, requirements_content, - f"Dependency {dependency} not found in requirements.txt") - - -if __name__ == '__main__': - unittest.main() \ No newline at end of file diff --git a/tests/test_serper_api_final.py b/tests/test_serper_api_final.py deleted file mode 100644 index 29adb52862a35be2cb8ba6b3c75044e59bd332bc..0000000000000000000000000000000000000000 --- a/tests/test_serper_api_final.py +++ /dev/null @@ -1,346 +0,0 @@ -""" -Comprehensive test script for Serper API integration. - -This script tests the Serper API integration with the GAIA agent, including: -1. Direct API connection test -2. Search functionality test -3. Integration with the GAIA agent (optional) - -Usage: - python tests/test_serper_api_final.py [--test-agent] [--verbose] -""" - -import os -import sys -import json -import time -import logging -import argparse -import requests -import traceback -from pathlib import Path -from typing import Dict, Any, List, Tuple, Optional - -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(name)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("serper_api_test") - -sys.path.append('.') - -def read_env_file(file_path='.env'): - """Read environment variables from .env file.""" - env_vars = {} - try: - with open(file_path, 'r') as f: - for line in f: - line = line.strip() - if not line or line.startswith('#'): - continue - if '=' in line: - key, value = line.split('=', 1) - env_vars[key] = value - return env_vars - except Exception as e: - logger.error(f"Error reading .env file: {str(e)}") - return {} - -def test_api_connection(api_key: str, api_url: str = "https://google.serper.dev/search") -> Tuple[bool, str]: - """ - Test direct connection to the Serper API. - - Args: - api_key: The Serper API key - api_url: The Serper API URL - - Returns: - Tuple[bool, str]: (success, message) - """ - logger.info("Testing direct connection to Serper API...") - - if not api_key: - return False, "Serper API key not provided" - - try: - headers = { - "X-API-KEY": api_key, - "Content-Type": "application/json" - } - - payload = { - "q": "test query", - "gl": "us", - "hl": "en", - "num": 1 - } - - logger.info(f"Sending request to {api_url}...") - response = requests.post( - api_url, - headers=headers, - json=payload, - timeout=10 - ) - - if response.status_code == 200: - return True, "Successfully connected to Serper API" - else: - return False, f"Failed to connect to Serper API. Status code: {response.status_code}, Response: {response.text}" - - except Exception as e: - logger.error(f"Error connecting to Serper API: {str(e)}") - logger.error(traceback.format_exc()) - return False, f"Error connecting to Serper API: {str(e)}" - -def test_search_functionality(api_key: str, query: str = "artificial intelligence latest developments", api_url: str = "https://google.serper.dev/search") -> Tuple[bool, str, Optional[Dict[str, Any]]]: - """ - Test the Serper search functionality. - - Args: - api_key: The Serper API key - query: The search query - api_url: The Serper API URL - - Returns: - Tuple[bool, str, Optional[Dict[str, Any]]]: (success, message, response_data) - """ - logger.info(f"Testing search functionality with query: {query}") - - if not api_key: - return False, "Serper API key not provided", None - - try: - headers = { - "X-API-KEY": api_key, - "Content-Type": "application/json" - } - - payload = { - "q": query, - "gl": "us", - "hl": "en", - "num": 5 - } - - start_time = time.time() - response = requests.post( - api_url, - headers=headers, - json=payload, - timeout=10 - ) - end_time = time.time() - - if response.status_code == 200: - response_data = response.json() - organic_results = response_data.get("organic", []) - result_count = len(organic_results) - - message = ( - f"Successfully tested search functionality.\n" - f"Query: {query}\n" - f"Results: {result_count}\n" - f"Time: {end_time - start_time:.2f} seconds" - ) - - return True, message, response_data - else: - return False, f"Failed to test search functionality. Status code: {response.status_code}, Response: {response.text}", None - - except Exception as e: - logger.error(f"Error testing search functionality: {str(e)}") - logger.error(traceback.format_exc()) - return False, f"Error testing search functionality: {str(e)}", None - -def test_agent_integration(api_key: str) -> Tuple[bool, str, Optional[Dict[str, Any]]]: - """ - Test the integration with the GAIA agent. - - Args: - api_key: The Serper API key - - Returns: - Tuple[bool, str, Optional[Dict[str, Any]]]: (success, message, results) - """ - logger.info("Testing integration with GAIA agent...") - - try: - from agent.agent import GaiaAgent - from tools.web_tools import SerperSearchTool - - serper_tool = SerperSearchTool() - serper_tool.api_key = api_key - - agent = GaiaAgent() - - agent.register_tool("serper_search", serper_tool.search) - - query = "What are the latest developments in artificial intelligence?" - - start_time = time.time() - response = agent.run(query, tools=["serper_search"]) - end_time = time.time() - - results = { - "query": query, - "response": response, - "processing_time": end_time - start_time - } - - message = ( - f"Successfully tested integration with GAIA agent.\n" - f"Query: {query}\n" - f"Processing time: {end_time - start_time:.2f} seconds" - ) - - return True, message, results - - except ImportError as e: - logger.error(f"Error importing required modules: {str(e)}") - logger.error(traceback.format_exc()) - return False, f"Error importing required modules: {str(e)}", None - - except Exception as e: - logger.error(f"Error testing integration with GAIA agent: {str(e)}") - logger.error(traceback.format_exc()) - return False, f"Error testing integration with GAIA agent: {str(e)}", None - -def run_tests(args: argparse.Namespace) -> Dict[str, Any]: - """ - Run all tests. - - Args: - args: Command-line arguments - - Returns: - Dict[str, Any]: Test results - """ - results = { - "api_connection": {}, - "search_functionality": {}, - "agent_integration": {} - } - - env_vars = read_env_file() - api_key = env_vars.get("SERPER_API_KEY", "") - - if not api_key: - logger.error("Serper API key not found in .env file") - results["error"] = "Serper API key not found in .env file" - return results - - logger.info(f"Using Serper API key: {api_key[:4]}...{api_key[-4:]}") - - connection_success, connection_message = test_api_connection(api_key) - results["api_connection"] = { - "success": connection_success, - "message": connection_message - } - - if not connection_success: - logger.error("Failed to connect to Serper API. Skipping remaining tests.") - return results - - search_success, search_message, search_data = test_search_functionality(api_key) - results["search_functionality"] = { - "success": search_success, - "message": search_message, - "data": search_data - } - - if args.test_agent: - agent_success, agent_message, agent_data = test_agent_integration(api_key) - results["agent_integration"] = { - "success": agent_success, - "message": agent_message, - "data": agent_data - } - else: - logger.info("Skipping agent integration test (use --test-agent to include)") - results["agent_integration"] = { - "success": None, - "message": "Test skipped", - "data": None - } - - return results - -def print_results(results: Dict[str, Any]) -> None: - """ - Print test results. - - Args: - results: Test results - """ - - if "error" in results: - return - - if results["api_connection"]["success"]: - else: - - if results["search_functionality"]["success"]: - - if results["search_functionality"]["data"] and results["search_functionality"]["data"].get("organic"): - first_result = results["search_functionality"]["data"]["organic"][0] - else: - - if results["agent_integration"].get("success") is True: - - if results["agent_integration"]["data"] and results["agent_integration"]["data"].get("response"): - response = results["agent_integration"]["data"]["response"] - elif results["agent_integration"].get("success") is False: - else: - - -def save_results(results: Dict[str, Any], output_file: str) -> None: - """ - Save test results to a file. - - Args: - results: Test results - output_file: Output file path - """ - try: - with open(output_file, "w") as f: - json.dump(results, f, indent=2) - - logger.info(f"Results saved to {output_file}") - except Exception as e: - logger.error(f"Error saving results to {output_file}: {str(e)}") - -def main(): - """Main function.""" - parser = argparse.ArgumentParser(description="Test Serper API integration") - parser.add_argument("--test-agent", "-a", action="store_true", help="Test integration with GAIA agent") - parser.add_argument("--verbose", "-v", action="store_true", help="Enable verbose logging") - parser.add_argument("--output", "-o", type=str, default="serper_api_test_results.json", help="Output file for test results") - args = parser.parse_args() - - if args.verbose: - logger.setLevel(logging.DEBUG) - if not any(isinstance(h, logging.StreamHandler) for h in logger.handlers): - console_handler = logging.StreamHandler() - console_handler.setLevel(logging.DEBUG) - console_handler.setFormatter(logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")) - logger.addHandler(console_handler) - - results = run_tests(args) - - print_results(results) - - save_results(results, args.output) - - if "error" in results: - return False - - return ( - results["api_connection"]["success"] and - results["search_functionality"]["success"] and - (results["agent_integration"]["success"] if results["agent_integration"].get("success") is not None else True) - ) - -if __name__ == "__main__": - success = main() - sys.exit(0 if success else 1) \ No newline at end of file diff --git a/tests/test_serper_integration.py b/tests/test_serper_integration.py deleted file mode 100644 index 79542a7afdbe16ccedf587d31b5566b18f61ccce..0000000000000000000000000000000000000000 --- a/tests/test_serper_integration.py +++ /dev/null @@ -1,310 +0,0 @@ -""" -Test script for Serper API integration. - -This script tests the Serper API integration with the GAIA agent to ensure -that the configuration changes are working properly. It performs the following tests: -1. Verifies connection to the Serper API -2. Tests the search functionality -3. Tests the Serper API in the context of the GAIA agent - -Usage: - python tests/test_serper_integration.py [--verbose] -""" - -import os -import sys -import json -import time -import logging -import argparse -import requests -import traceback -from typing import Dict, Any, List, Tuple, Optional - -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(name)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("serper_test") - -sys.path.append('.') - -try: - from dotenv import load_dotenv - load_dotenv() - logger.info("Loaded environment variables from .env file") -except ImportError: - logger.warning("python-dotenv not installed, using environment variables directly") - -try: - from agent.config import ( - SERPER_API_KEY, - SERPER_API_URL, - WEB_SEARCH_RESULT_COUNT - ) - logger.info(f"Imported configuration from agent.config") -except ImportError as e: - logger.error(f"Error importing configuration: {str(e)}") - logger.error(traceback.format_exc()) - sys.exit(1) - -def verify_serper_connection() -> Tuple[bool, str]: - """ - Verify connection to Serper API. - - Returns: - Tuple[bool, str]: (success, message) - """ - logger.info("Verifying Serper API connection...") - - if not SERPER_API_KEY: - return False, "Serper API key not set" - - try: - headers = { - "X-API-KEY": SERPER_API_KEY, - "Content-Type": "application/json" - } - - payload = { - "q": "test query", - "gl": "us", - "hl": "en", - "num": 1 - } - - response = requests.post( - SERPER_API_URL, - headers=headers, - json=payload, - timeout=10 - ) - - if response.status_code == 200: - return True, "Successfully connected to Serper API" - else: - return False, f"Failed to connect to Serper API. Status code: {response.status_code}, Response: {response.text}" - - except Exception as e: - logger.error(f"Error connecting to Serper API: {str(e)}") - logger.error(traceback.format_exc()) - return False, f"Error connecting to Serper API: {str(e)}" - -def test_serper_search(query: str = "artificial intelligence latest developments") -> Tuple[bool, str, Dict[str, Any]]: - """ - Test the Serper search functionality. - - Args: - query: The search query to test - - Returns: - Tuple[bool, str, Dict[str, Any]]: (success, message, response_data) - """ - logger.info(f"Testing Serper search with query: {query}") - - if not SERPER_API_KEY: - return False, "Serper API key not set", {} - - try: - headers = { - "X-API-KEY": SERPER_API_KEY, - "Content-Type": "application/json" - } - - payload = { - "q": query, - "gl": "us", - "hl": "en", - "num": WEB_SEARCH_RESULT_COUNT - } - - start_time = time.time() - response = requests.post( - SERPER_API_URL, - headers=headers, - json=payload, - timeout=10 - ) - end_time = time.time() - - if response.status_code == 200: - response_data = response.json() - organic_results = response_data.get("organic", []) - result_count = len(organic_results) - - first_result = organic_results[0] if organic_results else {} - - message = ( - f"Successfully tested Serper search.\n" - f"Query: {query}\n" - f"Results: {result_count}\n" - f"First result: {first_result.get('title', 'No title')}\n" - f"Time: {end_time - start_time:.2f} seconds" - ) - - return True, message, response_data - else: - return False, f"Failed to test Serper search. Status code: {response.status_code}, Response: {response.text}", {} - - except Exception as e: - logger.error(f"Error testing Serper search: {str(e)}") - logger.error(traceback.format_exc()) - return False, f"Error testing Serper search: {str(e)}", {} - -def test_gaia_agent_with_serper() -> Tuple[bool, str, Dict[str, Any]]: - """ - Test the GAIA agent with Serper API. - - Returns: - Tuple[bool, str, Dict[str, Any]]: (success, message, results) - """ - logger.info("Testing GAIA agent with Serper API...") - - try: - from agent.agent import GaiaAgent - from tools.web_tools import create_serper_search - - agent = GaiaAgent() - - serper_tool = create_serper_search() - agent.register_tool("serper_search", serper_tool.search) - - query = "What are the latest developments in artificial intelligence?" - - start_time = time.time() - response = agent.run(query, tools=["serper_search"]) - end_time = time.time() - - results = { - "query": query, - "response": response, - "processing_time": end_time - start_time - } - - message = ( - f"Successfully tested GAIA agent with Serper API.\n" - f"Query: {query}\n" - f"Response: {response[:100]}...\n" - f"Processing time: {end_time - start_time:.2f} seconds" - ) - - return True, message, results - - except Exception as e: - logger.error(f"Error testing GAIA agent with Serper API: {str(e)}") - logger.error(traceback.format_exc()) - return False, f"Error testing GAIA agent with Serper API: {str(e)}", {} - -def run_tests(args: argparse.Namespace) -> Dict[str, Any]: - """ - Run all tests. - - Args: - args: Command-line arguments - - Returns: - Dict[str, Any]: Test results - """ - results = { - "connection": {}, - "search": {}, - "agent": {} - } - - connection_success, connection_message = verify_serper_connection() - results["connection"] = { - "success": connection_success, - "message": connection_message - } - - if not connection_success: - logger.error("Failed to connect to Serper API. Skipping remaining tests.") - return results - - search_success, search_message, search_data = test_serper_search() - results["search"] = { - "success": search_success, - "message": search_message, - "data": search_data - } - - if args.test_agent: - agent_success, agent_message, agent_data = test_gaia_agent_with_serper() - results["agent"] = { - "success": agent_success, - "message": agent_message, - "data": agent_data - } - else: - logger.info("Skipping GAIA agent test (use --test-agent to include)") - results["agent"] = { - "success": None, - "message": "Test skipped", - "data": {} - } - - return results - -def print_results(results: Dict[str, Any]) -> None: - """ - Print test results. - - Args: - results: Test results - """ - - if results["connection"]["success"]: - else: - - if results["search"]["success"]: - else: - - if results["agent"].get("success"): - elif results["agent"].get("success") is False: - else: - - -def save_results(results: Dict[str, Any], output_file: str) -> None: - """ - Save test results to a file. - - Args: - results: Test results - output_file: Output file path - """ - with open(output_file, "w") as f: - json.dump(results, f, indent=2) - - logger.info(f"Results saved to {output_file}") - -def main(): - """Main function.""" - parser = argparse.ArgumentParser(description="Test Serper API integration") - parser.add_argument("--verbose", "-v", action="store_true", help="Enable verbose logging") - parser.add_argument("--output", "-o", type=str, default="serper_test_results.json", help="Output file for test results") - parser.add_argument("--test-agent", "-a", action="store_true", help="Test the GAIA agent with Serper API") - args = parser.parse_args() - - if args.verbose: - logger.setLevel(logging.DEBUG) - if not any(isinstance(h, logging.StreamHandler) for h in logger.handlers): - console_handler = logging.StreamHandler() - console_handler.setLevel(logging.DEBUG) - console_handler.setFormatter(logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")) - logger.addHandler(console_handler) - - results = run_tests(args) - - print_results(results) - - save_results(results, args.output) - - return all( - results["connection"]["success"] and - results["search"]["success"] and - (results["agent"]["success"] if results["agent"].get("success") is not None else True) - ) - -if __name__ == "__main__": - success = main() - sys.exit(0 if success else 1) \ No newline at end of file diff --git a/tests/test_supabase_connection.py b/tests/test_supabase_connection.py deleted file mode 100644 index f72e2545d73d40d5d2ee99cfbebe129e59f1891a..0000000000000000000000000000000000000000 --- a/tests/test_supabase_connection.py +++ /dev/null @@ -1,274 +0,0 @@ -""" -Test Supabase Connection for GAIA Memory - -This script tests the connection to Supabase and verifies that the required tables exist. -It performs basic CRUD operations on the test_connection table to ensure functionality. -""" - -import os -import sys -import json -import logging -import argparse -import time -from typing import Dict, Any, List, Optional - -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(name)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("supabase_connection_test") - -try: - from supabase import create_client, Client -except ImportError: - logger.error("Error importing supabase. Please install it with: pip install supabase") - sys.exit(1) - -try: - from agent.config import ( - SUPABASE_URL, - SUPABASE_KEY, - SUPABASE_PROJECT_ID, - SUPABASE_PROJECT_NAME - ) -except ImportError as e: - logger.error(f"Error importing configuration: {str(e)}") - sys.exit(1) - -class SupabaseConnectionTester: - """Class for testing Supabase connection and operations.""" - - def __init__(self, verbose: bool = False): - """ - Initialize the Supabase connection tester. - - Args: - verbose: Whether to enable verbose logging - """ - self.verbose = verbose - self.client = None - - if verbose: - logging.getLogger().setLevel(logging.DEBUG) - - self._initialize_client() - - def _initialize_client(self) -> None: - """Initialize the Supabase client.""" - logger.info("Initializing Supabase client...") - - if not SUPABASE_URL or not SUPABASE_KEY: - logger.error("Supabase URL or key not provided. Please set the SUPABASE_URL and SUPABASE_KEY environment variables.") - sys.exit(1) - - try: - self.client = create_client(SUPABASE_URL, SUPABASE_KEY) - logger.info("Supabase client initialized successfully") - logger.info(f"Connected to Supabase project: {SUPABASE_PROJECT_NAME} ({SUPABASE_PROJECT_ID})") - except Exception as e: - logger.error(f"Error initializing Supabase client: {str(e)}") - sys.exit(1) - - def test_connection(self) -> bool: - """ - Test the connection to Supabase. - - Returns: - bool: True if the connection is successful, False otherwise - """ - logger.info("Testing Supabase connection...") - - try: - result = self.client.table("test_connection").select("*").limit(1).execute() - logger.info("✅ Connection test successful") - return True - except Exception as e: - logger.error(f"❌ Connection test failed: {str(e)}") - return False - - def test_service_status(self) -> Dict[str, Any]: - """ - Test the Supabase service status. - - Returns: - Dict[str, Any]: The service status - """ - logger.info("Testing Supabase service status...") - - try: - result = self.client.rpc("get_service_status").execute() - status = result.data - logger.info(f"✅ Service status: {status}") - return status - except Exception as e: - logger.error(f"❌ Service status test failed: {str(e)}") - return {"status": "error", "message": str(e)} - - def test_crud_operations(self) -> Dict[str, bool]: - """ - Test CRUD operations on the test_connection table. - - Returns: - Dict[str, bool]: Dictionary with test results - """ - logger.info("Testing CRUD operations on test_connection table...") - - results = { - "create": False, - "read": False, - "update": False, - "delete": False - } - - try: - test_id = int(time.time()) - test_name = f"test_entry_{test_id}" - - logger.info(f"Creating test entry with ID {test_id}...") - create_result = self.client.table("test_connection").insert({ - "id": test_id, - "name": test_name - }).execute() - - if create_result.data: - results["create"] = True - logger.info("✅ Create operation successful") - else: - logger.warning("❌ Create operation failed") - return results - - logger.info(f"Reading test entry with ID {test_id}...") - read_result = self.client.table("test_connection").select("*").eq("id", test_id).execute() - - if read_result.data and len(read_result.data) > 0: - results["read"] = True - logger.info("✅ Read operation successful") - else: - logger.warning("❌ Read operation failed") - return results - - updated_name = f"{test_name}_updated" - logger.info(f"Updating test entry with ID {test_id}...") - update_result = self.client.table("test_connection").update({ - "name": updated_name - }).eq("id", test_id).execute() - - if update_result.data: - results["update"] = True - logger.info("✅ Update operation successful") - else: - logger.warning("❌ Update operation failed") - return results - - verify_result = self.client.table("test_connection").select("*").eq("id", test_id).execute() - if verify_result.data and verify_result.data[0]["name"] == updated_name: - logger.info("✅ Update verification successful") - else: - logger.warning("❌ Update verification failed") - results["update"] = False - return results - - logger.info(f"Deleting test entry with ID {test_id}...") - delete_result = self.client.table("test_connection").delete().eq("id", test_id).execute() - - if delete_result.data: - results["delete"] = True - logger.info("✅ Delete operation successful") - else: - logger.warning("❌ Delete operation failed") - return results - - verify_delete = self.client.table("test_connection").select("*").eq("id", test_id).execute() - if not verify_delete.data or len(verify_delete.data) == 0: - logger.info("✅ Delete verification successful") - else: - logger.warning("❌ Delete verification failed") - results["delete"] = False - - except Exception as e: - logger.error(f"Error testing CRUD operations: {str(e)}") - - return results - - def verify_tables_exist(self) -> Dict[str, bool]: - """ - Verify that all required tables exist. - - Returns: - Dict[str, bool]: Dictionary with test results - """ - logger.info("Verifying that required tables exist...") - - tables = [ - "gaia_memory", - "gaia_memory_conversations", - "gaia_memory_cache", - "gaia_memory_working", - "test_connection" - ] - - results = {} - - for table in tables: - try: - result = self.client.table(table).select("*").limit(1).execute() - results[table] = True - logger.info(f"✅ Table {table} exists") - except Exception as e: - results[table] = False - logger.warning(f"❌ Table {table} does not exist or cannot be accessed: {str(e)}") - - return results - - def run_all_tests(self) -> None: - """Run all Supabase connection tests.""" - - connection_result = self.test_connection() - - if not connection_result: - return - - service_status = self.test_service_status() - - table_results = self.verify_tables_exist() - - for table, result in table_results.items(): - status = "✅ Exists" if result else "❌ Missing" - - crud_results = self.test_crud_operations() - - for operation, result in crud_results.items(): - status = "✅ Passed" if result else "❌ Failed" - - - if connection_result: - else: - - tables_success = all(table_results.values()) - if tables_success: - else: - - crud_success = all(crud_results.values()) - if crud_success: - else: - - if not tables_success: - - if not crud_success: - - - -def main(): - """Main function.""" - parser = argparse.ArgumentParser(description="Test Supabase connection for GAIA memory") - parser.add_argument("--verbose", action="store_true", help="Enable verbose logging") - args = parser.parse_args() - - tester = SupabaseConnectionTester(verbose=args.verbose) - - tester.run_all_tests() - - -if __name__ == "__main__": - main() \ No newline at end of file diff --git a/tests/test_supabase_memory.py b/tests/test_supabase_memory.py deleted file mode 100644 index b1ee5cd022c487c57f19724902a8fba7c9f7a7ff..0000000000000000000000000000000000000000 --- a/tests/test_supabase_memory.py +++ /dev/null @@ -1,464 +0,0 @@ -""" -Test Supabase Memory Module for GAIA Agent - -This script tests the Supabase memory module implementation, including: -- SupabaseMemory class for basic memory operations -- ConversationMemory class for conversation history -- ResultCache class for caching results -- WorkingMemory class for storing intermediate results -""" - -import os -import sys -import json -import logging -import argparse -import time -import uuid -from typing import Dict, Any, List, Optional, Union, Tuple - -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(name)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("supabase_memory_test") - -try: - from memory.supabase_memory import ( - SupabaseMemory, - ConversationMemory, - ResultCache, - WorkingMemory, - MemoryEntry - ) - from agent.config import ( - get_memory_config, - SUPABASE_URL, - SUPABASE_KEY, - SUPABASE_PROJECT_ID, - SUPABASE_PROJECT_NAME - ) -except ImportError as e: - logger.error(f"Error importing memory components: {str(e)}") - sys.exit(1) - -class SupabaseMemoryTester: - """Class for testing the Supabase memory module.""" - - def __init__(self, verbose: bool = False): - """ - Initialize the Supabase memory tester. - - Args: - verbose: Whether to enable verbose logging - """ - self.verbose = verbose - self.session_id = str(uuid.uuid4()) - self.conversation_id = str(uuid.uuid4()) - - if verbose: - logging.getLogger().setLevel(logging.DEBUG) - - self._initialize_memory() - - def _initialize_memory(self) -> None: - """Initialize the memory components.""" - logger.info("Initializing memory components...") - - try: - memory_config = get_memory_config() - - memory_config["enabled"] = True - - self.memory = SupabaseMemory(config=memory_config) - self.conversation = ConversationMemory(self.memory, self.conversation_id) - self.cache = ResultCache(self.memory) - self.working = WorkingMemory(self.memory, self.session_id) - - logger.info("Memory components initialized successfully") - logger.info(f"Session ID: {self.session_id}") - logger.info(f"Conversation ID: {self.conversation_id}") - except Exception as e: - logger.error(f"Error initializing memory components: {str(e)}") - sys.exit(1) - - def test_basic_memory_operations(self) -> Dict[str, bool]: - """ - Test basic memory operations. - - Returns: - Dict[str, bool]: Dictionary with test results - """ - logger.info("Testing basic memory operations...") - - results = { - "store": False, - "retrieve": False, - "delete": False, - "clear": False, - "list_keys": False - } - - try: - test_key = f"test_key_{time.time()}" - test_value = f"test_value_{time.time()}" - test_metadata = {"test": True, "timestamp": time.time()} - - logger.info(f"Storing value with key {test_key}...") - store_result = self.memory.store(test_key, test_value, test_metadata) - - if store_result: - results["store"] = True - logger.info("✅ Store operation successful") - else: - logger.warning("❌ Store operation failed") - return results - - logger.info(f"Retrieving value with key {test_key}...") - retrieved_value = self.memory.retrieve(test_key) - - if retrieved_value == test_value: - results["retrieve"] = True - logger.info("✅ Retrieve operation successful") - else: - logger.warning(f"❌ Retrieve operation failed. Expected {test_value}, got {retrieved_value}") - return results - - logger.info("Listing keys...") - keys = self.memory.list_keys() - - if test_key in keys: - results["list_keys"] = True - logger.info("✅ List keys operation successful") - else: - logger.warning(f"❌ List keys operation failed. Expected {test_key} in {keys}") - return results - - logger.info(f"Deleting value with key {test_key}...") - delete_result = self.memory.delete(test_key) - - if delete_result: - results["delete"] = True - logger.info("✅ Delete operation successful") - else: - logger.warning("❌ Delete operation failed") - return results - - retrieved_after_delete = self.memory.retrieve(test_key) - if retrieved_after_delete is None: - logger.info("✅ Delete verification successful") - else: - logger.warning(f"❌ Delete verification failed. Expected None, got {retrieved_after_delete}") - results["delete"] = False - return results - - clear_test_key = f"clear_test_key_{time.time()}" - clear_test_value = f"clear_test_value_{time.time()}" - self.memory.store(clear_test_key, clear_test_value) - - logger.info("Clearing all values...") - clear_result = self.memory.clear() - - if clear_result: - results["clear"] = True - logger.info("✅ Clear operation successful") - else: - logger.warning("❌ Clear operation failed") - return results - - keys_after_clear = self.memory.list_keys() - if not keys_after_clear or clear_test_key not in keys_after_clear: - logger.info("✅ Clear verification successful") - else: - logger.warning(f"❌ Clear verification failed. Expected empty list, got {keys_after_clear}") - results["clear"] = False - - except Exception as e: - logger.error(f"Error testing basic memory operations: {str(e)}") - - return results - - def test_conversation_memory(self) -> Dict[str, bool]: - """ - Test conversation memory. - - Returns: - Dict[str, bool]: Dictionary with test results - """ - logger.info("Testing conversation memory...") - - results = { - "add_message": False, - "get_messages": False, - "clear": False - } - - try: - test_role = "user" - test_content = f"Test message at {time.time()}" - test_metadata = {"test": True, "timestamp": time.time()} - - logger.info(f"Adding message: {test_content}...") - add_result = self.conversation.add_message(test_role, test_content, test_metadata) - - if add_result: - results["add_message"] = True - logger.info("✅ Add message operation successful") - else: - logger.warning("❌ Add message operation failed") - return results - - logger.info("Getting messages...") - messages = self.conversation.get_messages() - - if messages and any(m.get("content") == test_content for m in messages): - results["get_messages"] = True - logger.info("✅ Get messages operation successful") - else: - logger.warning(f"❌ Get messages operation failed. Expected message with content {test_content}") - return results - - logger.info("Clearing conversation...") - clear_result = self.conversation.clear() - - if clear_result: - results["clear"] = True - logger.info("✅ Clear operation successful") - else: - logger.warning("❌ Clear operation failed") - return results - - messages_after_clear = self.conversation.get_messages() - if not messages_after_clear: - logger.info("✅ Clear verification successful") - else: - logger.warning(f"❌ Clear verification failed. Expected empty list, got {messages_after_clear}") - results["clear"] = False - - except Exception as e: - logger.error(f"Error testing conversation memory: {str(e)}") - - return results - - def test_result_cache(self) -> Dict[str, bool]: - """ - Test result cache. - - Returns: - Dict[str, bool]: Dictionary with test results - """ - logger.info("Testing result cache...") - - results = { - "cache_result": False, - "get_result": False, - "invalidate": False, - "clear": False - } - - try: - test_query = f"test_query_{time.time()}" - test_result = f"test_result_{time.time()}" - test_metadata = {"test": True, "timestamp": time.time()} - - logger.info(f"Caching result for query: {test_query}...") - cache_result = self.cache.cache_result(test_query, test_result, test_metadata) - - if cache_result: - results["cache_result"] = True - logger.info("✅ Cache result operation successful") - else: - logger.warning("❌ Cache result operation failed") - return results - - logger.info(f"Getting result for query: {test_query}...") - retrieved_result = self.cache.get_result(test_query) - - if retrieved_result == test_result: - results["get_result"] = True - logger.info("✅ Get result operation successful") - else: - logger.warning(f"❌ Get result operation failed. Expected {test_result}, got {retrieved_result}") - return results - - logger.info(f"Invalidating result for query: {test_query}...") - invalidate_result = self.cache.invalidate(test_query) - - if invalidate_result: - results["invalidate"] = True - logger.info("✅ Invalidate operation successful") - else: - logger.warning("❌ Invalidate operation failed") - return results - - retrieved_after_invalidate = self.cache.get_result(test_query) - if retrieved_after_invalidate is None: - logger.info("✅ Invalidate verification successful") - else: - logger.warning(f"❌ Invalidate verification failed. Expected None, got {retrieved_after_invalidate}") - results["invalidate"] = False - return results - - clear_test_query = f"clear_test_query_{time.time()}" - clear_test_result = f"clear_test_result_{time.time()}" - self.cache.cache_result(clear_test_query, clear_test_result) - - logger.info("Clearing all results...") - clear_result = self.cache.clear() - - if clear_result: - results["clear"] = True - logger.info("✅ Clear operation successful") - else: - logger.warning("❌ Clear operation failed") - return results - - retrieved_after_clear = self.cache.get_result(clear_test_query) - if retrieved_after_clear is None: - logger.info("✅ Clear verification successful") - else: - logger.warning(f"❌ Clear verification failed. Expected None, got {retrieved_after_clear}") - results["clear"] = False - - except Exception as e: - logger.error(f"Error testing result cache: {str(e)}") - - return results - - def test_working_memory(self) -> Dict[str, bool]: - """ - Test working memory. - - Returns: - Dict[str, bool]: Dictionary with test results - """ - logger.info("Testing working memory...") - - results = { - "store_intermediate_result": False, - "get_intermediate_result": False, - "list_intermediate_results": False, - "clear": False - } - - try: - test_key = f"test_key_{time.time()}" - test_value = f"test_value_{time.time()}" - test_metadata = {"test": True, "timestamp": time.time()} - - logger.info(f"Storing intermediate result with key {test_key}...") - store_result = self.working.store_intermediate_result(test_key, test_value, test_metadata) - - if store_result: - results["store_intermediate_result"] = True - logger.info("✅ Store intermediate result operation successful") - else: - logger.warning("❌ Store intermediate result operation failed") - return results - - logger.info(f"Getting intermediate result with key {test_key}...") - retrieved_value = self.working.get_intermediate_result(test_key) - - if retrieved_value == test_value: - results["get_intermediate_result"] = True - logger.info("✅ Get intermediate result operation successful") - else: - logger.warning(f"❌ Get intermediate result operation failed. Expected {test_value}, got {retrieved_value}") - return results - - logger.info("Listing intermediate results...") - intermediate_results = self.working.list_intermediate_results() - - if test_key in intermediate_results: - results["list_intermediate_results"] = True - logger.info("✅ List intermediate results operation successful") - else: - logger.warning(f"❌ List intermediate results operation failed. Expected {test_key} in {intermediate_results}") - return results - - logger.info("Clearing working memory...") - clear_result = self.working.clear() - - if clear_result: - results["clear"] = True - logger.info("✅ Clear operation successful") - else: - logger.warning("❌ Clear operation failed") - return results - - intermediate_results_after_clear = self.working.list_intermediate_results() - if not intermediate_results_after_clear: - logger.info("✅ Clear verification successful") - else: - logger.warning(f"❌ Clear verification failed. Expected empty dict, got {intermediate_results_after_clear}") - results["clear"] = False - - except Exception as e: - logger.error(f"Error testing working memory: {str(e)}") - - return results - - def run_all_tests(self) -> None: - """Run all Supabase memory tests.""" - - basic_results = self.test_basic_memory_operations() - - for operation, result in basic_results.items(): - status = "✅ Passed" if result else "❌ Failed" - - conversation_results = self.test_conversation_memory() - - for operation, result in conversation_results.items(): - status = "✅ Passed" if result else "❌ Failed" - - cache_results = self.test_result_cache() - - for operation, result in cache_results.items(): - status = "✅ Passed" if result else "❌ Failed" - - working_results = self.test_working_memory() - - for operation, result in working_results.items(): - status = "✅ Passed" if result else "❌ Failed" - - - basic_success = all(basic_results.values()) - if basic_success: - else: - - conversation_success = all(conversation_results.values()) - if conversation_success: - else: - - cache_success = all(cache_results.values()) - if cache_success: - else: - - working_success = all(working_results.values()) - if working_success: - else: - - overall_success = basic_success and conversation_success and cache_success and working_success - if overall_success: - else: - - if not basic_success: - if not conversation_success: - if not cache_success: - if not working_success: - - - -def main(): - """Main function.""" - parser = argparse.ArgumentParser(description="Test Supabase memory for GAIA agent") - parser.add_argument("--verbose", action="store_true", help="Enable verbose logging") - args = parser.parse_args() - - tester = SupabaseMemoryTester(verbose=args.verbose) - - tester.run_all_tests() - - -if __name__ == "__main__": - main() \ No newline at end of file diff --git a/tests/test_supabase_tables_comprehensive.py b/tests/test_supabase_tables_comprehensive.py deleted file mode 100644 index 4e5e8337e189ff9339f5b50b9a0cda5daad37f48..0000000000000000000000000000000000000000 --- a/tests/test_supabase_tables_comprehensive.py +++ /dev/null @@ -1,628 +0,0 @@ -""" -Comprehensive test script scrior Supabase tables in GAIA Ag.ent. - -This script tests all Supabase tables used by the GAIA asent: -- ged by the A(mA n ent:table) -- - gaia_memorw ikeot (workbng ) tbl - -Ivifisatda_a is stormem_wrkingco recabeand -It verifies that data is stored in the correct tables and measures performance. - -Usage: - python -m tests.test_supabase_tables_comprehensive -""" - -import os -import sys -import logging -import tmme -import uuid -et -from typing import Dict, Any, List, Optional, Tuple, Tuple - -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(name)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("tett_sest_supabase_tables_comprehe") - -sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) - -try: - from sp MMemory - ) - from agent.configmport get_memory_config - ) -fom gen.cfig iptgot_mimory_configx - rom memory.fix_supabase_tables import apply_monkey_patch, ensure_tables_exist -excelogger.error(f"Error importing required components: {str(e)}") - sys.exit(1) - -class SupabaseTablesTest: - """Comprehensive test class for Supabase tables.""" - - def __init__(self): - Compr"hen"ive "estInitialize test class.""".""" - - def __it__(self): - """nitializeth est class - self.memory_config = get_memory_config() self.memory_config = get_memory_config() - self.memory_config["enable "] = True - - # Initialize memory components - s l .base_memory = None - self.workingememory = None - self.sesslof_.d = smr(uuid.uuid4o)) - - # Table names - ry_c.table_names = { - "base": "gaia_memory"o - "working": "gaia_memory_working" - } - - # Test results - nsilf.gesults = { - "initialization": False, - "ta[le_exietencn"a {}, - b"lase_memory": {}, - "working_memery": {}, - "perfdrmance": {} - } - - ]def=initiTrize_memory(uelf -> bool - """Initialize memory components. - logger.info("# Initialing memory componints...") -z - ery: - seof.bmpe_memory = SupabaoeMemory(selfnmemory_config)ents - self.base_memory = None - felf.working_memory = WorkingMemory(self.base_memory, self.session_id)working_memory = None - self - if self.sase_memsry and self.baie_memory.initializodn - logger.info("✅ Memory components initialized successfully") - self.results["initialization"] = True - _r =urn True - else: - logger.error("❌ Msmoty componentsrinitializa(iun failed") - u silf.results["iditi.uization"] = Falsi -d r(tu)n False - - except Exceptin a : - er.error(f"Error itializin memory components: {str(e)}") - self.results["initialization"] = False - return False - - def check_table_existence(self) -> Dict[str, bool]: - Check if the required tables exist.""" - logger.info("Checking table exi#t nce...") - - resuTts = {} - - ia not selfbbase_memo y or not self.nase_memory.initialized: - logger.error("Memory companentm not initialized") - returns{"error":"Mmoy compnent not initializd"} - - self.table_names = { - "base": "gaia_memory", - "working": "gaia_memory_working" - } - - # Test results - self.results = { - "initialization": False, - "table_existence": {}, - "base_memory": {}, - "working_memory": {}, - "performance": {} - } - - def initialize_memory(self) -> bool: - """Initialize memory components.""" - logger.info("Initializing memory components...") - - try: - # Create the base memory implementation - self.base_memory = SupabaseMemory(self.memory_config) - - # Create working memory - self.working_memory = WorkingMemory(self.base_memory, self.session_id) - - # Check if memory components are initialized - if self.base_memory and self.base_memory.initialized: - logger.info("✅ Memory components initialized successfully") - self.results["initialization"] = True - return True - else: - logger.error("❌ Memory components initialization failed") - self.results["initialization"] = False - return False - - except Exception as e: - logger.error(f"Error initializing memory components: {str(e)}") - self.results["initialization"] = False - return False - - def check_table_existence(self) -> Dict[str, bool]: - """Check if the required tables exist.""" - logger.info("Checking table existence...") - - results = {} - - if not self.base_memory or not self.base_memory.initialized: - logger.error("Memory components not initialized") - return {"error": "Memory components not initialized"} - - self.session_id = str(uuid.uuid4()) - try: - # Try to query the table - response = self.base_memory.client.table(table_name).select("*").limit(1).execute() - - # Check if the query was successful - if hasattr(response, "data"): - logger.info(f"✅ Table {table_name} exists") - results[table_name] = True - else: - logger.warning(f"❌ Table {table_name} query failed") - results[table_name] = False - - except Exception as e: - logger.error(f"Error checking table {table_name}: {str(e)}") - results[table_name] = False - - self.results["table_existence"] = results - return results - - def test_base_memory(self) -> Dict[str, bool]: - """Test base memory operations.""" - logger.info("Testing base memory operations...") - - results = {} - - if not self.base_memory: - logger.error("Base memory not initialized") - return {"error": "Base memory not initialized"} - - # Test storing and retrieving data - try: - # Generate test data - test_key = f"test_base_{int(time.time())}" - test_value = { - "test_data": "This is test data for base memory", - "timestamp": time.time(), - "random_id": str(uuid.uuid4()) - } - - # Store the data - logger.info(f"Storing data in base memory: {test_key}") - start_time = time.time() - store_result = self.base_memory.store(test_key, test_value) - store_time = time.time() - start_time - - if store_result: - logger.info(f"✅ Store operation successful (took {store_time:.4f}s)") - results["store"] = True - results["store_time"] = store_time - else: - logger.warning("❌ Store operation failed") - results["store"] = False - - # Retrieve the data - logger.info(f"Retrieving data from base memory: {test_key}") - start_time = time.time() - retrieved_value = self.base_memory.retrieve(test_key) - retrieve_time = time.time() - start_time - - if retrieved_value and retrieved_value == test_value: - logger.info(f"✅ Retrieve operation successful (took {retrieve_time:.4f}s)") - results["retrieve"] = True - results["retrieve_time"] = retrieve_time - else: - logger.warning(f"❌ Retrieve operation failed or data mismatch") - logger.warning(f"Expected: {test_value}") - logger.warning(f"Got: {retrieved_value}") - results["retrieve"] = False - - # Verify the data was stored in the base memory table - logger.info("Verifying data was stored in the correct table...") - - if self.base_memory.client and self.base_memory.initialized: - try: - # Query the base memory table directly - response = self.base_memory.client.table(self.table_names["base"]).select("*").eq("key", test_key).execute() - - # Check if any entries were found - if response.data and len(response.data) > 0: - logger.info(f"✅ Data found in base memory table: {len(response.data)} entries") - results["correct_table"] = True - - # Check if the data matches - entry_data = response.data[0] - stored_value = entry_data.get("value") - - # Parse JSON if needed - if isinstance(stored_value, str): - try: - stored_value = json.loads(stored_value) - except json.JSONDecodeError: - # If it's not valid JSON, keep it as a string - pass - - if stored_value == test_value or stored_value == json.dumps(test_value): - logger.info("✅ Stored data matches expected value") - results["data_match"] = True - else: - logger.warning("❌ Stored data does not match expected value") - logger.warning(f"Expected: {test_value}") - logger.warning(f"Got: {stored_value}") - results["data_match"] = False - else: - logger.warning("❌ No data found in base memory table") - results["correct_table"] = False - - except Exception as e: - logger.error(f"Error verifying data storage: {str(e)}") - results["correct_table"] = False - - # Test deleting data - logger.info(f"Deleting data from base memory: {test_key}") - delete_result = self.base_memory.delete(test_key) - - if delete_result: - logger.info("✅ Delete operation successful") - results["delete"] = True - else: - logger.warning("❌ Delete operation failed") - results["delete"] = False - - # Verify the data was deleted - retrieved_after_delete = self.base_memory.retrieve(test_key) - if retrieved_after_delete is None: - logger.info("✅ Data was successfully deleted") - results["delete_verified"] = True - else: - logger.warning("❌ Data was not deleted") - results["delete_verified"] = False - - except Exception as e: - logger.error(f"Error testing base memory: {str(e)}") - results["error"] = str(e) - - self.results["base_memory"] = results - return results - - def test_working_memory(self) -> Dict[str, bool]: - """Test working memory operations.""" - logger.info("Testing working memory operations...") - - results = {} - - if not self.working_memory: - logger.error("Working memory not initialized") - return {"error": "Working memory not initialized"} - - # Test storing and retrieving data - try: - # Generate test data - test_key = f"test_working_{int(time.time())}" - test_value = { - "test_data": "This is test data for working memory", - "timestamp": time.time(), - "random_id": str(uuid.uuid4()) - } - - # Store the data - logger.info(f"Storing data in working memory: {test_key}") - start_time = time.time() - store_result = self.working_memory.store_intermediate_result(test_key, test_value) - store_time = time.time() - start_time - - if store_result: - logger.info(f"✅ Store operation successful (took {store_time:.4f}s)") - results["store"] = True - results["store_time"] = store_time - else: - logger.warning("❌ Store operation failed") - results["store"] = False - - # Retrieve the data - logger.info(f"Retrieving data from working memory: {test_key}") - start_time = time.time() - retrieved_value = self.working_memory.get_intermediate_result(test_key) - retrieve_time = time.time() - start_time - - if retrieved_value and retrieved_value == test_value: - logger.info(f"✅ Retrieve operation successful (took {retrieve_time:.4f}s)") - results["retrieve"] = True - results["retrieve_time"] = retrieve_time - else: - logger.warning(f"❌ Retrieve operation failed or data mismatch") - logger.warning(f"Expected: {test_value}") - logger.warning(f"Got: {retrieved_value}") - results["retrieve"] = False - - # Verify the data was stored in the working memory table - logger.info("Verifying data was stored in the correct table...") - - if self.base_memory.client and self.base_memory.initialized: - try: - # Query the working memory table directly - working_key = f"working:{self.session_id}:{test_key}" - response = self.base_memory.client.table(self.table_names["working"]).select("*").eq("key", working_key).execute() - - # Check if any entries were found - if response.data and len(response.data) > 0: - logger.info(f"✅ Data found in working memory table: {len(response.data)} entries") - results["correct_table"] = True - - # Check if the data matches - entry_data = response.data[0] - stored_value = entry_data.get("value") - - # Parse JSON if needed - if isinstance(stored_value, str): - try: - stored_value = json.loads(stored_value) - except json.JSONDecodeError: - # If it's not valid JSON, keep it as a string - pass - - if stored_value == test_value or stored_value == json.dumps(test_value): - logger.info("✅ Stored data matches expected value") - results["data_match"] = True - else: - logger.warning("❌ Stored data does not match expected value") - logger.warning(f"Expected: {test_value}") - logger.warning(f"Got: {stored_value}") - results["data_match"] = False - else: - logger.warning("❌ No data found in working memory table") - results["correct_table"] = False - - # Check if it was stored in the base memory table instead - response = self.base_memory.client.table(self.table_names["base"]).select("*").eq("key", working_key).execute() - - if response.data and len(response.data) > 0: - logger.warning("⚠️ Data was stored in base memory table instead of working memory table") - results["stored_in_base"] = True - else: - logger.warning("❌ Data was not stored in any table") - results["stored_in_base"] = False - - except Exception as e: - logger.error(f"Error verifying data storage: {str(e)}") - results["correct_table"] = False - - # Test listing intermediate results - logger.info("Listing intermediate results...") - list_results = self.working_memory.list_intermediate_results() - - if list_results and test_key in list_results: - logger.info(f"✅ List operation successful: {len(list_results)} results") - results["list"] = True - else: - logger.warning("❌ List operation failed or test key not found") - logger.warning(f"Expected key: {test_key}") - logger.warning(f"Got keys: {list(list_results.keys()) if list_results else None}") - results["list"] = False - - # Clean up - logger.info("Cleaning up test data...") - self.working_memory.clear() - - except Exception as e: - logger.error(f"Error testing working memory: {str(e)}") - results["error"] = str(e) - - self.results["working_memory"] = results - return results - - def test_performance(self) -> Dict[str, float]: - """Test performance of memory operations.""" - logger.info("Testing memory performance...") - - results = {} - - if not self.base_memory or not self.working_memory: - logger.error("Memory components not initialized") - return {"error": "Memory components not initialized"} - - # Test parameters - num_iterations = 5 - data_size = 1000 # characters - - # Generate test data - test_data = "x" * data_size - - # Test base memory performance - base_store_times = [] - base_retrieve_times = [] - - for i in range(num_iterations): - test_key = f"perf_base_{i}_{int(time.time())}" - - # Store - start_time = time.time() - self.base_memory.store(test_key, test_data) - base_store_times.append(time.time() - start_time) - - # Retrieve - start_time = time.time() - self.base_memory.retrieve(test_key) - base_retrieve_times.append(time.time() - start_time) - - # Clean up - self.base_memory.delete(test_key) - - # Test working memory performance - working_store_times = [] - working_retrieve_times = [] - - for i in range(num_iterations): - test_key = f"perf_working_{i}_{int(time.time())}" - - # Store - start_time = time.time() - self.working_memory.store_intermediate_result(test_key, test_data) - working_store_times.append(time.time() - start_time) - - # Retrieve - start_time = time.time() - self.working_memory.get_intermediate_result(test_key) - working_retrieve_times.append(time.time() - start_time) - - # Clean up - self.working_memory.clear() - - # Calculate average times - avg_base_store = sum(base_store_times) / len(base_store_times) - avg_base_retrieve = sum(base_retrieve_times) / len(base_retrieve_times) - avg_working_store = sum(working_store_times) / len(working_store_times) - avg_working_retrieve = sum(working_retrieve_times) / len(working_retrieve_times) - - logger.info(f"Base memory average store time: {avg_base_store:.4f}s") - logger.info(f"Base memory average retrieve time: {avg_base_retrieve:.4f}s") - logger.info(f"Working memory average store time: {avg_working_store:.4f}s") - logger.info(f"Working memory average retrieve time: {avg_working_retrieve:.4f}s") - - results["avg_base_store"] = avg_base_store - results["avg_base_retrieve"] = avg_base_retrieve - results["avg_working_store"] = avg_working_store - results["avg_working_retrieve"] = avg_working_retrieve - - self.results["performance"] = results - return results - - def run_all_tests(self) -> Dict[str, Dict[str, Any]]: - """Run all tests.""" - logger.info("Running all tests...") - - # Initialize memory components - if not self.initialize_memory(): - logger.error("Memory initialization failed, aborting tests") - return self.results - - # Check table existence - self.check_table_existence() - - # Apply the fix if needed - if not all(self.results["table_existence"].values()): - logger.info("Some tables don't exist, ensuring tables exist...") - ensure_tables_exist() - - # Apply the monkey patch - logger.info("Applying monkey patch...") - apply_monkey_patch() - - # Re-check table existence - self.check_table_existence() - - # Test base memory - self.test_base_memory() - - # Test working memory - self.test_working_memory() - - # Test performance - self.test_performance() - - return self.results - - def print_test_summary(self, results: Dict[str, Dict[str, Any]]) -> None: - """Print a summary of the test results.""" - - # Print initialization results - if results["initialization"]: - else: - - # Print table existence results - for table_name, exists in results["table_existence"].items(): - if exists: - else: - - # Print base memory results - base_results = results["base_memory"] - - if base_results.get("store", False): - else: - - if base_results.get("retrieve", False): - else: - - if base_results.get("correct_table", False): - else: - - if base_results.get("data_match", False): - else: - - if base_results.get("delete", False): - else: - - # Print working memory results - working_results = results["working_memory"] - - if working_results.get("store", False): - else: - - if working_results.get("retrieve", False): - else: - - if working_results.get("correct_table", False): - else: - - if working_results.get("stored_in_base", False): - - if working_results.get("data_match", False): - else: - - if working_results.get("list", False): - else: - - # Print performance results - perf_results = results["performance"] - - if "avg_base_store" in perf_results: - if "avg_base_retrieve" in perf_results: - if "avg_working_store" in perf_results: - if "avg_working_retrieve" in perf_results: - - # Print overall result - if (results["initialization"] and - all(results["table_existence"].values()) and - working_results.get("store", False) and - working_results.get("retrieve", False) and - working_results.get("correct_table", False) and - working_results.get("data_match", False)): - else: - - - def save_results_to_file(self) -> str: - """Save test results to a file.""" - # Create a timestamp for the filename - timestamp = datetime.datetime.now().strftime("%Y%m%d_%H%M%S") - filename = f"gaia_memory_test_results_{timestamp}.json" - - # Add timestamp to results - self.results["timestamp"] = timestamp - - # Save results to file - with open(filename, "w") as f: - json.dump(self.results, f, indent=2) - - logger.info(f"Test results saved to {filename}") - return filename - -def main(): - """Main entry point for the test script.""" - - # Run the tests - test = SupabaseTablesTest() - results = test.run_all_tests() - - # Print the summary - test.print_test_summary(results) - - # Save results to file - results_file = test.save_results_to_file() - -if __name__ == "__main__": - main() \ No newline at end of file diff --git a/tests/test_tools.py b/tests/test_tools.py deleted file mode 100644 index 280effff7de3b1603adcf33989ece76c9c31aac2..0000000000000000000000000000000000000000 --- a/tests/test_tools.py +++ /dev/null @@ -1,643 +0,0 @@ -""" -Unit tests for GAIA agent tools. -""" -import unittest -from unittest.mock import patch, MagicMock, ANY -import json -import sys -import os -from typing import Dict, Any, List, Optional -from pathlib import Path - -sys.path.append('.') - -from tools.web_tools import ( - WebSearchTool, DuckDuckGoSearchTool, SerperSearchTool, - WebContentExtractor, WebNavigator -) -from tools.reasoning_tools import ( - StepByStepReasoner, MathCalculator, FactVerifier -) -from tools.multimodal_tools import ( - ImageAnalyzer, ChartInterpreter, DocumentParser -) - - -class TestWebTools(unittest.TestCase): - """Test case for web tools.""" - - def setUp(self): - """Set up the test case with mocked dependencies.""" - self.requests_patcher = patch('tools.web_tools.requests') - self.mock_requests = self.requests_patcher.start() - - self.mock_response = MagicMock() - self.mock_response.text = "Test Page

Test content

" - self.mock_response.raise_for_status = MagicMock() - self.mock_requests.get.return_value = self.mock_response - self.mock_requests.post.return_value = self.mock_response - - self.ddgs_patcher = patch('tools.web_tools.DDGS') - self.mock_ddgs = self.ddgs_patcher.start() - self.mock_ddgs_instance = MagicMock() - self.mock_ddgs.return_value.__enter__.return_value = self.mock_ddgs_instance - - # Mock DuckDuckGo search results - self.mock_ddgs_instance.text.return_value = [ - {"title": "Result 1", "href": "https://example.com/1", "body": "Description 1"}, - {"title": "Result 2", "href": "https://example.com/2", "body": "Description 2"} - ] - - self.bs_patcher = patch('tools.web_tools.BeautifulSoup') - self.mock_bs = self.bs_patcher.start() - self.mock_soup = MagicMock() - self.mock_bs.return_value = self.mock_soup - - self.mock_soup.title = MagicMock() - self.mock_soup.title.string = "Test Page" - self.mock_soup.get_text.return_value = "Test content" - self.mock_soup.find_all.return_value = [ - MagicMock(href="/link1", get_text=MagicMock(return_value="Link 1")), - MagicMock(href="https://example.com/link2", get_text=MagicMock(return_value="Link 2")) - ] - - def tearDown(self): - """Clean up after the test.""" - self.requests_patcher.stop() - self.ddgs_patcher.stop() - self.bs_patcher.stop() - - def test_web_search_tool_base(self): - """Test the base WebSearchTool class.""" - tool = WebSearchTool() - self.assertEqual(tool.result_count, 5) - self.assertEqual(tool.timeout, 10) - - results = [ - {"title": "Title 1", "link": "https://example.com/1", "snippet": "Snippet 1"}, - {"title": "Title 2", "link": "https://example.com/2", "snippet": "Snippet 2"} - ] - formatted = tool._format_results(results) - self.assertEqual(len(formatted), 2) - self.assertEqual(formatted[0]["title"], "Title 1") - self.assertEqual(formatted[1]["link"], "https://example.com/2") - - with self.assertRaises(NotImplementedError): - tool.search("test query") - - def test_duckduckgo_search_tool(self): - """Test the DuckDuckGoSearchTool class.""" - tool = DuckDuckGoSearchTool() - self.assertEqual(tool.max_results, 5) - - results = tool.search("test query") - self.assertEqual(len(results), 2) - self.assertEqual(results[0]["title"], "Result 1") - self.assertEqual(results[0]["link"], "https://example.com/1") - self.assertEqual(results[0]["snippet"], "Description 1") - - self.mock_ddgs_instance.text.side_effect = Exception("Test error") - with self.assertRaises(Exception): - tool.search("test query") - - def test_serper_search_tool(self): - """Test the SerperSearchTool class.""" - # Mock Serper API response - self.mock_response.json.return_value = { - "organic": [ - {"title": "Result 1", "link": "https://example.com/1", "snippet": "Snippet 1"}, - {"title": "Result 2", "link": "https://example.com/2", "snippet": "Snippet 2"} - ] - } - - with patch('tools.web_tools.SERPER_API_KEY', 'test_key'): - tool = SerperSearchTool() - self.assertEqual(tool.api_key, 'test_key') - - results = tool.search("test query") - self.assertEqual(len(results), 2) - self.assertEqual(results[0]["title"], "Result 1") - self.assertEqual(results[0]["link"], "https://example.com/1") - - self.mock_requests.post.side_effect = Exception("Test error") - with self.assertRaises(Exception): - tool.search("test query") - - with patch('tools.web_tools.SERPER_API_KEY', None): - tool = SerperSearchTool() - with self.assertRaises(ValueError): - tool.search("test query") - - def test_web_content_extractor(self): - """Test the WebContentExtractor class.""" - tool = WebContentExtractor() - self.assertEqual(tool.timeout, 15) - self.assertEqual(tool.max_content_length, 10000) - - with patch('tools.web_tools.urlparse') as mock_urlparse: - mock_parsed_url = MagicMock() - mock_parsed_url.scheme = "https" - mock_parsed_url.netloc = "example.com" - mock_urlparse.return_value = mock_parsed_url - - content = tool.extract_content("https://example.com") - self.assertEqual(content["url"], "https://example.com") - self.assertEqual(content["title"], "Test Page") - self.assertEqual(content["content"], "Test content") - self.assertEqual(len(content["links"]), 2) - - mock_parsed_url.scheme = "" - with self.assertRaises(ValueError): - tool.extract_content("invalid-url") - - self.mock_requests.get.side_effect = Exception("Test error") - with self.assertRaises(Exception): - tool.extract_content("https://example.com") - - def test_web_navigator(self): - """Test the WebNavigator class.""" - tool = WebNavigator() - self.assertEqual(tool.timeout, 15) - self.assertEqual(tool.max_links, 3) - - tool.content_extractor = MagicMock() - tool.content_extractor.extract_content.return_value = { - "url": "https://example.com", - "title": "Test Page", - "content": "Test content", - "links": [ - {"text": "Link 1", "url": "https://example.com/link1"}, - {"text": "Link 2", "url": "https://example.com/link2"} - ] - } - - content = tool.navigate("https://example.com") - self.assertEqual(content["url"], "https://example.com") - self.assertEqual(content["title"], "Test Page") - tool.content_extractor.extract_content.assert_called_once_with("https://example.com") - - tool.content_extractor.extract_content.reset_mock() - results = tool.follow_links("https://example.com") - self.assertEqual(len(results), 3) # Initial page + 2 links - self.assertEqual(tool.content_extractor.extract_content.call_count, 3) - - tool.content_extractor.extract_content.reset_mock() - with patch('tools.web_tools.re.compile') as mock_compile: - mock_pattern = MagicMock() - mock_pattern.search.side_effect = lambda url: url.endswith("link1") - mock_compile.return_value = mock_pattern - - results = tool.follow_links("https://example.com", "link1") - self.assertEqual(len(results), 2) # Initial page + 1 matching link - self.assertEqual(tool.content_extractor.extract_content.call_count, 2) - - tool.content_extractor.extract_content.side_effect = [ - {"url": "https://example.com", "title": "Test Page", "content": "Test content", "links": [ - {"text": "Link 1", "url": "https://example.com/link1"}, - {"text": "Link 2", "url": "https://example.com/link2"} - ]}, - Exception("Test error"), - {"url": "https://example.com/link2", "title": "Link 2", "content": "Link 2 content", "links": []} - ] - - results = tool.follow_links("https://example.com") - self.assertEqual(len(results), 2) # Initial page + 1 successful link - - -class TestReasoningTools(unittest.TestCase): - """Test case for reasoning tools.""" - - def setUp(self): - """Set up the test case with mocked dependencies.""" - self.openai_patcher = patch('tools.reasoning_tools.ChatOpenAI') - self.mock_openai = self.openai_patcher.start() - self.mock_model = MagicMock() - self.mock_openai.return_value = self.mock_model - - self.prompt_patcher = patch('tools.reasoning_tools.PromptTemplate') - self.mock_prompt = self.prompt_patcher.start() - self.mock_template = MagicMock() - self.mock_prompt.from_template.return_value = self.mock_template - - self.parser_patcher = patch('tools.reasoning_tools.StrOutputParser') - self.mock_parser = self.parser_patcher.start() - self.mock_output_parser = MagicMock() - self.mock_parser.return_value = self.mock_output_parser - - self.mock_chain = MagicMock() - self.mock_template.__or__.return_value.__or__.return_value = self.mock_chain - - self.sympify_patcher = patch('tools.reasoning_tools.sympify') - self.mock_sympify = self.sympify_patcher.start() - self.solve_patcher = patch('tools.reasoning_tools.solve') - self.mock_solve = self.solve_patcher.start() - self.symbols_patcher = patch('tools.reasoning_tools.symbols') - self.mock_symbols = self.symbols_patcher.start() - - def tearDown(self): - """Clean up after the test.""" - self.openai_patcher.stop() - self.prompt_patcher.stop() - self.parser_patcher.stop() - self.sympify_patcher.stop() - self.solve_patcher.stop() - self.symbols_patcher.stop() - - def test_step_by_step_reasoner(self): - """Test the StepByStepReasoner class.""" - reasoner = StepByStepReasoner() - self.mock_openai.assert_called_once() - - self.mock_chain.invoke.return_value = json.dumps({ - "steps": [ - { - "step_number": 1, - "reasoning": "First step", - "intermediate_conclusion": "First conclusion" - }, - { - "step_number": 2, - "reasoning": "Second step", - "intermediate_conclusion": "Second conclusion" - } - ], - "final_conclusion": "Final answer", - "confidence": 0.95 - }) - - result = reasoner.reason("What is 2+2?") - self.assertEqual(result["final_conclusion"], "Final answer") - self.assertEqual(len(result["steps"]), 2) - self.assertEqual(result["confidence"], 0.95) - - result = reasoner.reason("What is 2+2?", "Context information") - self.mock_chain.invoke.assert_called_with({ - "question": "What is 2+2?", - "context_text": "Context: Context information" - }) - - self.mock_chain.invoke.side_effect = Exception("Test error") - with self.assertRaises(Exception): - reasoner.reason("What is 2+2?") - - self.mock_chain.invoke.side_effect = None - self.mock_chain.invoke.return_value = "Chain of thought reasoning" - - result = reasoner.chain_of_thought("What is 2+2?") - self.assertEqual(result, "Chain of thought reasoning") - - self.mock_chain.invoke.side_effect = Exception("Test error") - with self.assertRaises(Exception): - reasoner.chain_of_thought("What is 2+2?") - - def test_math_calculator(self): - """Test the MathCalculator class.""" - calculator = MathCalculator() - - mock_result = MagicMock() - mock_result.evalf.return_value = 4.0 - mock_result.is_integer.return_value = True - self.mock_sympify.return_value = mock_result - - result = calculator.calculate("2 + 2") - self.assertEqual(result["expression"], "2 + 2") - self.assertEqual(result["result"], 4) - self.assertTrue(len(result["steps"]) > 0) - - self.mock_sympify.side_effect = Exception("Test error") - with self.assertRaises(Exception): - calculator.calculate("2 + 2") - - self.mock_sympify.side_effect = None - - mock_var = MagicMock() - self.mock_symbols.return_value = mock_var - mock_expr = MagicMock() - self.mock_sympify.return_value = mock_expr - mock_solution = MagicMock() - mock_solution.is_real = True - float_mock = MagicMock(return_value=5.0) - mock_solution.__float__ = float_mock - self.mock_solve.return_value = [mock_solution] - - result = calculator.solve_equation("x + 2 = 7") - self.assertEqual(result["equation"], "x + 2 = 7") - self.assertEqual(result["variable"], "x") - self.assertEqual(result["solutions"], [5]) - self.assertTrue(len(result["steps"]) > 0) - - self.mock_solve.side_effect = Exception("Test error") - with self.assertRaises(Exception): - calculator.solve_equation("x + 2 = 7") - - def test_fact_verifier(self): - """Test the FactVerifier class.""" - verifier = FactVerifier() - self.mock_openai.assert_called_once() - - self.mock_chain.invoke.return_value = json.dumps({ - "is_verifiable": True, - "verification_result": "supported", - "confidence": 0.95, - "reasoning": "The evidence supports the claim", - "missing_information": "None" - }) - - result = verifier.verify("The Earth is round", "Scientific evidence shows the Earth is round") - self.assertEqual(result["verification_result"], "supported") - self.assertEqual(result["confidence"], 0.95) - - result = verifier.verify("The Earth is round") - self.mock_chain.invoke.assert_called_with({ - "claim": "The Earth is round", - "evidence": "No evidence provided." - }) - - self.mock_chain.invoke.side_effect = Exception("Test error") - with self.assertRaises(Exception): - verifier.verify("The Earth is round") - - self.mock_chain.invoke.side_effect = None - - self.mock_chain.invoke.return_value = json.dumps({ - "relationship": "contradictory", - "confidence": 0.95, - "reasoning": "The claims contradict each other", - "implications": "Both claims cannot be true simultaneously" - }) - - result = verifier.compare_claims("The Earth is round", "The Earth is flat") - self.assertEqual(result["relationship"], "contradictory") - self.assertEqual(result["confidence"], 0.95) - - self.mock_chain.invoke.side_effect = Exception("Test error") - with self.assertRaises(Exception): - verifier.compare_claims("The Earth is round", "The Earth is flat") - - -class TestMultimodalTools(unittest.TestCase): - """Test case for multimodal tools.""" - - def setUp(self): - """Set up the test case with mocked dependencies.""" - self.openai_patcher = patch('tools.multimodal_tools.ChatOpenAI') - self.mock_openai = self.openai_patcher.start() - self.mock_model = MagicMock() - self.mock_openai.return_value = self.mock_model - - self.prompt_patcher = patch('tools.multimodal_tools.PromptTemplate') - self.mock_prompt = self.prompt_patcher.start() - self.mock_template = MagicMock() - self.mock_prompt.from_template.return_value = self.mock_template - - self.parser_patcher = patch('tools.multimodal_tools.StrOutputParser') - self.mock_parser = self.parser_patcher.start() - self.mock_output_parser = MagicMock() - self.mock_parser.return_value = self.mock_output_parser - - self.mock_chain = MagicMock() - self.mock_template.__or__.return_value.__or__.return_value = self.mock_chain - - self.pil_patcher = patch('tools.multimodal_tools.Image') - self.mock_pil = self.pil_patcher.start() - self.mock_image = MagicMock() - self.mock_pil.open.return_value = self.mock_image - - self.os_path_patcher = patch('tools.multimodal_tools.os.path') - self.mock_os_path = self.os_path_patcher.start() - self.mock_os_path.exists.return_value = True - - self.pdf2image_patcher = patch('tools.multimodal_tools.pdf2image') - self.mock_pdf2image = self.pdf2image_patcher.start() - self.pytesseract_patcher = patch('tools.multimodal_tools.pytesseract') - self.mock_pytesseract = self.pytesseract_patcher.start() - self.docx2txt_patcher = patch('tools.multimodal_tools.docx2txt') - self.mock_docx2txt = self.docx2txt_patcher.start() - - def tearDown(self): - """Clean up after the test.""" - self.openai_patcher.stop() - self.prompt_patcher.stop() - self.parser_patcher.stop() - self.pil_patcher.stop() - self.os_path_patcher.stop() - self.pdf2image_patcher.stop() - self.pytesseract_patcher.stop() - self.docx2txt_patcher.stop() - - def test_image_analyzer(self): - """Test the ImageAnalyzer class.""" - analyzer = ImageAnalyzer() - self.mock_openai.assert_called_once() - - self.mock_chain.invoke.return_value = json.dumps({ - "description": "A detailed description of the image", - "subjects": ["Subject 1", "Subject 2"], - "text_content": "Text in the image", - "context": "The overall context", - "tags": ["Tag 1", "Tag 2"] - }) - - result = analyzer.analyze_image("test_image.jpg") - self.assertEqual(result["description"], "A detailed description of the image") - self.assertEqual(len(result["subjects"]), 2) - self.assertEqual(result["text_content"], "Text in the image") - - result = analyzer.analyze_image("test_image.jpg", "Custom prompt") - self.mock_prompt.from_template.assert_called_with("Custom prompt") - - self.mock_os_path.exists.return_value = False - with self.assertRaises(FileNotFoundError): - analyzer.analyze_image("nonexistent.jpg") - - self.mock_os_path.exists.return_value = True - - self.mock_chain.invoke.return_value = "Not valid JSON" - result = analyzer.analyze_image("test_image.jpg") - self.assertEqual(result["description"], "Not valid JSON") - self.assertEqual(result["subjects"], []) - - self.mock_chain.invoke.side_effect = Exception("Test error") - with self.assertRaises(Exception): - analyzer.analyze_image("test_image.jpg") - - self.mock_chain.invoke.side_effect = None - - self.mock_chain.invoke.return_value = json.dumps({ - "objects": [ - { - "name": "Object 1", - "confidence": 0.95, - "location": "Top left" - }, - { - "name": "Object 2", - "confidence": 0.85, - "location": "Bottom right" - } - ], - "scene_type": "Indoor", - "object_count": 2 - }) - - result = analyzer.detect_objects("test_image.jpg") - self.assertEqual(len(result["objects"]), 2) - self.assertEqual(result["objects"][0]["name"], "Object 1") - self.assertEqual(result["scene_type"], "Indoor") - self.assertEqual(result["object_count"], 2) - - self.mock_chain.invoke.return_value = "Not valid JSON" - result = analyzer.detect_objects("test_image.jpg") - self.assertEqual(result["objects"], []) - self.assertEqual(result["scene_type"], "Unknown") - self.assertEqual(result["object_count"], 0) - - def test_chart_interpreter(self): - """Test the ChartInterpreter class.""" - interpreter = ChartInterpreter() - self.mock_openai.assert_called_once() - - self.mock_chain.invoke.return_value = json.dumps({ - "chart_type": "Bar chart", - "title": "Sales by Region", - "axes": { - "x_axis": "Region", - "y_axis": "Sales ($)" - }, - "data_points": [ - {"category": "North", "value": "1000"}, - {"category": "South", "value": "1500"} - ], - "trends": ["South has higher sales than North"], - "insights": "South region is performing better", - "confidence": 0.95 - }) - - result = interpreter.interpret_chart("test_chart.jpg") - self.assertEqual(result["chart_type"], "Bar chart") - self.assertEqual(result["title"], "Sales by Region") - self.assertEqual(len(result["data_points"]), 2) - self.assertEqual(result["insights"], "South region is performing better") - - self.mock_os_path.exists.return_value = False - with self.assertRaises(FileNotFoundError): - interpreter.interpret_chart("nonexistent.jpg") - - self.mock_os_path.exists.return_value = True - - self.mock_chain.invoke.return_value = "Not valid JSON" - result = interpreter.interpret_chart("test_chart.jpg") - self.assertEqual(result["chart_type"], "Unknown") - self.assertEqual(result["insights"], "Not valid JSON") - - self.mock_chain.invoke.side_effect = Exception("Test error") - with self.assertRaises(Exception): - interpreter.interpret_chart("test_chart.jpg") - - self.mock_chain.invoke.side_effect = None - - self.mock_chain.invoke.return_value = json.dumps({ - "chart_type": "Bar chart", - "data": [ - {"x": "North", "y": "1000", "category": "Region"}, - {"x": "South", "y": "1500", "category": "Region"} - ], - "data_table": [ - ["Region", "Sales"], - ["North", "1000"], - ["South", "1500"] - ], - "confidence": 0.95, - "notes": "Data extracted successfully" - }) - - result = interpreter.extract_data("test_chart.jpg") - self.assertEqual(result["chart_type"], "Bar chart") - self.assertEqual(len(result["data"]), 2) - self.assertEqual(len(result["data_table"]), 3) - self.assertEqual(result["confidence"], 0.95) - - self.mock_chain.invoke.return_value = "Not valid JSON" - result = interpreter.extract_data("test_chart.jpg") - self.assertEqual(result["chart_type"], "Unknown") - self.assertEqual(result["data"], []) - self.assertEqual(result["data_table"], []) - - def test_document_parser(self): - """Test the DocumentParser class.""" - parser = DocumentParser() - self.mock_openai.assert_called_once() - - with patch('tools.multimodal_tools.Path') as mock_path: - mock_path_instance = MagicMock() - mock_path_instance.suffix = '.txt' - mock_path.return_value = mock_path_instance - - with patch('builtins.open', unittest.mock.mock_open(read_data="Test document content")) as mock_open: - parser._summarize_text = MagicMock(return_value="Test summary") - - result = parser.parse_document("test_document.txt") - self.assertEqual(result["document_path"], "test_document.txt") - self.assertEqual(result["file_type"], ".txt") - self.assertEqual(result["text_content"], "Test document content") - self.assertEqual(result["summary"], "Test summary") - - self.mock_os_path.exists.return_value = False - with self.assertRaises(FileNotFoundError): - parser.parse_document("nonexistent.txt") - - self.mock_os_path.exists.return_value = True - - mock_path_instance.suffix = '.docx' - self.mock_docx2txt.process.return_value = "Test DOCX content" - - result = parser.parse_document("test_document.docx") - self.assertEqual(result["file_type"], ".docx") - self.assertEqual(result["text_content"], "Test DOCX content") - - mock_path_instance.suffix = '.pdf' - parser._parse_pdf = MagicMock(return_value="Test PDF content") - - result = parser.parse_document("test_document.pdf") - self.assertEqual(result["file_type"], ".pdf") - self.assertEqual(result["text_content"], "Test PDF content") - - mock_path_instance.suffix = '.xyz' - with self.assertRaises(ValueError): - parser.parse_document("test_document.xyz") - - parser.parse_document = MagicMock(return_value={ - "text_content": "Name: John Doe\nAge: 30\nOccupation: Engineer" - }) - - schema = { - "type": "object", - "properties": { - "name": {"type": "string"}, - "age": {"type": "number"}, - "occupation": {"type": "string"} - } - } - - self.mock_chain.invoke.return_value = json.dumps({ - "name": "John Doe", - "age": 30, - "occupation": "Engineer" - }) - - result = parser.extract_structured_data("test_document.txt", schema) - self.assertEqual(result["name"], "John Doe") - self.assertEqual(result["age"], 30) - self.assertEqual(result["occupation"], "Engineer") - - self.mock_chain.invoke.return_value = "Not valid JSON" - with self.assertRaises(ValueError): - parser.extract_structured_data("test_document.txt", schema) - - self.mock_chain.invoke.side_effect = Exception("Test error") - with self.assertRaises(Exception): - parser.extract_structured_data("test_document.txt", schema) - - -if __name__ == "__main__": - unittest.main() \ No newline at end of file diff --git a/tests/test_utils.py b/tests/test_utils.py deleted file mode 100644 index dfa0f1f9884b36eec5cd9dcb444a373ab9f3a625..0000000000000000000000000000000000000000 --- a/tests/test_utils.py +++ /dev/null @@ -1,424 +0,0 @@ -""" -Unit tests for GAIA agent utilities. -""" -import unittest -import json -import sys -from typing import Dict, Any, List, Optional - -sys.path.append('.') - -from utils.formatting import ( - extract_answer, extract_number, extract_date, extract_boolean, - extract_list, extract_entity, extract_structured_data, - format_answer, validate_answer_format, process_answer, - FORMAT_TYPES -) -from utils.parsing import ( - extract_key_information, identify_required_tools, - determine_question_complexity, parse_question, - COMPLEXITY_LEVELS -) - - -class TestFormattingUtils(unittest.TestCase): - """Test case for formatting utilities.""" - - def test_extract_answer(self): - """Test answer extraction from responses.""" - response = "The answer is 42." - self.assertEqual(extract_answer(response), "42") - - response = "42" - self.assertEqual(extract_answer(response), "42") - - response = "In conclusion: Paris is the capital of France." - self.assertEqual(extract_answer(response), "Paris is the capital of France") - - response = "Therefore: The result is negative." - self.assertEqual(extract_answer(response), "The result is negative") - - response = "The calculation gives us $123.45." - self.assertEqual(extract_answer(response, FORMAT_TYPES["NUMBER"]), "123.45") - - response = "The event occurred on January 15, 2023." - self.assertEqual(extract_answer(response, FORMAT_TYPES["DATE"]), "January 15, 2023") - - response = "This is a complex question. There are many factors to consider. The most important factor is time." - self.assertEqual(extract_answer(response), "The most important factor is time") - - def test_extract_number(self): - """Test number extraction from text.""" - self.assertEqual(extract_number("The answer is 42"), "42") - - self.assertEqual(extract_number("The result is 3.14159"), "3.14159") - - self.assertEqual(extract_number("The cost is $99.99"), "$99.99") - - self.assertEqual(extract_number("The population is 1,234,567"), "1,234,567") - - self.assertEqual(extract_number("The growth rate is 5.2%"), "5.2%") - - self.assertEqual(extract_number("The temperature is -10 degrees"), "-10") - - self.assertEqual(extract_number("First value is 10, second is 20, third is 30"), "30") - - self.assertEqual(extract_number("There is no number here"), "") - - def test_extract_date(self): - """Test date extraction from text.""" - self.assertEqual(extract_date("The date is 12/25/2023"), "12/25/2023") - - self.assertEqual(extract_date("The date is 2023/12/25"), "2023/12/25") - - # Test Month DD, YYYY format - self.assertEqual(extract_date("The event is on December 25, 2023"), "December 25, 2023") - - self.assertEqual(extract_date("The event is on 25 December 2023"), "25 December 2023") - - self.assertEqual(extract_date("The meeting is at 14:30"), "14:30") - - self.assertEqual(extract_date("The date is: January 1, 2023"), "January 1, 2023") - - self.assertEqual(extract_date("There is no date here"), "") - - def test_extract_boolean(self): - """Test boolean extraction from text.""" - self.assertEqual(extract_boolean("Yes, that is correct"), "Yes") - self.assertEqual(extract_boolean("No, that is incorrect"), "No") - - self.assertEqual(extract_boolean("The answer is yes"), "Yes") - self.assertEqual(extract_boolean("The answer is no"), "No") - - self.assertEqual(extract_boolean("Affirmative, the statement is true"), "Yes") - self.assertEqual(extract_boolean("Negative, the statement is false"), "No") - - self.assertEqual(extract_boolean("True, the statement is correct"), "Yes") - self.assertEqual(extract_boolean("False, the statement is incorrect"), "No") - - self.assertEqual(extract_boolean("The statement is correct and accurate"), "Yes") - self.assertEqual(extract_boolean("The statement is incorrect and contains errors"), "No") - - self.assertEqual(extract_boolean("It depends on various factors"), "Unknown") - - def test_extract_list(self): - """Test list extraction from text.""" - self.assertEqual(extract_list("The list is: apple, banana, orange"), "apple, banana, orange") - - self.assertEqual(extract_list("The fruits are apple, banana and orange"), "apple, banana, orange") - - self.assertEqual(extract_list("The colors are: red; blue; green"), "red, blue, green") - - self.assertEqual(extract_list("The items are:\n- Item 1\n- Item 2\n- Item 3"), "Item 1, Item 2, Item 3") - - self.assertEqual(extract_list("The steps are:\n1. Step one\n2. Step two\n3. Step three"), "Step one, Step two, Step three") - - self.assertEqual(extract_list("There is no list here"), "") - - def test_extract_entity(self): - """Test entity extraction from text.""" - self.assertEqual(extract_entity("The person is John Smith"), "John Smith") - - self.assertEqual(extract_entity("The location is New York City"), "New York City") - - self.assertEqual(extract_entity("The company is Microsoft Corporation"), "Microsoft Corporation") - - self.assertEqual(extract_entity("Albert Einstein developed the theory of relativity"), "Albert Einstein") - - self.assertEqual(extract_entity("there are no entities in this text"), "") - - def test_extract_structured_data(self): - """Test structured data extraction from text.""" - json_text = """Here is the data: - ```json - {"name": "John", "age": 30, "city": "New York"} - ``` - """ - expected = json.dumps({"name": "John", "age": 30, "city": "New York"}, indent=2) - self.assertEqual(extract_structured_data(json_text), expected) - - json_text = """Here is the data: {"name": "John", "age": 30, "city": "New York"}""" - expected = json.dumps({"name": "John", "age": 30, "city": "New York"}, indent=2) - self.assertEqual(extract_structured_data(json_text), expected) - - self.assertEqual(extract_structured_data("This is not JSON: {name: John}"), "") - - self.assertEqual(extract_structured_data("There is no JSON here"), "") - - def test_format_answer(self): - """Test answer formatting.""" - self.assertEqual(format_answer(" This is a text ", FORMAT_TYPES["TEXT"]), "This is a text") - - self.assertEqual(format_answer("The answer is $42.00", FORMAT_TYPES["NUMBER"]), "42") - self.assertEqual(format_answer("3.14159", FORMAT_TYPES["NUMBER"]), "3.14159") - - self.assertEqual(format_answer("The date is January 1, 2023", FORMAT_TYPES["DATE"]), "January 1, 2023") - - self.assertEqual(format_answer("Yes, that is correct", FORMAT_TYPES["BOOLEAN"]), "Yes") - self.assertEqual(format_answer("No, that is incorrect", FORMAT_TYPES["BOOLEAN"]), "No") - - self.assertEqual(format_answer("The items are: apple, banana, orange", FORMAT_TYPES["LIST"]), "apple, banana, orange") - - self.assertEqual(format_answer("The person is John Smith", FORMAT_TYPES["ENTITY"]), "John Smith") - - json_text = """{"name": "John", "age": 30}""" - expected = json.dumps({"name": "John", "age": 30}, indent=2) - self.assertEqual(format_answer(json_text, FORMAT_TYPES["STRUCTURED"]), expected) - - def test_validate_answer_format(self): - """Test answer format validation.""" - result = validate_answer_format("42", FORMAT_TYPES["NUMBER"]) - self.assertTrue(result["is_valid"]) - self.assertEqual(result["formatted_answer"], "42") - - result = validate_answer_format("not a number", FORMAT_TYPES["NUMBER"]) - self.assertFalse(result["is_valid"]) - - result = validate_answer_format("January 1, 2023", FORMAT_TYPES["DATE"]) - self.assertTrue(result["is_valid"]) - - result = validate_answer_format("not a date", FORMAT_TYPES["DATE"]) - self.assertFalse(result["is_valid"]) - - result = validate_answer_format("Yes", FORMAT_TYPES["BOOLEAN"]) - self.assertTrue(result["is_valid"]) - - result = validate_answer_format("Maybe", FORMAT_TYPES["BOOLEAN"]) - self.assertFalse(result["is_valid"]) - - result = validate_answer_format("apple, banana, orange", FORMAT_TYPES["LIST"]) - self.assertTrue(result["is_valid"]) - - result = validate_answer_format("single item", FORMAT_TYPES["LIST"]) - self.assertFalse(result["is_valid"]) - - result = validate_answer_format("John Smith", FORMAT_TYPES["ENTITY"]) - self.assertTrue(result["is_valid"]) - - result = validate_answer_format("not an entity", FORMAT_TYPES["ENTITY"]) - self.assertFalse(result["is_valid"]) - - result = validate_answer_format('{"name": "John", "age": 30}', FORMAT_TYPES["STRUCTURED"]) - self.assertTrue(result["is_valid"]) - - result = validate_answer_format("not JSON", FORMAT_TYPES["STRUCTURED"]) - self.assertFalse(result["is_valid"]) - - result = validate_answer_format("any text", FORMAT_TYPES["TEXT"]) - self.assertTrue(result["is_valid"]) - - def test_process_answer(self): - """Test complete answer processing.""" - result = process_answer("The answer is 42.", FORMAT_TYPES["NUMBER"]) - self.assertEqual(result["extracted_answer"], "42") - self.assertEqual(result["formatted_answer"], "42") - self.assertTrue(result["is_valid"]) - - result = process_answer("The event occurred on January 1, 2023.", FORMAT_TYPES["DATE"]) - self.assertEqual(result["extracted_answer"], "January 1, 2023") - self.assertEqual(result["formatted_answer"], "January 1, 2023") - self.assertTrue(result["is_valid"]) - - result = process_answer("The answer is text.", FORMAT_TYPES["NUMBER"]) - self.assertFalse(result["is_valid"]) - - result = process_answer("This is a text answer.") - self.assertEqual(result["format_type"], FORMAT_TYPES["TEXT"]) - self.assertTrue(result["is_valid"]) - - -class TestParsingUtils(unittest.TestCase): - """Test case for parsing utilities.""" - - def test_extract_key_information(self): - """Test key information extraction from questions.""" - question = "What is the capital of France?" - info = extract_key_information(question) - self.assertEqual(info["question_type"], "factual") - self.assertEqual(info["expected_answer_type"], "unknown") - self.assertIn("France", info["entities"]) - self.assertIn("capital", info["keywords"]) - - question = "Who was Albert Einstein?" - info = extract_key_information(question) - self.assertEqual(info["question_type"], "factual") - self.assertEqual(info["expected_answer_type"], "person") - self.assertIn("Albert", info["entities"]) - self.assertIn("Einstein", info["entities"]) - - question = "Where is the Eiffel Tower located?" - info = extract_key_information(question) - self.assertEqual(info["question_type"], "factual") - self.assertEqual(info["expected_answer_type"], "location") - self.assertIn("Eiffel", info["entities"]) - self.assertIn("Tower", info["entities"]) - - question = "When was the Declaration of Independence signed?" - info = extract_key_information(question) - self.assertEqual(info["question_type"], "factual") - self.assertEqual(info["expected_answer_type"], "time") - self.assertIn("Declaration", info["entities"]) - self.assertIn("Independence", info["entities"]) - - question = "How many planets are in our solar system?" - info = extract_key_information(question) - self.assertEqual(info["question_type"], "explanatory") - self.assertEqual(info["expected_answer_type"], "number") - self.assertIn("planets", info["keywords"]) - self.assertIn("solar", info["keywords"]) - self.assertIn("system", info["keywords"]) - - question = "Why does the sun rise in the east?" - info = extract_key_information(question) - self.assertEqual(info["question_type"], "explanatory") - self.assertEqual(info["expected_answer_type"], "reason") - self.assertIn("sun", info["keywords"]) - self.assertIn("rise", info["keywords"]) - self.assertIn("east", info["keywords"]) - - question = "How does photosynthesis work?" - info = extract_key_information(question) - self.assertEqual(info["question_type"], "explanatory") - self.assertEqual(info["expected_answer_type"], "process") - self.assertIn("photosynthesis", info["keywords"]) - self.assertIn("work", info["keywords"]) - - question = "List the main characters in Romeo and Juliet." - info = extract_key_information(question) - self.assertEqual(info["question_type"], "list") - self.assertEqual(info["expected_answer_type"], "list") - self.assertIn("Romeo", info["entities"]) - self.assertIn("Juliet", info["entities"]) - self.assertIn("characters", info["keywords"]) - - question = "Is water wet?" - info = extract_key_information(question) - self.assertEqual(info["question_type"], "yes_no") - self.assertIn("water", info["keywords"]) - self.assertIn("wet", info["keywords"]) - - question = "Compare the economies of the United States and China." - info = extract_key_information(question) - self.assertEqual(info["question_type"], "comparative") - self.assertIn("United", info["entities"]) - self.assertIn("States", info["entities"]) - self.assertIn("China", info["entities"]) - self.assertIn("economies", info["keywords"]) - self.assertIn("compare", info["keywords"]) - - question = "What were the major events in 2020?" - info = extract_key_information(question) - self.assertIn(("in", "2020"), info["temporal_references"]) - - question = "List only the mammals found in Africa." - info = extract_key_information(question) - self.assertIn("only the mammals", info["constraints"]) - self.assertIn("in africa", info["constraints"]) - - def test_identify_required_tools(self): - """Test tool identification from questions.""" - question = "What is the latest news about climate change?" - tools = identify_required_tools(question) - self.assertIn("web_search", tools) - - question = "Extract the main points from the webpage at example.com." - tools = identify_required_tools(question) - self.assertIn("web_content", tools) - - question = "Why do leaves change color in the fall?" - tools = identify_required_tools(question) - self.assertIn("reasoning", tools) - - question = "Calculate the area of a circle with radius 5 cm." - tools = identify_required_tools(question) - self.assertIn("math", tools) - - question = "Verify if the Earth is flat." - tools = identify_required_tools(question) - self.assertIn("fact_verification", tools) - - question = "Describe what you see in this image." - tools = identify_required_tools(question) - self.assertIn("image_analysis", tools) - - question = "Interpret this graph showing temperature trends." - tools = identify_required_tools(question) - self.assertIn("chart_interpretation", tools) - - question = "Extract the key points from this PDF document." - tools = identify_required_tools(question) - self.assertIn("document_parsing", tools) - - question = "Analyze this image and verify if the information is accurate." - tools = identify_required_tools(question) - self.assertIn("image_analysis", tools) - self.assertIn("fact_verification", tools) - - question = "Tell me about elephants." - tools = identify_required_tools(question) - self.assertIn("reasoning", tools) - self.assertIn("web_search", tools) - - def test_determine_question_complexity(self): - """Test question complexity determination.""" - question = "What is the capital of France?" - complexity = determine_question_complexity(question) - self.assertEqual(complexity["complexity_level"], "SIMPLE") - self.assertEqual(complexity["complexity_score"], 1) - self.assertFalse(complexity["requires_complex_reasoning"]) - - question = "What are the main factors that contributed to the economic growth of Japan after World War II?" - complexity = determine_question_complexity(question) - self.assertEqual(complexity["complexity_level"], "MODERATE") - self.assertEqual(complexity["complexity_score"], 2) - - question = "Analyze how climate change impacts both the global economy and biodiversity, and explain the relationship between these effects." - complexity = determine_question_complexity(question) - self.assertEqual(complexity["complexity_level"], "COMPLEX") - self.assertEqual(complexity["complexity_score"], 3) - self.assertTrue(complexity["requires_complex_reasoning"]) - self.assertTrue(complexity["multi_domain"]) - - question = "Compare and contrast the economic policies of the United States before and after the 2008 financial crisis. How did these policies affect global markets, and what lessons can be applied to current economic challenges?" - complexity = determine_question_complexity(question) - self.assertEqual(complexity["complexity_level"], "VERY_COMPLEX") - self.assertEqual(complexity["complexity_score"], 4) - self.assertTrue(complexity["requires_complex_reasoning"]) - self.assertTrue(complexity["multi_domain"]) - - question = "Explain how quantum physics relates to computer science in the development of quantum computing." - complexity = determine_question_complexity(question) - self.assertIn("science", complexity["domains_involved"]) - self.assertIn("technology", complexity["domains_involved"]) - - question = "If global temperatures rise by 2 degrees Celsius, what would be the impact on coastal cities?" - complexity = determine_question_complexity(question) - self.assertTrue(complexity["complexity_score"] >= 2) - - def test_parse_question(self): - """Test comprehensive question parsing.""" - question = "Why did the Industrial Revolution begin in Great Britain rather than in other European countries during the 18th century?" - parsed = parse_question(question) - - self.assertEqual(parsed["question"], question) - self.assertIn("key_information", parsed) - self.assertIn("required_tools", parsed) - self.assertIn("complexity", parsed) - - self.assertEqual(parsed["key_information"]["question_type"], "explanatory") - self.assertEqual(parsed["key_information"]["expected_answer_type"], "reason") - self.assertIn("Industrial", parsed["key_information"]["entities"]) - self.assertIn("Revolution", parsed["key_information"]["entities"]) - self.assertIn("Great", parsed["key_information"]["entities"]) - self.assertIn("Britain", parsed["key_information"]["entities"]) - self.assertIn("European", parsed["key_information"]["entities"]) - - self.assertIn("reasoning", parsed["required_tools"]) - - self.assertTrue(parsed["complexity"]["complexity_score"] >= 2) - self.assertTrue(parsed["complexity"]["requires_complex_reasoning"]) - self.assertIn("history", parsed["complexity"]["domains_involved"]) - - -if __name__ == "__main__": - unittest.main() \ No newline at end of file diff --git a/tests/verify_connections.py b/tests/verify_connections.py deleted file mode 100644 index 20ac90b2c649d955f98a7a2fb83882a80f3a0f54..0000000000000000000000000000000000000000 --- a/tests/verify_connections.py +++ /dev/null @@ -1,251 +0,0 @@ -""" -Connection verification script for the GAIA agent. - -This script verifies connections to all external services used by the GAIA agent: -- OpenAI API -- Supabase -- Serper API -- Hugging Face API - -It provides detailed feedback on which connections are working and which are not. -""" - -import os -import sys -import json -import logging -import requests -from typing import Dict, Any, List, Tuple - -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(name)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("connection_verifier") - -sys.path.append('.') - -try: - from agent.config import ( - OPENAI_API_KEY, - OPENAI_API_BASE, - SUPABASE_URL, - SUPABASE_KEY, - SERPER_API_KEY, - SERPER_API_URL, - HF_TOKEN, - HUGGINGFACE_API_URL - ) -except ImportError as e: - logger.error(f"Error importing configuration: {str(e)}") - sys.exit(1) - -def verify_openai_connection() -> Tuple[bool, str]: - """ - Verify connection to OpenAI API. - - Returns: - Tuple[bool, str]: (success, message) - """ - logger.info("Verifying OpenAI API connection...") - - if not OPENAI_API_KEY: - return False, "OpenAI API key not set" - - try: - headers = { - "Content-Type": "application/json", - "Authorization": f"Bearer {OPENAI_API_KEY}" - } - - response = requests.get( - f"{OPENAI_API_BASE}/models", - headers=headers - ) - - if response.status_code == 200: - models = response.json().get("data", []) - model_count = len(models) - return True, f"Successfully connected to OpenAI API. {model_count} models available." - else: - return False, f"Failed to connect to OpenAI API. Status code: {response.status_code}, Response: {response.text}" - - except Exception as e: - return False, f"Error connecting to OpenAI API: {str(e)}" - -def verify_supabase_connection() -> Tuple[bool, str]: - """ - Verify connection to Supabase. - - Returns: - Tuple[bool, str]: (success, message) - """ - logger.info("Verifying Supabase connection...") - - if not SUPABASE_URL or not SUPABASE_KEY: - return False, "Supabase URL or key not set" - - try: - try: - from supabase import create_client, Client - except ImportError: - return False, "Supabase Python client not installed. Run 'pip install supabase'" - - # Create Supabase client - supabase = create_client(SUPABASE_URL, SUPABASE_KEY) - - response = supabase.rpc("get_service_status").execute() - - if hasattr(response, "data"): - return True, "Successfully connected to Supabase" - else: - return False, f"API returned unexpected response: {response}" - - except Exception as e: - return False, f"Error connecting to Supabase: {str(e)}" - -def verify_serper_connection() -> Tuple[bool, str]: - """ - Verify connection to Serper API. - - Returns: - Tuple[bool, str]: (success, message) - """ - logger.info("Verifying Serper API connection...") - - if not SERPER_API_KEY: - return False, "Serper API key not set" - - try: - headers = { - "X-API-KEY": SERPER_API_KEY, - "Content-Type": "application/json" - } - - payload = { - "q": "test query", - "gl": "us", - "hl": "en", - "num": 1 - } - - response = requests.post( - SERPER_API_URL, - headers=headers, - json=payload - ) - - if response.status_code == 200: - return True, "Successfully connected to Serper API" - else: - return False, f"Failed to connect to Serper API. Status code: {response.status_code}, Response: {response.text}" - - except Exception as e: - return False, f"Error connecting to Serper API: {str(e)}" - -def verify_huggingface_connection() -> Tuple[bool, str]: - """ - Verify connection to Hugging Face API. - - Returns: - Tuple[bool, str]: (success, message) - """ - logger.info("Verifying Hugging Face API connection...") - - if not HF_TOKEN: - return False, "Hugging Face token not set" - - try: - headers = { - "Authorization": f"Bearer {HF_TOKEN}" - } - - response = requests.get( - "https://huggingface.co/api/models", - headers=headers, - params={"limit": 1} - ) - - if response.status_code == 200: - return True, "Successfully connected to Hugging Face API" - else: - return False, f"Failed to connect to Hugging Face API. Status code: {response.status_code}, Response: {response.text}" - - except Exception as e: - return False, f"Error connecting to Hugging Face API: {str(e)}" - -def verify_all_connections() -> Dict[str, Dict[str, Any]]: - """ - Verify all connections. - - Returns: - Dict[str, Dict[str, Any]]: Connection verification results - """ - results = {} - - # Verify OpenAI connection - openai_success, openai_message = verify_openai_connection() - results["openai"] = { - "success": openai_success, - "message": openai_message - } - - # Verify Supabase connection - supabase_success, supabase_message = verify_supabase_connection() - results["supabase"] = { - "success": supabase_success, - "message": supabase_message - } - - # Verify Serper connection - serper_success, serper_message = verify_serper_connection() - results["serper"] = { - "success": serper_success, - "message": serper_message - } - - # Verify Hugging Face connection - hf_success, hf_message = verify_huggingface_connection() - results["huggingface"] = { - "success": hf_success, - "message": hf_message - } - - return results - -def print_results(results: Dict[str, Dict[str, Any]]) -> None: - """ - Print connection verification results. - - Args: - results: Connection verification results - """ - - all_success = True - - for service, result in results.items(): - success = result["success"] - message = result["message"] - - status = "✅ Connected" if success else "❌ Failed" - all_success = all_success and success - - - - if all_success: - else: - - -def main(): - """Main function.""" - logger.info("Starting connection verification...") - - results = verify_all_connections() - - print_results(results) - - return all(result["success"] for result in results.values()) - -if __name__ == "__main__": - success = main() - sys.exit(0 if success else 1) \ No newline at end of file diff --git a/tools/__init__.py b/tools/__init__.py deleted file mode 100644 index 4646965d770ccd0add26546d2b491c3539ece881..0000000000000000000000000000000000000000 --- a/tools/__init__.py +++ /dev/null @@ -1,683 +0,0 @@ -""" -Tools module for GAIA implementation. - -Contains implementations of various tools for web browsing, reasoning, and multimodal processing. -Includes a tool registry for dynamic tool selection based on question requirements. -""" - -import logging -from typing import Dict, Any, List, Optional, Union, Callable, Type, Set - -from tools.web_tools import ( - DuckDuckGoSearchTool, - SerperSearchTool, - WebContentExtractor, - WebNavigator, - create_duckduckgo_search, - create_serper_search, - create_web_content_extractor, - create_web_navigator -) - -from tools.reasoning_tools import ( - StepByStepReasoner, - MathCalculator, - FactVerifier, - create_step_by_step_reasoner, - create_math_calculator, - create_fact_verifier -) - -from tools.perplexity_tool import ( - PerplexityTool, - create_perplexity_tool -) - -from tools.multimodal_tools import ( - ImageAnalyzer, - ChartInterpreter, - DocumentParser, - create_image_analyzer, - create_chart_interpreter, - create_document_parser -) - -logger = logging.getLogger("gaia_agent.tools") - -class ToolMetadata: - """Metadata for a tool in the registry.""" - - def __init__( - self, - name: str, - description: str, - category: str, - factory: Callable[[], Any], - keywords: List[str], - input_schema: Dict[str, Any], - output_schema: Dict[str, Any], - requires_api_key: bool = False, - example_uses: List[str] = None - ): - """ - Initialize tool metadata. - - Args: - name: The name of the tool - description: A description of what the tool does - category: The category of the tool (web, reasoning, multimodal) - factory: A factory function that creates an instance of the tool - keywords: Keywords for tool selection - input_schema: Schema describing the tool's input parameters - output_schema: Schema describing the tool's output format - requires_api_key: Whether the tool requires an API key - example_uses: Example use cases for the tool - """ - self.name = name - self.description = description - self.category = category - self.factory = factory - self.keywords = keywords - self.input_schema = input_schema - self.output_schema = output_schema - self.requires_api_key = requires_api_key - self.example_uses = example_uses or [] - self._instance = None - - def get_instance(self) -> Any: - """ - Get or create an instance of the tool. - - Returns: - An instance of the tool - """ - if self._instance is None: - self._instance = self.factory() - return self._instance - - -class ToolRegistry: - """Registry for managing available tools.""" - - def __init__(self): - """Initialize the tool registry.""" - self.tools: Dict[str, ToolMetadata] = {} - self.categories: Dict[str, List[str]] = { - "web": [], - "reasoning": [], - "multimodal": [] - } - - def register_tool(self, metadata: ToolMetadata) -> None: - """ - Register a tool in the registry. - - Args: - metadata: The tool metadata - """ - self.tools[metadata.name] = metadata - - if metadata.category in self.categories: - self.categories[metadata.category].append(metadata.name) - else: - self.categories[metadata.category] = [metadata.name] - - - def get_tool(self, name: str) -> Optional[Any]: - """ - Get a tool instance by name. - - Args: - name: The name of the tool - - Returns: - The tool instance, or None if not found - """ - if name not in self.tools: - logger.warning(f"Tool not found: {name}") - return None - - return self.tools[name].get_instance() - - def get_tool_metadata(self, name: str) -> Optional[ToolMetadata]: - """ - Get tool metadata by name. - - Args: - name: The name of the tool - - Returns: - The tool metadata, or None if not found - """ - return self.tools.get(name) - - def list_tools(self, category: Optional[str] = None) -> List[str]: - """ - List available tools, optionally filtered by category. - - Args: - category: Optional category to filter by - - Returns: - List of tool names - """ - if category: - return self.categories.get(category, []) - - return list(self.tools.keys()) - - def select_tools(self, requirements: List[str]) -> List[str]: - """ - Select appropriate tools based on requirements. - - Args: - requirements: List of tool requirements or keywords - - Returns: - List of selected tool names - """ - selected_tools = set() - - for req in requirements: - req_lower = req.lower() - - if req_lower in self.tools: - selected_tools.add(req_lower) - continue - - for name, metadata in self.tools.items(): - if any(keyword.lower() in req_lower for keyword in metadata.keywords): - selected_tools.add(name) - - return list(selected_tools) - - def get_tools_by_question(self, question: str) -> List[str]: - """ - Select appropriate tools based on a question. - - Args: - question: The question to analyze - - Returns: - List of selected tool names - """ - question_lower = question.lower() - selected_tools = set() - - if any(term in question_lower for term in ["search", "find", "look up", "latest", "current", "news"]): - selected_tools.add("duckduckgo_search") - selected_tools.add("perplexity_search") - - if any(term in question_lower for term in ["website", "webpage", "url", "link", "extract", "content"]): - selected_tools.add("web_content_extractor") - - if any(term in question_lower for term in ["navigate", "browse", "follow", "links"]): - selected_tools.add("web_navigator") - - if any(term in question_lower for term in ["reason", "explain", "why", "how", "analyze"]): - selected_tools.add("step_by_step_reasoner") - selected_tools.add("perplexity_search") - - if any(term in question_lower for term in ["calculate", "compute", "solve", "equation", "math", "formula"]): - selected_tools.add("math_calculator") - - if any(term in question_lower for term in ["verify", "check", "fact", "true", "false", "accurate"]): - selected_tools.add("fact_verifier") - selected_tools.add("perplexity_search") - - if any(term in question_lower for term in ["image", "picture", "photo", "describe", "visual", "see"]): - selected_tools.add("image_analyzer") - - if any(term in question_lower for term in ["chart", "graph", "plot", "diagram", "visualization", "trend"]): - selected_tools.add("chart_interpreter") - - if any(term in question_lower for term in ["document", "pdf", "docx", "text", "extract", "parse", "read"]): - selected_tools.add("document_parser") - - if not selected_tools: - selected_tools.add("step_by_step_reasoner") - selected_tools.add("perplexity_search") - - return list(selected_tools) - - -registry = ToolRegistry() - -registry.register_tool( - ToolMetadata( - name="duckduckgo_search", - description="Search the web using DuckDuckGo", - category="web", - factory=create_duckduckgo_search, - keywords=["search", "web", "find", "lookup", "information", "duckduckgo"], - input_schema={ - "query": { - "type": "string", - "description": "The search query" - } - }, - output_schema={ - "type": "array", - "items": { - "type": "object", - "properties": { - "title": {"type": "string"}, - "link": {"type": "string"}, - "snippet": {"type": "string"} - } - } - }, - example_uses=[ - "Finding current information about a topic", - "Searching for specific facts or data", - "Looking up definitions or explanations" - ] - ) -) - -registry.register_tool( - ToolMetadata( - name="serper_search", - description="Search the web using Serper API", - category="web", - factory=create_serper_search, - keywords=["search", "web", "find", "lookup", "information", "serper", "google"], - input_schema={ - "query": { - "type": "string", - "description": "The search query" - } - }, - output_schema={ - "type": "array", - "items": { - "type": "object", - "properties": { - "title": {"type": "string"}, - "link": {"type": "string"}, - "snippet": {"type": "string"} - } - } - }, - requires_api_key=True, - example_uses=[ - "Finding current information about a topic", - "Searching for specific facts or data", - "Looking up definitions or explanations" - ] - ) -) - -registry.register_tool( - ToolMetadata( - name="web_content_extractor", - description="Extract content from a web page", - category="web", - factory=create_web_content_extractor, - keywords=["extract", "content", "webpage", "website", "url", "scrape"], - input_schema={ - "url": { - "type": "string", - "description": "The URL to extract content from" - } - }, - output_schema={ - "type": "object", - "properties": { - "url": {"type": "string"}, - "title": {"type": "string"}, - "content": {"type": "string"}, - "links": {"type": "array"}, - "metadata": {"type": "object"} - } - }, - example_uses=[ - "Extracting the main content from an article", - "Getting the text content of a webpage", - "Analyzing the structure of a webpage" - ] - ) -) - -registry.register_tool( - ToolMetadata( - name="web_navigator", - description="Navigate and scrape web pages", - category="web", - factory=create_web_navigator, - keywords=["navigate", "browse", "follow", "links", "scrape", "crawl"], - input_schema={ - "url": { - "type": "string", - "description": "The URL to navigate to" - }, - "link_pattern": { - "type": "string", - "description": "Optional regex pattern to match links", - "required": False - } - }, - output_schema={ - "type": "array", - "items": { - "type": "object", - "properties": { - "url": {"type": "string"}, - "title": {"type": "string"}, - "content": {"type": "string"}, - "links": {"type": "array"}, - "metadata": {"type": "object"} - } - } - }, - example_uses=[ - "Following links on a webpage", - "Crawling a website for specific information", - "Navigating through multiple pages" - ] - ) -) - -registry.register_tool( - ToolMetadata( - name="step_by_step_reasoner", - description="Perform step-by-step reasoning on a question", - category="reasoning", - factory=create_step_by_step_reasoner, - keywords=["reason", "analyze", "step", "logic", "thinking", "solve"], - input_schema={ - "question": { - "type": "string", - "description": "The question to reason about" - }, - "context": { - "type": "string", - "description": "Optional context information", - "required": False - } - }, - output_schema={ - "type": "object", - "properties": { - "steps": { - "type": "array", - "items": { - "type": "object", - "properties": { - "step_number": {"type": "integer"}, - "reasoning": {"type": "string"}, - "intermediate_conclusion": {"type": "string"} - } - } - }, - "final_conclusion": {"type": "string"}, - "confidence": {"type": "number"} - } - }, - example_uses=[ - "Breaking down complex problems", - "Analyzing logical arguments", - "Solving multi-step reasoning tasks" - ] - ) -) - -registry.register_tool( - ToolMetadata( - name="math_calculator", - description="Perform mathematical calculations", - category="reasoning", - factory=create_math_calculator, - keywords=["calculate", "compute", "math", "equation", "solve", "formula", "arithmetic"], - input_schema={ - "expression": { - "type": "string", - "description": "The mathematical expression to evaluate" - } - }, - output_schema={ - "type": "object", - "properties": { - "expression": {"type": "string"}, - "result": {"type": "number"}, - "steps": {"type": "array"} - } - }, - example_uses=[ - "Evaluating mathematical expressions", - "Solving equations", - "Performing arithmetic calculations" - ] - ) -) - -registry.register_tool( - ToolMetadata( - name="fact_verifier", - description="Verify facts and claims", - category="reasoning", - factory=create_fact_verifier, - keywords=["verify", "fact", "check", "claim", "true", "false", "accuracy"], - input_schema={ - "claim": { - "type": "string", - "description": "The claim to verify" - }, - "evidence": { - "type": "string", - "description": "Optional evidence to use for verification", - "required": False - } - }, - output_schema={ - "type": "object", - "properties": { - "is_verifiable": {"type": "boolean"}, - "verification_result": {"type": "string"}, - "confidence": {"type": "number"}, - "reasoning": {"type": "string"}, - "missing_information": {"type": "string"} - } - }, - example_uses=[ - "Checking the accuracy of statements", - "Verifying claims against evidence", - "Determining the reliability of information" - ] - ) -) - -registry.register_tool( - ToolMetadata( - name="image_analyzer", - description="Analyze and describe images", - category="multimodal", - factory=create_image_analyzer, - keywords=["image", "picture", "photo", "visual", "analyze", "describe"], - input_schema={ - "image_path": { - "type": "string", - "description": "Path to the image file" - }, - "prompt": { - "type": "string", - "description": "Optional specific prompt for analysis", - "required": False - } - }, - output_schema={ - "type": "object", - "properties": { - "description": {"type": "string"}, - "subjects": {"type": "array"}, - "text_content": {"type": "string"}, - "context": {"type": "string"}, - "tags": {"type": "array"} - } - }, - example_uses=[ - "Describing the content of images", - "Identifying objects in photos", - "Extracting text from images", - "Analyzing visual content" - ] - ) -) - -registry.register_tool( - ToolMetadata( - name="chart_interpreter", - description="Interpret charts and graphs", - category="multimodal", - factory=create_chart_interpreter, - keywords=["chart", "graph", "plot", "diagram", "visualization", "interpret"], - input_schema={ - "chart_path": { - "type": "string", - "description": "Path to the chart image file" - } - }, - output_schema={ - "type": "object", - "properties": { - "chart_type": {"type": "string"}, - "title": {"type": "string"}, - "axes": {"type": "object"}, - "data_points": {"type": "array"}, - "trends": {"type": "array"}, - "insights": {"type": "string"}, - "confidence": {"type": "number"} - } - }, - example_uses=[ - "Interpreting data visualizations", - "Extracting insights from charts", - "Analyzing trends in graphs", - "Understanding statistical plots" - ] - ) -) - -registry.register_tool( - ToolMetadata( - name="document_parser", - description="Parse and extract information from documents", - category="multimodal", - factory=create_document_parser, - keywords=["document", "pdf", "docx", "text", "extract", "parse"], - input_schema={ - "document_path": { - "type": "string", - "description": "Path to the document file" - } - }, - output_schema={ - "type": "object", - "properties": { - "document_path": {"type": "string"}, - "file_type": {"type": "string"}, - "text_content": {"type": "string"}, - "summary": {"type": "string"}, - "word_count": {"type": "integer"}, - "character_count": {"type": "integer"} - } - }, - example_uses=[ - "Extracting text from PDF documents", - "Parsing Word documents", - "Summarizing document content", - "Extracting structured data from documents" - ] - ) -) - -# Register Perplexity tool -registry.register_tool( - ToolMetadata( - name="perplexity_search", - description="Search the web and get reasoning using Perplexity API", - category="web", - factory=create_perplexity_tool, - keywords=["search", "web", "find", "lookup", "information", "perplexity", "reasoning", "online"], - input_schema={ - "query": { - "type": "string", - "description": "The search query" - }, - "max_tokens": { - "type": "integer", - "description": "Maximum number of tokens in the response", - "required": False - }, - "system_prompt": { - "type": "string", - "description": "Optional system prompt to guide the response", - "required": False - } - }, - output_schema={ - "type": "object", - "properties": { - "query": {"type": "string"}, - "content": {"type": "string"}, - "citations": {"type": "array"}, - "usage": {"type": "object"}, - "model": {"type": "string"} - } - }, - requires_api_key=True, - example_uses=[ - "Searching for current information with reasoning", - "Getting detailed explanations with citations", - "Answering complex questions with web search" - ] - ) -) - -def get_tool(name: str) -> Optional[Any]: - """ - Get a tool instance by name. - - Args: - name: The name of the tool - - Returns: - The tool instance, or None if not found - """ - return registry.get_tool(name) - -def list_tools(category: Optional[str] = None) -> List[str]: - """ - List available tools, optionally filtered by category. - - Args: - category: Optional category to filter by - - Returns: - List of tool names - """ - return registry.list_tools(category) - -def select_tools_for_question(question: str) -> List[str]: - """ - Select appropriate tools based on a question. - - Args: - question: The question to analyze - - Returns: - List of selected tool names - """ - return registry.get_tools_by_question(question) - -def get_tool_metadata(name: str) -> Optional[ToolMetadata]: - """ - Get tool metadata by name. - - Args: - name: The name of the tool - - Returns: - The tool metadata, or None if not found - """ - return registry.get_tool_metadata(name) \ No newline at end of file diff --git a/tools/__pycache__/__init__.cpython-312.pyc b/tools/__pycache__/__init__.cpython-312.pyc deleted file mode 100644 index 4529ceb1be745d3537c7d44c6fdcd9b4ed4aeaf2..0000000000000000000000000000000000000000 Binary files a/tools/__pycache__/__init__.cpython-312.pyc and /dev/null differ diff --git a/tools/__pycache__/__init__.cpython-313.pyc b/tools/__pycache__/__init__.cpython-313.pyc deleted file mode 100644 index 516d752bb04f04fe61be93e5eeaf9218fc0111c5..0000000000000000000000000000000000000000 Binary files a/tools/__pycache__/__init__.cpython-313.pyc and /dev/null differ diff --git a/tools/__pycache__/arxiv_tool.cpython-313.pyc b/tools/__pycache__/arxiv_tool.cpython-313.pyc deleted file mode 100644 index cc074bae00deb55618a48d16e8cf579253a05f29..0000000000000000000000000000000000000000 Binary files a/tools/__pycache__/arxiv_tool.cpython-313.pyc and /dev/null differ diff --git a/tools/__pycache__/multimodal_tools.cpython-313.pyc b/tools/__pycache__/multimodal_tools.cpython-313.pyc deleted file mode 100644 index 90ae4a5491e4c2b8edad2ef4b649de1ea619c518..0000000000000000000000000000000000000000 Binary files a/tools/__pycache__/multimodal_tools.cpython-313.pyc and /dev/null differ diff --git a/tools/__pycache__/perplexity_tool.cpython-313.pyc b/tools/__pycache__/perplexity_tool.cpython-313.pyc deleted file mode 100644 index 5b7b36d522b460f2c16530e3bebc5fc114152c0c..0000000000000000000000000000000000000000 Binary files a/tools/__pycache__/perplexity_tool.cpython-313.pyc and /dev/null differ diff --git a/tools/__pycache__/reasoning_tools.cpython-313.pyc b/tools/__pycache__/reasoning_tools.cpython-313.pyc deleted file mode 100644 index 5cb868051fa5de11c429c68aee726945f1dd7096..0000000000000000000000000000000000000000 Binary files a/tools/__pycache__/reasoning_tools.cpython-313.pyc and /dev/null differ diff --git a/tools/__pycache__/web_tools.cpython-312.pyc b/tools/__pycache__/web_tools.cpython-312.pyc deleted file mode 100644 index 66b761caddfbd5e35b3bac87f3dd1a79ffe2a835..0000000000000000000000000000000000000000 Binary files a/tools/__pycache__/web_tools.cpython-312.pyc and /dev/null differ diff --git a/tools/__pycache__/web_tools.cpython-313.pyc b/tools/__pycache__/web_tools.cpython-313.pyc deleted file mode 100644 index ccaad40b1649b9ac6d5f188313a62bb7edb07462..0000000000000000000000000000000000000000 Binary files a/tools/__pycache__/web_tools.cpython-313.pyc and /dev/null differ diff --git a/tools/arxiv_tool.py b/tools/arxiv_tool.py deleted file mode 100644 index fcde528b26fd96c9908224ff1b275dabe3d5f5ae..0000000000000000000000000000000000000000 --- a/tools/arxiv_tool.py +++ /dev/null @@ -1,197 +0,0 @@ -""" -arXiv search tool for the GAIA agent. - -This module provides a tool for searching academic papers on arXiv.org. -It uses the arxiv Python package to query the arXiv API and retrieve -paper information. - -The tool handles API responses and errors, and formats results in a -consistent way for use by the GAIA agent. -""" - -import logging -import time -import traceback -from typing import Dict, Any, List, Optional, Union, Tuple -from datetime import datetime - -try: - import arxiv -except ImportError: - arxiv = None - -from agent.config import get_tool_config - -logger = logging.getLogger("gaia_agent.tools.arxiv") - -class ArxivSearchTool: - """Tool for searching academic papers on arXiv.""" - - def __init__(self, config: Optional[Dict[str, Any]] = None): - """ - Initialize the arXiv search tool. - - Args: - config: Optional configuration dictionary - """ - self.config = config or get_tool_config().get("arxiv", {}) - self.max_results = self.config.get("max_results", 3) - - if arxiv is None: - logger.warning("arXiv package not installed. Install with: pip install arxiv") - - def search(self, query: str, max_results: Optional[int] = None) -> List[Dict[str, Any]]: - """ - Search arXiv for papers matching the query. - - Args: - query: The search query - max_results: Maximum number of results to return (overrides config) - - Returns: - List of paper information dictionaries - - Raises: - Exception: If an error occurs during the search - """ - - if arxiv is None: - raise ImportError("arXiv package not installed. Install with: pip install arxiv") - - max_results = max_results or self.max_results - - try: - client = arxiv.Client() - - search = arxiv.Search( - query=query, - max_results=max_results, - sort_by=arxiv.SortCriterion.Relevance - ) - - results = list(client.results(search)) - - - formatted_results = [] - for paper in results: - published = paper.published - if published: - published_str = published.strftime("%Y-%m-%d") - else: - published_str = "Unknown" - - authors = [author.name for author in paper.authors] - authors_str = ", ".join(authors) - - formatted_result = { - "title": paper.title, - "authors": authors_str, - "summary": paper.summary, - "published": published_str, - "url": paper.entry_id, - "pdf_url": paper.pdf_url, - "categories": paper.categories, - "comment": paper.comment, - "journal_ref": paper.journal_ref, - "doi": paper.doi - } - - formatted_results.append(formatted_result) - - return formatted_results - - except Exception as e: - logger.error(f"Error searching arXiv: {str(e)}") - logger.error(traceback.format_exc()) - raise Exception(f"arXiv search failed: {str(e)}") - - def get_paper_by_id(self, paper_id: str) -> Dict[str, Any]: - """ - Get a specific paper by its arXiv ID. - - Args: - paper_id: The arXiv ID (e.g., "2103.00020") - - Returns: - Dictionary containing paper information - - Raises: - Exception: If an error occurs during the retrieval - """ - - if arxiv is None: - raise ImportError("arXiv package not installed. Install with: pip install arxiv") - - try: - client = arxiv.Client() - - search = arxiv.Search(id_list=[paper_id]) - - results = list(client.results(search)) - - if not results: - raise ValueError(f"No paper found with ID: {paper_id}") - - paper = results[0] - - published = paper.published - if published: - published_str = published.strftime("%Y-%m-%d") - else: - published_str = "Unknown" - - authors = [author.name for author in paper.authors] - authors_str = ", ".join(authors) - - result = { - "title": paper.title, - "authors": authors_str, - "summary": paper.summary, - "published": published_str, - "url": paper.entry_id, - "pdf_url": paper.pdf_url, - "categories": paper.categories, - "comment": paper.comment, - "journal_ref": paper.journal_ref, - "doi": paper.doi - } - - return result - - except Exception as e: - logger.error(f"Error getting arXiv paper {paper_id}: {str(e)}") - logger.error(traceback.format_exc()) - raise Exception(f"arXiv paper retrieval failed: {str(e)}") - - def search_by_category(self, category: str, max_results: Optional[int] = None) -> List[Dict[str, Any]]: - """ - Search for recent papers in a specific category. - - Args: - category: The arXiv category (e.g., "cs.AI") - max_results: Maximum number of results to return (overrides config) - - Returns: - List of paper information dictionaries - - Raises: - Exception: If an error occurs during the search - """ - - query = f"cat:{category}" - - return self.search(query, max_results) - -def create_arxiv_search() -> ArxivSearchTool: - """Create an arXiv search tool instance.""" - return ArxivSearchTool() - -if __name__ == "__main__": - logging.basicConfig(level=logging.INFO) - - arxiv_tool = create_arxiv_search() - - try: - results = arxiv_tool.search("large language models") - for i, paper in enumerate(results): - except Exception as e: diff --git a/tools/multimodal_tools.py b/tools/multimodal_tools.py deleted file mode 100644 index 916fbec701825beef8378c573557ee07384157c3..0000000000000000000000000000000000000000 --- a/tools/multimodal_tools.py +++ /dev/null @@ -1,1253 +0,0 @@ -""" -Multimodal tools for the GAIA agent. - -This module provides tools for processing and analyzing various media formats, including: -- Image analysis and description -- Chart/graph interpretation -- Document parsing -- YouTube video analysis and transcript extraction - -All tools handle errors gracefully and provide detailed error messages. -""" - -import logging -import traceback -import json -import os -import tempfile -import re -import time -from typing import Dict, Any, List, Optional, Union, BinaryIO, Tuple -from pathlib import Path -from enum import Enum - -# Configure module-level logger with more detailed format -logger = logging.getLogger("gaia_agent.tools.multimodal") - -# Define error severity levels for better categorization -class ErrorSeverity(Enum): - """Enum for categorizing error severity levels.""" - INFO = "INFO" # Informational, not critical - WARNING = "WARNING" # Potential issue, but operation can continue - ERROR = "ERROR" # Operation failed but system can continue - CRITICAL = "CRITICAL" # System cannot function properly - -try: - from PIL import Image - import numpy as np -except ImportError: - Image = None - np = None - -try: - import pytesseract - import pdf2image - import docx2txt -except ImportError: - pytesseract = None - pdf2image = None - docx2txt = None - -# Import required modules -import requests -from requests.exceptions import RequestException, Timeout, ConnectionError as RequestsConnectionError - -# Try to import YouTube transcript API -try: - from youtube_transcript_api import YouTubeTranscriptApi, TranscriptsDisabled, NoTranscriptFound, VideoUnavailable - # Some exceptions might not be available in all versions - try: - from youtube_transcript_api import NoTranscriptAvailable, TranslationLanguageNotAvailable - except ImportError: - NoTranscriptAvailable = Exception - TranslationLanguageNotAvailable = Exception - - try: - from youtube_transcript_api import CookiePathInvalid, NotTranslatable - except ImportError: - CookiePathInvalid = Exception - NotTranslatable = Exception - - # Define TooManyRequests if not available - TooManyRequests = Exception -except ImportError as e: - logger.error(f"Failed to import youtube_transcript_api: {str(e)}") - YouTubeTranscriptApi = None - TranscriptsDisabled = Exception - NoTranscriptFound = Exception - VideoUnavailable = Exception - NoTranscriptAvailable = Exception - TranslationLanguageNotAvailable = Exception - CookiePathInvalid = Exception - NotTranslatable = Exception - TooManyRequests = Exception - -from agent.config import get_model_config, get_tool_config -from langchain_openai import ChatOpenAI -from langchain.prompts import PromptTemplate -from langchain_core.output_parsers import StrOutputParser - -class ImageAnalyzer: - """Tool for analyzing and describing images.""" - - def __init__(self, config: Optional[Dict[str, Any]] = None): - """ - Initialize the image analyzer. - - Args: - config: Optional configuration dictionary - """ - self.model_config = config or get_model_config() - self.model = ChatOpenAI( - model=self.model_config.get("vision_model", "gpt-4o"), - temperature=self.model_config.get("temperature", 0.1), - max_tokens=self.model_config.get("max_tokens", 4096) - ) - - if Image is None: - logger.warning("PIL not installed. Install with: pip install pillow") - - def analyze_image(self, image_path: str, prompt: Optional[str] = None) -> Dict[str, Any]: - """ - Analyze an image and provide a description. - - Args: - image_path: Path to the image file - prompt: Optional specific prompt for analysis - - Returns: - Dictionary containing the analysis results - - Raises: - Exception: If an error occurs during analysis - """ - - if Image is None: - raise ImportError("PIL not installed. Install with: pip install pillow") - - try: - if not os.path.exists(image_path): - raise FileNotFoundError(f"Image file not found: {image_path}") - - image = Image.open(image_path) - - default_prompt = """Analyze this image in detail. Describe: - 1. The main subject(s) - 2. Important visual elements - 3. Any text visible in the image - 4. The overall context or setting - - Provide your analysis in the following JSON format: - { - "description": "A detailed description of the image", - "subjects": ["List of main subjects"], - "text_content": "Any text visible in the image", - "context": "The overall context or setting", - "tags": ["Relevant tags or categories"] - } - - JSON Response:""" - - analysis_prompt = prompt if prompt else default_prompt - - prompt_template = PromptTemplate.from_template(analysis_prompt) - - chain = prompt_template | self.model | StrOutputParser() - - result = chain.invoke({"image": image}) - - try: - parsed_result = json.loads(result) - return parsed_result - except json.JSONDecodeError: - logger.warning("Image analysis result is not valid JSON, returning as plain text") - return { - "description": result, - "subjects": [], - "text_content": "", - "context": "", - "tags": [] - } - - except Exception as e: - logger.error(f"Error analyzing image: {str(e)}") - logger.error(traceback.format_exc()) - raise Exception(f"Image analysis failed: {str(e)}") - - def detect_objects(self, image_path: str) -> Dict[str, Any]: - """ - Detect and identify objects in an image. - - Args: - image_path: Path to the image file - - Returns: - Dictionary containing detected objects with locations - - Raises: - Exception: If an error occurs during detection - """ - - if Image is None: - raise ImportError("PIL not installed. Install with: pip install pillow") - - try: - if not os.path.exists(image_path): - raise FileNotFoundError(f"Image file not found: {image_path}") - - image = Image.open(image_path) - - detection_prompt = """Detect and identify objects in this image. - - For each object, provide: - 1. The object name/category - 2. A confidence score (0-1) - 3. An approximate location description - - Provide your analysis in the following JSON format: - { - "objects": [ - { - "name": "Object name", - "confidence": 0.95, - "location": "Description of location in the image" - }, - ... - ], - "scene_type": "Indoor/Outdoor/Other", - "object_count": 5 - } - - JSON Response:""" - - prompt_template = PromptTemplate.from_template(detection_prompt) - - chain = prompt_template | self.model | StrOutputParser() - - result = chain.invoke({"image": image}) - - try: - parsed_result = json.loads(result) - return parsed_result - except json.JSONDecodeError: - logger.warning("Object detection result is not valid JSON, returning empty result") - return { - "objects": [], - "scene_type": "Unknown", - "object_count": 0 - } - - except Exception as e: - logger.error(f"Error detecting objects: {str(e)}") - logger.error(traceback.format_exc()) - raise Exception(f"Object detection failed: {str(e)}") - - -class ChartInterpreter: - """Tool for interpreting charts and graphs.""" - - def __init__(self, config: Optional[Dict[str, Any]] = None): - """ - Initialize the chart interpreter. - - Args: - config: Optional configuration dictionary - """ - self.model_config = config or get_model_config() - self.model = ChatOpenAI( - model=self.model_config.get("vision_model", "gpt-4o"), - temperature=self.model_config.get("temperature", 0.1), - max_tokens=self.model_config.get("max_tokens", 4096) - ) - - if Image is None: - logger.warning("PIL not installed. Install with: pip install pillow") - - def interpret_chart(self, chart_path: str) -> Dict[str, Any]: - """ - Interpret a chart or graph image. - - Args: - chart_path: Path to the chart image file - - Returns: - Dictionary containing the interpretation results - - Raises: - Exception: If an error occurs during interpretation - """ - - if Image is None: - raise ImportError("PIL not installed. Install with: pip install pillow") - - try: - if not os.path.exists(chart_path): - raise FileNotFoundError(f"Chart file not found: {chart_path}") - - chart_image = Image.open(chart_path) - - interpretation_prompt = """Interpret this chart or graph in detail. Provide: - - 1. The type of chart/graph (bar, line, pie, scatter, etc.) - 2. The title and axes labels (if present) - 3. The key data points or trends - 4. A summary of the main insights - - Provide your interpretation in the following JSON format: - { - "chart_type": "Type of chart/graph", - "title": "Chart title if present", - "axes": { - "x_axis": "X-axis label and units", - "y_axis": "Y-axis label and units" - }, - "data_points": [ - {"category": "Category name", "value": "Value"} - ], - "trends": ["List of identified trends"], - "insights": "Summary of main insights from the chart", - "confidence": 0.95 - } - - JSON Response:""" - - prompt_template = PromptTemplate.from_template(interpretation_prompt) - - chain = prompt_template | self.model | StrOutputParser() - - result = chain.invoke({"image": chart_image}) - - try: - parsed_result = json.loads(result) - return parsed_result - except json.JSONDecodeError: - logger.warning("Chart interpretation result is not valid JSON, returning as plain text") - return { - "chart_type": "Unknown", - "title": "", - "axes": {"x_axis": "", "y_axis": ""}, - "data_points": [], - "trends": [], - "insights": result, - "confidence": 0.5 - } - - except Exception as e: - logger.error(f"Error interpreting chart: {str(e)}") - logger.error(traceback.format_exc()) - raise Exception(f"Chart interpretation failed: {str(e)}") - - def extract_data(self, chart_path: str) -> Dict[str, Any]: - """ - Extract numerical data from a chart or graph. - - Args: - chart_path: Path to the chart image file - - Returns: - Dictionary containing the extracted data - - Raises: - Exception: If an error occurs during data extraction - """ - - if Image is None: - raise ImportError("PIL not installed. Install with: pip install pillow") - - try: - if not os.path.exists(chart_path): - raise FileNotFoundError(f"Chart file not found: {chart_path}") - - chart_image = Image.open(chart_path) - - extraction_prompt = """Extract the numerical data from this chart or graph. - - Provide the data in a structured format that could be used to recreate the chart. - Be as precise as possible with the numerical values. - - Provide your extraction in the following JSON format: - { - "chart_type": "Type of chart/graph", - "data": [ - {"x": "x-value", "y": "y-value", "category": "category if applicable"} - ], - "data_table": [ - ["Header1", "Header2", "Header3"], - ["Value1", "Value2", "Value3"], - ... - ], - "confidence": 0.95, - "notes": "Any notes about the extraction process or uncertainties" - } - - JSON Response:""" - - prompt_template = PromptTemplate.from_template(extraction_prompt) - - chain = prompt_template | self.model | StrOutputParser() - - result = chain.invoke({"image": chart_image}) - - try: - parsed_result = json.loads(result) - return parsed_result - except json.JSONDecodeError: - logger.warning("Chart data extraction result is not valid JSON, returning empty result") - return { - "chart_type": "Unknown", - "data": [], - "data_table": [], - "confidence": 0.5, - "notes": "Failed to parse extraction result as JSON" - } - - except Exception as e: - logger.error(f"Error extracting data from chart: {str(e)}") - logger.error(traceback.format_exc()) - raise Exception(f"Chart data extraction failed: {str(e)}") - - -class DocumentParser: - """Tool for parsing and extracting information from documents.""" - - def __init__(self, config: Optional[Dict[str, Any]] = None): - """ - Initialize the document parser. - - Args: - config: Optional configuration dictionary - """ - self.config = config or get_tool_config().get("document_parsing", {}) - self.model_config = get_model_config() - self.model = ChatOpenAI( - model=self.model_config.get("text_model", "gpt-4o"), - temperature=self.model_config.get("temperature", 0.1), - max_tokens=self.model_config.get("max_tokens", 4096) - ) - - if pytesseract is None: - logger.warning("Pytesseract not installed. Install with: pip install pytesseract") - if pdf2image is None: - logger.warning("pdf2image not installed. Install with: pip install pdf2image") - if docx2txt is None: - logger.warning("docx2txt not installed. Install with: pip install docx2txt") - - def parse_document(self, document_path: str) -> Dict[str, Any]: - """ - Parse a document and extract its content. - - Args: - document_path: Path to the document file - - Returns: - Dictionary containing the parsed content - - Raises: - Exception: If an error occurs during parsing - """ - - try: - if not os.path.exists(document_path): - raise FileNotFoundError(f"Document file not found: {document_path}") - - file_extension = Path(document_path).suffix.lower() - - if file_extension == '.pdf': - if pdf2image is None or pytesseract is None: - raise ImportError("pdf2image and pytesseract are required for PDF parsing. Install with: pip install pdf2image pytesseract") - text = self._parse_pdf(document_path) - elif file_extension == '.docx': - if docx2txt is None: - raise ImportError("docx2txt is required for DOCX parsing. Install with: pip install docx2txt") - text = docx2txt.process(document_path) - elif file_extension in ['.txt', '.md', '.csv']: - with open(document_path, 'r', encoding='utf-8') as file: - text = file.read() - else: - raise ValueError(f"Unsupported file type: {file_extension}") - - max_length = self.config.get("max_text_length", 10000) - if len(text) > max_length: - text = text[:max_length] + "..." - - summary = self._summarize_text(text) - - return { - "document_path": document_path, - "file_type": file_extension, - "text_content": text, - "summary": summary, - "word_count": len(text.split()), - "character_count": len(text) - } - - except Exception as e: - logger.error(f"Error parsing document: {str(e)}") - logger.error(traceback.format_exc()) - raise Exception(f"Document parsing failed: {str(e)}") - - def extract_structured_data(self, document_path: str, schema: Dict[str, Any]) -> Dict[str, Any]: - """ - Extract structured data from a document based on a schema. - - Args: - document_path: Path to the document file - schema: Schema defining the data to extract - - Returns: - Dictionary containing the extracted structured data - - Raises: - Exception: If an error occurs during extraction - """ - - try: - parsed_doc = self.parse_document(document_path) - text_content = parsed_doc["text_content"] - - schema_str = json.dumps(schema, indent=2) - extraction_prompt = f"""Extract structured data from the following document according to this schema: - - {schema_str} - - Document content: - {text_content} - - Extract the requested information and provide it in a valid JSON format matching the schema. - - JSON Response:""" - - prompt_template = PromptTemplate.from_template(extraction_prompt) - - chain = prompt_template | self.model | StrOutputParser() - - result = chain.invoke({}) - - try: - parsed_result = json.loads(result) - return parsed_result - except json.JSONDecodeError: - logger.warning("Structured data extraction result is not valid JSON, returning as plain text") - return { - "extraction_result": result, - "error": "Failed to parse extraction result as JSON" - } - - except Exception as e: - logger.error(f"Error extracting structured data: {str(e)}") - logger.error(traceback.format_exc()) - raise Exception(f"Structured data extraction failed: {str(e)}") - - def _parse_pdf(self, pdf_path: str) -> str: - """ - Parse a PDF file and extract its text content. - - Args: - pdf_path: Path to the PDF file - - Returns: - Extracted text content - """ - images = pdf2image.convert_from_path(pdf_path) - - text_content = [] - for image in images: - text = pytesseract.image_to_string(image) - text_content.append(text) - - return "\n\n".join(text_content) - - def _summarize_text(self, text: str) -> str: - """ - Generate a summary of text content. - - Args: - text: The text to summarize - - Returns: - Summary of the text - """ - try: - summarization_prompt = """Summarize the following text in a concise manner. - Focus on the key points and main ideas. - - Text: - {text} - - Summary:""" - - prompt_template = PromptTemplate.from_template(summarization_prompt) - - chain = prompt_template | self.model | StrOutputParser() - - summary = chain.invoke({"text": text}) - - return summary - - except Exception as e: - logger.warning(f"Error generating summary: {str(e)}") - return "Summary generation failed." - - -def create_image_analyzer() -> ImageAnalyzer: - """Create an image analyzer instance.""" - return ImageAnalyzer() - -def create_chart_interpreter() -> ChartInterpreter: - """Create a chart interpreter instance.""" - return ChartInterpreter() - -def create_document_parser() -> DocumentParser: - """Create a document parser instance.""" - return DocumentParser() - - -class YouTubeVideoTool: - """Tool for analyzing YouTube videos and extracting transcripts.""" - - def __init__(self, config: Optional[Dict[str, Any]] = None): - """ - Initialize the YouTube video tool. - - Args: - config: Optional configuration dictionary - """ - self.config = config or get_tool_config().get("youtube_video", {}) - self.model_config = get_model_config() - - # Initialize with default values - self.max_retries = self.config.get("max_retries", 3) - self.retry_delay = self.config.get("retry_delay", 2) - self.fallback_enabled = self.config.get("fallback_enabled", True) - self.timeout = self.config.get("request_timeout", 10) - - # Initialize LLM for transcript processing - try: - self.model = ChatOpenAI( - model=self.model_config.get("text_model", "gpt-4o"), - temperature=self.model_config.get("temperature", 0.1), - max_tokens=self.model_config.get("max_tokens", 4096) - ) - except Exception as e: - logger.error(f"Failed to initialize LLM model: {str(e)}", exc_info=True) - self.model = None - - # Check for required dependencies - if YouTubeTranscriptApi is None: - logger.warning("youtube_transcript_api not installed. Install with: pip install youtube-transcript-api") - - def extract_video_id(self, url: str) -> str: - """ - Extract the YouTube video ID from a URL. - - Args: - url: YouTube video URL - - Returns: - YouTube video ID - - Raises: - ValueError: If the video ID cannot be extracted - """ - if not url: - logger.error("Empty URL provided to extract_video_id", extra={"severity": ErrorSeverity.ERROR.value}) - raise ValueError("Empty URL provided") - - logger.debug(f"Attempting to extract video ID from URL: {url}") - - # Regular expressions for different YouTube URL formats - patterns = [ - r'(?:youtube\.com\/watch\?v=|youtu\.be\/|youtube\.com\/embed\/|youtube\.com\/v\/|youtube\.com\/e\/|youtube\.com\/watch\?.*v=|youtube\.com\/watch\?.*&v=)([^&\n?#]+)', - r'youtube\.com\/shorts\/([^&\n?#]+)' - ] - - for pattern in patterns: - match = re.search(pattern, url) - if match: - video_id = match.group(1) - logger.debug(f"Successfully extracted video ID: {video_id}") - return video_id - - logger.error(f"Could not extract video ID from URL: {url}", extra={"severity": ErrorSeverity.ERROR.value}) - raise ValueError(f"Could not extract video ID from URL: {url}") - - def extract_transcript(self, video_id_or_url: str, language: Optional[str] = None) -> Dict[str, Any]: - """ - Extract transcript from a YouTube video. - - Args: - video_id_or_url: YouTube video ID or URL - language: Optional language code for transcript (e.g., 'en', 'fr') - - Returns: - Dictionary containing the transcript and metadata - - Raises: - Exception: If an error occurs during transcript extraction - """ - if YouTubeTranscriptApi is None: - error_msg = "youtube_transcript_api not installed. Install with: pip install youtube-transcript-api" - logger.error(error_msg, extra={"severity": ErrorSeverity.CRITICAL.value}) - return self._create_error_response( - video_id_or_url, - error_msg, - "ImportError", - severity=ErrorSeverity.CRITICAL - ) - - video_id = None - try: - # Check if input is a URL or a video ID - if isinstance(video_id_or_url, str) and ("youtube.com" in video_id_or_url or "youtu.be" in video_id_or_url): - try: - video_id = self.extract_video_id(video_id_or_url) - logger.info(f"Extracted video ID '{video_id}' from URL: {video_id_or_url}") - except ValueError as e: - logger.error(f"Failed to extract video ID: {str(e)}", extra={"severity": ErrorSeverity.ERROR.value}) - return self._create_error_response( - video_id_or_url, - f"Invalid YouTube URL format: {str(e)}", - "ValueError", - severity=ErrorSeverity.ERROR - ) - else: - video_id = video_id_or_url - logger.info(f"Using provided video ID: {video_id}") - - # Validate video ID format - if not self._is_valid_video_id_format(video_id): - logger.warning(f"Video ID '{video_id}' has unusual format", extra={"severity": ErrorSeverity.WARNING.value}) - - # Get video metadata with retries - metadata = self._get_metadata_with_retries(video_id) - - # Get transcript with retries and fallback - transcript_result = self._get_transcript_with_retries_and_fallback(video_id, language) - - # Check if transcript extraction failed - if "error" in transcript_result: - # Merge metadata with error response - error_response = transcript_result - error_response.update({ - "title": metadata.get("title", "Unknown"), - "channel": metadata.get("channel", "Unknown"), - "thumbnail_url": metadata.get("thumbnail_url", ""), - }) - return error_response - - # Format transcript with timestamps - formatted_transcript = self._format_transcript(transcript_result["transcript_list"]) - - # Process transcript to identify speakers (if possible) - processed_transcript = self._process_transcript_with_speakers(formatted_transcript) - - return { - "video_id": video_id, - "title": metadata.get("title", ""), - "channel": metadata.get("channel", ""), - "transcript": formatted_transcript, - "processed_transcript": processed_transcript, - "duration_seconds": metadata.get("duration", 0), - "language": transcript_result.get("language", language or "auto-detected"), - "transcript_source": transcript_result.get("source", "youtube_api") - } - - except TranscriptsDisabled as e: - error_msg = f"Transcripts are disabled for this video: {str(e)}" - logger.error(error_msg, extra={"severity": ErrorSeverity.WARNING.value}) - return self._create_error_response( - video_id or video_id_or_url, - error_msg, - "TranscriptsDisabled", - severity=ErrorSeverity.WARNING, - suggestion="This video has disabled transcripts. Try another video or use a different method to analyze the content." - ) - - except NoTranscriptFound as e: - error_msg = f"No transcript found for this video: {str(e)}" - logger.error(error_msg, extra={"severity": ErrorSeverity.WARNING.value}) - return self._create_error_response( - video_id or video_id_or_url, - error_msg, - "NoTranscriptFound", - severity=ErrorSeverity.WARNING, - suggestion="This video doesn't have captions. Try another video or use a different method to analyze the content." - ) - - except VideoUnavailable as e: - error_msg = f"Video is unavailable: {str(e)}" - logger.error(error_msg, extra={"severity": ErrorSeverity.ERROR.value}) - return self._create_error_response( - video_id or video_id_or_url, - error_msg, - "VideoUnavailable", - severity=ErrorSeverity.ERROR, - suggestion="The video may be private, removed, or region-restricted." - ) - - except TooManyRequests as e: - error_msg = f"Too many requests to YouTube API: {str(e)}" - logger.error(error_msg, extra={"severity": ErrorSeverity.ERROR.value}) - return self._create_error_response( - video_id or video_id_or_url, - error_msg, - "TooManyRequests", - severity=ErrorSeverity.ERROR, - suggestion="Try again later or reduce the frequency of requests." - ) - - except (RequestException, Timeout, RequestsConnectionError) as e: - error_msg = f"Network error while accessing YouTube: {str(e)}" - logger.error(error_msg, extra={"severity": ErrorSeverity.ERROR.value}) - return self._create_error_response( - video_id or video_id_or_url, - error_msg, - type(e).__name__, - severity=ErrorSeverity.ERROR, - suggestion="Check your internet connection and try again." - ) - - except Exception as e: - error_msg = f"Error extracting transcript: {str(e)}" - logger.error(error_msg, exc_info=True, extra={"severity": ErrorSeverity.ERROR.value}) - return self._create_error_response( - video_id if video_id else video_id_or_url, - error_msg, - type(e).__name__, - severity=ErrorSeverity.ERROR - ) - - def _is_valid_video_id_format(self, video_id: str) -> bool: - """ - Check if the video ID has a valid format. - - Args: - video_id: YouTube video ID to validate - - Returns: - True if the format appears valid, False otherwise - """ - # YouTube IDs are typically 11 characters and contain alphanumeric chars, underscores, and hyphens - if not video_id or not isinstance(video_id, str): - return False - - # Check length (most YouTube IDs are 11 characters) - if len(video_id) != 11: - return False - - # Check for valid characters (alphanumeric, underscore, hyphen) - return bool(re.match(r'^[A-Za-z0-9_-]+$', video_id)) - - def _create_error_response( - self, - video_id_or_url: str, - error_message: str, - error_type: str, - severity: ErrorSeverity = ErrorSeverity.ERROR, - suggestion: Optional[str] = None - ) -> Dict[str, Any]: - """ - Create a standardized error response. - - Args: - video_id_or_url: The video ID or URL that caused the error - error_message: Detailed error message - error_type: Type of error - severity: Error severity level - suggestion: Optional suggestion for resolving the error - - Returns: - Dictionary with standardized error information - """ - # Extract video_id if URL was provided - video_id = video_id_or_url - if isinstance(video_id_or_url, str) and ("youtube.com" in video_id_or_url or "youtu.be" in video_id_or_url): - try: - video_id = self.extract_video_id(video_id_or_url) - except ValueError: - # Keep original value if extraction fails - pass - - response = { - "video_id": video_id, - "error": error_message, - "error_type": error_type, - "severity": severity.value, - "success": False - } - - if suggestion: - response["suggestion"] = suggestion - - return response - - def get_video_metadata(self, video_id: str) -> Dict[str, Any]: - """ - Get metadata for a YouTube video. - - Args: - video_id: YouTube video ID - - Returns: - Dictionary containing video metadata - """ - logger.info(f"Getting metadata for video ID: {video_id}") - - try: - # Use YouTube's oEmbed API to get basic metadata - oembed_url = f"https://www.youtube.com/oembed?url=https://www.youtube.com/watch?v={video_id}&format=json" - response = requests.get(oembed_url, timeout=self.timeout) - - if response.status_code == 200: - metadata = response.json() - logger.debug(f"Successfully retrieved metadata for video {video_id}") - return { - "title": metadata.get("title", ""), - "channel": metadata.get("author_name", ""), - "thumbnail_url": metadata.get("thumbnail_url", ""), - "duration": 0 # YouTube's oEmbed API doesn't provide duration - } - else: - logger.warning( - f"Failed to get metadata: HTTP {response.status_code}", - extra={"severity": ErrorSeverity.WARNING.value} - ) - # Fallback to minimal metadata - return { - "title": "Unknown", - "channel": "Unknown", - "thumbnail_url": "", - "duration": 0, - "metadata_error": f"HTTP {response.status_code}" - } - - except Timeout as e: - logger.warning(f"Timeout while fetching video metadata: {str(e)}", extra={"severity": ErrorSeverity.WARNING.value}) - return { - "title": "Unknown (Request Timeout)", - "channel": "Unknown", - "thumbnail_url": "", - "duration": 0, - "metadata_error": f"Request timeout: {str(e)}" - } - - except RequestsConnectionError as e: - logger.warning(f"Connection error while fetching video metadata: {str(e)}", extra={"severity": ErrorSeverity.WARNING.value}) - return { - "title": "Unknown (Connection Error)", - "channel": "Unknown", - "thumbnail_url": "", - "duration": 0, - "metadata_error": f"Connection error: {str(e)}" - } - - except Exception as e: - logger.warning( - f"Error fetching video metadata: {str(e)}", - exc_info=True, - extra={"severity": ErrorSeverity.WARNING.value} - ) - return { - "title": "Unknown", - "channel": "Unknown", - "thumbnail_url": "", - "duration": 0, - "metadata_error": str(e) - } - - def _get_metadata_with_retries(self, video_id: str) -> Dict[str, Any]: - """ - Get video metadata with retry mechanism. - - Args: - video_id: YouTube video ID - - Returns: - Dictionary containing video metadata - """ - for attempt in range(1, self.max_retries + 1): - try: - metadata = self.get_video_metadata(video_id) - if metadata.get("title") != "Unknown" or attempt == self.max_retries: - return metadata - - logger.warning(f"Metadata attempt {attempt} returned unknown title, retrying...") - time.sleep(self.retry_delay) - - except Exception as e: - logger.warning(f"Metadata attempt {attempt} failed: {str(e)}") - if attempt < self.max_retries: - time.sleep(self.retry_delay) - else: - return { - "title": "Unknown", - "channel": "Unknown", - "thumbnail_url": "", - "duration": 0, - "metadata_error": f"All {self.max_retries} attempts failed" - } - - # This should not be reached - return { - "title": "Unknown", - "channel": "Unknown", - "thumbnail_url": "", - "duration": 0 - } - - def _get_transcript_with_retries_and_fallback(self, video_id: str, language: Optional[str] = None) -> Dict[str, Any]: - """ - Get transcript with retries and fallback mechanisms. - - Args: - video_id: YouTube video ID - language: Optional language code - - Returns: - Dictionary with transcript list and metadata or error information - """ - logger.info(f"Attempting to get transcript for video {video_id} with language {language or 'auto'}") - - # Try primary method with retries - for attempt in range(1, self.max_retries + 1): - try: - logger.debug(f"Transcript extraction attempt {attempt}/{self.max_retries}") - - if language: - transcript_list = YouTubeTranscriptApi.get_transcript(video_id, languages=[language]) - return {"transcript_list": transcript_list, "language": language, "source": "youtube_api"} - else: - transcript_list = YouTubeTranscriptApi.get_transcript(video_id) - return {"transcript_list": transcript_list, "source": "youtube_api"} - - except (TranscriptsDisabled, NoTranscriptFound, VideoUnavailable) as e: - # Don't retry for these specific exceptions - logger.warning(f"Transcript unavailable (attempt {attempt}): {str(e)}") - raise - - except Exception as e: - if attempt < self.max_retries: - logger.warning(f"Transcript extraction failed (attempt {attempt}): {str(e)}. Retrying...") - time.sleep(self.retry_delay) - else: - logger.error(f"All {self.max_retries} transcript extraction attempts failed: {str(e)}") - - # Try fallback methods if enabled - if self.fallback_enabled: - return self._try_transcript_fallbacks(video_id, language) - - # If no fallbacks or all fallbacks failed - raise - - # This should not be reached due to the exception handling above - return {"error": "Failed to extract transcript after retries", "error_type": "ExtractionFailure"} - - def _try_transcript_fallbacks(self, video_id: str, language: Optional[str] = None) -> Dict[str, Any]: - """ - Try alternative methods to get transcript when primary method fails. - - Args: - video_id: YouTube video ID - language: Optional language code - - Returns: - Dictionary with transcript list and metadata or error information - """ - logger.info(f"Trying fallback methods for video {video_id}") - - fallback_methods = [ - self._try_fallback_auto_generated, - self._try_fallback_alternative_language, - ] - - for method in fallback_methods: - try: - result = method(video_id, language) - if result and "transcript_list" in result: - logger.info(f"Fallback method {method.__name__} succeeded") - return result - except Exception as e: - logger.warning(f"Fallback method {method.__name__} failed: {str(e)}") - continue - - # All fallbacks failed - return { - "error": "Failed to extract transcript with all available methods", - "error_type": "AllFallbacksFailed", - "severity": ErrorSeverity.ERROR.value, - "suggestion": "This video may not have any available transcripts or captions." - } - - def _try_fallback_auto_generated(self, video_id: str, language: Optional[str] = None) -> Dict[str, Any]: - """ - Try to get auto-generated transcript as fallback. - - Args: - video_id: YouTube video ID - language: Optional language code - - Returns: - Dictionary with transcript list and metadata or None if failed - """ - logger.info(f"Trying to get auto-generated transcript for {video_id}") - try: - # Try to get auto-generated transcript - transcript_list = YouTubeTranscriptApi.get_transcript( - video_id, - languages=['en'] if not language else [language], - continue_after_error=True - ) - - if transcript_list: - return { - "transcript_list": transcript_list, - "source": "auto_generated", - "language": language or "en" - } - return None - except Exception as e: - logger.warning(f"Auto-generated transcript fallback failed: {str(e)}") - return None - - def _try_fallback_alternative_language(self, video_id: str, language: Optional[str] = None) -> Dict[str, Any]: - """ - Try to get transcript in alternative languages as fallback. - - Args: - video_id: YouTube video ID - language: Optional language code - - Returns: - Dictionary with transcript list and metadata or None if failed - """ - logger.info(f"Trying to get transcript in alternative languages for {video_id}") - try: - # Get available transcript languages - transcript_list = YouTubeTranscriptApi.list_transcripts(video_id) - - # Try each available language - for transcript in transcript_list: - try: - fetched_transcript = transcript.fetch() - actual_language = transcript.language_code - - logger.info(f"Found transcript in language: {actual_language}") - - return { - "transcript_list": fetched_transcript, - "source": "alternative_language", - "language": actual_language - } - except Exception as e: - logger.debug(f"Failed to fetch transcript in {transcript.language_code}: {str(e)}") - continue - - return None - except Exception as e: - logger.warning(f"Alternative language fallback failed: {str(e)}") - return None - - def _format_transcript(self, transcript_list: List[Dict[str, Any]]) -> str: - """ - Format transcript with timestamps. - - Args: - transcript_list: List of transcript segments - - Returns: - Formatted transcript string - """ - if not transcript_list: - logger.warning("Empty transcript list provided to _format_transcript") - return "" - - formatted_lines = [] - - try: - for item in transcript_list: - start_time = item.get('start', 0) - text = item.get('text', '') - - # Convert seconds to MM:SS format - minutes = int(start_time // 60) - seconds = int(start_time % 60) - timestamp = f"[{minutes:02d}:{seconds:02d}]" - - formatted_lines.append(f"{timestamp} {text}") - - return "\n".join(formatted_lines) - - except Exception as e: - logger.error(f"Error formatting transcript: {str(e)}", exc_info=True) - # Return a basic format as fallback - return "\n".join([f"[??:??] {item.get('text', '')}" for item in transcript_list]) - - def _process_transcript_with_speakers(self, transcript: str) -> str: - """ - Process transcript to identify speakers if possible. - - Args: - transcript: Formatted transcript string - - Returns: - Processed transcript with speaker identification if possible - """ - if not transcript: - logger.warning("Empty transcript provided to _process_transcript_with_speakers") - return "" - - if not self.model: - logger.warning("LLM model not available for speaker identification") - return transcript - - try: - # Check if transcript is long enough to process - if len(transcript) < 100: - logger.info("Transcript too short for speaker identification") - return transcript - - # Use LLM to identify potential speakers in the transcript - logger.info("Processing transcript to identify speakers") - - # Limit transcript length to avoid token limits - max_length = 8000 # Adjust based on model token limits - if len(transcript) > max_length: - logger.warning(f"Transcript too long ({len(transcript)} chars), truncating to {max_length} chars for speaker identification") - processed_transcript = transcript[:max_length] + "..." - else: - processed_transcript = transcript - - prompt = f""" - Analyze this YouTube video transcript and identify different speakers if possible. - Format the transcript with speaker labels (e.g., "Speaker 1:", "Speaker 2:"). - If you cannot confidently identify different speakers, return the transcript as is. - - Transcript: - {processed_transcript} - - Processed transcript with speakers: - """ - - prompt_template = PromptTemplate.from_template(prompt) - chain = prompt_template | self.model | StrOutputParser() - - # Set timeout to avoid hanging - result = chain.invoke({}) - - # Validate result - if not result or len(result) < len(processed_transcript) / 2: - logger.warning("Speaker identification returned suspiciously short result, using original transcript") - return transcript - - logger.info("Successfully processed transcript with speaker identification") - return result - - except Exception as e: - logger.error(f"Error processing transcript with speakers: {str(e)}", exc_info=True, - extra={"severity": ErrorSeverity.WARNING.value}) - return transcript # Return original transcript if processing fails - - -def create_youtube_video_tool() -> YouTubeVideoTool: - """Create a YouTube video tool instance.""" - return YouTubeVideoTool() \ No newline at end of file diff --git a/tools/perplexity_tool.py b/tools/perplexity_tool.py deleted file mode 100644 index 848cc1dba5e162510464e2cb8d830d2580874164..0000000000000000000000000000000000000000 --- a/tools/perplexity_tool.py +++ /dev/null @@ -1,199 +0,0 @@ -""" -Perplexity API tool for the GAIA agent. - -This module provides a tool for querying the Perplexity API with the sonar-reasoning model, -which is particularly useful for real-time web search and reasoning tasks. - -The tool handles API responses and errors, and extracts content and citations from responses. -""" - -import os -import json -import logging -import urllib.request -import urllib.error -from typing import Dict, Any, List, Optional, Union - -logger = logging.getLogger("gaia_agent.tools.perplexity") - -if not logger.handlers: - handler = logging.StreamHandler() - handler.setFormatter(logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')) - logger.addHandler(handler) - logger.setLevel(logging.INFO) - -class PerplexityTool: - """Tool for querying the Perplexity API.""" - - def __init__(self, config: Optional[Dict[str, Any]] = None): - """ - Initialize the Perplexity API tool. - - Args: - config: Optional configuration dictionary - """ - self.config = config or {} - - self.api_key = self.config.get("api_key") or os.getenv("PERPLEXITY_API_KEY") - if not self.api_key: - logger.warning("Perplexity API key not found in config or environment variables") - - self.api_url = "https://api.perplexity.ai" - self.model = "sonar-reasoning" - self.timeout = 30 - - self.endpoint = f"{self.api_url}/chat/completions" - - - def query(self, - question: str, - max_tokens: int = 500, - system_prompt: Optional[str] = None) -> Dict[str, Any]: - """ - Send a query to the Perplexity API. - - Args: - question: The question to ask - max_tokens: Maximum number of tokens in the response - system_prompt: Optional system prompt - - Returns: - Dictionary containing the query results - - Raises: - Exception: If an error occurs during the query - """ - - if not self.api_key: - raise Exception("Perplexity API key not available") - - try: - messages = [] - - if system_prompt: - messages.append({"role": "system", "content": system_prompt}) - - messages.append({"role": "user", "content": question}) - - payload = { - "model": self.model, - "messages": messages, - "max_tokens": max_tokens - } - - payload_bytes = json.dumps(payload).encode('utf-8') - - headers = { - "Authorization": f"Bearer {self.api_key}", - "Content-Type": "application/json" - } - - req = urllib.request.Request( - self.endpoint, - data=payload_bytes, - headers=headers, - method="POST" - ) - - with urllib.request.urlopen(req, timeout=self.timeout) as response: - response_data = response.read().decode('utf-8') - api_response = json.loads(response_data) - - content = self._extract_content(api_response) - citations = self._extract_citations(api_response) - usage = self._get_usage(api_response) - - result = { - "query": question, - "content": content, - "citations": citations, - "usage": usage, - "model": self.model, - "raw_response": api_response - } - - - return result - - except urllib.error.HTTPError as e: - logger.error(f"HTTP Error {e.code} when querying Perplexity API: {e.reason}") - try: - error_data = e.read().decode('utf-8') - logger.error(f"Error details: {error_data}") - except: - logger.error("Could not read error details") - raise Exception(f"Perplexity API HTTP error: {e.code} {e.reason}") - - except urllib.error.URLError as e: - logger.error(f"URL Error when querying Perplexity API: {e.reason}") - raise Exception(f"Perplexity API connection error: {e.reason}") - - except Exception as e: - logger.error(f"Error querying Perplexity API: {str(e)}") - raise Exception(f"Perplexity API error: {str(e)}") - - def search(self, query: str, max_tokens: int = 500) -> Dict[str, Any]: - """ - Search the web using the Perplexity API. - - Args: - query: The search query - max_tokens: Maximum number of tokens in the response - - Returns: - Dictionary containing the search results - - Raises: - Exception: If an error occurs during the search - """ - - system_prompt = """You are a helpful web search assistant. - Search the web for accurate and up-to-date information to answer the user's query. - Provide a comprehensive answer with relevant facts and details. - Include citations to your sources.""" - - return self.query(query, max_tokens, system_prompt) - - def _extract_content(self, response: Dict[str, Any]) -> Optional[str]: - """ - Extract content from a Perplexity API response. - - Args: - response: The API response dictionary - - Returns: - The extracted content, or None if not found - """ - if "choices" in response and len(response["choices"]) > 0: - if "message" in response["choices"][0]: - return response["choices"][0]["message"].get("content") - return None - - def _extract_citations(self, response: Dict[str, Any]) -> List[str]: - """ - Extract citations from a Perplexity API response. - - Args: - response: The API response dictionary - - Returns: - List of citation URLs - """ - return response.get("citations", []) - - def _get_usage(self, response: Dict[str, Any]) -> Dict[str, Any]: - """ - Extract usage information from a Perplexity API response. - - Args: - response: The API response dictionary - - Returns: - Dictionary containing usage information - """ - return response.get("usage", {}) - - -def create_perplexity_tool() -> PerplexityTool: - """Create a Perplexity tool instance.""" - return PerplexityTool() \ No newline at end of file diff --git a/tools/reasoning_tools.py b/tools/reasoning_tools.py deleted file mode 100644 index 9df6a0fa6280c422a2a74ae17ab4f9796976c266..0000000000000000000000000000000000000000 --- a/tools/reasoning_tools.py +++ /dev/null @@ -1,428 +0,0 @@ -""" -Reasoning tools for the GAIA agent. - -This module provides tools for complex reasoning tasks, including: -- Step-by-step reasoning -- Mathematical calculations -- Fact verification - -All tools handle errors gracefully and provide detailed error messages. -""" - -import logging -import traceback -import re -import math -import json -from typing import Dict, Any, List, Optional, Union, Tuple, Callable -from datetime import datetime - -import operator -from sympy import symbols, sympify, solve, simplify -import numpy as np - -from agent.config import get_model_config, get_tool_config -from langchain_openai import ChatOpenAI -from langchain.prompts import PromptTemplate -from langchain_core.output_parsers import StrOutputParser - -logger = logging.getLogger("gaia_agent.tools.reasoning") - -class StepByStepReasoner: - """Tool for performing step-by-step reasoning.""" - - def __init__(self, config: Optional[Dict[str, Any]] = None): - """ - Initialize the step-by-step reasoner. - - Args: - config: Optional configuration dictionary - """ - self.model_config = config or get_model_config() - self.model = ChatOpenAI( - model=self.model_config.get("reasoning_model", "gpt-4o"), - temperature=self.model_config.get("temperature", 0.1), - max_tokens=self.model_config.get("max_tokens", 4096) - ) - - def reason(self, question: str, context: Optional[str] = None) -> Dict[str, Any]: - """ - Perform step-by-step reasoning on a question. - - Args: - question: The question to reason about - context: Optional context information - - Returns: - Dictionary containing the reasoning steps and conclusion - - Raises: - Exception: If an error occurs during reasoning - """ - - try: - prompt_template = """You are an expert at step-by-step reasoning. - - Your task is to solve the following problem by breaking it down into clear, logical steps. - - {context_text} - - Question: {question} - - Think through this step-by-step and provide your reasoning in the following JSON format: - {{ - "steps": [ - {{ - "step_number": 1, - "reasoning": "First step of reasoning", - "intermediate_conclusion": "Conclusion from this step" - }}, - ... - ], - "final_conclusion": "The final answer based on all steps", - "confidence": 0.95 // A number between 0 and 1 indicating your confidence - }} - - JSON Response:""" - - context_text = f"Context: {context}" if context else "No additional context provided." - - prompt = PromptTemplate.from_template(prompt_template) - - chain = prompt | self.model | StrOutputParser() - - result = chain.invoke({ - "question": question, - "context_text": context_text - }) - - parsed_result = json.loads(result) - - - return parsed_result - - except Exception as e: - logger.error(f"Error during step-by-step reasoning: {str(e)}") - logger.error(traceback.format_exc()) - raise Exception(f"Step-by-step reasoning failed: {str(e)}") - - def chain_of_thought(self, question: str, context: Optional[str] = None) -> str: - """ - Perform chain-of-thought reasoning and return a textual response. - - Args: - question: The question to reason about - context: Optional context information - - Returns: - String containing the reasoning and conclusion - - Raises: - Exception: If an error occurs during reasoning - """ - - try: - prompt_template = """You are an expert at chain-of-thought reasoning. - - Your task is to solve the following problem by thinking step-by-step. - - {context_text} - - Question: {question} - - Let's think through this step-by-step:""" - - context_text = f"Context: {context}" if context else "No additional context provided." - - prompt = PromptTemplate.from_template(prompt_template) - - chain = prompt | self.model | StrOutputParser() - - result = chain.invoke({ - "question": question, - "context_text": context_text - }) - - - return result - - except Exception as e: - logger.error(f"Error during chain-of-thought reasoning: {str(e)}") - logger.error(traceback.format_exc()) - raise Exception(f"Chain-of-thought reasoning failed: {str(e)}") - - -class MathCalculator: - """Tool for performing mathematical calculations.""" - - def __init__(self): - """Initialize the math calculator.""" - self.operations = { - '+': operator.add, - '-': operator.sub, - '*': operator.mul, - '/': operator.truediv, - '^': operator.pow, - '**': operator.pow, - '%': operator.mod, - '//': operator.floordiv - } - - def calculate(self, expression: str) -> Dict[str, Any]: - """ - Evaluate a mathematical expression. - - Args: - expression: The mathematical expression to evaluate - - Returns: - Dictionary containing the result and steps - - Raises: - Exception: If an error occurs during calculation - """ - - try: - cleaned_expr = self._clean_expression(expression) - - result = float(sympify(cleaned_expr).evalf()) - - if result.is_integer(): - result = int(result) - - return { - "expression": expression, - "result": result, - "steps": [f"Evaluated expression: {cleaned_expr}", f"Result: {result}"] - } - - except Exception as e: - logger.error(f"Error calculating expression: {str(e)}") - logger.error(traceback.format_exc()) - raise Exception(f"Mathematical calculation failed: {str(e)}") - - def solve_equation(self, equation: str, variable: str = 'x') -> Dict[str, Any]: - """ - Solve a mathematical equation for a variable. - - Args: - equation: The equation to solve (e.g., "x + 5 = 10") - variable: The variable to solve for (default: 'x') - - Returns: - Dictionary containing the solution and steps - - Raises: - Exception: If an error occurs during solving - """ - - try: - if '=' in equation: - left_side, right_side = equation.split('=', 1) - equation_sympy = f"({left_side.strip()}) - ({right_side.strip()})" - else: - equation_sympy = equation - - var = symbols(variable) - - expr = sympify(equation_sympy) - solutions = solve(expr, var) - - formatted_solutions = [] - for sol in solutions: - if sol.is_real: - try: - float_sol = float(sol) - if float_sol.is_integer(): - formatted_solutions.append(int(float_sol)) - else: - formatted_solutions.append(float_sol) - except: - formatted_solutions.append(str(sol)) - else: - formatted_solutions.append(str(sol)) - - return { - "equation": equation, - "variable": variable, - "solutions": formatted_solutions, - "steps": [ - f"Parsed equation: {equation}", - f"Rearranged to: {equation_sympy} = 0", - f"Solved for {variable}", - f"Solutions: {', '.join(map(str, formatted_solutions))}" - ] - } - - except Exception as e: - logger.error(f"Error solving equation: {str(e)}") - logger.error(traceback.format_exc()) - raise Exception(f"Equation solving failed: {str(e)}") - - def _clean_expression(self, expression: str) -> str: - """ - Clean a mathematical expression for evaluation. - - Args: - expression: The expression to clean - - Returns: - Cleaned expression - """ - expression = expression.strip() - - replacements = { - 'sin': 'math.sin', - 'cos': 'math.cos', - 'tan': 'math.tan', - 'log': 'math.log', - 'exp': 'math.exp', - 'sqrt': 'math.sqrt', - 'pi': 'math.pi', - 'e': 'math.e' - } - - for old, new in replacements.items(): - expression = re.sub(r'\b' + old + r'\b', new, expression) - - return expression - - -class FactVerifier: - """Tool for verifying facts and claims.""" - - def __init__(self, config: Optional[Dict[str, Any]] = None): - """ - Initialize the fact verifier. - - Args: - config: Optional configuration dictionary - """ - self.model_config = config or get_model_config() - self.model = ChatOpenAI( - model=self.model_config.get("reasoning_model", "gpt-4o"), - temperature=self.model_config.get("temperature", 0.1), - max_tokens=self.model_config.get("max_tokens", 4096) - ) - - def verify(self, claim: str, evidence: Optional[str] = None) -> Dict[str, Any]: - """ - Verify a claim against evidence. - - Args: - claim: The claim to verify - evidence: Optional evidence to use for verification - - Returns: - Dictionary containing verification results - - Raises: - Exception: If an error occurs during verification - """ - - try: - prompt_template = """You are an expert fact checker. - - Your task is to verify the following claim based on the provided evidence. - - Claim: {claim} - - Evidence: {evidence} - - Analyze the claim and evidence carefully. Provide your verification in the following JSON format: - {{ - "is_verifiable": true/false, // Whether the claim can be verified with the given evidence - "verification_result": "supported"/"contradicted"/"insufficient_evidence", - "confidence": 0.95, // A number between 0 and 1 indicating your confidence - "reasoning": "Your step-by-step reasoning process", - "missing_information": "What additional information would help verify this claim" - }} - - JSON Response:""" - - evidence_text = evidence if evidence else "No evidence provided." - - prompt = PromptTemplate.from_template(prompt_template) - - chain = prompt | self.model | StrOutputParser() - - result = chain.invoke({ - "claim": claim, - "evidence": evidence_text - }) - - parsed_result = json.loads(result) - - - return parsed_result - - except Exception as e: - logger.error(f"Error during fact verification: {str(e)}") - logger.error(traceback.format_exc()) - raise Exception(f"Fact verification failed: {str(e)}") - - def compare_claims(self, claim1: str, claim2: str) -> Dict[str, Any]: - """ - Compare two claims for consistency. - - Args: - claim1: The first claim - claim2: The second claim - - Returns: - Dictionary containing comparison results - - Raises: - Exception: If an error occurs during comparison - """ - - try: - prompt_template = """You are an expert at analyzing logical consistency. - - Your task is to compare the following two claims and determine if they are consistent, contradictory, or unrelated. - - Claim 1: {claim1} - - Claim 2: {claim2} - - Analyze the claims carefully. Provide your analysis in the following JSON format: - {{ - "relationship": "consistent"/"contradictory"/"unrelated", - "confidence": 0.95, // A number between 0 and 1 indicating your confidence - "reasoning": "Your step-by-step reasoning process", - "implications": "What the relationship between these claims implies" - }} - - JSON Response:""" - - prompt = PromptTemplate.from_template(prompt_template) - - chain = prompt | self.model | StrOutputParser() - - result = chain.invoke({ - "claim1": claim1, - "claim2": claim2 - }) - - parsed_result = json.loads(result) - - - return parsed_result - - except Exception as e: - logger.error(f"Error during claim comparison: {str(e)}") - logger.error(traceback.format_exc()) - raise Exception(f"Claim comparison failed: {str(e)}") - - -def create_step_by_step_reasoner() -> StepByStepReasoner: - """Create a step-by-step reasoner instance.""" - return StepByStepReasoner() - -def create_math_calculator() -> MathCalculator: - """Create a math calculator instance.""" - return MathCalculator() - -def create_fact_verifier() -> FactVerifier: - """Create a fact verifier instance.""" - return FactVerifier() \ No newline at end of file diff --git a/tools/web_tools.py b/tools/web_tools.py deleted file mode 100644 index a20def5dfece8b498470da309242b5f9dd4ae8d1..0000000000000000000000000000000000000000 --- a/tools/web_tools.py +++ /dev/null @@ -1,1570 +0,0 @@ -""" -Web browsing tools for the GAIA agent. - -This module provides tools for web search, content extraction, and URL navigation. -It includes implementations for: -- Web search using DuckDuckGo and Serper -- Web page content extraction -- URL navigation and scraping -- Result filtering and ranking based on relevance -- Browser-based direct website viewing -- Unified library-based search across multiple providers - -All tools handle errors gracefully and provide detailed error messages. -""" - -import logging -import time -import json -import requests -import os -import re -from typing import Dict, Any, List, Optional, Union, Tuple -from urllib.parse import urlparse, quote_plus -import traceback -import re -from collections import Counter - -from bs4 import BeautifulSoup - -# For DuckDuckGo search -try: - from duckduckgo_search import DDGS -except ImportError: - DDGS = None - -# For arXiv search -try: - import arxiv -except ImportError: - arxiv = None - -from agent.config import ( - get_tool_config, - SERPER_API_KEY, - SERPER_API_URL, - USER_AGENT, - PERPLEXITY_API_KEY -) - -logger = logging.getLogger("gaia_agent.tools.web") - -class WebSearchTool: - """Base class for web search tools.""" - - def __init__(self, config: Optional[Dict[str, Any]] = None): - """ - Initialize the web search tool. - - Args: - config: Optional configuration dictionary - """ - self.config = config or get_tool_config().get("web_search", {}) - self.result_count = self.config.get("result_count", 5) - self.timeout = self.config.get("timeout", 10) - - def search(self, query: str) -> List[Dict[str, str]]: - """ - Search the web for the given query. - - Args: - query: The search query - - Returns: - List of search results - - Raises: - NotImplementedError: This method must be implemented by subclasses - """ - raise NotImplementedError("Subclasses must implement search method") - - def _format_results(self, results: List[Dict[str, Any]]) -> List[Dict[str, Any]]: - """ - Format search results into a standard format. - - Args: - results: Raw search results - - Returns: - Formatted search results - """ - formatted_results = [] - for result in results: - formatted_result = { - "title": result.get("title", ""), - "link": result.get("link", ""), - "snippet": result.get("snippet", "") - } - formatted_results.append(formatted_result) - - return formatted_results[:self.result_count] - - def filter_results(self, results: List[Dict[str, Any]], query: str) -> List[Dict[str, Any]]: - """ - Filter search results based on relevance to the query. - - Args: - results: Search results to filter - query: The original search query - - Returns: - Filtered search results - """ - if not results: - return [] - - # Extract keywords from the query - query_keywords = set(re.findall(r'\b\w+\b', query.lower())) - - # Filter out common words - common_words = {'the', 'a', 'an', 'and', 'or', 'but', 'in', 'on', 'at', 'to', 'for', 'with', 'by', 'about'} - query_keywords = query_keywords - common_words - - filtered_results = [] - for result in results: - title = result.get("title", "").lower() - snippet = result.get("snippet", "").lower() - - # Count keyword occurrences in title and snippet - title_keywords = set(re.findall(r'\b\w+\b', title)) - common_words - snippet_keywords = set(re.findall(r'\b\w+\b', snippet)) - common_words - - # Calculate relevance score - title_matches = len(query_keywords.intersection(title_keywords)) - snippet_matches = len(query_keywords.intersection(snippet_keywords)) - - # Title matches are weighted more heavily - relevance_score = (title_matches * 2) + snippet_matches - - # Add relevance score to result - result["relevance_score"] = relevance_score - - # Only include results with at least some relevance - if relevance_score > 0: - filtered_results.append(result) - # If no relevance found but we have exact phrase matches, include it - elif any(phrase.lower() in title or phrase.lower() in snippet - for phrase in re.findall(r'"([^"]*)"', query)): - result["relevance_score"] = 1 - filtered_results.append(result) - - # Sort by relevance score (descending) - filtered_results.sort(key=lambda x: x.get("relevance_score", 0), reverse=True) - - # If no results passed the filter, return the original results - # but still add relevance scores - if not filtered_results and results: - for result in results: - if "relevance_score" not in result: - result["relevance_score"] = 0 - return results - - return filtered_results - - -class DuckDuckGoSearchTool(WebSearchTool): - """Tool for searching the web using DuckDuckGo.""" - - def __init__(self, config: Optional[Dict[str, Any]] = None): - """ - Initialize the DuckDuckGo search tool. - - Args: - config: Optional configuration dictionary - """ - super().__init__(config) - self.ddg_config = get_tool_config().get("duckduckgo", {}) - self.max_results = self.ddg_config.get("max_results", 5) - self.ddg_timeout = self.ddg_config.get("timeout", 10) - - if DDGS is None: - logger.warning("DuckDuckGo search package not installed. Install with: pip install duckduckgo-search") - - def search(self, query: str) -> List[Dict[str, Any]]: - """ - Search the web using DuckDuckGo. - - Args: - query: The search query - - Returns: - List of search results - - Raises: - Exception: If an error occurs during the search - """ - - if DDGS is None: - raise ImportError("DuckDuckGo search package not installed. Install with: pip install duckduckgo-search") - - try: - # Standard search - with DDGS() as ddgs: - results = list(ddgs.text( - query, - max_results=self.max_results, - timelimit=self.ddg_timeout - )) - - formatted_results = [] - for result in results: - formatted_result = { - "title": result.get("title", ""), - "link": result.get("href", ""), - "snippet": result.get("body", "") - } - formatted_results.append(formatted_result) - - # Filter and rank results by relevance - filtered_results = self.filter_results(formatted_results, query) - - return filtered_results[:self.result_count] - - except Exception as e: - logger.error(f"Error searching DuckDuckGo: {str(e)}") - logger.error(traceback.format_exc()) - # Return empty results instead of raising exception - logger.info(f"Returning empty results due to DuckDuckGo search failure") - return [] - - -class SerperSearchTool(WebSearchTool): - """Tool for searching the web using Serper API.""" - - def __init__(self, config: Optional[Dict[str, Any]] = None): - """ - Initialize the Serper search tool. - - Args: - config: Optional configuration dictionary - """ - super().__init__(config) - self.api_key = SERPER_API_KEY - self.api_url = SERPER_API_URL - - if not self.api_key: - logger.warning("Serper API key not found. Set SERPER_API_KEY environment variable.") - - def search(self, query: str) -> List[Dict[str, Any]]: - """ - Search the web using Serper API. - - Args: - query: The search query - - Returns: - List of search results - - Raises: - Exception: If an error occurs during the search - """ - - if not self.api_key: - logger.warning("Serper API key not found. Set SERPER_API_KEY environment variable.") - return [] - - try: - # Standard search - headers = { - "X-API-KEY": self.api_key, - "Content-Type": "application/json" - } - - payload = { - "q": query, - "num": self.result_count * 2 # Request more results for better filtering - } - - response = requests.post( - self.api_url, - headers=headers, - json=payload, - timeout=self.timeout - ) - - response.raise_for_status() - - data = response.json() - - organic_results = data.get("organic", []) - - formatted_results = [] - for result in organic_results: - formatted_result = { - "title": result.get("title", ""), - "link": result.get("link", ""), - "snippet": result.get("snippet", "") - } - formatted_results.append(formatted_result) - - # Filter and rank results by relevance - filtered_results = self.filter_results(formatted_results, query) - - return filtered_results[:self.result_count] - - except requests.exceptions.RequestException as e: - logger.error(f"Error searching Serper: {str(e)}") - logger.error(traceback.format_exc()) - # Return empty results instead of raising exception - logger.info(f"Returning empty results due to Serper search failure: {str(e)}") - return [] - - except Exception as e: - logger.error(f"Error processing Serper results: {str(e)}") - logger.error(traceback.format_exc()) - # Return empty results instead of raising exception - logger.info(f"Returning empty results due to Serper processing failure: {str(e)}") - return [] - - -class ArxivSearchTool(WebSearchTool): - """Tool for searching academic papers on arXiv.""" - - def __init__(self, config: Optional[Dict[str, Any]] = None): - """ - Initialize the arXiv search tool. - - Args: - config: Optional configuration dictionary - """ - super().__init__(config) - self.arxiv_config = get_tool_config().get("arxiv", {}) - self.max_results = self.arxiv_config.get("max_results", 3) - - if arxiv is None: - logger.warning("arXiv package not installed. Install with: pip install arxiv") - - def search(self, query: str) -> List[Dict[str, Any]]: - """ - Search arXiv for papers matching the query. - - Args: - query: The search query - - Returns: - List of search results - - Raises: - Exception: If an error occurs during the search - """ - - if arxiv is None: - raise ImportError("arXiv package not installed. Install with: pip install arxiv") - - try: - client = arxiv.Client() - - search = arxiv.Search( - query=query, - max_results=self.max_results, - sort_by=arxiv.SortCriterion.Relevance - ) - - results = list(client.results(search)) - - formatted_results = [] - for paper in results: - published = paper.published - if published: - published_str = published.strftime("%Y-%m-%d") - else: - published_str = "Unknown" - - authors = [author.name for author in paper.authors] - authors_str = ", ".join(authors) - - formatted_result = { - "title": paper.title, - "link": paper.entry_id, - "snippet": paper.summary[:200] + "..." if len(paper.summary) > 200 else paper.summary, - "authors": authors_str, - "published": published_str, - "pdf_url": paper.pdf_url, - "categories": paper.categories, - "source": "arxiv" - } - - formatted_results.append(formatted_result) - - # Filter and rank results by relevance - filtered_results = self.filter_results(formatted_results, query) - - return filtered_results[:self.result_count] - - except Exception as e: - logger.error(f"Error searching arXiv: {str(e)}") - logger.error(traceback.format_exc()) - # Return empty results instead of raising exception - logger.info(f"Returning empty results due to arXiv search failure") - return [] - - -class WebContentExtractor: - """Tool for extracting content from web pages.""" - - def __init__(self, config: Optional[Dict[str, Any]] = None): - """ - Initialize the web content extractor. - - Args: - config: Optional configuration dictionary - """ - self.config = config or get_tool_config().get("web_scraping", {}) - self.timeout = self.config.get("timeout", 15) - self.max_content_length = self.config.get("max_content_length", 10000) - self.user_agent = USER_AGENT - - def extract_content(self, url: str) -> Dict[str, Any]: - """ - Extract content from a web page. - - Args: - url: The URL to extract content from - - Returns: - Dictionary containing the extracted content - - Raises: - Exception: If an error occurs during extraction - """ - - try: - parsed_url = urlparse(url) - if not parsed_url.scheme or not parsed_url.netloc: - raise ValueError(f"Invalid URL: {url}") - - headers = {"User-Agent": self.user_agent} - response = requests.get(url, headers=headers, timeout=self.timeout) - response.raise_for_status() - - soup = BeautifulSoup(response.text, "html.parser") - - title = soup.title.string if soup.title else "" - - for script in soup(["script", "style"]): - script.extract() - - text = soup.get_text() - - lines = (line.strip() for line in text.splitlines()) - - chunks = (phrase.strip() for line in lines for phrase in line.split(" ")) - - text = "\n".join(chunk for chunk in chunks if chunk) - - # Extract specific information based on the URL and content - extracted_info = {} - - # Truncate text if it's too long - if len(text) > self.max_content_length: - text = text[:self.max_content_length] + "..." - - links = [] - for link in soup.find_all("a", href=True): - href = link["href"] - if href.startswith("/"): - href = f"{parsed_url.scheme}://{parsed_url.netloc}{href}" - links.append({ - "text": link.get_text().strip(), - "url": href - }) - - metadata = {} - for meta in soup.find_all("meta"): - if meta.get("name") and meta.get("content"): - metadata[meta["name"]] = meta["content"] - - return { - "url": url, - "title": title, - "content": text, - "links": links[:self.config.get("max_links", 10)], - "metadata": metadata, - "extracted_info": extracted_info - } - - except requests.exceptions.RequestException as e: - logger.error(f"Error fetching URL {url}: {str(e)}") - logger.error(traceback.format_exc()) - raise Exception(f"Failed to fetch URL {url}: {str(e)}") - - except Exception as e: - logger.error(f"Error extracting content from {url}: {str(e)}") - logger.error(traceback.format_exc()) - raise Exception(f"Content extraction failed for {url}: {str(e)}") - - -class WebNavigator: - """Tool for navigating and scraping web pages.""" - - def __init__(self, config: Optional[Dict[str, Any]] = None): - """ - Initialize the web navigator. - - Args: - config: Optional configuration dictionary - """ - self.config = config or get_tool_config().get("web_scraping", {}) - self.timeout = self.config.get("timeout", 15) - self.max_links = self.config.get("max_links", 3) - self.user_agent = USER_AGENT - self.content_extractor = WebContentExtractor(config) - - def navigate(self, url: str) -> Dict[str, Any]: - """ - Navigate to a URL and extract its content. - - Args: - url: The URL to navigate to - - Returns: - Dictionary containing the page content - - Raises: - Exception: If an error occurs during navigation - """ - return self.content_extractor.extract_content(url) - - def follow_links(self, url: str, link_pattern: Optional[str] = None) -> List[Dict[str, Any]]: - """ - Navigate to a URL and follow links matching a pattern. - - Args: - url: The starting URL - link_pattern: Optional regex pattern to match links - - Returns: - List of dictionaries containing content from followed links - - Raises: - Exception: If an error occurs during navigation - """ - - try: - initial_page = self.navigate(url) - - links = initial_page.get("links", []) - - if link_pattern: - pattern = re.compile(link_pattern) - links = [link for link in links if pattern.search(link["url"])] - - links = links[:self.max_links] - - results = [initial_page] - for link in links: - try: - link_url = link["url"] - link_content = self.navigate(link_url) - results.append(link_content) - except Exception as e: - logger.warning(f"Error following link {link['url']}: {str(e)}") - - return results - - except Exception as e: - logger.error(f"Error following links from {url}: {str(e)}") - logger.error(traceback.format_exc()) - raise Exception(f"Link following failed for {url}: {str(e)}") - - -# Create a unified browser-based search tool for any website -class BrowserSearchTool: - """Tool for searching any website using browser_action to view content directly.""" - - def __init__(self, config: Optional[Dict[str, Any]] = None): - """ - Initialize the unified browser search tool. - - Args: - config: Optional configuration dictionary - """ - self.config = config or get_tool_config().get("browser_search", {}) - - # Initialize fallback tools and perplexity tool for unified_search - self.fallback_tools = [] - self.perplexity_tool = None - - # Define search URL templates for common websites - self.search_templates = { - "wikipedia": "https://en.wikipedia.org/wiki/Special:Search?search={query}", - "arxiv": "https://arxiv.org/search/?query={query}&searchtype=all", - "nytimes": "https://www.nytimes.com/search?query={query}", - "google": "https://www.google.com/search?q={query}", - "youtube": "https://www.youtube.com/results?search_query={query}", - "github": "https://github.com/search?q={query}", - "twitter": "https://twitter.com/search?q={query}", - "reddit": "https://www.reddit.com/search/?q={query}", - "scholar": "https://scholar.google.com/scholar?q={query}", - "pubmed": "https://pubmed.ncbi.nlm.nih.gov/?term={query}", - "universetoday": "https://www.universetoday.com/?s={query}", - "malko": "https://www.malkocompetition.com/winners?q={query}" - } - - def search(self, query: str, source: Optional[str] = None) -> List[Dict[str, Any]]: - """ - Search a specific website or determine the best site based on the query. - This method is designed to be used with the browser_action tool. - - Args: - query: The search query - source: Optional specific source to search (e.g., "wikipedia", "arxiv", "nytimes") - - Returns: - List of search results with browser_action instructions - """ - try: - # Format the query for URL - search_term = query.replace(" ", "+") - - # Determine the source if not specified - if not source: - source = self._detect_source_from_query(query) - - # Get the search URL - search_url = self._get_search_url(source, search_term) - - # Get source-specific instructions - instructions = self._get_instructions_for_source(source) - - return [{ - "title": f"{source.title()} Search: {query}", - "link": search_url, - "snippet": f"To search {source.title()} for '{query}', use the browser_action tool to open the link.", - "source": source.lower(), - "relevance_score": 10.0, - "instructions": instructions - }] - - except Exception as e: - logger.error(f"Error in BrowserSearchTool: {str(e)}") - logger.error(traceback.format_exc()) - - return [{ - "title": "Browser Search Error", - "link": "https://www.google.com", - "snippet": f"Error searching: {str(e)}", - "source": source or "unknown", - "relevance_score": 0.0, - "error": str(e) - }] - - def _detect_source_from_query(self, query: str) -> str: - """ - Detect the most appropriate source based on the query content. - - Args: - query: The search query - - Returns: - String identifying the best source for this query - """ - query_lower = query.lower() - - # Special handling for GAIA assessment questions - if "spinosaurus" in query_lower and ("wikipedia" in query_lower or "wiki" in query_lower): - return "wikipedia" - elif "universe today" in query_lower or ("nasa" in query_lower and "award" in query_lower): - return "universetoday" - elif "mercedes sosa" in query_lower and "albums" in query_lower: - return "google" - elif "malko competition" in query_lower or "malko" in query_lower: - return "malko" - - # Check for specific website mentions - if "wikipedia" in query_lower or "wiki" in query_lower: - return "wikipedia" - elif "youtube" in query_lower or "video" in query_lower: - return "youtube" - elif "arxiv" in query_lower or "paper" in query_lower or "research" in query_lower: - return "arxiv" - elif "google" in query_lower: - return "google" - elif "scholar" in query_lower or "academic" in query_lower: - return "scholar" - elif "pubmed" in query_lower or "medical" in query_lower: - return "pubmed" - elif "github" in query_lower or "code" in query_lower or "repository" in query_lower: - return "github" - elif "twitter" in query_lower or "tweet" in query_lower: - return "twitter" - elif "reddit" in query_lower: - return "reddit" - elif "news" in query_lower or "nytimes" in query_lower: - return "nytimes" - - # Default fallback - return "google" - - def _get_search_url(self, source: str, query: str) -> str: - """ - Get the search URL for the given source and query. - - Args: - source: The source to search (e.g., "wikipedia", "arxiv") - query: The formatted search query - - Returns: - The complete search URL - """ - template = self.search_templates.get(source, self.search_templates["google"]) - return template.replace("{query}", query) - - def _get_instructions_for_source(self, source: str) -> str: - """ - Get browser_action instructions for the given source. - - Args: - source: The source to get instructions for - - Returns: - Instructions for using browser_action with this source - """ - instructions = { - "wikipedia": "Use browser_action to open the Wikipedia search page and read the article.", - "arxiv": "Use browser_action to open the arXiv search page and download or read papers.", - "google": "Use browser_action to open Google search results and explore relevant links.", - "youtube": "Use browser_action to open YouTube search results and watch videos.", - "github": "Use browser_action to open GitHub search results and explore repositories.", - "twitter": "Use browser_action to open Twitter search results and read tweets.", - "reddit": "Use browser_action to open Reddit search results and read discussions.", - "scholar": "Use browser_action to open Google Scholar search results and read academic papers.", - "pubmed": "Use browser_action to open PubMed search results and read medical research.", - "nytimes": "Use browser_action to open New York Times search results and read news articles." - } - - return instructions.get(source, f"Use browser_action to open the {source} search results.") - - def _is_youtube_video_question(self, query: str) -> bool: - """ - Determine if a query is specifically asking about a YouTube video. - - Args: - query: The search query - - Returns: - True if the query is about a YouTube video, False otherwise - """ - query_lower = query.lower() - - # Check for YouTube URL patterns - if "youtube.com/watch" in query_lower or "youtu.be/" in query_lower: - return True - - # Check for YouTube-related keywords - youtube_keywords = ["youtube video", "youtube transcript", "youtube channel"] - return any(keyword in query_lower for keyword in youtube_keywords) - - def unified_search(self, query: str) -> List[Dict[str, Any]]: - """ - Search for the given query using the most appropriate search tools. - - This method intelligently routes queries to the most appropriate search tools: - 1. It handles YouTube-related queries with the YouTube tool when available - 2. It prioritizes Perplexity for high-quality results when available - 3. It routes Wikipedia-specific queries to the Wikipedia tool - 4. It falls back to other search tools when needed - - Args: - query: The search query - - Returns: - List of search results - """ - # Check for YouTube-related queries first - if self._is_youtube_video_question(query): - # Look for a YouTube tool in the fallback tools - youtube_tool = None - for tool in self.fallback_tools: - if tool.__class__.__name__ == "YouTubeVideoTool": - youtube_tool = tool - break - - if youtube_tool: - try: - logger.info(f"Using YouTube tool for query: {query}") - # Extract video ID or URL from the query - import re - video_id_match = re.search(r'(?:youtube\.com\/watch\?v=|youtu\.be\/)([a-zA-Z0-9_-]+)', query) - if video_id_match: - video_id = video_id_match.group(1) - transcript = youtube_tool.extract_transcript(video_id) - - # Format the YouTube result as a search result - return [{ - "title": f"YouTube Video Transcript: {video_id}", - "link": f"https://www.youtube.com/watch?v={video_id}", - "snippet": transcript[:500] + "..." if len(transcript) > 500 else transcript, - "source": "youtube", - "relevance_score": 10.0, - "full_content": transcript # Include the full transcript - }] - except Exception as e: - logger.warning(f"YouTube tool failed: {str(e)}") - # Continue to other tools - - # Next, try to use Perplexity for all queries if available - # Perplexity provides high-quality results for most questions - if self.perplexity_tool: - try: - logger.info(f"Using Perplexity for query: {query}") - perplexity_results = self.perplexity_tool.search(query) - - # If we got valid results from Perplexity, format them - if perplexity_results and isinstance(perplexity_results, dict) and "content" in perplexity_results: - content = perplexity_results["content"] - - # Format the Perplexity result as a search result - return [{ - "title": "Perplexity AI Search Result", - "link": "https://perplexity.ai/", - "snippet": content[:500] + "..." if len(content) > 500 else content, - "source": "perplexity", - "relevance_score": 10.0, - "full_content": content # Include the full content - }] - except Exception as e: - logger.warning(f"Perplexity search failed: {str(e)}") - # Continue to fallback tools - - # Note: We don't prioritize the Wikipedia tool here anymore - # Perplexity already handles Wikipedia queries well, and we've already tried it above - # If Perplexity failed, we'll fall back to other tools including Wikipedia - - # Fall back to regular search tools - for tool in self.fallback_tools: - try: - results = tool.search(query) - if results: # Only return if we got actual results - return results - except Exception as e: - logger.warning(f"Fallback search tool failed: {str(e)}") - - # If all tools failed, return empty results - logger.warning(f"All search tools failed for query: {query}") - return [] - -def create_duckduckgo_search() -> DuckDuckGoSearchTool: - """Create a DuckDuckGo search tool instance.""" - return DuckDuckGoSearchTool() - -def create_serper_search() -> SerperSearchTool: - """Create a Serper search tool instance.""" - return SerperSearchTool() - -def create_web_content_extractor() -> WebContentExtractor: - """Create a web content extractor instance.""" - return WebContentExtractor() - -def create_web_navigator() -> WebNavigator: - """Create a web navigator instance.""" - return WebNavigator() - -class LibrarySearchTool(WebSearchTool): - """Tool for searching using imported Python libraries (DuckDuckGo and arXiv).""" - - def __init__(self, config: Optional[Dict[str, Any]] = None): - """ - Initialize the library search tool. - - Args: - config: Optional configuration dictionary - """ - super().__init__(config) - self.library_config = get_tool_config().get("library_search", {}) - self.max_results = self.library_config.get("max_results", 5) - self.timeout = self.library_config.get("timeout", 10) - - # Check for required libraries - if DDGS is None: - logger.warning("DuckDuckGo search package not installed. Install with: pip install duckduckgo-search") - - if arxiv is None: - logger.warning("arXiv package not installed. Install with: pip install arxiv") - - def _is_academic_query(self, query: str) -> bool: - """ - Determine if a query is likely to be academic/research-oriented. - - Args: - query: The search query - - Returns: - True if the query appears to be academic, False otherwise - """ - query_lower = query.lower() - - # Check for academic keywords - academic_keywords = [ - "paper", "research", "study", "journal", "publication", "arxiv", - "conference", "proceedings", "thesis", "dissertation", "academic", - "preprint", "article", "scientific", "author", "published", - "doi", "cite", "citation", "references", "bibliography" - ] - - # Check for academic fields - academic_fields = [ - "physics", "mathematics", "computer science", "cs.", "math.", "phys.", - "biology", "chemistry", "neuroscience", "psychology", "economics", - "machine learning", "artificial intelligence", "ai", "ml", "nlp", - "deep learning", "neural network", "quantum", "algorithm", "theorem" - ] - - # Check if query contains academic keywords or fields - has_academic_keyword = any(keyword in query_lower for keyword in academic_keywords) - has_academic_field = any(field in query_lower for field in academic_fields) - - # Check for patterns like "Author et al." or "Author, Year" - has_citation_pattern = bool(re.search(r'\b[A-Z][a-z]+ et al\.', query)) or \ - bool(re.search(r'\b[A-Z][a-z]+,? \(\d{4}\)', query)) - - return has_academic_keyword or has_academic_field or has_citation_pattern - - def search(self, query: str) -> List[Dict[str, Any]]: - """ - Search using the appropriate library based on query type. - - Args: - query: The search query - - Returns: - List of search results - - Raises: - Exception: If an error occurs during the search - """ - # Determine which library to use based on query type - if self._is_academic_query(query): - logger.info(f"Using arXiv for academic query: {query}") - return self._search_arxiv(query) - else: - logger.info(f"Using DuckDuckGo for general query: {query}") - return self._search_duckduckgo(query) - - def _search_duckduckgo(self, query: str) -> List[Dict[str, Any]]: - """ - Search the web using DuckDuckGo library. - - Args: - query: The search query - - Returns: - List of search results - """ - if DDGS is None: - logger.error("DuckDuckGo search package not installed") - return [] - - try: - # Standard search - with DDGS() as ddgs: - results = list(ddgs.text( - query, - max_results=self.max_results, - timelimit=self.timeout - )) - - formatted_results = [] - for result in results: - formatted_result = { - "title": result.get("title", ""), - "link": result.get("href", ""), - "snippet": result.get("body", ""), - "source": "duckduckgo" - } - formatted_results.append(formatted_result) - - # Filter and rank results by relevance - filtered_results = self.filter_results(formatted_results, query) - - return filtered_results[:self.result_count] - - except Exception as e: - logger.error(f"Error searching DuckDuckGo: {str(e)}") - logger.error(traceback.format_exc()) - return [] - - def _search_arxiv(self, query: str) -> List[Dict[str, Any]]: - """ - Search academic papers using arXiv library. - - Args: - query: The search query - - Returns: - List of search results - """ - if arxiv is None: - logger.error("arXiv package not installed") - return [] - - try: - # Clean the query for arXiv search - # Remove special characters that might cause issues with arXiv API - clean_query = re.sub(r'[^\w\s\-\+\:\(\)]', '', query) - - # Search arXiv - search = arxiv.Search( - query=clean_query, - max_results=self.max_results, - sort_by=arxiv.SortCriterion.Relevance - ) - - results = [] - for paper in search.results(): - # Format authors - authors = ", ".join([author.name for author in paper.authors]) - - # Format abstract (snippet) - abstract = paper.summary.replace("\n", " ") - if len(abstract) > 300: - abstract = abstract[:300] + "..." - - result = { - "title": paper.title, - "link": paper.entry_id, - "snippet": abstract, - "authors": authors, - "published": paper.published.strftime("%Y-%m-%d") if paper.published else "", - "pdf_url": paper.pdf_url, - "source": "arxiv", - "categories": [cat for cat in paper.categories], - "relevance_score": 1 # Default score, will be updated by filter_results - } - results.append(result) - - # Filter and rank results by relevance - filtered_results = self.filter_results(results, query) - - return filtered_results[:self.result_count] - - except Exception as e: - logger.error(f"Error searching arXiv: {str(e)}") - logger.error(traceback.format_exc()) - return [] -def calculate_query_relevance(text: str, query: str) -> float: - """ - Calculate the relevance of a text to a query. - - This function computes a relevance score between 0.0 and 1.0 based on: - 1. Keyword matching - 2. Phrase matching - 3. Term frequency - - Args: - text: The text to evaluate - query: The query to compare against - - Returns: - Float between 0.0 and 1.0 representing relevance score - """ - if not text or not query: - return 0.0 - - # Normalize text and query - text_lower = text.lower() - query_lower = query.lower() - - # Extract keywords from query (remove common words) - common_words = {'the', 'a', 'an', 'and', 'or', 'but', 'in', 'on', 'at', 'to', 'for', 'with', 'by', 'about'} - query_words = [word for word in re.findall(r'\b\w+\b', query_lower) if word not in common_words] - - # Count keyword matches - keyword_matches = sum(1 for word in query_words if word in text_lower) - keyword_score = keyword_matches / max(len(query_words), 1) - - # Check for exact phrases (quoted or not) - phrases = re.findall(r'"([^"]*)"', query) or [query] - phrase_matches = sum(1 for phrase in phrases if phrase.lower() in text_lower) - phrase_score = phrase_matches / len(phrases) - - # Calculate term frequency - term_counts = Counter(re.findall(r'\b\w+\b', text_lower)) - query_term_freq = sum(term_counts.get(word, 0) for word in query_words) - term_freq_score = min(1.0, query_term_freq / max(len(text_lower.split()), 1) * 5) - - # Combine scores with weights - final_score = (keyword_score * 0.5) + (phrase_score * 0.3) + (term_freq_score * 0.2) - - return final_score - -def create_perplexity_tool(): - """ - Create a Perplexity tool instance. - - This function imports the PerplexityTool from tools.perplexity_tool - and creates an instance with default configuration. - - Returns: - PerplexityTool: An instance of the Perplexity tool - """ - from tools.perplexity_tool import PerplexityTool - return PerplexityTool() - -def create_library_search() -> LibrarySearchTool: - """ - Create a library search tool instance that uses Python libraries. - """ - return LibrarySearchTool() -def create_wikipedia_search(): - """ - Create a Wikipedia search tool instance. - - Note: This is a placeholder function. The actual WikipediaSearchTool - implementation is not available in this version. - """ - logger.warning("WikipediaSearchTool is not implemented in this version") - return None - -class EnhancedWebSearchTool: - """ - Tool for enhanced web search that intelligently routes queries to appropriate search tools. - - This is a simplified implementation to support the ApiSearchTool. - """ - - def __init__(self, config: Optional[Dict[str, Any]] = None): - """ - Initialize the enhanced web search tool. - - Args: - config: Optional configuration dictionary - """ - self.config = config or {} - self.fallback_tools = [] - - def add_fallback_tool(self, tool): - """ - Add a fallback search tool. - - Args: - tool: The search tool to add - """ - if tool is not None: - self.fallback_tools.append(tool) - - def search(self, query: str) -> List[Dict[str, Any]]: - """ - Search using the most appropriate tool based on the query. - - Args: - query: The search query - - Returns: - List of search results - """ - for tool in self.fallback_tools: - try: - results = tool.search(query) - if results: - return results - except Exception as e: - logger.warning(f"Fallback search tool failed: {str(e)}") - - return [] - -def create_enhanced_web_search(): - """ - Create an enhanced web search tool instance that intelligently routes GAIA assessment questions. - - This tool prioritizes the ApiSearchTool which has been optimized for GAIA assessment questions. - The ApiSearchTool intelligently selects between Perplexity and Serper APIs based on the query type - and includes special handling for specific GAIA assessment questions: - - - "What albums did Mercedes Sosa release between 2000 and 2009?" - Uses Perplexity with enhanced query - - "Who nominated the Spinosaurus article for featured status on Wikipedia?" - Uses Serper with Wikipedia focus - - "What is the NASA award number mentioned in the Universe Today article about exoplanet research?" - Uses Serper - - "Who are the recent recipients of the Malko Competition?" - Uses Perplexity with enhanced query - - Returns: - EnhancedWebSearchTool: An instance of the enhanced web search tool - """ - config = get_tool_config().get("enhanced_web_search", {}) - enhanced_tool = EnhancedWebSearchTool(config) - - # Try to add ApiSearchTool first (highest priority) - # This tool has been optimized for GAIA assessment questions - try: - api_search_tool = create_api_search() - enhanced_tool.add_fallback_tool(api_search_tool) - logger.info("Added ApiSearch tool to EnhancedWebSearchTool (optimized for GAIA assessment)") - except Exception as e: - logger.warning(f"Failed to add ApiSearch tool: {str(e)}") - - # Fall back to individual API tools if ApiSearchTool fails - # Try to add Perplexity - from tools.perplexity_tool import create_perplexity_tool - try: - perplexity_api_key = os.environ.get("PERPLEXITY_API_KEY") - if perplexity_api_key: - perplexity_tool = create_perplexity_tool() - enhanced_tool.add_fallback_tool(perplexity_tool) - logger.info("Added Perplexity tool to EnhancedWebSearchTool") - else: - logger.warning("Perplexity API key not available, skipping Perplexity tool") - except Exception as e: - logger.warning(f"Failed to add Perplexity tool: {str(e)}") - - # Add YouTube tool for video-related queries - from tools.multimodal_tools import create_youtube_video_tool - try: - youtube_tool = create_youtube_video_tool() - enhanced_tool.add_fallback_tool(youtube_tool) - logger.info("Added YouTube tool to EnhancedWebSearchTool") - except Exception as e: - logger.warning(f"Failed to add YouTube tool: {str(e)}") - - # Add Wikipedia tool (good for Wikipedia-specific questions) - try: - wikipedia_tool = create_wikipedia_search() - if wikipedia_tool: - enhanced_tool.add_fallback_tool(wikipedia_tool) - logger.info("Added Wikipedia tool to EnhancedWebSearchTool") - else: - logger.warning("Wikipedia tool not available, skipping") - except Exception as e: - logger.warning(f"Failed to add Wikipedia fallback: {str(e)}") - - # Add Serper only if ApiSearchTool failed (to avoid duplication) - if not any(isinstance(tool, ApiSearchTool) for tool in enhanced_tool.fallback_tools): - try: - serper_api_key = os.environ.get("SERPER_API_KEY") - if serper_api_key: - serper_tool = create_serper_search() - enhanced_tool.add_fallback_tool(serper_tool) - logger.info("Added Serper tool to EnhancedWebSearchTool") - else: - logger.warning("Serper API key not available, skipping Serper tool") - except Exception as e: - logger.warning(f"Failed to add Serper fallback: {str(e)}") - - # Add LibrarySearchTool (uses both DuckDuckGo and arXiv based on query type) - try: - library_tool = create_library_search() - enhanced_tool.add_fallback_tool(library_tool) - logger.info("Added LibrarySearch tool to EnhancedWebSearchTool") - except Exception as e: - logger.warning(f"Failed to add LibrarySearch tool: {str(e)}") - - # If LibrarySearchTool fails, fall back to DuckDuckGo directly - try: - duckduckgo_tool = create_duckduckgo_search() - enhanced_tool.add_fallback_tool(duckduckgo_tool) - logger.info("Added DuckDuckGo tool to EnhancedWebSearchTool") - except Exception as e: - logger.warning(f"Failed to add DuckDuckGo fallback: {str(e)}") - - return enhanced_tool - - -class ApiSearchTool(WebSearchTool): - """ - Tool for searching using external API services (Perplexity and Serper). - - This tool intelligently selects between Perplexity API (sonar-reasoning model) - and Serper API (Google search results) based on the query type. Complex, reasoning-based - queries are routed to Perplexity, while factual and simple queries go to Serper. - - The tool requires API keys for both services and internet access. It provides - higher quality results than traditional web search but depends on external services. - - API keys must be set in environment variables: - - PERPLEXITY_API_KEY: For accessing Perplexity's sonar-reasoning model - - SERPER_API_KEY: For accessing Google search results via Serper - """ - - def __init__(self, config: Optional[Dict[str, Any]] = None): - """ - Initialize the API search tool. - - Args: - config: Optional configuration dictionary with settings for the API search tool - """ - super().__init__(config) - self.api_config = config or get_tool_config().get("api_search", {}) - - # Initialize API keys from environment variables or config - # First check environment variables - self.perplexity_api_key = os.environ.get("PERPLEXITY_API_KEY", "") - self.serper_api_key = os.environ.get("SERPER_API_KEY", "") - - # If not found in environment, use config values - if not self.perplexity_api_key: - perplexity_config = get_tool_config().get("perplexity", {}) - self.perplexity_api_key = perplexity_config.get("api_key", PERPLEXITY_API_KEY) - - if not self.serper_api_key: - serper_config = get_tool_config().get("serper", {}) - self.serper_api_key = serper_config.get("api_key", SERPER_API_KEY) - - # Check for API keys and log warnings if missing - if not self.perplexity_api_key: - logger.warning("Perplexity API key not found. Set PERPLEXITY_API_KEY environment variable.") - - if not self.serper_api_key: - logger.warning("Serper API key not found. Set SERPER_API_KEY environment variable.") - - # Initialize API tools - self.perplexity_tool = None - self.serper_tool = None - - # Try to initialize Perplexity tool - if self.perplexity_api_key: - try: - from tools.perplexity_tool import create_perplexity_tool - self.perplexity_tool = create_perplexity_tool() - logger.info("Perplexity tool initialized successfully") - except Exception as e: - logger.error(f"Failed to initialize Perplexity tool: {str(e)}") - logger.debug(traceback.format_exc()) - - # Try to initialize Serper tool - if self.serper_api_key: - try: - self.serper_tool = SerperSearchTool() - logger.info("Serper tool initialized successfully") - except Exception as e: - logger.error(f"Failed to initialize Serper tool: {str(e)}") - logger.debug(traceback.format_exc()) - - def search(self, query: str) -> List[Dict[str, Any]]: - """ - Search using the most appropriate API based on the query type. - - This method intelligently routes queries to either Perplexity or Serper - based on the complexity and nature of the query. Complex, reasoning-based - queries go to Perplexity, while factual and simple queries go to Serper. - - The method analyzes the query to determine the most appropriate search API - based on the query complexity and content. - - Args: - query: The search query - - Returns: - List of search results with standardized format - - Raises: - Exception: If an error occurs during the search process - """ - # Check if we have any API tools available - if not self.perplexity_tool and not self.serper_tool: - logger.error("No API search tools available. Set PERPLEXITY_API_KEY or SERPER_API_KEY environment variables.") - return [] - - # Determine which API to use based on query type - - # Determine which API to use based on query type - if self._is_complex_query(query) and self.perplexity_tool: - logger.info(f"Using Perplexity API for complex query: {query}") - try: - results = self._search_with_perplexity(query) - if results: - return results - # If Perplexity returns empty results, fall back to Serper - logger.info(f"Perplexity returned empty results, falling back to Serper for query: {query}") - except Exception as e: - logger.error(f"Perplexity search failed: {str(e)}, falling back to Serper") - # Fall back to Serper on exception - - # Fall back to Serper if available - if self.serper_tool: - logger.info(f"Falling back to Serper API for query: {query}") - return self._search_with_serper(query) - - # For non-complex queries or if Perplexity is not available - if self.serper_tool: - logger.info(f"Using Serper API for query: {query}") - return self._search_with_serper(query) - elif self.perplexity_tool: - logger.info(f"Using Perplexity API as fallback: {query}") - return self._search_with_perplexity(query) - else: - logger.error("No API search tools available for this query") - return [] - - def _is_complex_query(self, query: str) -> bool: - """ - Determine if a query is complex and would benefit from Perplexity's reasoning capabilities. - - This method analyzes the query to determine if it requires reasoning, explanation, - or detailed analysis that would benefit from Perplexity's sonar-reasoning model. - - Args: - query: The search query - - Returns: - True if the query is complex, False otherwise - """ - query_lower = query.lower() - - # Check for question words that indicate reasoning or explanation is needed - question_words = [ - "why", "how", "explain", "what is", "what are", "what happens", - "compare", "difference between", "pros and cons", "advantages", - "disadvantages", "analyze", "evaluate", "summarize", "describe", - "reason", "cause", "effect", "impact", "influence", "relationship" - ] - - # Check for complex query indicators that suggest detailed analysis is required - complex_indicators = [ - "in detail", "step by step", "comprehensive", "thoroughly", "in depth", - "reasoning", "analysis", "implications", "consequences", "relationship between", - "impact of", "effects of", "causes of", "explain why", "explain how", - "compare and contrast", "similarities and differences", "advantages and disadvantages", - "elaborate on", "provide context", "historical perspective", "future implications" - ] - - # Check if query contains question words or complex indicators - has_question_word = any(word in query_lower for word in question_words) - has_complex_indicator = any(indicator in query_lower for indicator in complex_indicators) - - # Check if query is a long, detailed question (more than 10 words) - is_long_query = len(query.split()) > 10 - - return has_question_word or has_complex_indicator or is_long_query - - def _search_with_perplexity(self, query: str) -> List[Dict[str, Any]]: - """ - Search using the Perplexity API with sonar-reasoning model. - - This method sends the query to Perplexity's API and formats the response - into a standardized search result format. It includes special handling for - GAIA assessment questions to ensure optimal results. - - Args: - query: The search query - - Returns: - List of search results with Perplexity's response - """ - try: - if not self.perplexity_tool: - logger.error("Perplexity tool not initialized") - return [] - - # Use Perplexity's search method - perplexity_results = self.perplexity_tool.search(query) - - # Check if we got valid results - if not perplexity_results or not isinstance(perplexity_results, dict) or "content" not in perplexity_results: - logger.warning("Invalid or empty results from Perplexity API") - return [] - - # Extract content and citations - content = perplexity_results.get("content", "") - citations = perplexity_results.get("citations", []) - - # Format as search results - formatted_result = { - "title": "Perplexity AI Search Result", - "link": "https://perplexity.ai/", - "snippet": content[:300] + "..." if len(content) > 300 else content, - "source": "perplexity", - "relevance_score": 10.0, # High relevance for Perplexity results - "full_content": content, - "citations": citations - } - - return [formatted_result] - - except Exception as e: - logger.error(f"Error searching with Perplexity: {str(e)}") - logger.error(traceback.format_exc()) - return [] - - def _search_with_serper(self, query: str) -> List[Dict[str, Any]]: - """ - Search using the Serper API for Google search results. - - This method sends the query to Serper's API and formats the response - into a standardized search result format. It includes special handling for - GAIA assessment questions to ensure optimal results. - - Args: - query: The search query - - Returns: - List of search results from Google via Serper - """ - try: - if not self.serper_tool: - logger.error("Serper tool not initialized") - return [] - - # Use Serper's search method - serper_results = self.serper_tool.search(query) - - # Check if we got valid results - if not serper_results or not isinstance(serper_results, list): - logger.warning("Invalid or empty results from Serper API") - return [] - - # Add source information to each result - for result in serper_results: - result["source"] = "serper" - - # Add relevance score if not present - if "relevance_score" not in result: - result["relevance_score"] = 8.0 # Default good score for Serper results - - # Apply general domain-based relevance boosting - link = result.get("link", "") - if "wikipedia.org" in link: - result["relevance_score"] = 9.0 # Wikipedia is generally reliable - elif ".edu" in link or ".gov" in link: - result["relevance_score"] = 9.5 # Educational and government sources are highly reliable - - return serper_results - - except Exception as e: - logger.error(f"Error searching with Serper: {str(e)}") - logger.error(traceback.format_exc()) - return [] - - -def create_api_search() -> ApiSearchTool: - """ - Create an API search tool instance that uses Perplexity and Serper APIs. - - This function creates and returns an ApiSearchTool instance that intelligently - routes queries between Perplexity's sonar-reasoning model and Serper's Google - search API based on the query type. - - Returns: - ApiSearchTool: An initialized API search tool - - Note: - Requires PERPLEXITY_API_KEY and/or SERPER_API_KEY environment variables to be set. - The tool will work with either one or both APIs available. - """ - config = get_tool_config().get("api_search", {}) - return ApiSearchTool(config) - - -def create_browser_search() -> BrowserSearchTool: - """ - Create and configure a BrowserSearchTool instance. - - This factory function creates a BrowserSearchTool with the appropriate configuration - from the tool config. It handles any necessary setup and initialization. - - Returns: - BrowserSearchTool: A configured instance of the browser search tool - """ - config = get_tool_config().get("browser_search", {}) - return BrowserSearchTool(config) - - -def create_library_search() -> LibrarySearchTool: - """ - Create and configure a LibrarySearchTool instance. - - This factory function creates a LibrarySearchTool with the appropriate configuration - from the tool config. It handles any necessary setup and initialization. - - Returns: - LibrarySearchTool: A configured instance of the library search tool - """ - config = get_tool_config().get("library_search", {}) - return LibrarySearchTool(config) \ No newline at end of file diff --git a/utils/__init__.py b/utils/__init__.py deleted file mode 100644 index 668e6afd2145250bc7ac1683f689a561eaa81fce..0000000000000000000000000000000000000000 --- a/utils/__init__.py +++ /dev/null @@ -1,142 +0,0 @@ -""" -Utilities module for GAIA implementation. -Contains helper functions for question parsing, answer formatting, and other utilities. -""" - -import logging -from typing import Dict, Any, List, Optional, Union - -from utils.parsing import ( - extract_key_information, - identify_required_tools, - determine_question_complexity, - parse_question, - COMPLEXITY_LEVELS -) - -from utils.formatting import ( - extract_answer, - format_answer, - validate_answer_format, - process_answer, - FORMAT_TYPES -) - -logger = logging.getLogger("gaia_agent.utils") - -def clean_text(text: str) -> str: - """ - Clean text by removing extra whitespace and normalizing. - - Args: - text: The text to clean - - Returns: - Cleaned text - """ - if not text: - return "" - - cleaned = " ".join(text.split()) - return cleaned.strip() - -def truncate_text(text: str, max_length: int = 1000, add_ellipsis: bool = True) -> str: - """ - Truncate text to a maximum length. - - Args: - text: The text to truncate - max_length: Maximum length in characters - add_ellipsis: Whether to add ellipsis if truncated - - Returns: - Truncated text - """ - if not text or len(text) <= max_length: - return text - - truncated = text[:max_length] - if add_ellipsis: - truncated += "..." - - return truncated - -def merge_dicts(dict1: Dict[str, Any], dict2: Dict[str, Any]) -> Dict[str, Any]: - """ - Merge two dictionaries, with dict2 values taking precedence. - - Args: - dict1: First dictionary - dict2: Second dictionary (takes precedence) - - Returns: - Merged dictionary - """ - result = dict1.copy() - result.update(dict2) - return result - -def get_answer_format(question: str) -> str: - """ - Determine the expected answer format based on the question. - - Args: - question: The question to analyze - - Returns: - Expected answer format type - """ - question_lower = question.lower() - - if any(term in question_lower for term in ["how many", "how much", "count", "sum", "total", "calculate"]): - return FORMAT_TYPES["NUMBER"] - - if any(term in question_lower for term in ["when", "what date", "what time", "what year"]): - return FORMAT_TYPES["DATE"] - - if any(term in question_lower for term in ["is it", "are there", "does it", "can it", "will it", "should it"]): - if question_lower.startswith(("is ", "are ", "does ", "do ", "can ", "will ", "should ")): - return FORMAT_TYPES["BOOLEAN"] - - if any(term in question_lower for term in ["list", "enumerate", "what are", "examples of", "types of"]): - return FORMAT_TYPES["LIST"] - - if any(term in question_lower for term in ["who", "person", "which company", "which organization"]): - return FORMAT_TYPES["ENTITY"] - - return FORMAT_TYPES["TEXT"] - -def analyze_question(question: str) -> Dict[str, Any]: - """ - Comprehensive analysis of a question. - - Args: - question: The question to analyze - - Returns: - Dictionary with analysis results - """ - parsed = parse_question(question) - - expected_format = get_answer_format(question) - - return { - "question": question, - "parsed_info": parsed, - "expected_format": expected_format, - "required_tools": parsed["required_tools"], - "complexity": parsed["complexity"]["complexity_level"] - } - -def format_response(response: str, expected_format: Optional[str] = None) -> Dict[str, Any]: - """ - Format a response according to the expected format. - - Args: - response: The response to format - expected_format: Optional expected format type - - Returns: - Dictionary with formatted response - """ - return process_answer(response, expected_format) \ No newline at end of file diff --git a/utils/formatting.py b/utils/formatting.py deleted file mode 100644 index 1416eff3650c87ddb133d33dd20ff3ef87bc4f76..0000000000000000000000000000000000000000 --- a/utils/formatting.py +++ /dev/null @@ -1,470 +0,0 @@ -""" -Answer formatting utilities for GAIA implementation. - -This module provides functions for extracting, formatting, and validating answers -to ensure they match the expected format (number, few words, comma-separated list, etc.). -""" - -import re -import logging -import json -from typing import Dict, Any, List, Optional, Union, Tuple, Set - -logger = logging.getLogger("gaia_agent.utils.formatting") - -FORMAT_TYPES = { - "TEXT": "text", # Free-form text - "NUMBER": "number", # Numerical answer - "DATE": "date", # Date or time - "BOOLEAN": "boolean", # Yes/No or True/False - "LIST": "list", # Comma-separated list - "ENTITY": "entity", # Named entity (person, place, etc.) - "STRUCTURED": "structured" # JSON or structured data -} - -def extract_answer(response: str, expected_format: Optional[str] = None) -> str: - """ - Extract the answer from a response. - - Args: - response: The full response text - expected_format: Optional expected format type - - Returns: - The extracted answer - """ - - if len(response.split()) < 20: - return response.strip() - - answer_patterns = [ - r'(?:The\s+answer\s+is:?\s+)([^.]+)', - r'(?:In\s+conclusion:?\s+)([^.]+)', - r'(?:To\s+summarize:?\s+)([^.]+)', - r'(?:Therefore:?\s+)([^.]+)', - r'(?:Thus:?\s+)([^.]+)', - r'(?:In\s+summary:?\s+)([^.]+)', - r'(?:The\s+result\s+is:?\s+)([^.]+)' - ] - - for pattern in answer_patterns: - match = re.search(pattern, response, re.IGNORECASE) - if match: - return match.group(1).strip() - - if expected_format: - if expected_format == FORMAT_TYPES["NUMBER"]: - return extract_number(response) - elif expected_format == FORMAT_TYPES["DATE"]: - return extract_date(response) - elif expected_format == FORMAT_TYPES["BOOLEAN"]: - return extract_boolean(response) - elif expected_format == FORMAT_TYPES["LIST"]: - return extract_list(response) - elif expected_format == FORMAT_TYPES["ENTITY"]: - return extract_entity(response) - elif expected_format == FORMAT_TYPES["STRUCTURED"]: - return extract_structured_data(response) - - sentences = re.split(r'[.!?]\s+', response) - if sentences: - return sentences[-1].strip() - - return response.strip() - -def extract_number(text: str) -> str: - """ - Extract a numerical answer from text. - - Args: - text: The text to extract from - - Returns: - The extracted number as a string - """ - number_pattern = r'(?:[\$€£¥])?(?:[\-+])?(?:\d{1,3}(?:,\d{3})+|\d+)(?:\.\d+)?(?:\s*%)?' - - answer_number_patterns = [ - rf'(?:The\s+(?:answer|result|number|value|amount)\s+is:?\s+)({number_pattern})', - rf'(?:equals\s+)({number_pattern})', - rf'(?:approximately\s+)({number_pattern})', - rf'(?:about\s+)({number_pattern})' - ] - - for pattern in answer_number_patterns: - match = re.search(pattern, text, re.IGNORECASE) - if match: - return match.group(1).strip() - - matches = re.findall(number_pattern, text) - if matches: - return matches[-1].strip() - - return "" - -def extract_date(text: str) -> str: - """ - Extract a date or time from text. - - Args: - text: The text to extract from - - Returns: - The extracted date as a string - """ - date_patterns = [ - r'\d{1,2}[-/]\d{1,2}[-/]\d{2,4}', # MM/DD/YYYY or DD/MM/YYYY - r'\d{4}[-/]\d{1,2}[-/]\d{1,2}', # YYYY/MM/DD - r'(?:Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec)[a-z]*\s+\d{1,2}(?:st|nd|rd|th)?,?\s+\d{4}', # Month DD, YYYY - r'\d{1,2}(?:st|nd|rd|th)?\s+(?:of\s+)?(?:Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec)[a-z]*,?\s+\d{4}', # DD Month YYYY - r'\d{1,2}:\d{2}(?::\d{2})?\s*(?:AM|PM|am|pm)?' # HH:MM:SS AM/PM - ] - - answer_date_patterns = [ - rf'(?:The\s+(?:date|time)\s+is:?\s+)({")|(".join(date_patterns)})', - rf'(?:occurred\s+on\s+)({")|(".join(date_patterns)})', - rf'(?:happened\s+on\s+)({")|(".join(date_patterns)})', - rf'(?:scheduled\s+for\s+)({")|(".join(date_patterns)})' - ] - - for pattern in answer_date_patterns: - match = re.search(pattern, text, re.IGNORECASE) - if match: - return match.group(1).strip() - - for pattern in date_patterns: - match = re.search(pattern, text) - if match: - return match.group(0).strip() - - return "" - -def extract_boolean(text: str) -> str: - """ - Extract a boolean answer (yes/no, true/false) from text. - - Args: - text: The text to extract from - - Returns: - The extracted boolean as a string - """ - text_lower = text.lower() - - yes_patterns = [ - r'^yes', - r'^affirmative', - r'^correct', - r'^true', - r'the answer is yes', - r'the answer is affirmative', - r'the answer is true', - r'the answer is correct' - ] - - no_patterns = [ - r'^no', - r'^negative', - r'^incorrect', - r'^false', - r'the answer is no', - r'the answer is negative', - r'the answer is false', - r'the answer is incorrect' - ] - - for pattern in yes_patterns: - if re.search(pattern, text_lower): - return "Yes" - - for pattern in no_patterns: - if re.search(pattern, text_lower): - return "No" - - positive_terms = ["can", "does", "is", "will", "should", "would", "could", "positive", "affirmative"] - negative_terms = ["cannot", "can't", "doesn't", "does not", "isn't", "is not", "won't", "will not", - "shouldn't", "should not", "wouldn't", "would not", "couldn't", "could not", - "negative", "never"] - - positive_count = sum(1 for term in positive_terms if term in text_lower) - negative_count = sum(1 for term in negative_terms if term in text_lower) - - if positive_count > negative_count: - return "Yes" - elif negative_count > positive_count: - return "No" - - return "Unknown" - -def extract_list(text: str) -> str: - """ - Extract a comma-separated list from text. - - Args: - text: The text to extract from - - Returns: - The extracted list as a comma-separated string - """ - list_patterns = [ - r'(?:The\s+list\s+is:?\s+)(.*?)(?:\.|\n|$)', - r'(?:The\s+items\s+are:?\s+)(.*?)(?:\.|\n|$)', - r'(?:The\s+elements\s+are:?\s+)(.*?)(?:\.|\n|$)', - r'(?:They\s+are:?\s+)(.*?)(?:\.|\n|$)' - ] - - for pattern in list_patterns: - match = re.search(pattern, text, re.IGNORECASE) - if match: - items = match.group(1).strip() - items = re.sub(r'\s*,\s*', ', ', items) # Normalize commas - items = re.sub(r'\s+and\s+', ', ', items) # Replace "and" with comma - items = re.sub(r'\s*;\s*', ', ', items) # Replace semicolons with commas - return items - - bullet_pattern = r'(?:^|\n)(?:\d+\.|[-•*])\s*(.*?)(?:$|\n)' - bullet_matches = re.findall(bullet_pattern, text) - if bullet_matches: - return ', '.join(item.strip() for item in bullet_matches) - - comma_pattern = r'(?:^|:)\s*((?:\w+(?:\s+\w+)*,\s*){2,}(?:\w+(?:\s+\w+)*))' - comma_match = re.search(comma_pattern, text) - if comma_match: - return comma_match.group(1).strip() - - return "" - -def extract_entity(text: str) -> str: - """ - Extract a named entity (person, place, organization) from text. - - Args: - text: The text to extract from - - Returns: - The extracted entity as a string - """ - entity_patterns = [ - r'(?:The\s+(?:person|individual)\s+is:?\s+)([A-Z][a-z]+(?:\s+[A-Z][a-z]+)*)', - r'(?:The\s+(?:place|location)\s+is:?\s+)([A-Z][a-z]+(?:\s+[A-Z][a-z]+)*)', - r'(?:The\s+(?:organization|company)\s+is:?\s+)([A-Z][a-z]+(?:\s+[A-Z][a-z]+)*)', - r'(?:The\s+(?:entity)\s+is:?\s+)([A-Z][a-z]+(?:\s+[A-Z][a-z]+)*)' - ] - - for pattern in entity_patterns: - match = re.search(pattern, text) - if match: - return match.group(1).strip() - - proper_noun_pattern = r'([A-Z][a-z]+(?:\s+[A-Z][a-z]+)*)' - matches = re.findall(proper_noun_pattern, text) - - if matches: - common_words = {"The", "A", "An", "This", "That", "These", "Those", "It", "Its", "I", "My", "Me", "You", "Your"} - filtered_matches = [match for match in matches if match not in common_words] - - if filtered_matches: - return filtered_matches[0].strip() - - return "" - -def extract_structured_data(text: str) -> str: - """ - Extract structured data (JSON) from text. - - Args: - text: The text to extract from - - Returns: - The extracted structured data as a string - """ - json_pattern = r'```(?:json)?\s*({[\s\S]*?})```' - match = re.search(json_pattern, text) - if match: - json_str = match.group(1).strip() - try: - parsed = json.loads(json_str) - return json.dumps(parsed, indent=2) - except json.JSONDecodeError: - logger.warning("Found JSON-like block but it's not valid JSON") - - curly_pattern = r'({[\s\S]*?})' - matches = re.findall(curly_pattern, text) - for match in matches: - try: - parsed = json.loads(match) - return json.dumps(parsed, indent=2) - except json.JSONDecodeError: - continue - - return "" - -def format_answer(answer: str, format_type: str) -> str: - """ - Format an answer according to the specified format type. - - Args: - answer: The answer to format - format_type: The desired format type - - Returns: - The formatted answer - """ - - if format_type == FORMAT_TYPES["TEXT"]: - return answer.strip() - - elif format_type == FORMAT_TYPES["NUMBER"]: - number_str = extract_number(answer) - if not number_str: - return answer.strip() - - clean_number = re.sub(r'[^\d\.\-\+]', '', number_str) - try: - num = float(clean_number) - if num.is_integer(): - return str(int(num)) - return str(num) - except ValueError: - return number_str - - elif format_type == FORMAT_TYPES["DATE"]: - date_str = extract_date(answer) - if date_str: - return date_str - return answer.strip() - - elif format_type == FORMAT_TYPES["BOOLEAN"]: - bool_str = extract_boolean(answer) - if bool_str in ["Yes", "No", "Unknown"]: - return bool_str - return answer.strip() - - elif format_type == FORMAT_TYPES["LIST"]: - list_str = extract_list(answer) - if list_str: - items = [item.strip() for item in list_str.split(',')] - return ', '.join(items) - return answer.strip() - - elif format_type == FORMAT_TYPES["ENTITY"]: - entity_str = extract_entity(answer) - if entity_str: - return entity_str - return answer.strip() - - elif format_type == FORMAT_TYPES["STRUCTURED"]: - json_str = extract_structured_data(answer) - if json_str: - return json_str - return answer.strip() - - return answer.strip() - -def validate_answer_format(answer: str, expected_format: str) -> Dict[str, Any]: - """ - Validate that an answer matches the expected format. - - Args: - answer: The answer to validate - expected_format: The expected format type - - Returns: - Dictionary with validation results - """ - - result = { - "is_valid": False, - "formatted_answer": answer, - "error": None - } - - if expected_format == FORMAT_TYPES["NUMBER"]: - clean_answer = re.sub(r'[^\d\.\-\+]', '', answer) - try: - float(clean_answer) - result["is_valid"] = True - result["formatted_answer"] = clean_answer - except ValueError: - result["is_valid"] = False - result["error"] = "Answer is not a valid number" - - elif expected_format == FORMAT_TYPES["DATE"]: - date_patterns = [ - r'\d{1,2}[-/]\d{1,2}[-/]\d{2,4}', - r'\d{4}[-/]\d{1,2}[-/]\d{1,2}', - r'(?:Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec)[a-z]*\s+\d{1,2}(?:st|nd|rd|th)?,?\s+\d{4}', - r'\d{1,2}(?:st|nd|rd|th)?\s+(?:of\s+)?(?:Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec)[a-z]*,?\s+\d{4}' - ] - if any(re.search(pattern, answer) for pattern in date_patterns): - result["is_valid"] = True - else: - result["is_valid"] = False - result["error"] = "Answer is not a valid date" - - elif expected_format == FORMAT_TYPES["BOOLEAN"]: - bool_answer = extract_boolean(answer) - if bool_answer in ["Yes", "No"]: - result["is_valid"] = True - result["formatted_answer"] = bool_answer - else: - result["is_valid"] = False - result["error"] = "Answer is not a clear Yes/No" - - elif expected_format == FORMAT_TYPES["LIST"]: - list_items = [item.strip() for item in answer.split(',')] - if len(list_items) > 1: - result["is_valid"] = True - result["formatted_answer"] = ', '.join(list_items) - else: - result["is_valid"] = False - result["error"] = "Answer is not a comma-separated list" - - elif expected_format == FORMAT_TYPES["ENTITY"]: - if re.match(r'^[A-Z][a-z]+(?:\s+[A-Z][a-z]+)*$', answer): - result["is_valid"] = True - else: - result["is_valid"] = False - result["error"] = "Answer is not a valid named entity" - - elif expected_format == FORMAT_TYPES["STRUCTURED"]: - try: - json.loads(answer) - result["is_valid"] = True - except json.JSONDecodeError: - result["is_valid"] = False - result["error"] = "Answer is not valid JSON" - - else: # FORMAT_TYPES["TEXT"] or any other - result["is_valid"] = True - - return result - -def process_answer(response: str, expected_format: Optional[str] = None) -> Dict[str, Any]: - """ - Process an answer: extract, format, and validate. - - Args: - response: The full response text - expected_format: Optional expected format type - - Returns: - Dictionary with processed answer information - """ - - format_type = expected_format if expected_format else FORMAT_TYPES["TEXT"] - - extracted_answer = extract_answer(response, format_type) - - formatted_answer = format_answer(extracted_answer, format_type) - - validation_result = validate_answer_format(formatted_answer, format_type) - - return { - "original_response": response, - "extracted_answer": extracted_answer, - "formatted_answer": formatted_answer, - "format_type": format_type, - "is_valid": validation_result["is_valid"], - "error": validation_result["error"] - } \ No newline at end of file diff --git a/utils/parsing.py b/utils/parsing.py deleted file mode 100644 index bbd152d340841e3dced8e7c8d1b20b4fc8085112..0000000000000000000000000000000000000000 --- a/utils/parsing.py +++ /dev/null @@ -1,268 +0,0 @@ -""" -Question parsing utilities for GAIA implementation. - -This module provides functions for extracting key information from questions, -identifying required tools, and determining question complexity. -""" - -import re -import logging -from typing import Dict, Any, List, Optional, Union, Tuple, Set -import json - -logger = logging.getLogger("gaia_agent.utils.parsing") - -COMPLEXITY_LEVELS = { - "SIMPLE": 1, # Single fact or straightforward question - "MODERATE": 2, # Multiple facts or moderate reasoning - "COMPLEX": 3, # Complex reasoning or multiple steps - "VERY_COMPLEX": 4 # Advanced reasoning, multiple domains, or specialized knowledge -} - -def extract_key_information(question: str) -> Dict[str, Any]: - """ - Extract key information from a question. - - Args: - question: The question to analyze - - Returns: - Dictionary containing extracted information - """ - - result = { - "question_text": question, - "entities": [], - "keywords": [], - "question_type": "", - "constraints": [], - "temporal_references": [], - "expected_answer_type": "unknown" - } - - capitalized_words = re.findall(r'\b[A-Z][a-zA-Z]*\b', question) - result["entities"] = [word for word in capitalized_words if len(word) > 1] - - stop_words = {"a", "an", "the", "is", "are", "was", "were", "be", "been", - "being", "to", "of", "and", "or", "in", "on", "at", "by", - "for", "with", "about", "against", "between", "into", - "through", "during", "before", "after", "above", "below", - "from", "up", "down", "out", "off", "over", "under", "again", - "further", "then", "once", "here", "there", "when", "where", - "why", "how", "all", "any", "both", "each", "few", "more", - "most", "other", "some", "such", "no", "nor", "not", "only", - "own", "same", "so", "than", "too", "very", "s", "t", "can", - "will", "just", "don", "should", "now", "d", "ll", "m", "o", - "re", "ve", "y", "ain", "aren", "couldn", "didn", "doesn", - "hadn", "hasn", "haven", "isn", "ma", "mightn", "mustn", - "needn", "shan", "shouldn", "wasn", "weren", "won", "wouldn"} - - words = re.findall(r'\b[a-zA-Z]+\b', question.lower()) - result["keywords"] = [word for word in words if word not in stop_words and len(word) > 2] - - if re.search(r'\b(what|who|where|when|which)\b', question.lower()): - result["question_type"] = "factual" - elif re.search(r'\b(why|how)\b', question.lower()): - result["question_type"] = "explanatory" - elif re.search(r'\b(is|are|was|were|do|does|did|can|could|will|would|should|has|have)\b', question.lower()): - result["question_type"] = "yes_no" - elif re.search(r'\b(list|name|give|enumerate)\b', question.lower()): - result["question_type"] = "list" - elif re.search(r'\b(compare|contrast|difference|similarities)\b', question.lower()): - result["question_type"] = "comparative" - else: - result["question_type"] = "other" - - time_patterns = [ - r'\b(today|yesterday|tomorrow|now|current|latest|recent)\b', - r'\b(in\s+\d{4})\b', # in 2023 - r'\b(\d{4}s)\b', # 1990s - r'\b(last|this|next)\s+(day|week|month|year|decade)\b', - r'\b(january|february|march|april|may|june|july|august|september|october|november|december)\b', - r'\b(jan|feb|mar|apr|jun|jul|aug|sep|oct|nov|dec)\b' - ] - - for pattern in time_patterns: - matches = re.findall(pattern, question.lower()) - if matches: - result["temporal_references"].extend(matches) - - if re.search(r'\b(who|person)\b', question.lower()): - result["expected_answer_type"] = "person" - elif re.search(r'\b(where|location|place)\b', question.lower()): - result["expected_answer_type"] = "location" - elif re.search(r'\b(when|date|time|year)\b', question.lower()): - result["expected_answer_type"] = "time" - elif re.search(r'\b(how\s+many|count|number|sum|total)\b', question.lower()): - result["expected_answer_type"] = "number" - elif re.search(r'\b(why|reason|cause)\b', question.lower()): - result["expected_answer_type"] = "reason" - elif re.search(r'\b(how|process|steps|procedure)\b', question.lower()): - result["expected_answer_type"] = "process" - elif re.search(r'\b(list|examples|types)\b', question.lower()): - result["expected_answer_type"] = "list" - - constraint_patterns = [ - r'(only|just)\s+([^.,;!?]*)', - r'(at\s+least|at\s+most|more\s+than|less\s+than|exactly)\s+(\d+)', - r'(between)\s+(\d+)\s+and\s+(\d+)', - r'(not|except|excluding)\s+([^.,;!?]*)', - r'(from|in)\s+([^.,;!?]*)', - r'(before|after|during)\s+([^.,;!?]*)' - ] - - for pattern in constraint_patterns: - matches = re.findall(pattern, question.lower()) - if matches: - result["constraints"].extend([' '.join(match) for match in matches]) - - return result - -def identify_required_tools(question: str) -> List[str]: - """ - Identify tools required to answer a question. - - Args: - question: The question to analyze - - Returns: - List of required tool names - """ - - question_lower = question.lower() - required_tools = set() - - if any(term in question_lower for term in ["search", "find", "look up", "latest", "current", "news", "information about"]): - required_tools.add("web_search") - - if any(term in question_lower for term in ["website", "webpage", "url", "link", "extract", "content"]): - required_tools.add("web_content") - - if any(term in question_lower for term in ["reason", "explain", "why", "how", "analyze", "understand", "interpret"]): - required_tools.add("reasoning") - - if any(term in question_lower for term in ["calculate", "compute", "solve", "equation", "math", "formula", "number"]): - required_tools.add("math") - - if any(term in question_lower for term in ["verify", "check", "fact", "true", "false", "accurate", "correct"]): - required_tools.add("fact_verification") - - if any(term in question_lower for term in ["image", "picture", "photo", "describe", "visual", "see", "look at"]): - required_tools.add("image_analysis") - - if any(term in question_lower for term in ["chart", "graph", "plot", "diagram", "visualization", "trend", "data"]): - required_tools.add("chart_interpretation") - - if any(term in question_lower for term in ["document", "pdf", "docx", "text", "extract", "parse", "read"]): - required_tools.add("document_parsing") - - if not required_tools: - required_tools.add("reasoning") - required_tools.add("web_search") - - return list(required_tools) - -def determine_question_complexity(question: str) -> Dict[str, Any]: - """ - Determine the complexity of a question. - - Args: - question: The question to analyze - - Returns: - Dictionary containing complexity assessment - """ - - question_lower = question.lower() - - complexity_score = 1 # Start with SIMPLE - - - if len(question.split()) > 15: - complexity_score += 1 - if len(question.split()) > 30: - complexity_score += 1 - - if question.count("?") > 1: - complexity_score += 1 - - complex_reasoning_terms = [ - "why", "how", "explain", "analyze", "compare", "contrast", - "evaluate", "assess", "interpret", "synthesize", "relationship", - "impact", "effect", "cause", "implication", "consequence" - ] - if any(term in question_lower for term in complex_reasoning_terms): - complexity_score += 1 - - domains = { - "science": ["physics", "chemistry", "biology", "scientific", "experiment"], - "math": ["math", "equation", "calculation", "formula", "compute"], - "history": ["history", "historical", "ancient", "century", "era", "period"], - "geography": ["geography", "country", "region", "map", "location"], - "literature": ["book", "author", "novel", "character", "literary"], - "arts": ["art", "painting", "music", "artist", "composition"], - "technology": ["technology", "computer", "software", "hardware", "digital"], - "business": ["business", "economy", "market", "finance", "industry"], - "politics": ["politics", "government", "policy", "election", "law"] - } - - domain_count = 0 - for domain, terms in domains.items(): - if any(term in question_lower for term in terms): - domain_count += 1 - - if domain_count > 1: - complexity_score += 1 - if domain_count > 2: - complexity_score += 1 - - temporal_terms = [ - "before", "after", "during", "while", "simultaneously", - "previously", "subsequently", "meanwhile", "throughout", - "initially", "finally", "eventually", "ultimately" - ] - if any(term in question_lower for term in temporal_terms): - complexity_score += 1 - - conditional_terms = [ - "if", "unless", "assuming", "provided that", "in case", - "depending on", "subject to", "given that", "on condition" - ] - if any(term in question_lower for term in conditional_terms): - complexity_score += 1 - - complexity_score = min(complexity_score, 4) - - complexity_level = "" - for level, score in COMPLEXITY_LEVELS.items(): - if complexity_score == score: - complexity_level = level - - return { - "complexity_level": complexity_level, - "complexity_score": complexity_score, - "requires_multiple_tools": len(identify_required_tools(question)) > 1, - "requires_complex_reasoning": any(term in question_lower for term in complex_reasoning_terms), - "multi_domain": domain_count > 1, - "domains_involved": [domain for domain, terms in domains.items() if any(term in question_lower for term in terms)] - } - -def parse_question(question: str) -> Dict[str, Any]: - """ - Comprehensive parsing of a question. - - Args: - question: The question to parse - - Returns: - Dictionary containing all parsed information - """ - - result = { - "question": question, - "key_information": extract_key_information(question), - "required_tools": identify_required_tools(question), - "complexity": determine_question_complexity(question) - } - - return result \ No newline at end of file diff --git a/validate_app.py b/validate_app.py deleted file mode 100644 index c41a39da375a120cf2e20fce5e61bbcf3855bcfd..0000000000000000000000000000000000000000 --- a/validate_app.py +++ /dev/null @@ -1,23 +0,0 @@ -import py_compile -import sys - -def validate_syntax(filename): - try: - py_compile.compile(filename, doraise=True) - print(f"✓ {filename} has valid Python syntax") - return True - except py_compile.PyCompileError as e: - print(f"✗ Syntax error in {filename}:") - print(e) - return False - except Exception as e: - print(f"✗ Error checking {filename}:") - print(e) - return False - -if __name__ == "__main__": - filename = "app.py" - if validate_syntax(filename): - sys.exit(0) # Success - else: - sys.exit(1) # Error \ No newline at end of file diff --git a/validate_app_simple.py b/validate_app_simple.py deleted file mode 100644 index 9e019c59632c6d22b8f2be1a90ddbbe5b7999bc7..0000000000000000000000000000000000000000 --- a/validate_app_simple.py +++ /dev/null @@ -1,39 +0,0 @@ -""" -Simple script to check app.py for syntax errors and identify line numbers -""" -import os -import sys -import traceback - -def check_file_syntax(file_path): - print(f"Checking syntax for: {file_path}") - - try: - with open(file_path, 'r', encoding='utf-8') as f: - source = f.read() - - # Try to compile the source code - compiled = compile(source, file_path, 'exec') - print(f"✓ {file_path} has valid Python syntax") - return True - except SyntaxError as e: - print(f"✗ Syntax error in {file_path}:") - print(f" Line {e.lineno}, Column {e.offset}: {e.text}") - print(f" {e}") - return False - except Exception as e: - print(f"✗ Error checking {file_path}:") - print(f" {e}") - traceback.print_exc() - return False - -if __name__ == "__main__": - app_path = "app.py" - if not os.path.exists(app_path): - print(f"File not found: {app_path}") - sys.exit(1) - - if check_file_syntax(app_path): - sys.exit(0) - else: - sys.exit(1) \ No newline at end of file diff --git a/verify_components.py b/verify_components.py deleted file mode 100644 index 6dfb0b9713cdd6ad57d459b29563a4051373b013..0000000000000000000000000000000000000000 --- a/verify_components.py +++ /dev/null @@ -1,359 +0,0 @@ -""" -Step-by-step verification of GAIA components -This script tests each component individually to identify issues -""" -import os -import sys -import time -import logging -import traceback -from typing import Dict, Any, List - -# Configure logging -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(name)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("verify_components") - -class ComponentVerifier: - """Verifies each component of the GAIA system step by step""" - - def __init__(self): - self.results = {} - self.env_vars_set = False - self.supabase_connected = False - - def verify_environment(self): - """Verify environment variables are set""" - try: - logger.info("Checking environment variables...") - required_vars = ["OPENAI_API_KEY", "SUPABASE_URL", "SUPABASE_KEY"] - missing = [] - - for var in required_vars: - if not os.environ.get(var): - missing.append(var) - - if missing: - logger.error(f"Missing required environment variables: {', '.join(missing)}") - return False - - logger.info("All required environment variables are set") - self.env_vars_set = True - return True - except Exception as e: - logger.error(f"Error checking environment variables: {e}") - return False - - def verify_imports(self): - """Verify that all necessary modules can be imported""" - try: - logger.info("Verifying imports...") - - # 1. Try importing OpenAI - try: - from openai import OpenAI - logger.info("OpenAI module imported successfully") - self.results["openai_import"] = True - except Exception as e: - logger.error(f"Failed to import OpenAI: {e}") - self.results["openai_import"] = False - - # 2. Try importing Supabase - try: - from supabase import create_client - logger.info("Supabase module imported successfully") - self.results["supabase_import"] = True - except Exception as e: - logger.error(f"Failed to import Supabase: {e}") - self.results["supabase_import"] = False - - # 3. Try importing agent.config - try: - from agent.config import get_memory_config - logger.info("Agent config module imported successfully") - self.results["agent_config_import"] = True - except Exception as e: - logger.error(f"Failed to import agent.config: {e}") - self.results["agent_config_import"] = False - - # 4. Try importing memory module - try: - from memory.supabase_memory import ( - SupabaseMemory, - ConversationMemory, - ResultCache, - WorkingMemory - ) - logger.info("Memory module imported successfully") - self.results["memory_import"] = True - except Exception as e: - logger.error(f"Failed to import memory module: {e}") - self.results["memory_import"] = False - - # 5. Try importing memory fix - try: - from memory.fix_supabase_tables import apply_monkey_patch, ensure_tables_exist - logger.info("Memory fix module imported successfully") - self.results["memory_fix_import"] = True - except Exception as e: - logger.error(f"Failed to import memory fix module: {e}") - self.results["memory_fix_import"] = False - - # 6. Try importing agent.graph - try: - from agent.graph import create_agent_graph, run_agent_graph - logger.info("Agent graph module imported successfully") - self.results["agent_graph_import"] = True - except Exception as e: - logger.error(f"Failed to import agent.graph: {e}") - self.results["agent_graph_import"] = False - - # 7. Try importing agent - try: - from agent.agent import GAIAAgent - logger.info("GAIAAgent imported successfully") - self.results["agent_import"] = True - except Exception as e: - logger.error(f"Failed to import GAIAAgent: {e}") - self.results["agent_import"] = False - - # 8. Try importing tools - try: - from tools.web_tools import duckduckgo_search - logger.info("Web tools imported successfully") - self.results["web_tools_import"] = True - except Exception as e: - logger.error(f"Failed to import web tools: {e}") - self.results["web_tools_import"] = False - - return True - except Exception as e: - logger.error(f"Error verifying imports: {e}") - return False - - def verify_supabase_connection(self): - """Verify connection to Supabase""" - if not self.env_vars_set: - logger.error("Environment variables not verified, skipping Supabase connection test") - return False - - try: - logger.info("Verifying Supabase connection...") - from supabase import create_client - - supabase_url = os.environ.get("SUPABASE_URL") - supabase_key = os.environ.get("SUPABASE_KEY") - - client = create_client(supabase_url, supabase_key) - - # Try a simple query - response = client.table("_memory").select("count(*)", count="exact").execute() - count = response.count if hasattr(response, 'count') else 0 - - logger.info(f"Successfully connected to Supabase, found {count} records in _memory table") - self.supabase_connected = True - return True - except Exception as e: - logger.error(f"Failed to connect to Supabase: {e}") - logger.error(traceback.format_exc()) - return False - - def apply_supabase_fixes(self): - """Verify and apply Supabase table fixes""" - if not self.supabase_connected: - logger.error("Supabase connection not verified, skipping table fixes") - return False - - try: - logger.info("Applying Supabase table fixes...") - from memory.fix_supabase_tables import apply_monkey_patch, ensure_tables_exist - - # Ensure tables exist - ensure_tables_exist() - logger.info("Successfully ensured all required tables exist") - - # Apply monkey patch - apply_monkey_patch() - logger.info("Successfully applied monkey patch for Supabase tables") - - return True - except Exception as e: - logger.error(f"Failed to apply Supabase fixes: {e}") - logger.error(traceback.format_exc()) - return False - - def initialize_memory(self): - """Initialize and test memory system""" - try: - logger.info("Initializing memory system...") - from memory.supabase_memory import SupabaseMemory - from agent.config import get_memory_config - - # Get memory config - memory_config = get_memory_config() - memory_config["enabled"] = True - - # Initialize memory - memory = SupabaseMemory(memory_config) - - # Test basic operations - test_key = f"test_key_{int(time.time())}" - test_value = {"message": "Test value", "timestamp": time.time()} - - # Store - memory.store(test_key, test_value) - logger.info(f"Successfully stored test value with key: {test_key}") - - # Retrieve - retrieved = memory.retrieve(test_key) - - if retrieved and retrieved.get("message") == "Test value": - logger.info("Successfully retrieved test value") - return True - else: - logger.error(f"Retrieved value doesn't match. Got: {retrieved}") - return False - except Exception as e: - logger.error(f"Failed to initialize memory: {e}") - logger.error(traceback.format_exc()) - return False - - def test_memory_subtypes(self): - """Test all memory subtypes""" - try: - logger.info("Testing memory subtypes...") - from memory.supabase_memory import ( - SupabaseMemory, - ConversationMemory, - ResultCache, - WorkingMemory - ) - from agent.config import get_memory_config - - # Initialize base memory - memory_config = get_memory_config() - memory_config["enabled"] = True - memory = SupabaseMemory(memory_config) - - # Test session ID - session_id = f"test_session_{int(time.time())}" - - # Test conversation memory - try: - conversation = ConversationMemory(memory, session_id) - conversation.add_message("user", "Test question") - conversation.add_message("assistant", "Test answer") - messages = conversation.get_messages() - - if len(messages) >= 2: - logger.info("ConversationMemory working correctly") - self.results["conversation_memory"] = True - else: - logger.error(f"ConversationMemory not working correctly. Got {len(messages)} messages") - self.results["conversation_memory"] = False - except Exception as e: - logger.error(f"Error testing ConversationMemory: {e}") - self.results["conversation_memory"] = False - - # Test result cache - try: - cache = ResultCache(memory) - cache_key = f"test_cache_{int(time.time())}" - cache.cache_result(cache_key, "Test cache result") - cached = cache.get_result(cache_key) - - if cached == "Test cache result": - logger.info("ResultCache working correctly") - self.results["result_cache"] = True - else: - logger.error(f"ResultCache not working correctly. Got: {cached}") - self.results["result_cache"] = False - except Exception as e: - logger.error(f"Error testing ResultCache: {e}") - self.results["result_cache"] = False - - # Test working memory - try: - working = WorkingMemory(memory, session_id) - working.store_intermediate_result("test_step", "Test intermediate result") - results = working.list_intermediate_results() - - if results and any(r.get("step") == "test_step" for r in results): - logger.info("WorkingMemory working correctly") - self.results["working_memory"] = True - else: - logger.error(f"WorkingMemory not working correctly. Got: {results}") - self.results["working_memory"] = False - except Exception as e: - logger.error(f"Error testing WorkingMemory: {e}") - self.results["working_memory"] = False - - # Overall result - return (self.results.get("conversation_memory", False) and - self.results.get("result_cache", False) and - self.results.get("working_memory", False)) - - except Exception as e: - logger.error(f"Failed to test memory subtypes: {e}") - logger.error(traceback.format_exc()) - return False - - def run_all_tests(self): - """Run all verification tests in sequence""" - logger.info("=== STARTING COMPONENT VERIFICATION ===") - - # Step 1: Verify environment variables - self.results["environment"] = self.verify_environment() - - # Step 2: Verify imports - self.results["imports"] = self.verify_imports() - - # Step 3: Verify Supabase connection - if self.results["environment"]: - self.results["supabase_connection"] = self.verify_supabase_connection() - else: - logger.warning("Skipping Supabase connection test due to environment variables issue") - self.results["supabase_connection"] = False - - # Step 4: Apply Supabase fixes - if self.results["supabase_connection"]: - self.results["supabase_fixes"] = self.apply_supabase_fixes() - else: - logger.warning("Skipping Supabase fixes due to connection issue") - self.results["supabase_fixes"] = False - - # Step 5: Initialize memory - if self.results["supabase_fixes"]: - self.results["memory_initialize"] = self.initialize_memory() - else: - logger.warning("Skipping memory initialization due to Supabase fixes issue") - self.results["memory_initialize"] = False - - # Step 6: Test memory subtypes - if self.results["memory_initialize"]: - self.results["memory_subtypes"] = self.test_memory_subtypes() - else: - logger.warning("Skipping memory subtypes test due to initialization issue") - self.results["memory_subtypes"] = False - - # Print summary - logger.info("\n=== COMPONENT VERIFICATION RESULTS ===") - for component, result in self.results.items(): - status = "SUCCESS" if result else "FAILED" - logger.info(f"{component}: {status}") - - # Return overall result - return all(self.results.values()) - -if __name__ == "__main__": - verifier = ComponentVerifier() - success = verifier.run_all_tests() - - if success: - logger.info("\n✅ All components verified successfully!") - sys.exit(0) - else: - logger.error("\n❌ Component verification failed!") - sys.exit(1) \ No newline at end of file diff --git a/verify_deployment_readiness.py b/verify_deployment_readiness.py deleted file mode 100644 index 3fb13ba8883c7ffeecb8b5bbcc82df4fd4888fe5..0000000000000000000000000000000000000000 --- a/verify_deployment_readiness.py +++ /dev/null @@ -1,338 +0,0 @@ -""" -Pre-deployment verification script for GAIA Agent - -This script performs basic checks on the project structure and environment -without requiring API connections. It provides recommendations for fixing -any detected issues before deployment. -""" - -import os -import sys -import logging -import json -from datetime import datetime -from pathlib import Path - -# Configure logging -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(name)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("verify_deployment") - -# Required files for the GAIA Agent -REQUIRED_FILES = [ - "app.py", - "requirements.txt", - "README.md", - "agent/agent.py", - "agent/config.py", - "agent/graph.py", - "memory/supabase_memory.py", - "memory/supabase_memory_fix.py" -] - -# Required environment variables -REQUIRED_ENV_VARS = [ - "OPENAI_API_KEY", - "SUPABASE_URL", - "SUPABASE_KEY" -] - -# Optional environment variables -OPTIONAL_ENV_VARS = [ - "SERPER_API_KEY", - "HF_TOKEN", - "SPACE_ID" -] - -def check_file_structure(): - """Check if all required files are present.""" - logger.info("Checking file structure...") - - missing_files = [] - for file_path in REQUIRED_FILES: - if not os.path.exists(file_path): - missing_files.append(file_path) - logger.error(f"❌ Missing required file: {file_path}") - else: - logger.info(f"✅ Found required file: {file_path}") - - return len(missing_files) == 0, missing_files - -def check_environment_variables(): - """Check if all required environment variables are set.""" - logger.info("Checking environment variables...") - - missing_required = [] - missing_optional = [] - - for var_name in REQUIRED_ENV_VARS: - var_value = os.getenv(var_name, "") - if not var_value: - missing_required.append(var_name) - logger.error(f"❌ Missing required environment variable: {var_name}") - else: - # Safely mask the actual value for security - try: - if len(var_value) > 8: - masked_value = var_value[:4] + "*" * (len(var_value) - 8) + var_value[-4:] - else: - masked_value = "****" - logger.info(f"✅ Environment variable set: {var_name}={masked_value}") - except Exception as e: - logger.info(f"✅ Environment variable set: {var_name}=****") - - for var_name in OPTIONAL_ENV_VARS: - var_value = os.getenv(var_name, "") - if not var_value: - missing_optional.append(var_name) - logger.warning(f"⚠️ Missing optional environment variable: {var_name}") - else: - # Safely mask the actual value for security - try: - if len(var_value) > 8: - masked_value = var_value[:4] + "*" * (len(var_value) - 8) + var_value[-4:] - else: - masked_value = "****" - logger.info(f"✅ Environment variable set: {var_name}={masked_value}") - except Exception as e: - logger.info(f"✅ Environment variable set: {var_name}=****") - - return len(missing_required) == 0, missing_required, missing_optional - -def check_gradio_interface(): - """Check if Gradio interface is properly configured.""" - logger.info("Checking Gradio interface configuration...") - - issues = [] - - # Check app.py for basic Gradio imports and structure - try: - with open("app.py", "r") as f: - content = f.read() - - if "import gradio as gr" not in content: - issues.append("Missing Gradio import in app.py") - logger.error("❌ Missing Gradio import in app.py") - else: - logger.info("✅ Gradio import found in app.py") - - if "with gr.Blocks" not in content: - issues.append("Missing Gradio Blocks interface in app.py") - logger.error("❌ Missing Gradio Blocks interface in app.py") - else: - logger.info("✅ Gradio Blocks interface found in app.py") - - # Check for the fixed Accordion update - if "outputs=[gr.Accordion.update" in content: - issues.append("Invalid Accordion update method in app.py") - logger.error("❌ Invalid Accordion update method in app.py") - else: - logger.info("✅ Accordion update method looks correct in app.py") - except Exception as e: - issues.append(f"Error checking app.py: {str(e)}") - logger.error(f"❌ Error checking app.py: {str(e)}") - - # Check requirements.txt for Gradio - try: - with open("requirements.txt", "r") as f: - content = f.read() - - if "gradio" not in content: - issues.append("Missing Gradio in requirements.txt") - logger.error("❌ Missing Gradio in requirements.txt") - else: - logger.info("✅ Gradio found in requirements.txt") - except Exception as e: - issues.append(f"Error checking requirements.txt: {str(e)}") - logger.error(f"❌ Error checking requirements.txt: {str(e)}") - - return len(issues) == 0, issues - -def check_agent_configuration(): - """Check if agent is properly configured.""" - logger.info("Checking agent configuration...") - - issues = [] - - # Check config.py for required elements - try: - with open("agent/config.py", "r") as f: - content = f.read() - - if "OPENAI_API_KEY" not in content: - issues.append("Missing OPENAI_API_KEY reference in agent/config.py") - logger.error("❌ Missing OPENAI_API_KEY reference in agent/config.py") - else: - logger.info("✅ OPENAI_API_KEY reference found in agent/config.py") - - if "SUPABASE_URL" not in content: - issues.append("Missing SUPABASE_URL reference in agent/config.py") - logger.error("❌ Missing SUPABASE_URL reference in agent/config.py") - else: - logger.info("✅ SUPABASE_URL reference found in agent/config.py") - - if "SUPABASE_KEY" not in content: - issues.append("Missing SUPABASE_KEY reference in agent/config.py") - logger.error("❌ Missing SUPABASE_KEY reference in agent/config.py") - else: - logger.info("✅ SUPABASE_KEY reference found in agent/config.py") - except Exception as e: - issues.append(f"Error checking agent/config.py: {str(e)}") - logger.error(f"❌ Error checking agent/config.py: {str(e)}") - - return len(issues) == 0, issues - -def check_requirements(): - """Check if requirements.txt has all necessary packages.""" - logger.info("Checking requirements.txt...") - - required_packages = [ - "openai", - "gradio", - "langchain", - "supabase", - "requests" - ] - - issues = [] - - try: - with open("requirements.txt", "r") as f: - content = f.read().lower() - - for package in required_packages: - if package.lower() not in content: - issues.append(f"Missing {package} in requirements.txt") - logger.error(f"❌ Missing {package} in requirements.txt") - else: - logger.info(f"✅ {package} found in requirements.txt") - except Exception as e: - issues.append(f"Error checking requirements.txt: {str(e)}") - logger.error(f"❌ Error checking requirements.txt: {str(e)}") - - return len(issues) == 0, issues - -def generate_report(results): - """Generate a deployment readiness report.""" - logger.info("Generating deployment readiness report...") - - report = { - "timestamp": datetime.now().isoformat(), - "status": "ready" if all(result[0] for result in results.values()) else "not_ready", - "results": { - key: { - "passed": result[0], - "issues": result[1] if len(result) > 1 else [] - } for key, result in results.items() - }, - "recommendations": [] - } - - # Generate recommendations based on results - if not results["file_structure"][0]: - missing_files = results["file_structure"][1] - for file in missing_files: - report["recommendations"].append(f"Create the missing file: {file}") - - if not results["environment_variables"][0]: - missing_vars = results["environment_variables"][1] - for var in missing_vars: - report["recommendations"].append(f"Set the required environment variable: {var}") - - if not results["gradio_interface"][0]: - issues = results["gradio_interface"][1] - for issue in issues: - report["recommendations"].append(f"Fix Gradio interface issue: {issue}") - - if not results["agent_configuration"][0]: - issues = results["agent_configuration"][1] - for issue in issues: - report["recommendations"].append(f"Fix agent configuration issue: {issue}") - - if not results["requirements"][0]: - issues = results["requirements"][1] - for issue in issues: - report["recommendations"].append(f"Fix requirements issue: {issue}") - - # Save report to file - with open("deployment_readiness_report.json", "w") as f: - json.dump(report, f, indent=2) - - return report - -def main(): - """Main entry point for the verification script.""" - try: - logger.info("===== GAIA Agent Deployment Readiness Verification =====") - logger.info(f"Starting verification at: {datetime.now().isoformat()}") - - # Run all checks with error handling - results = {} - - try: - results["file_structure"] = check_file_structure() - except Exception as e: - logger.error(f"Error checking file structure: {str(e)}") - results["file_structure"] = (False, [f"Exception: {str(e)}"]) - - try: - results["environment_variables"] = check_environment_variables() - except Exception as e: - logger.error(f"Error checking environment variables: {str(e)}") - results["environment_variables"] = (False, [f"Exception: {str(e)}"]) - - try: - results["gradio_interface"] = check_gradio_interface() - except Exception as e: - logger.error(f"Error checking Gradio interface: {str(e)}") - results["gradio_interface"] = (False, [f"Exception: {str(e)}"]) - - try: - results["agent_configuration"] = check_agent_configuration() - except Exception as e: - logger.error(f"Error checking agent configuration: {str(e)}") - results["agent_configuration"] = (False, [f"Exception: {str(e)}"]) - - try: - results["requirements"] = check_requirements() - except Exception as e: - logger.error(f"Error checking requirements: {str(e)}") - results["requirements"] = (False, [f"Exception: {str(e)}"]) - - # Generate report with error handling - try: - report = generate_report(results) - except Exception as e: - logger.error(f"Error generating report: {str(e)}") - return 1 - - # Print summary with error handling - try: - logger.info("\n===== Verification Results Summary =====") - - for check_name, result in results.items(): - status = "✅ PASSED" if result[0] else "❌ FAILED" - logger.info(f"{status}: {check_name}") - - logger.info("\n===== Recommendations =====") - - if not report["recommendations"]: - logger.info("✅ No issues found! The GAIA Agent is ready for deployment.") - else: - for i, recommendation in enumerate(report["recommendations"], 1): - logger.info(f"{i}. {recommendation}") - - logger.info(f"\nDetailed report saved to: deployment_readiness_report.json") - - return 0 if report["status"] == "ready" else 1 - except Exception as e: - logger.error(f"Error printing summary: {str(e)}") - return 1 - except Exception as e: - logger.error(f"Critical error in verification script: {str(e)}") - return 1 - -if __name__ == "__main__": - sys.exit(main()) \ No newline at end of file diff --git a/verify_gaia_assessment.py b/verify_gaia_assessment.py deleted file mode 100644 index 435243d79ae274147dfe93eed4521788d38fd976..0000000000000000000000000000000000000000 --- a/verify_gaia_assessment.py +++ /dev/null @@ -1,732 +0,0 @@ -""" -GAIA Agent Final Integration Test - -This script provides a comprehensive integration test for the GAIA agent, -verifying that all components work together correctly, especially the -Supabase tables fix. It tests: - -1. All memory types (base, working, conversation, cache) -2. Multi-turn conversations -3. Caching for repeated questions -4. Complex reasoning with working memory -5. Table usage verification -6. Performance metrics - -The script runs a series of tests that exercise all aspects of the GAIA agent -and provides detailed output on the results, including table content before -and after operations. - -Usage: - python verify_gaia_assessment.py [options] - -Options: - --verbose Enable verbose logging - --show-all Show all table entries - --metrics Show detailed performance metrics - --help Show this help message - -Environment Variables Required: - OPENAI_API_KEY: Your OpenAI API key for generating responses - SUPABASE_URL: Your Supabase URL - SUPABASE_KEY: Your Supabase key for accessing the database - -Author: GAIA Team -Date: April 30, 2025 -""" - -import os -import sys -import json -import time -import uuid -import logging -import argparse -import traceback -from datetime import datetime -from typing import Dict, Any, List, Optional, Tuple, Union - -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(name)s - %(levelname)s - %(message)s" -) -logger = logging.getLogger("gaia_verification") - -console_handler = logging.StreamHandler() -console_handler.setLevel(logging.INFO) -formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s") -console_handler.setFormatter(formatter) -logger.addHandler(console_handler) - -try: - from agent.agent import GAIAAgent - from agent.config import get_memory_config - from agent.config import SUPABASE_URL, SUPABASE_KEY - - from memory.supabase_memory import ( - SupabaseMemory, - WorkingMemory, - ConversationMemory, - ResultCache - ) - - from memory.fix_supabase_tables import apply_monkey_patch, ensure_tables_exist - - # Import Supabase client - from supabase import create_client, Client - - # Import OpenAI for simulating agent responses - from openai import OpenAI - - logger.info("All components imported successfully") -except ImportError as e: - logger.error(f"Error importing required components: {str(e)}") - logger.error(f"Traceback: {traceback.format_exc()}") - sys.exit(1) - -def check_environment_variables() -> bool: - """ - Check if all required environment variables are set. - - Returns: - bool: True if all required variables are set, False otherwise - """ - required_vars = { - "OPENAI_API_KEY": os.getenv("OPENAI_API_KEY"), - "SUPABASE_KEY": os.getenv("SUPABASE_KEY") - } - - missing_vars = [var for var, value in required_vars.items() if not value] - if missing_vars: - logger.error(f"Missing required environment variables: {', '.join(missing_vars)}") - for var in missing_vars: - logger.error(f"Please set {var} in your environment or .env file") - return False - - return True - -def get_table_content(client: Client, table_name: str) -> List[Dict[str, Any]]: - """ - Get the content of a Supabase table. - - Args: - client (Client): The Supabase client - table_name (str): The name of the table to query - - Returns: - List[Dict[str, Any]]: The table content as a list of dictionaries - """ - try: - response = client.table(table_name).select("*").execute() - if response.data: - logger.info(f"Found {len(response.data)} entries in table {table_name}") - return response.data - else: - logger.info(f"No entries found in table {table_name}") - return [] - except Exception as e: - logger.error(f"Error getting table content: {str(e)}") - return [] - -def print_table_summary(client: Client, table_name: str, description: str = "", - show_all: bool = False, max_entries: int = 5) -> None: - """ - Print a summary of a table's content. - - Args: - client (Client): The Supabase client - table_name (str): The name of the table to summarize - description (str, optional): A description of the table. Defaults to "". - show_all (bool, optional): Whether to show all entries. Defaults to False. - max_entries (int, optional): Maximum number of entries to show. Defaults to 5. - """ - content = get_table_content(client, table_name) - - - if content: - display_content = content if show_all else content[:max_entries] - - for i, entry in enumerate(display_content): - - value = entry.get('value', 'N/A') - if value and value != 'N/A': - if isinstance(value, str) and len(value) > 100 and not show_all: - value = value[:100] + "..." - - if entry.get('expires_at'): - # Process entries with expiration date - pass - - if not show_all and len(content) > max_entries: - logger.info(f"... and {len(content) - max_entries} more entries (use --show-all to see all)") - else: - logger.info(f"No entries found in {table_name}") -class GAIAVerifier: - """ - Class for verifying the GAIA agent's functionality with Supabase tables. - """ - - def __init__(self, verbose: bool = False, show_all: bool = False, metrics: bool = False): - """ - Initialize the GAIA verifier. - - Args: - verbose (bool): Whether to enable verbose logging - show_all (bool): Whether to show all table entries - metrics (bool): Whether to show detailed performance metrics - """ - self.verbose = verbose - self.show_all = show_all - self.metrics = metrics - - if verbose: - logger.setLevel(logging.DEBUG) - console_handler.setLevel(logging.DEBUG) - - logger.info("Applying fix for Supabase tables...") - try: - ensure_tables_exist() - apply_monkey_patch() - logger.info("Supabase tables fix applied successfully") - except Exception as e: - logger.error(f"Error applying Supabase tables fix: {str(e)}") - logger.error(f"Traceback: {traceback.format_exc()}") - raise - - try: - self.supabase_client = create_client(SUPABASE_URL, SUPABASE_KEY) - logger.info("Supabase client created successfully") - except Exception as e: - logger.error(f"Error creating Supabase client: {str(e)}") - logger.error(f"Traceback: {traceback.format_exc()}") - raise - - logger.info("Initializing memory system...") - - try: - self.memory_config = get_memory_config() - self.memory_config["enabled"] = True - except Exception as e: - logger.error(f"Error getting memory configuration: {str(e)}") - logger.error(f"Traceback: {traceback.format_exc()}") - raise - - try: - self.memory = SupabaseMemory(self.memory_config) - logger.info("Memory system initialized successfully") - except Exception as e: - logger.error(f"Error initializing memory system: {str(e)}") - logger.error(f"Traceback: {traceback.format_exc()}") - raise - - self.session_id = str(uuid.uuid4()) - self.memory.session_id = self.session_id - logger.info(f"Session ID: {self.session_id}") - - self.conversation_memory = ConversationMemory(self.memory, self.session_id) - self.result_cache = ResultCache(self.memory) - self.working_memory = WorkingMemory(self.memory, self.session_id) - - self.base_table = self.memory.table_name - self.working_table = f"{self.base_table}_working" - self.conversations_table = f"{self.base_table}_conversations" - self.cache_table = f"{self.base_table}_cache" - - self.tables = { - self.base_table: "Main memory table", - self.working_table: "Working memory table", - self.conversations_table: "Conversation history table", - self.cache_table: "Result cache table" - } - - try: - self.agent = GAIAAgent() - logger.info("GAIA agent initialized successfully") - except Exception as e: - logger.error(f"Error initializing GAIA agent: {str(e)}") - logger.error(f"Traceback: {traceback.format_exc()}") - raise - - def get_initial_table_state(self) -> Dict[str, int]: - """ - Get the initial state of all tables. - - Returns: - Dict[str, int]: Dictionary with table names and entry counts - """ - logger.info("Getting initial table state...") - initial_state = {} - - for table_name in self.tables.keys(): - initial_state[table_name] = len(get_table_content(self.supabase_client, table_name)) - - return initial_state - - def get_final_table_state(self) -> Dict[str, int]: - """ - Get the final state of all tables. - - Returns: - Dict[str, int]: Dictionary with table names and entry counts - """ - logger.info("Getting final table state...") - final_state = {} - - for table_name in self.tables.keys(): - final_state[table_name] = len(get_table_content(self.supabase_client, table_name)) - - return final_state - - def print_table_state_comparison(self, initial_state: Dict[str, int], final_state: Dict[str, int]) -> None: - """ - Print a comparison of initial and final table states. - - Args: - initial_state (Dict[str, int]): Initial table state - final_state (Dict[str, int]): Final table state - """ - for table_name in self.tables.keys(): - initial = initial_state.get(table_name, 0) - final = final_state.get(table_name, 0) - new_entries = final - initial - - logger.info(f"Table {table_name}: {initial} -> {final} entries ({new_entries} new)") - - - - def print_detailed_table_content(self) -> None: - """Print detailed content of all tables.""" - if self.show_all: - for table_name, description in self.tables.items(): - print_table_summary(self.supabase_client, table_name, description, self.show_all) - def test_simple_question(self) -> Dict[str, Any]: - """ - Test the agent with a simple question. - - Returns: - Dict[str, Any]: Test results - """ - logger.info("Testing simple question...") - - initial_state = self.get_initial_table_state() - - question = "What is the capital of France?" - - start_time = time.time() - try: - answer = self.agent(question) - execution_time = time.time() - start_time - - - final_state = self.get_final_table_state() - - self.print_table_state_comparison(initial_state, final_state) - - self.print_detailed_table_content() - - return { - "question": question, - "answer": answer, - "execution_time": execution_time, - "initial_state": initial_state, - "final_state": final_state - } - except Exception as e: - logger.error(f"Error processing simple question: {str(e)}") - logger.error(f"Traceback: {traceback.format_exc()}") - execution_time = time.time() - start_time - - return { - "question": question, - "error": str(e), - "execution_time": execution_time, - "initial_state": initial_state, - "final_state": self.get_final_table_state() - } - - def test_cached_question(self) -> Dict[str, Any]: - """ - Test the agent with a cached question. - - Returns: - Dict[str, Any]: Test results - """ - logger.info("Testing cached question...") - - initial_state = self.get_initial_table_state() - - question = "What is the capital of France?" - - start_time = time.time() - try: - answer = self.agent(question) - execution_time = time.time() - start_time - - - final_state = self.get_final_table_state() - - self.print_table_state_comparison(initial_state, final_state) - - self.print_detailed_table_content() - - return { - "question": question, - "answer": answer, - "execution_time": execution_time, - "initial_state": initial_state, - "final_state": final_state - } - except Exception as e: - logger.error(f"Error processing cached question: {str(e)}") - logger.error(f"Traceback: {traceback.format_exc()}") - execution_time = time.time() - start_time - - return { - "question": question, - "error": str(e), - "execution_time": execution_time, - "initial_state": initial_state, - "final_state": self.get_final_table_state() - } - - def test_follow_up_question(self) -> Dict[str, Any]: - """ - Test the agent with a follow-up question. - - Returns: - Dict[str, Any]: Test results - """ - logger.info("Testing follow-up question...") - - initial_state = self.get_initial_table_state() - - question = "What is the population of that city?" - - start_time = time.time() - try: - answer = self.agent(question) - execution_time = time.time() - start_time - - - final_state = self.get_final_table_state() - - self.print_table_state_comparison(initial_state, final_state) - - self.print_detailed_table_content() - - return { - "question": question, - "answer": answer, - "execution_time": execution_time, - "initial_state": initial_state, - "final_state": final_state - } - except Exception as e: - logger.error(f"Error processing follow-up question: {str(e)}") - logger.error(f"Traceback: {traceback.format_exc()}") - execution_time = time.time() - start_time - - return { - "question": question, - "error": str(e), - "execution_time": execution_time, - "initial_state": initial_state, - "final_state": self.get_final_table_state() - } - - def test_complex_question(self) -> Dict[str, Any]: - """ - Test the agent with a complex question that exercises all memory types. - - Returns: - Dict[str, Any]: Test results - """ - logger.info("Testing complex question...") - - initial_state = self.get_initial_table_state() - - question = """ - I need a comprehensive analysis of renewable energy trends. - First, what are the top 3 renewable energy sources by global capacity? - Second, which countries are leading in renewable energy adoption? - Finally, what are the projected growth rates for solar and wind energy over the next decade? - """ - - start_time = time.time() - try: - answer = self.agent(question) - execution_time = time.time() - start_time - - - final_state = self.get_final_table_state() - - self.print_table_state_comparison(initial_state, final_state) - - self.print_detailed_table_content() - - return { - "question": question, - "answer": answer, - "execution_time": execution_time, - "initial_state": initial_state, - "final_state": final_state - } - except Exception as e: - logger.error(f"Error processing complex question: {str(e)}") - logger.error(f"Traceback: {traceback.format_exc()}") - execution_time = time.time() - start_time - - return { - "question": question, - "error": str(e), - "execution_time": execution_time, - "initial_state": initial_state, - "final_state": self.get_final_table_state() - } - - def test_multi_turn_conversation(self) -> Dict[str, Any]: - """ - Test the agent with a multi-turn conversation. - - Returns: - Dict[str, Any]: Test results - """ - logger.info("Testing multi-turn conversation...") - - initial_state = self.get_initial_table_state() - - conversation = [] - - question1 = "What are the main challenges in artificial intelligence development?" - - start_time = time.time() - try: - answer1 = self.agent(question1) - execution_time1 = time.time() - start_time - - - conversation.append({ - "question": question1, - "answer": answer1, - "execution_time": execution_time1 - }) - - question2 = "How does deep learning address some of these challenges?" - - start_time = time.time() - answer2 = self.agent(question2) - execution_time2 = time.time() - start_time - - - conversation.append({ - "question": question2, - "answer": answer2, - "execution_time": execution_time2 - }) - - question3 = "What ethical considerations should be taken into account when developing AI systems?" - - start_time = time.time() - answer3 = self.agent(question3) - execution_time3 = time.time() - start_time - - - conversation.append({ - "question": question3, - "answer": answer3, - "execution_time": execution_time3 - }) - - final_state = self.get_final_table_state() - - self.print_table_state_comparison(initial_state, final_state) - - self.print_detailed_table_content() - - messages = self.conversation_memory.get_messages() - - for i, message in enumerate(messages): - role = message.get("role", "unknown") - content = message.get("content", "") - logger.debug(f"Message {i}: {role} - {content[:50]}...") - - return { - "conversation": conversation, - "messages": messages, - "initial_state": initial_state, - "final_state": final_state - } - except Exception as e: - logger.error(f"Error processing multi-turn conversation: {str(e)}") - logger.error(f"Traceback: {traceback.format_exc()}") - - return { - "error": str(e), - "conversation": conversation, - "initial_state": initial_state, - "final_state": self.get_final_table_state() - } - - def verify_working_memory_usage(self) -> Dict[str, Any]: - """ - Verify that working memory is being used for complex reasoning. - - Returns: - Dict[str, Any]: Verification results - """ - logger.info("Verifying working memory usage...") - - working_memory_content = get_table_content(self.supabase_client, self.working_table) - - has_entries = len(working_memory_content) > 0 - - working_memory_keys = self.working_memory.list_intermediate_results() - - - examples = {} - for key, value in working_memory_keys.items(): - if isinstance(value, str) and len(value) > 100: - examples[key] = value[:100] + "..." - else: - examples[key] = value - - for key, value in examples.items(): - logger.debug(f"Working memory key: {key}, value: {value}") - - return { - "has_entries": has_entries, - "keys": list(working_memory_keys.keys()), - "examples": examples - } - - def verify_conversation_memory_usage(self) -> Dict[str, Any]: - """ - Verify that conversation memory is being used. - - Returns: - Dict[str, Any]: Verification results - """ - logger.info("Verifying conversation memory usage...") - - conversation_memory_content = get_table_content(self.supabase_client, self.conversations_table) - - has_entries = len(conversation_memory_content) > 0 - messages = self.conversation_memory.get_messages() - - logger.info(f"Found {len(messages)} messages in conversation memory") - - - return { - "has_entries": has_entries, - "message_count": len(messages), - "messages": messages - } - - def verify_cache_usage(self) -> Dict[str, Any]: - """ - Verify that the result cache is being used. - - Returns: - Dict[str, Any]: Verification results - """ - logger.info("Verifying cache usage...") - - cache_content = get_table_content(self.supabase_client, self.cache_table) - has_entries = len(cache_content) > 0 - - logger.info(f"Found {len(cache_content)} entries in result cache") - - - return { - "has_entries": has_entries, - "entry_count": len(cache_content) - } - - def run_all_tests(self) -> Dict[str, Any]: - """ - Run all verification tests. - - Returns: - Dict[str, Any]: All test results - """ - results = {} - - logger.info("Running all verification tests...") - - results["simple_question"] = self.test_simple_question() - - results["cached_question"] = self.test_cached_question() - - results["follow_up_question"] = self.test_follow_up_question() - - results["complex_question"] = self.test_complex_question() - - results["multi_turn_conversation"] = self.test_multi_turn_conversation() - - results["working_memory_verification"] = self.verify_working_memory_usage() - - results["conversation_memory_verification"] = self.verify_conversation_memory_usage() - - results["cache_verification"] = self.verify_cache_usage() - - return results - -def main(): - """Main entry point for the script.""" - parser = argparse.ArgumentParser(description="Run GAIA agent final integration test") - parser.add_argument("--verbose", action="store_true", - help="Enable verbose logging") - parser.add_argument("--show-all", action="store_true", - help="Show all table entries") - parser.add_argument("--metrics", action="store_true", - help="Show detailed performance metrics") - - args = parser.parse_args() - - if len(sys.argv) == 1: - parser.print_help() - return - - if not check_environment_variables(): - return - - try: - verifier = GAIAVerifier( - verbose=args.verbose, - show_all=args.show_all, - metrics=args.metrics - ) - - results = verifier.run_all_tests() - - timestamp = datetime.now().strftime("%Y%m%d_%H%M%S") - filename = f"gaia_assessment_results_{timestamp}.json" - - serializable_results = { - "timestamp": timestamp, - "tests": {} - } - - for test_name, test_result in results.items(): - if "messages" in test_result: - test_result["messages"] = [ - {k: v for k, v in message.items() if isinstance(v, (str, int, float, bool, list, dict))} - for message in test_result["messages"] - ] - - serializable_results["tests"][test_name] = test_result - - with open(filename, "w") as f: - json.dump(serializable_results, f, indent=2) - - logger.info(f"Results saved to {filename}") - logger.info("All verification tests completed successfully") - - except Exception as e: - logger.error(f"Error running GAIA agent verification: {str(e)}") - logger.error(f"Traceback: {traceback.format_exc()}") - return 1 - - return 0 - -if __name__ == "__main__": - sys.exit(main()) \ No newline at end of file diff --git a/verify_minimal.py b/verify_minimal.py deleted file mode 100644 index 4008ead9cec8e35262261a40b9ad4ed06fd5a868..0000000000000000000000000000000000000000 --- a/verify_minimal.py +++ /dev/null @@ -1,120 +0,0 @@ -""" -Minimal deployment verification script for GAIA Agent -""" - -import os -import sys -from datetime import datetime -import traceback - -def print_header(message): - print("\n" + "=" * 50) - print(message) - print("=" * 50) - -def check_file_structure(): - """Check if critical files are present.""" - print_header("Checking file structure") - - critical_files = [ - "app.py", - "requirements.txt", - "agent/config.py", - "agent/agent.py" - ] - - missing_files = [] - for file_path in critical_files: - if not os.path.exists(file_path): - missing_files.append(file_path) - print(f"[FAIL] Missing critical file: {file_path}") - else: - print(f"[PASS] Found file: {file_path}") - - return len(missing_files) == 0, missing_files - -def check_env_vars(): - """Check if critical environment variables are accessible.""" - print_header("Checking environment variables") - - critical_vars = [ - "OPENAI_API_KEY", - "SUPABASE_URL", - "SUPABASE_KEY" - ] - - missing_vars = [] - for var_name in critical_vars: - var_value = os.getenv(var_name, "") - if not var_value: - missing_vars.append(var_name) - print(f"[FAIL] Missing environment variable: {var_name}") - else: - print(f"[PASS] Found environment variable: {var_name}") - - return len(missing_vars) == 0, missing_vars - -def check_packages(): - """Check if requirements.txt contains critical packages.""" - print_header("Checking requirements.txt packages") - - critical_packages = [ - "openai", - "gradio", - "langchain" - ] - - if not os.path.exists("requirements.txt"): - print("[FAIL] requirements.txt file not found") - return False, ["requirements.txt file not found"] - - try: - with open("requirements.txt", "r") as f: - content = f.read().lower() - - missing_packages = [] - for package in critical_packages: - if package.lower() not in content: - missing_packages.append(package) - print(f"[FAIL] Missing package in requirements.txt: {package}") - else: - print(f"[PASS] Found package in requirements.txt: {package}") - - return len(missing_packages) == 0, missing_packages - except Exception as e: - print(f"[ERROR] Error reading requirements.txt: {str(e)}") - return False, [f"Error reading requirements.txt: {str(e)}"] - -def main(): - """Run minimal verification checks.""" - try: - print_header(f"GAIA Agent Minimal Verification - {datetime.now()}") - - # Run checks - file_check = check_file_structure() - env_check = check_env_vars() - pkg_check = check_packages() - - # Print summary - print_header("Verification Summary") - print(f"File structure check: {'[PASS]' if file_check[0] else '[FAIL]'}") - print(f"Environment variables check: {'[PASS]' if env_check[0] else '[FAIL]'}") - print(f"Package requirements check: {'[PASS]' if pkg_check[0] else '[FAIL]'}") - - # Overall status - if file_check[0] and env_check[0] and pkg_check[0]: - print("\n[SUCCESS] All checks passed. Basic deployment verification successful.") - return 0 - else: - print("\n[FAILURE] Some checks failed. Please address the issues before deployment.") - return 1 - - except Exception as e: - print_header("ERROR OCCURRED") - print(f"Error during verification: {str(e)}") - print("\nTraceback:") - traceback.print_exc() - return 1 - -if __name__ == "__main__": - sys.exit(main()) \ No newline at end of file diff --git a/verify_production.py b/verify_production.py deleted file mode 100644 index ab0967afa7a7e3cbe114896240f5db05213dc42c..0000000000000000000000000000000000000000 --- a/verify_production.py +++ /dev/null @@ -1,462 +0,0 @@ -""" -GAIA Production Verification Script for Hugging Face Space - -This script tests all critical components of the GAIA system after deployment: -- OpenAI API connection -- Supabase memory integration -- Search tools (Serper, DuckDuckGo) -- Agent workflow - -Usage: - python verify_production.py [--verbose] - -Author: DevOps Team -Date: April 30, 2025 -""" - -import os -import sys -import json -import time -import argparse -import logging -from typing import Dict, Any, List, Optional, Tuple - -# Configure logging -logging.basicConfig( - level=logging.INFO, - format="%(asctime)s - %(name)s - %(levelname)s - %(message)s", - handlers=[logging.StreamHandler()] -) - -logger = logging.getLogger("GAIA-Verifier") - -# Import required components with error handling -try: - # Try importing OpenAI - from openai import OpenAI - has_openai = True -except ImportError: - logger.error("OpenAI SDK not installed. Run: pip install openai") - has_openai = False - -try: - # Try importing Supabase - from supabase import create_client, Client - has_supabase = True -except ImportError: - logger.error("Supabase SDK not installed. Run: pip install supabase") - has_supabase = False - -# Import agent components with error handling -try: - from agent.agent import Agent - from agent.config import get_config - has_agent = True -except ImportError: - logger.error("Agent module not found or incomplete") - has_agent = False - -class ProductionVerifier: - """Production verification test suite for GAIA system.""" - - def __init__(self, verbose: bool = False): - """Initialize the verifier with configuration.""" - self.verbose = verbose - if verbose: - logger.setLevel(logging.DEBUG) - - self.results = { - "openai": {"success": False, "message": "Not tested"}, - "supabase": {"success": False, "message": "Not tested"}, - "search_tools": {"success": False, "message": "Not tested"}, - "agent_workflow": {"success": False, "message": "Not tested"} - } - - # Check environment variables - self.env_vars = self._check_environment_variables() - - # Initialize clients - self.openai_client = None - self.supabase_client = None - self.agent = None - - if has_openai and self.env_vars["openai"]: - try: - self.openai_client = OpenAI(api_key=os.getenv("OPENAI_API_KEY")) - logger.debug("OpenAI client initialized") - except Exception as e: - logger.error(f"Error initializing OpenAI client: {str(e)}") - - if has_supabase and self.env_vars["supabase"]: - try: - supabase_url = os.getenv("SUPABASE_URL") - supabase_key = os.getenv("SUPABASE_KEY") - self.supabase_client = create_client(supabase_url, supabase_key) - logger.debug("Supabase client initialized") - except Exception as e: - logger.error(f"Error initializing Supabase client: {str(e)}") - - if has_agent: - try: - config = get_config() - self.agent = Agent() - logger.debug("Agent initialized") - except Exception as e: - logger.error(f"Error initializing Agent: {str(e)}") - - def _check_environment_variables(self) -> Dict[str, bool]: - """Check if required environment variables are set.""" - env_checks = { - "openai": bool(os.getenv("OPENAI_API_KEY")), - "supabase": bool(os.getenv("SUPABASE_URL") and os.getenv("SUPABASE_KEY")), - "serper": bool(os.getenv("SERPER_API_KEY")), - "perplexity": bool(os.getenv("PERPLEXITY_API_KEY")), - } - - missing_vars = [key for key, value in env_checks.items() if not value] - if missing_vars: - logger.warning(f"Missing environment variables for: {', '.join(missing_vars)}") - else: - logger.info("All required environment variables are set") - - return env_checks - - def test_openai_connection(self) -> Dict[str, Any]: - """Test OpenAI API connection.""" - logger.info("Testing OpenAI API connection...") - - if not has_openai: - return {"success": False, "message": "OpenAI SDK not installed"} - - if not self.env_vars["openai"]: - return {"success": False, "message": "OpenAI API key not set"} - - if not self.openai_client: - return {"success": False, "message": "OpenAI client not initialized"} - - try: - # Test listing models - models = self.openai_client.models.list() - model_count = len(models.data) - - # Test a basic completion - response = self.openai_client.chat.completions.create( - model="gpt-3.5-turbo", - messages=[ - {"role": "system", "content": "You are a helpful assistant."}, - {"role": "user", "content": "Say hello world"} - ], - max_tokens=20 - ) - - response_text = response.choices[0].message.content - - logger.info(f"OpenAI API connection successful. Available models: {model_count}") - logger.debug(f"OpenAI response: {response_text}") - - return { - "success": True, - "message": "OpenAI API connection successful", - "model_count": model_count, - "response": response_text - } - except Exception as e: - logger.error(f"Error testing OpenAI API: {str(e)}") - return {"success": False, "message": f"Error: {str(e)}"} - - def test_supabase_connection(self) -> Dict[str, Any]: - """Test Supabase connection and memory tables.""" - logger.info("Testing Supabase connection...") - - if not has_supabase: - return {"success": False, "message": "Supabase SDK not installed"} - - if not self.env_vars["supabase"]: - return {"success": False, "message": "Supabase credentials not set"} - - if not self.supabase_client: - return {"success": False, "message": "Supabase client not initialized"} - - try: - # Query available tables - tables_query = self.supabase_client.table("information_schema.tables")\ - .select("table_name")\ - .eq("table_schema", "public")\ - .execute() - - table_list = [record.get('table_name') for record in tables_query.data] - - # Check for required tables - memory_tables = ["gaia_memory", "gaia_memory_working", "gaia_memory_conversation"] - missing_tables = [table for table in memory_tables if table not in table_list] - - if missing_tables: - return { - "success": False, - "message": f"Missing tables: {', '.join(missing_tables)}", - "available_tables": table_list - } - - # Test writing to memory table - test_data = { - "session_id": f"test-{int(time.time())}", - "key": "test-key", - "value": json.dumps({"test": "value"}), - "metadata": json.dumps({"created_at": time.time()}) - } - - insert_result = self.supabase_client.table("gaia_memory").insert(test_data).execute() - - logger.info("Supabase connection successful. All required tables exist.") - - return { - "success": True, - "message": "Supabase connection successful", - "tables": table_list, - "write_test": "successful" - } - except Exception as e: - logger.error(f"Error testing Supabase: {str(e)}") - return {"success": False, "message": f"Error: {str(e)}"} - - def test_search_tools(self) -> Dict[str, Any]: - """Test search tools integration.""" - logger.info("Testing search tools...") - - results = { - "serper": {"success": False, "message": "Not tested"}, - "duckduckgo": {"success": False, "message": "Not tested"}, - } - - # Test Serper - if self.env_vars["serper"]: - try: - import requests - - url = "https://google.serper.dev/search" - payload = json.dumps({ - "q": "What is the capital of France", - "gl": "us", - "hl": "en" - }) - headers = { - 'X-API-KEY': os.getenv("SERPER_API_KEY"), - 'Content-Type': 'application/json' - } - - response = requests.request("POST", url, headers=headers, data=payload) - - if response.status_code == 200: - results["serper"] = { - "success": True, - "message": "Serper API connection successful" - } - logger.info("Serper API connection successful") - else: - results["serper"] = { - "success": False, - "message": f"Serper API error: {response.status_code} - {response.text}" - } - logger.error(f"Serper API error: {response.status_code}") - except Exception as e: - results["serper"] = {"success": False, "message": f"Error: {str(e)}"} - logger.error(f"Error testing Serper API: {str(e)}") - - # Test if the agent has the search tools registered - if has_agent and self.agent: - try: - tool_registry = self.agent.tool_registry - tools = tool_registry.get_tools() - - search_tools = [tool for tool in tools if "search" in tool.lower()] - - if search_tools: - logger.info(f"Agent has search tools: {', '.join(search_tools)}") - - # Mark overall search tools success if at least one search tool works - if results["serper"]["success"]: - return { - "success": True, - "message": "Search tools integration successful", - "tools": search_tools, - "details": results - } - except Exception as e: - logger.error(f"Error checking agent search tools: {str(e)}") - - # If we reached here, overall search wasn't successful - success = any(result["success"] for result in results.values()) - return { - "success": success, - "message": "Search tools partially working" if success else "Search tools integration failed", - "details": results - } - - def test_agent_workflow(self) -> Dict[str, Any]: - """Test the agent workflow with a simple query.""" - logger.info("Testing agent workflow...") - - if not has_agent: - return {"success": False, "message": "Agent module not installed"} - - if not self.agent: - return {"success": False, "message": "Agent not initialized"} - - try: - # Try a simple factual query that doesn't need external tools - response = self.agent.run("What is 2+2?") - - # If we got a response, consider it successful - if response and isinstance(response, str) and len(response) > 0: - logger.info("Agent workflow test successful") - return { - "success": True, - "message": "Agent workflow successful", - "response": response[:100] + "..." if len(response) > 100 else response - } - else: - logger.error("Agent returned empty or invalid response") - return { - "success": False, - "message": "Agent returned empty or invalid response" - } - except Exception as e: - logger.error(f"Error testing agent workflow: {str(e)}") - return {"success": False, "message": f"Error: {str(e)}"} - - def run_tests(self) -> Dict[str, Any]: - """Run all verification tests.""" - logger.info("Starting GAIA production verification...") - - # Test OpenAI - self.results["openai"] = self.test_openai_connection() - - # Test Supabase - self.results["supabase"] = self.test_supabase_connection() - - # Test search tools - self.results["search_tools"] = self.test_search_tools() - - # Test agent workflow - self.results["agent_workflow"] = self.test_agent_workflow() - - # Calculate overall success - success_count = sum(1 for result in self.results.values() if result["success"]) - total_count = len(self.results) - - overall_success = success_count == total_count - overall_message = ( - f"All {success_count}/{total_count} components verified successfully!" - if overall_success else - f"{success_count}/{total_count} components verified. Some components failed verification." - ) - - logger.info(overall_message) - - return { - "success": overall_success, - "message": overall_message, - "components": self.results, - "timestamp": time.time() - } - - def print_results(self, results: Dict[str, Any]) -> None: - """Print verification results in a readable format.""" - print("\n" + "="*50) - print("GAIA PRODUCTION VERIFICATION RESULTS") - print("="*50) - - print(f"\nOverall Status: {'✅ PASSED' if results['success'] else '❌ FAILED'}") - print(f"Message: {results['message']}") - print("\nComponent Results:") - - for component, result in results["components"].items(): - status = "✅ PASSED" if result["success"] else "❌ FAILED" - print(f"\n{component.upper()}: {status}") - print(f" {result['message']}") - - # Print additional details if available and verbose mode is on - if self.verbose: - for key, value in result.items(): - if key not in ["success", "message"]: - if isinstance(value, dict): - print(f" {key}:") - for k, v in value.items(): - print(f" {k}: {v}") - else: - print(f" {key}: {value}") - - print("\n" + "="*50) - - def save_results(self, results: Dict[str, Any], filename: str = "production_verification_report.md") -> str: - """Save verification results to a Markdown file.""" - with open(filename, "w") as f: - f.write("# GAIA Production Verification Report\n\n") - f.write(f"**Date:** {time.strftime('%Y-%m-%d %H:%M:%S')}\n\n") - - f.write(f"## Overall Status: {'✅ PASSED' if results['success'] else '❌ FAILED'}\n\n") - f.write(f"**Summary:** {results['message']}\n\n") - - f.write("## Component Results\n\n") - - for component, result in results["components"].items(): - status = "✅ PASSED" if result["success"] else "❌ FAILED" - f.write(f"### {component.upper()}: {status}\n\n") - f.write(f"- {result['message']}\n\n") - - # Write additional details - for key, value in result.items(): - if key not in ["success", "message"]: - if isinstance(value, dict): - f.write(f"**{key}:**\n\n") - for k, v in value.items(): - f.write(f"- {k}: {v}\n") - f.write("\n") - elif isinstance(value, list): - f.write(f"**{key}:**\n\n") - for item in value: - f.write(f"- {item}\n") - f.write("\n") - else: - f.write(f"**{key}:** {value}\n\n") - - f.write("## Environment Information\n\n") - - # Add information about environment variables (without sensitive data) - f.write("### Environment Variables\n\n") - for var, exists in self.env_vars.items(): - f.write(f"- {var}: {'✅ Set' if exists else '❌ Missing'}\n") - - f.write("\n### System Information\n\n") - f.write(f"- Python Version: {sys.version.split()[0]}\n") - f.write(f"- Platform: {sys.platform}\n") - - # Add Hugging Face Space info if available - hf_space = os.getenv("HF_SPACE_ID") - if hf_space: - f.write(f"- HF Space: {hf_space}\n") - - logger.info(f"Verification results saved to {filename}") - return filename - -def main(): - """Main function to run the verification script.""" - parser = argparse.ArgumentParser(description="GAIA Production Verification Script") - parser.add_argument("--verbose", action="store_true", help="Enable verbose output") - parser.add_argument("--output", default="production_verification_report.md", - help="Output file for verification report") - args = parser.parse_args() - - verifier = ProductionVerifier(verbose=args.verbose) - results = verifier.run_tests() - verifier.print_results(results) - report_path = verifier.save_results(results, args.output) - - print(f"\nVerification report saved to: {report_path}") - - # Return appropriate exit code - return 0 if results["success"] else 1 - -if __name__ == "__main__": - sys.exit(main()) \ No newline at end of file