---
title: Template Final Assignment
emoji: 🕵🏻♂️
colorFrom: indigo
colorTo: indigo
sdk: gradio
sdk_version: 6.14.0
app_file: app.py
pinned: false
hf_oauth: true
hf_oauth_expiration_minutes: 480
---
Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
# 🌍 GAIA Benchmark Agent
**An autonomous, multi-modal agent that tackles the GAIA reasoning benchmark.**
> [!NOTE]
> Developed as part of the **Hugging Face Agents Course (Unit 4: GAIA)**.
## 📖 Overview
GAIA tasks require multi-step reasoning, tool use, and the ability to process diverse file types (documents, spreadsheets, audio, images, code). This agent uses a **LangGraph** state machine to plan, retrieve few-shot context, call tools, and produce a strictly-formatted answer.
The solver model is **`Qwen/Qwen3-32B`** via Hugging Face Inference Providers; the formatter (separate node) reuses the same model bound to a single `emit_final_answer` tool to enforce the strict GAIA output contract. A Supabase + BM25 hybrid retriever surfaces similar past tasks as few-shot exemplars.
## 🚀 Key Features
- **🧠 Plan-Execute-Observe-Refine loop** driven by LangGraph.
- **🪜 Two-stage output**: a solver node reasons freely, then a dedicated formatter node returns the GAIA-compliant answer via a Pydantic-shaped tool call — eliminating regex parsing of the LLM's free text.
- **📂 Multi-modal file processing**:
- **Documents**: PDF, Word (`.docx`), PowerPoint (`.pptx`), Text
- **Data**: Excel (`.xlsx`, multi-sheet), CSV, JSON-LD, PDB (Protein Data Bank)
- **Media**: Audio transcription (`openai/whisper-large-v3` via HF), image analysis (`Qwen/Qwen3-VL-32B-Instruct`)
- **Code**: Python source files (read & execute)
- **Archives**: ZIP extraction + inspection
- **🔍 Hybrid RAG**: Vector search (Supabase RPC over `Alibaba-NLP/gte-modernbert-base`) **+** BM25 over a local 165-question corpus, fused with Reciprocal Rank Fusion, then reranked with a ModernBERT cross-encoder.
- **🌐 Web tooling**: DuckDuckGo, Tavily, Wikipedia, ArXiv, full-page extraction (Trafilatura), YouTube transcripts.
- **🛠️ Modular tool layout**: tools are organised by domain in `gaia/tools/` (basic, web, files, media, dispatcher) so adding a new capability is a single-file change.
## 🏗️ Architecture
```mermaid
graph TD
START --> FileDL["File Downloader
(fetches /files/{task_id})"]
FileDL --> Retriever["Retriever
(Vector + BM25 + RRF)"]
Retriever --> Reranker["Reranker
(ModernBERT cross-encoder)"]
Reranker --> Processor["Solver
(Qwen3-32B + tools)"]
Processor -->|tool call| Tools["Tool Node"]
Tools --> Processor
Processor -->|done| Formatter["Formatter
(emit_final_answer tool)"]
Formatter --> END
```
| Node | Role |
| :--- | :--- |
| **`file_downloader_node`** | If the question has an associated file, download from `{api.base_url}/files/{task_id}` and cache on disk under `data/task_files/{task_id}/`. |
| **`retriever_node`** | Hybrid search: Supabase vector RPC + local BM25 over `data/metadata.jsonl`, fused with RRF. Returns up to 20 candidate task IDs. |
| **`reranker_node`** | `Alibaba-NLP/gte-reranker-modernbert-base` re-scores candidates and injects the top-K as few-shot examples (Question + Final Answer + Solution Steps). |
| **`processor_node`** | Qwen3-32B with all tools bound. Reasons, calls tools, loops until satisfied. |
| **`tools`** | LangGraph `ToolNode` executing the chosen tool, then returning control to the processor. |
| **`formatter_node`** | A second Qwen3-32B call bound to a single `emit_final_answer(answer: str)` tool — produces the strictly-formatted value the GAIA scorer compares against. |
## 🛠️ Stack
| Category | Libraries | Purpose |
| :--- | :--- | :--- |
| **Orchestration** | `langgraph`, `langchain`, `langchain-huggingface` | State graph, tool binding, structured output. |
| **LLM / VLM / ASR** | `huggingface_hub` Inference API | `Qwen/Qwen3-32B`, `Qwen/Qwen3-VL-32B-Instruct`, `openai/whisper-large-v3`. |
| **Embeddings & Vector Store** | `sentence-transformers`, `supabase` | Semantic search via `Alibaba-NLP/gte-modernbert-base`. |
| **Keyword search** | `bm25s` | Local BM25 index over the 165-question GAIA corpus. |
| **Documents** | `pypdf`, `python-docx`, `python-pptx`, `openpyxl` | Office formats. |
| **Data** | `polars`, `biopython` | Tabular and PDB structural analysis. |
| **Media** | `pillow`, `librosa`, `soundfile` | Image + audio I/O. |
| **Web** | `ddgs`, `tavily-python`, `wikipedia`, `arxiv`, `trafilatura`, `youtube-transcript-api` | Search, page extraction, captions. |
| **UI** | `gradio` | Evaluation runner (HF Space entry point). |
## 💻 Installation & Setup
```bash
git clone
cd gaia
uv sync # or: pip install -r requirements.txt
cp .env.example .env # populate the keys below
```
Required environment variables (place in `.env` or set as HF Space Secrets):
| Variable | Required? | Purpose |
| :--- | :--- | :--- |
| `HF_INFERENCE_KEY` | yes | Hugging Face token — must have **"Make calls to Inference Providers"** permission. |
| `SUPABASE_URL` | yes (if `retrievers.enable_vector_search: true`) | Supabase project URL. |
| `SUPABASE_SERVICE_KEY` | yes (same) | Supabase `service_role` key. |
| `TAVILY_API_KEY` | optional | Only needed when the agent picks `tavily_web_search`. |
## 🎮 Usage
Start the Gradio interface:
```bash
python app.py
```
The UI requires a Hugging Face login. Click **"Run Evaluation & Submit All Answers"** to fetch the GAIA question set, run the agent on each, and submit to the scoring API.
To (re)populate the Supabase vector store from the local corpus:
```bash
python scripts/create_vector_database.py
```
## 📂 Project Structure
```
.
├── app.py # HF Space entry point (Gradio)
├── config.yaml # All tunable parameters
├── pyproject.toml / requirements.txt # Dependencies (uv + pip parity)
├── gaia/ # Application package
│ ├── agent.py # LangGraph nodes, graph, formatter
│ ├── states.py # AgentState TypedDict
│ ├── utils.py # config / prompt loaders, BM25, RRF, answer + youtube helpers
│ ├── prompts/
│ │ ├── prompt.yaml # Solver system prompt
│ │ └── vlm_prompt.yaml # analyze_image system prompt
│ └── tools/
│ ├── __init__.py # Aggregates tools_list
│ ├── basic.py # calculator, python_eval
│ ├── web.py # ddg / tavily / wiki / arxiv / fetch_webpage / youtube_transcript
│ ├── files.py # PDF, DOCX, PPTX, TXT, CSV, XLSX, JSON-LD, PDB, Python, ZIP
│ ├── media.py # analyze_image (VLM), transcribe_audio (ASR), shared HF client
│ └── dispatcher.py # read_file extension router
├── scripts/
│ └── create_vector_database.py # One-shot embedder for Supabase
├── notebooks/ # Exploratory work
├── data/
│ └── metadata.jsonl # Local GAIA corpus (165 examples)
└── models/ # HF model cache (gitignored)
```
All tunable knobs — model IDs, retrieval depth, thinking mode, recursion limit — live in `config.yaml`; no code change required to swap models or tweak retrieval.