README.md

metadata

title: Example App Hackathon Gustave Eiffel 2026
emoji: 🤖
colorFrom: blue
colorTo: purple
sdk: gradio
sdk_version: 4.44.1
python_version: '3.11'
app_file: app.py
pinned: false
license: apache-2.0

🗼 RAG Chat API — Gustave Eiffel Hackathon 2026

A complete Retrieval-Augmented Generation (RAG) system deployed as a Hugging Face Space, with a /query API endpoint designed for the RAG evaluation system.

---

Table of Contents

1. Overview
2. Architecture
3. Step-by-Step Explanation
4. API Endpoints
5. Setup & Deployment
6. Adding Binary Files to the HF Space
7. Configuration
8. How It Works (Detailed)

---

Overview

This application demonstrates how to build a production-ready RAG system within the Hugging Face ecosystem. It covers:

Requirement	Solution
LLM API calls	Azure OpenAI (gpt-5 via REST)
Text → Embeddings	Azure OpenAI (text-embedding-3-small via REST)
Vector Store	ChromaDB (persistent, runs in-process)
API Endpoint	FastAPI with POST /query
UI	Gradio Blocks (chat + document ingestion)

---

Architecture

┌─────────────────────────────────────────────────────────────┐
│                    Hugging Face Space                         │
│                                                              │
│  ┌──────────┐     ┌──────────────┐     ┌───────────────┐   │
│  │  Gradio  │     │   FastAPI    │     │   ChromaDB    │   │
│  │   UI     │────▶│  /query      │────▶│  Vector Store │   │
│  │          │     │  /ingest     │     │  (persistent) │   │
│  └──────────┘     └──────┬───────┘     └───────────────┘   │
│                          │                      ▲            │
│                          ▼                      │            │
│               ┌──────────────────┐    ┌─────────────────┐   │
│               │  Azure OpenAI    │    │  Azure OpenAI   │   │
│               │  GPT-5 (LLM)    │    │  text-embedding │   │
│               │                  │    │  -3-small       │   │
│               └──────────────────┘    └─────────────────┘   │
└─────────────────────────────────────────────────────────────┘

---

Step-by-Step Explanation

Step 1: Document Ingestion & Chunking

Before we can answer questions, we need to prepare our knowledge base.

1. Load documents — Read text files from sample_documents/ directory
2. Chunk text — Split documents into smaller overlapping chunks (512 tokens, 50 token overlap) using RecursiveCharacterTextSplitter. This ensures each chunk fits within the embedding model's context window while maintaining semantic coherence.

splitter = RecursiveCharacterTextSplitter(
    chunk_size=512,
    chunk_overlap=50,
    separators=["\n\n", "\n", ". ", " ", ""],
)
chunks = splitter.split_text(document_text)

Step 2: Generate Embeddings

Convert text chunks into dense vector representations that capture semantic meaning.

1. Call Azure OpenAI — We use the text-embedding-3-small model via the Azure OpenAI embeddings endpoint
2. Encode text — Each chunk is transformed into a fixed-size vector where semantically similar texts are closer together in vector space

import requests as http_requests

headers = {"api-key": AZURE_API_KEY, "Content-Type": "application/json"}
payload = {"input": ["chunk 1 text", "chunk 2 text"], "model": "text-embedding-3-small"}
resp = http_requests.post(EMBEDDING_ENDPOINT_URL, headers=headers, json=payload)
embeddings = [item["embedding"] for item in resp.json()["data"]]

Step 3: Store in Vector Database (ChromaDB)

Persist embeddings in a vector store optimized for similarity search.

1. Initialize ChromaDB — Create a persistent client that stores data on disk (survives Space restarts)
2. Create collection — A named collection with cosine similarity metric
3. Add documents — Store embeddings alongside the original text and metadata

import chromadb

client = chromadb.PersistentClient(path="./data/chroma_db")
collection = client.get_or_create_collection(
    name="rag_documents",
    metadata={"hnsw:space": "cosine"},
)
collection.add(
    ids=["doc_0", "doc_1"],
    embeddings=embeddings.tolist(),
    documents=["chunk 1 text", "chunk 2 text"],
    metadatas=[{"source": "file.txt"}, {"source": "file.txt"}],
)

Step 4: Query & Retrieval

When a user asks a question, find the most relevant context.

1. Embed the query — Use the same Azure OpenAI embedding model to convert the question to a vector
2. Similarity search — Find the top-K nearest vectors in ChromaDB (cosine similarity)
3. Return context — Extract the original text chunks for the closest matches

query_embedding = generate_embeddings(["What is the Eiffel Tower?"])[0]
results = collection.query(
    query_embeddings=[query_embedding],
    n_results=3,
)

Step 5: LLM Generation (Augmented Response)

Combine retrieved context with the user's question and generate an answer.

1. Build prompt — Load the template from prompts/rag_prompt.txt, inject retrieved context and the user's question
2. Call Azure OpenAI — Send the prompt to the Azure OpenAI chat/completions endpoint (gpt-5)
3. Return response — The LLM generates an answer grounded in the provided context

The prompt template (prompts/rag_prompt.txt):

You are a helpful assistant. Answer the user's question based ONLY on the provided context.
If the context does not contain enough information to answer, say "I don't have enough information to answer this question."
Always be concise and factual.

Context:
{context}

Question: {question}

The template is loaded once at startup and sent as the user message to the chat endpoint:

RAG_PROMPT_TEMPLATE = Path("prompts/rag_prompt.txt").read_text(encoding="utf-8")

# At query time:
prompt = RAG_PROMPT_TEMPLATE.format(context=context_text, question=user_query)
headers = {"api-key": AZURE_API_KEY, "Content-Type": "application/json"}
payload = {
    "model": "gpt-5",
    "messages": [{"role": "user", "content": prompt}],
    "max_completion_tokens": 512,
    "temperature": 0.7,
    "top_p": 0.95,
}
resp = requests.post(LLM_ENDPOINT_URL, headers=headers, json=payload)
answer = resp.json()["choices"][0]["message"]["content"]

Tip: Edit prompts/rag_prompt.txt to tune the model's behaviour (tone, language, output format) without touching application code.

Step 6: API Endpoint (/query)

The FastAPI endpoint ties everything together for the evaluation system.

@app.post("/query")
async def query_endpoint(request: QueryRequest):
    # 1. Retrieve relevant context
    # 2. Build augmented prompt
    # 3. Generate LLM response
    # 4. Return answer + sources
    result = rag_query(request.query, top_k=request.top_k)
    return JSONResponse(content=result)

---

API Endpoints

POST /query

The primary endpoint for the RAG evaluation system.

Request:

{
    "query": "What materials is the Eiffel Tower made of?",
    "top_k": 3
}

Response:

{
    "answer": "The Eiffel Tower is made of wrought iron (puddled iron)...",
    "sources": [
        {"source": "eiffel_tower.txt", "score": 0.87},
        {"source": "paris_landmarks.txt", "score": 0.72}
    ],
    "query": "What materials is the Eiffel Tower made of?"
}

POST /ingest

Add new documents to the knowledge base.

Request:

{
    "text": "The Eiffel Tower was built in 1889...",
    "source": "my_document.txt"
}

Response:

{
    "status": "success",
    "chunks_added": 5,
    "total_chunks": 42
}

GET /health

System health check.

Response:

{
    "status": "healthy",
    "documents_in_store": 42,
    "embedding_model": "text-embedding-3-small",
    "llm_model": "gpt-5"
}

---

Setup & Deployment

Prerequisites

• Python 3.11+
• A Hugging Face account with an API token

Local Development

# Clone the repository
git clone https://huggingface.co/spaces/YOUR_USERNAME/Example-App-Hackathon-Gustave-Eiffel-2026
cd Example-App-Hackathon-Gustave-Eiffel-2026

Set Up a Python Virtual Environment

Using a virtual environment isolates project dependencies from your global Python installation.

# Create the virtual environment (Python 3.11+ required)
python -m venv ~/.venv/hackathon-eiffel

# Activate it
# macOS / Linux
source ~/.venv/hackathon-eiffel/bin/activate
# Windows (PowerShell)
~\.venv\hackathon-eiffel\Scripts\Activate.ps1
# Windows (Command Prompt)
~\.venv\hackathon-eiffel\Scripts\activate.bat

Once your virtual environment is active, install dependencies and run the app:

# Install dependencies
pip install -r requirements.txt

# Set your Azure API key
# macOS / Linux
export AZURE_API_KEY="your_azure_api_key_here"
# Windows (PowerShell)
$env:AZURE_API_KEY = "your_azure_api_key_here"

# Run the application
python app.py
# Server starts at http://localhost:7860

Tip: To deactivate the virtual environment when you are done, run deactivate (venv) or conda deactivate (conda).

Deploy to Hugging Face Spaces

1. Create a new Space on huggingface.co (select Gradio SDK)
2. Add AZURE_API_KEY as a Space Secret (Settings → Secrets)
3. Push this code to the Space repository
4. The Space automatically installs dependencies and starts the app

Testing the API

# Health check
curl http://localhost:7860/health

# Query the RAG system
curl -X POST http://localhost:7860/query \
  -H "Content-Type: application/json" \
  -d '{"query": "What is the Eiffel Tower?", "top_k": 3}'

# Ingest a new document
curl -X POST http://localhost:7860/ingest \
  -H "Content-Type: application/json" \
  -d '{"text": "Your document text here...", "source": "new_doc.txt"}'

---

Adding Binary Files to the HF Space

Large or binary files (PDFs, pre-built ChromaDB databases, datasets, model weights) are stored in a Hugging Face bucket and mounted into the Space container. The hf sync command keeps your local folder in sync with the bucket.

Note: Git LFS is not supported for Hugging Face Spaces persistent storage. Use the bucket + hf sync workflow described here instead.

The /data Shared Folder

Each Hugging Face Space has a persistent storage bucket that is automatically mounted at /data inside the running container. This folder is the single shared location where the application reads and writes all persistent files — the ChromaDB vector store, ingested documents, and any binary assets.

Path in container	Purpose
/data/chroma_db/	ChromaDB vector store (survives Space restarts)
/data/sample_documents/	Text/PDF files auto-ingested on startup
/data/DataSet/	Training and evaluation datasets

Default bucket naming convention: A Space at
https://huggingface.co/spaces/<org>/<space-name>
gets a default storage bucket at
https://huggingface.co/buckets/<org>/<space-name>-storage

For this project:

• Space: https://huggingface.co/spaces/millimanfrance/Example-App-Hackathon-Gustave-Eiffel-2026
• Bucket: https://huggingface.co/buckets/millimanfrance/Example-App-Hackathon-Gustave-Eiffel-2026-storage

The hf:// URI for use with the CLI is:
hf://buckets/millimanfrance/Example-App-Hackathon-Gustave-Eiffel-2026-storage

Prerequisites

# Install the hf CLI (requires uv)
uv tool install "huggingface_hub[cli]"

# Authenticate (needs Write access to the bucket)
export HF_TOKEN="hf_your_token_here"
# or interactively:
hf auth login

Step 1 — Attach the Storage Bucket to Your Space

1. Open your Space on huggingface.co
2. Go to Settings → Persistent Storage
3. Under "Attach storage", select the existing bucket millimanfrance/Example-App-Hackathon-Gustave-Eiffel-2026-storage (or create a new one)
4. Save — Hugging Face will mount the bucket at /data inside the Space container

Step 2 — Sync Your Local ./data with the Space Bucket

Place all persistent files under a local ./data folder. The expected structure is:

./data/
├── chroma_db/          # Binary files and ChromaDB vector store
│   └── chroma.sqlite3
./train_data/           # Training/test datasets (PDFs, CSVs, etc.)
    ├── Automobile - Train/
    ├── Climatique-Train/
    └── ...

Upload (local → bucket):

# Mirror ./data to the bucket — removes remote files deleted locally
hf sync ./data hf://buckets/millimanfrance/Example-App-Hackathon-Gustave-Eiffel-2026-storage --delete

# Sync only a specific sub-folder (e.g., the vector store)
hf sync ./data/chroma_db hf://buckets/millimanfrance/Example-App-Hackathon-Gustave-Eiffel-2026-storage/chroma_db

# Sync only specific file types
hf sync ./data hf://buckets/millimanfrance/Example-App-Hackathon-Gustave-Eiffel-2026-storage \
  --include "*.pdf" --include "*.sqlite3"

--delete flag: Without it, hf sync only uploads new/changed files and never removes anything from the remote. Add --delete to make the bucket an exact mirror of your local ./data.

hf sync is incremental — it computes checksums and only transfers files that have changed.

Step 3 — Download the Bucket to Another Machine

To pull the latest bucket contents back to a local ./data folder (e.g., on a new dev machine):

hf sync hf://buckets/millimanfrance/Example-App-Hackathon-Gustave-Eiffel-2026-storage ./data

Step 4 — Sync Hackathon Training Data

The shared training dataset for this hackathon is stored in a separate read-only bucket. Sync it to a local ./train_data folder (it will be mounted at /train_data inside the Space):

# Download all training data locally
hf sync hf://buckets/millimanfrance/Hackathon2026TrainData ./train_data

# Download only a specific category
hf sync hf://buckets/millimanfrance/Hackathon2026TrainData/Automobile ./train_data/Automobile

Note: Hackathon2026TrainData is a shared read-only bucket. Do not attempt to push to it.

To make this data available inside your Space, push it to your Space's own bucket under a train_data/ sub-folder:

hf sync ./train_data hf://buckets/millimanfrance/Example-App-Hackathon-Gustave-Eiffel-2026-storage/train_data

It will then be accessible inside the container at /data/train_data/.

Accessing Files in the Space Application

Once the bucket is mounted, files appear under /data inside the Space container.

The application resolves the data root automatically via a single DATA_DIR constant in app.py:

from pathlib import Path

# /data when running in HF Spaces (bucket mount), ./data for local dev
DATA_DIR = Path("/data") if Path("/data").is_dir() else Path("./data")

CHROMA_PERSIST_DIR = str(DATA_DIR / "chroma_db")
SAMPLE_DOCS_DIR = DATA_DIR / "sample_documents"

All persistent data — the vector store, sample documents, datasets — lives under DATA_DIR so a single hf sync ./data ... covers everything.

Useful hf sync Flags

Flag	Description
--include "*.pdf"	Only sync files matching the pattern
--exclude "*.tmp"	Skip files matching the pattern
--delete	Remove files in the destination that no longer exist in the source
--dry-run	Preview what would be transferred without actually doing it

# Preview before committing
hf sync ./data hf://buckets/millimanfrance/Example-App-Hackathon-Gustave-Eiffel-2026-storage --dry-run

---

Configuration

The application loads model configuration with the following priority:

1. ./data/config.json — the live runtime config (read at startup)
2. ./config.json (project root) — example/template file used as fallback when no runtime config exists; never put real credentials here

Step 1 — Create data/config.json

Copy the root template to ./data/ and fill in your Azure OpenAI resource details:

cp config.json data/config.json

Then edit data/config.json:

{
    "embedding": {
        "endpoint_url": "https://<your-resource>.openai.azure.com/openai/deployments/<your-embedding-deployment>/embeddings?api-version=2024-12-01-preview",
        "model": "text-embedding-3-small"
    },
    "llm": {
        "endpoint_url": "https://<your-resource>.openai.azure.com/openai/deployments/<your-deployment>/chat/completions?api-version=2024-12-01-preview",
        "model": "gpt-5",
        "max_completion_tokens": 512,
        "temperature": 0.7,
        "top_p": 0.95
    }
}

Field	What to put
<your-resource>	Your Azure OpenAI resource name (from the Azure Portal)
<your-embedding-deployment>	The deployment name for your embedding model
<your-deployment>	The deployment name for your LLM
model	Must match the model name used when creating the deployment

Note: The endpoint_url for embeddings ends in /embeddings and the one for the LLM ends in /chat/completions. Keep those suffixes intact.

Step 2 — Set the API Key

The API key is not stored in config.json. Set it as an environment variable:

# macOS / Linux
export AZURE_API_KEY="your_azure_api_key_here"

# Windows (PowerShell)
$env:AZURE_API_KEY = "your_azure_api_key_here"

When deploying to Hugging Face Spaces, add it as a Space Secret (Settings → Secrets → New secret → name it AZURE_API_KEY).

Step 3 — Push data/config.json to the HF Bucket

data/config.json lives under ./data so a normal bucket sync includes it automatically:

hf sync ./data hf://buckets/millimanfrance/Example-App-Hackathon-Gustave-Eiffel-2026-storage --delete

The Space container will then find it at /data/config.json on next restart.

Do not commit data/config.json to Git — it contains endpoint URLs tied to your Azure resource. Add data/config.json to .gitignore. The root config.json is safe to commit as it only contains placeholder values.

Tunable Parameters

The following constants in app.py can also be adjusted directly:

Variable	Default	Description
AZURE_API_KEY	(env var / Space Secret)	Azure OpenAI API key (shared by LLM and embedding endpoints)
EMBEDDING_MODEL_NAME	text-embedding-3-small	Azure OpenAI embedding model
LLM_MODEL_NAME	gpt-5	Azure OpenAI LLM for answer generation
LLM_MAX_TOKENS	512	Maximum tokens in the LLM response
LLM_TEMPERATURE	0.7	Sampling temperature for the LLM
LLM_TOP_P	0.95	Top-p (nucleus) sampling for the LLM
DATA_DIR	/data (HF Spaces) or ./data (local)	Root directory for all persistent data
CHROMA_PERSIST_DIR	DATA_DIR/chroma_db	ChromaDB storage path
SAMPLE_DOCS_DIR	DATA_DIR/sample_documents	Documents ingested on startup
CHUNK_SIZE	512	Text chunk size in characters
CHUNK_OVERLAP	50	Overlap between chunks
TOP_K_RESULTS	3	Number of context chunks to retrieve

---

How It Works (Detailed)

Why RAG?

Large Language Models have a knowledge cutoff and can hallucinate. RAG solves this by:

• Grounding responses in actual documents
• Updating knowledge without retraining
• Providing source attribution for answers

Why ChromaDB in HF Spaces?

ChromaDB is ideal for Hugging Face Spaces because:

• Runs in-process (no external database needed)
• Supports persistent storage (survives Space restarts)
• Uses HNSW index for fast approximate nearest neighbor search
• Zero configuration required

Why Azure OpenAI Embeddings?

• High-quality embeddings — text-embedding-3-small produces state-of-the-art embeddings with excellent semantic understanding
• No local model loading — Eliminates GPU/memory requirements on the Space
• Scalable — Handles large batch embedding requests via the API
• Consistent — Same model used across environments (dev, staging, production)

Evaluation Integration

The /query endpoint is designed to be called by external RAG evaluation frameworks. It returns:

• The generated answer for correctness evaluation
• Source documents for faithfulness/groundedness checks
• Similarity scores for retrieval quality assessment

---

Troubleshooting

ChromaDB Telemetry Error

If you see capture() takes 1 positional argument but 3 were given in the logs, it is a version incompatibility between ChromaDB and the posthog library. Telemetry is disabled at startup (anonymized_telemetry=False) so this error should no longer appear. If it persists after updating dependencies, pin posthog<3.0 in requirements.txt.

LLM / Embedding API Errors

If the /query endpoint logs a 503 Service Unavailable or 401 Unauthorized:

1. Verify AZURE_API_KEY is set correctly as an environment variable or Space Secret
2. Check that the endpoint URLs in config.json match your Azure OpenAI deployment
3. Ensure your Azure OpenAI deployment is active and the model name matches the deployment name
4. The app calls the Azure OpenAI-compatible chat/completions and embeddings endpoints directly via REST

SSL Certificate Verification Error

If you see an error like the one below when running python app.py, your machine is most likely behind a corporate proxy that performs SSL inspection and has its own root CA that Python does not trust by default.

ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify
failed: unable to get local issuer certificate

Fix — point requests to your corporate CA bundle:

# macOS / Linux — use the system bundle (adjust path for your distro)
export REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt   # Debian/Ubuntu
export REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt     # RHEL/CentOS

# Or point to a specific corporate certificate file
export REQUESTS_CA_BUNDLE=/path/to/corporate-ca.crt

# Windows (PowerShell)
$env:REQUESTS_CA_BUNDLE = "C:\path\to\corporate-ca.crt"

Set the variable before running python app.py. The requests library (used for Azure OpenAI API calls) reads REQUESTS_CA_BUNDLE automatically — no code change is needed.

How to export your corporate CA certificate: Open the failing URL (https://huggingface.co) in your browser, click the padlock icon → View Certificate → export the root CA as a .crt / .pem file, then point REQUESTS_CA_BUNDLE to that file.

hf sync 401 Unauthorized Error

If hf sync fails with:

Error: Client error '401 Unauthorized' for url
'https://huggingface.co/api/buckets/.../tree?recursive=true'
Invalid username or password.

the hf CLI has not been authenticated. Log in with your Hugging Face token:

hf auth login
# Paste your token when prompted (needs read + write access to the bucket)

Or set the token as an environment variable so the CLI picks it up automatically without an interactive prompt:

export HF_TOKEN="hf_your_token_here"
hf sync ./data hf://buckets/millimanfrance/Example-App-Hackathon-Gustave-Eiffel-2026-storage

Generate a token: Go to huggingface.co → Settings → Access Tokens → New token. Select Write role so the CLI can both read and upload to the bucket.

Check current login state:

hf auth whoami

---

hf sync SSL Certificate Verification Error

The hf CLI (installed via uv tool install huggingface_hub) uses httpx internally instead of requests, so it ignores REQUESTS_CA_BUNDLE. If hf sync fails with:

httpcore.ConnectError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed:
unable to get local issuer certificate

you must set the certificate bundle through the variables that httpx (and the underlying ssl module) respects:

# Option A — point to your corporate CA bundle file (recommended)
export SSL_CERT_FILE=/path/to/corporate-ca.crt
export REQUESTS_CA_BUNDLE=/path/to/corporate-ca.crt   # keep this too for other tools

hf sync ./data hf://buckets/millimanfrance/Example-App-Hackathon-Gustave-Eiffel-2026-storage

# Option B — append your CA cert to the system bundle and point there
cat /path/to/corporate-ca.crt >> /etc/ssl/certs/ca-certificates.crt
export SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt

SSL_CERT_FILE overrides the default CA store for Python's ssl module, which httpx / httpcore uses directly.

If you need to set these variables permanently (e.g., in a shared dev environment), add them to your shell profile:

# ~/.bashrc or ~/.zshrc
export SSL_CERT_FILE=/path/to/corporate-ca.crt
export REQUESTS_CA_BUNDLE=/path/to/corporate-ca.crt

Finding your corporate CA cert on Linux:

# List all trusted CAs and look for your company's entry
awk -v cmd='openssl x509 -noout -subject' '/BEGIN CERT/{close(cmd)}; {print | cmd}' \
  /etc/ssl/certs/ca-certificates.crt | grep -i "your-company-name"

# Or export the cert directly from the proxy
echo | openssl s_client -connect huggingface.co:443 -showcerts 2>/dev/null \
  | openssl x509 -outform PEM > /tmp/hf-chain.pem
export SSL_CERT_FILE=/tmp/hf-chain.pem

---

License

Apache 2.0