Spaces:

purposeanalytics
/

organic-chatbot

Running

daniel-simeone

swap model

eb02516 10 months ago

9.76 kB

---
title: RAG Chatbot
emoji: 🤖
colorFrom: blue
colorTo: purple
sdk: gradio
sdk_version: 6.3.0
app_file: app.py
pinned: false
---
# Hugging Face RAG Chatbot

A Retrieval-Augmented Generation (RAG) chatbot that can ingest PDFs and URLs, then answer questions based on the ingested documents. Built for deployment on Hugging Face Spaces.

## Features

- 📄 **PDF Ingestion**: Upload and process PDF documents
- 🌐 **URL Ingestion**: Extract and process content from web URLs
- 🔍 **Vector Search**: Semantic search using sentence transformers
- 💬 **Chatbot Interface**: Interactive Gradio interface for querying documents
- 🚀 **Hugging Face Ready**: Configured for easy deployment to Hugging Face Spaces

## Setup

### Local Development

1. **Install dependencies:**
   ```bash
   pip install -r requirements.txt
   ```

2. **Run the application:**
   ```bash
   python app.py
   ```

3. **Access the interface:**
   - Open your browser to `http://localhost:7860`

### Usage

1. **Ingest Documents (Run this first or periodically to update):**
   - Add PDF files to the `pdfs/` folder
   - Edit `ingest_documents.py` and add your URLs to the `URLS` list
   - Run the ingestion script:
     ```bash
     py ingest_documents.py
     ```
   - Wait for processing to complete (this creates/updates the vector store)
2. **Chat with Documents:**
   - Run the chatbot app:
     ```bash
     py app.py
     ```
   - Open your browser to `http://localhost:7860`
   - Toggle "Use RAG" to enable/disable document retrieval
   - Ask questions about your ingested documents
   - The chatbot will retrieve relevant context and generate answers
## Deployment to Hugging Face Spaces

### Option 1: Using Hugging Face CLI

1. **Install Hugging Face CLI:**
   ```bash
   pip install huggingface_hub
   ```

2. **Login to Hugging Face:**
   ```bash
   huggingface-cli login
   ```

3. **Create a new Space:**
   - Go to https://huggingface.co/new-space
   - Choose a name and select "Gradio" as the SDK
   - Create the space

4. **Clone and push your code:**
   ```bash
   git clone https://huggingface.co/spaces/YOUR_USERNAME/YOUR_SPACE_NAME
   cd YOUR_SPACE_NAME
   # Copy your files here
   git add .
   git commit -m "Initial commit"
   git push
   ```

### Option 2: Using Git

1. **Initialize git repository:**
   ```bash
   git init
   git add .
   git commit -m "Initial commit"
   ```

2. **Add Hugging Face remote:**
   ```bash
   git remote add origin https://huggingface.co/spaces/YOUR_USERNAME/YOUR_SPACE_NAME
   git push -u origin main
   ```

### Required Files for Hugging Face Spaces

Your Space needs these files:
- `app.py` - Main Gradio application
- `requirements.txt` - Python dependencies
- `README.md` - This file (optional but recommended)

### Optional: Add app.py to README

For Hugging Face Spaces, you can also add a `app.py` reference in your README:

```yaml
---
title: RAG Chatbot
emoji: 🤖
colorFrom: blue
colorTo: purple
sdk: gradio
sdk_version: 4.0.0
app_file: app.py
pinned: false
---
```
## Configuration
### Setting Up Hugging Face Token (Required)
The chatbot uses Hugging Face Inference API to access high-quality models. You need to set up an API token:
1. **Get your token:**
   - Go to https://huggingface.co/settings/tokens
   - Create a new token with "Read" permissions
   - Copy the token
2. **For local development:**
   - Set environment variable: `export HF_TOKEN=your_token_here` (Linux/Mac)
   - Or: `set HF_TOKEN=your_token_here` (Windows)
   - Or create a `.env` file with `HF_TOKEN=your_token_here`
3. **For Hugging Face Spaces:**
   - Go to your Space → Settings → Secrets
   - Add a new secret: Name = `HF_TOKEN`, Value = your token
   - The app will automatically use this token
### Chatbot Model and Finding Available Models
The chatbot uses the Hugging Face Inference API. **Which models you can use depends on which providers you have enabled.**
**How to see which models are available to you:**
1. **Browse models that support Inference API**
   - https://huggingface.co/inference/models — lists providers and models
   - https://huggingface.co/models?inference_provider=hf-inference — filter Hub models by “HF Inference API”
2. **Enable providers (required)**  
   See **“How to enable a provider and model”** below.
3. **Pick a chat model**
   - From the links above, choose a **text generation / chat** model that’s supported by a provider you enabled. Note its **model id** (e.g. `meta-llama/Llama-3.2-1B-Instruct`).
4. **Use it in the app**
   - In `app.py`, pass that model id when creating the chatbot:
   ```python
   chatbot = RAGChatbot(model_name="meta-llama/Llama-3.2-1B-Instruct")  # use an id you enabled
   ```
   - The app also tries fallbacks (Phi-2, Zephyr, Qwen) by default; if none are available, enable a provider that supports at least one of them, or set `model_name` to a model you enabled as above.

### How to enable a provider and model

1. **Log in** to [Hugging Face](https://huggingface.co).

2. **Open Inference Provider settings** (one of these, depending on the current UI):
   - https://huggingface.co/settings/inference-providers  
   - https://huggingface.co/settings/inference-api  

3. **Enable a provider**
   - On that page you’ll see a list of **providers** (e.g. Hugging Face, Together, Groq, etc.).
   - **Turn on** the provider that serves your model (e.g. **Together** for `ServiceNow-AI/Apriel-1.6-15b-Thinker:together`).
   - You can set the **order** of providers; “auto” uses this order to pick which provider handles the request.

4. **Credits / billing**
   - Free accounts get a small amount of monthly credits; usage is deducted from that.
   - If you use a third-party provider (e.g. Together), you can either use HF-routed billing (credits on your HF account) or add a **custom provider API key** (e.g. Together API key) in the same settings so that provider is billed directly.

5. **Confirm the model**
   - Browse models for that provider:  
     [Together models on the Hub](https://huggingface.co/models?inference_provider=together&sort=trending)  
   - Open the model page (e.g. `ServiceNow-AI/Apriel-1.6-15b-Thinker`) and check the inference widget; if you see “Together” and can run it, that model is available with your enabled provider.
6. **Use it in this app**
   - The app default is already `ServiceNow-AI/Apriel-1.6-15b-Thinker:together`.  
   - Ensure **Together** is enabled in your Inference Provider settings and you have credits (or a Together API key). Then restart the app and send a message.

### Embedding Model

The default embedding model is **all-mpnet-base-v2**, which provides high-quality embeddings for better retrieval.

To change the embedding model, edit both `app.py` and `ingest_documents.py`:

```python
# In app.py
chatbot = RAGChatbot(embedding_model="all-mpnet-base-v2")
# In ingest_documents.py
ingestion = DocumentIngestion(embedding_model="all-mpnet-base-v2")
```

**Note:** If you change the embedding model, you must re-run `ingest_documents.py` to rebuild the vector store.

### Ingestion Parameters

The ingestion system uses optimized parameters:
- **Chunk size**: 600 characters (for precise retrieval)
- **Chunk overlap**: 150 characters (to avoid cutting sentences)
- **Retrieval count**: 5 chunks (for comprehensive context)

These parameters are set in `ingestion.py` and can be adjusted if needed.

## Project Structure

```
.
├── app.py                  # Main Gradio chatbot application
├── ingest_documents.py     # Standalone script to ingest PDFs and URLs
├── ingestion.py            # Document ingestion and vector store module
├── requirements.txt        # Python dependencies
├── README.md              # This file
├── pdfs/                  # Folder for PDF files (add your PDFs here)
│   └── README.md
└── data/
    └── vector_store/     # Saved vector store (created after ingestion)
        ├── index.faiss
        ├── documents.pkl
        └── embeddings.pkl
```

## How It Works

The chatbot uses **Retrieval-Augmented Generation (RAG)**:

1. **Document Ingestion**: PDFs and URLs are processed into chunks and embedded using sentence transformers
2. **Vector Search**: When you ask a question, the system searches for the most relevant document chunks
3. **Answer Generation**: The retrieved context is sent to Mistral-7B-Instruct via Inference API, which synthesizes a coherent answer based on the context

This approach combines the accuracy of document retrieval with the natural language capabilities of a large language model.

## Limitations

- Vector store is stored locally (not persistent on Hugging Face Spaces by default)
- Large documents may take time to process
- Some URLs may be blocked or require authentication
- Requires HF_TOKEN for Inference API access (free tier available)
- If you change embedding model or chunk parameters, you must re-run ingestion
## Troubleshooting
### Out of Memory Errors
- Use smaller models
- Reduce chunk size in `ingestion.py`
- Process fewer documents at once
### URL Fetching Issues
- Some websites block automated requests
- Try different URLs or use PDF uploads instead
### Inference API Issues
- Verify your `HF_TOKEN` is set correctly
- Check that the token has "Read" permissions
- Ensure you have API access (free tier available)
- If you get rate limit errors, you may need to upgrade your Hugging Face account

### Ingestion Issues
- If you changed embedding model or chunk parameters, re-run `ingest_documents.py`
- Ensure you have enough disk space for the vector store
- Large documents may take time to process

## License

This project is open source and available under the MIT License.

	---
	title: RAG Chatbot
	emoji: 🤖
	colorFrom: blue
	colorTo: purple
	sdk: gradio
	sdk_version: 6.3.0
	app_file: app.py
	pinned: false
	---

	# Hugging Face RAG Chatbot

	A Retrieval-Augmented Generation (RAG) chatbot that can ingest PDFs and URLs, then answer questions based on the ingested documents. Built for deployment on Hugging Face Spaces.

	## Features

	- 📄 PDF Ingestion: Upload and process PDF documents
	- 🌐 URL Ingestion: Extract and process content from web URLs
	- 🔍 Vector Search: Semantic search using sentence transformers
	- 💬 Chatbot Interface: Interactive Gradio interface for querying documents
	- 🚀 Hugging Face Ready: Configured for easy deployment to Hugging Face Spaces

	## Setup

	### Local Development

	1. Install dependencies:
	```bash
	pip install -r requirements.txt
	```

	2. Run the application:
	```bash
	python app.py
	```

	3. Access the interface:
	- Open your browser to `http://localhost:7860`

	### Usage

	1. Ingest Documents (Run this first or periodically to update):
	- Add PDF files to the `pdfs/` folder
	- Edit `ingest_documents.py` and add your URLs to the `URLS` list
	- Run the ingestion script:
	```bash
	py ingest_documents.py
	```
	- Wait for processing to complete (this creates/updates the vector store)

	2. Chat with Documents:
	- Run the chatbot app:
	```bash
	py app.py
	```
	- Open your browser to `http://localhost:7860`
	- Toggle "Use RAG" to enable/disable document retrieval
	- Ask questions about your ingested documents
	- The chatbot will retrieve relevant context and generate answers

	## Deployment to Hugging Face Spaces

	### Option 1: Using Hugging Face CLI

	1. Install Hugging Face CLI:
	```bash
	pip install huggingface_hub
	```

	2. Login to Hugging Face:
	```bash
	huggingface-cli login
	```

	3. Create a new Space:
	- Go to https://huggingface.co/new-space
	- Choose a name and select "Gradio" as the SDK
	- Create the space

	4. Clone and push your code:
	```bash
	git clone https://huggingface.co/spaces/YOUR_USERNAME/YOUR_SPACE_NAME
	cd YOUR_SPACE_NAME
	# Copy your files here
	git add .
	git commit -m "Initial commit"
	git push
	```

	### Option 2: Using Git

	1. Initialize git repository:
	```bash
	git init
	git add .
	git commit -m "Initial commit"
	```

	2. Add Hugging Face remote:
	```bash
	git remote add origin https://huggingface.co/spaces/YOUR_USERNAME/YOUR_SPACE_NAME
	git push -u origin main
	```

	### Required Files for Hugging Face Spaces

	Your Space needs these files:
	- `app.py` - Main Gradio application
	- `requirements.txt` - Python dependencies
	- `README.md` - This file (optional but recommended)

	### Optional: Add app.py to README

	For Hugging Face Spaces, you can also add a `app.py` reference in your README:

	```yaml
	---
	title: RAG Chatbot
	emoji: 🤖
	colorFrom: blue
	colorTo: purple
	sdk: gradio
	sdk_version: 4.0.0
	app_file: app.py
	pinned: false
	---
	```

	## Configuration

	### Setting Up Hugging Face Token (Required)

	The chatbot uses Hugging Face Inference API to access high-quality models. You need to set up an API token:

	1. Get your token:
	- Go to https://huggingface.co/settings/tokens
	- Create a new token with "Read" permissions
	- Copy the token

	2. For local development:
	- Set environment variable: `export HF_TOKEN=your_token_here` (Linux/Mac)
	- Or: `set HF_TOKEN=your_token_here` (Windows)
	- Or create a `.env` file with `HF_TOKEN=your_token_here`

	3. For Hugging Face Spaces:
	- Go to your Space → Settings → Secrets
	- Add a new secret: Name = `HF_TOKEN`, Value = your token
	- The app will automatically use this token

	### Chatbot Model and Finding Available Models

	The chatbot uses the Hugging Face Inference API. Which models you can use depends on which providers you have enabled.

	How to see which models are available to you:

	1. Browse models that support Inference API
	- https://huggingface.co/inference/models — lists providers and models
	- https://huggingface.co/models?inference_provider=hf-inference — filter Hub models by “HF Inference API”

	2. Enable providers (required)
	See “How to enable a provider and model” below.

	3. Pick a chat model
	- From the links above, choose a text generation / chat model that’s supported by a provider you enabled. Note its model id (e.g. `meta-llama/Llama-3.2-1B-Instruct`).

	4. Use it in the app
	- In `app.py`, pass that model id when creating the chatbot:
	```python
	chatbot = RAGChatbot(model_name="meta-llama/Llama-3.2-1B-Instruct") # use an id you enabled
	```
	- The app also tries fallbacks (Phi-2, Zephyr, Qwen) by default; if none are available, enable a provider that supports at least one of them, or set `model_name` to a model you enabled as above.

	### How to enable a provider and model

	1. Log in to [Hugging Face](https://huggingface.co).

	2. Open Inference Provider settings (one of these, depending on the current UI):
	- https://huggingface.co/settings/inference-providers
	- https://huggingface.co/settings/inference-api

	3. Enable a provider
	- On that page you’ll see a list of providers (e.g. Hugging Face, Together, Groq, etc.).
	- Turn on the provider that serves your model (e.g. Together for `ServiceNow-AI/Apriel-1.6-15b-Thinker:together`).
	- You can set the order of providers; “auto” uses this order to pick which provider handles the request.

	4. Credits / billing
	- Free accounts get a small amount of monthly credits; usage is deducted from that.
	- If you use a third-party provider (e.g. Together), you can either use HF-routed billing (credits on your HF account) or add a custom provider API key (e.g. Together API key) in the same settings so that provider is billed directly.

	5. Confirm the model
	- Browse models for that provider:
	[Together models on the Hub](https://huggingface.co/models?inference_provider=together&sort=trending)
	- Open the model page (e.g. `ServiceNow-AI/Apriel-1.6-15b-Thinker`) and check the inference widget; if you see “Together” and can run it, that model is available with your enabled provider.

	6. Use it in this app
	- The app default is already `ServiceNow-AI/Apriel-1.6-15b-Thinker:together`.
	- Ensure Together is enabled in your Inference Provider settings and you have credits (or a Together API key). Then restart the app and send a message.

	### Embedding Model

	The default embedding model is all-mpnet-base-v2, which provides high-quality embeddings for better retrieval.

	To change the embedding model, edit both `app.py` and `ingest_documents.py`:

	```python
	# In app.py
	chatbot = RAGChatbot(embedding_model="all-mpnet-base-v2")

	# In ingest_documents.py
	ingestion = DocumentIngestion(embedding_model="all-mpnet-base-v2")
	```

	Note: If you change the embedding model, you must re-run `ingest_documents.py` to rebuild the vector store.

	### Ingestion Parameters

	The ingestion system uses optimized parameters:
	- Chunk size: 600 characters (for precise retrieval)
	- Chunk overlap: 150 characters (to avoid cutting sentences)
	- Retrieval count: 5 chunks (for comprehensive context)

	These parameters are set in `ingestion.py` and can be adjusted if needed.

	## Project Structure

	```
	.
	├── app.py # Main Gradio chatbot application
	├── ingest_documents.py # Standalone script to ingest PDFs and URLs
	├── ingestion.py # Document ingestion and vector store module
	├── requirements.txt # Python dependencies
	├── README.md # This file
	├── pdfs/ # Folder for PDF files (add your PDFs here)
	│ └── README.md
	└── data/
	└── vector_store/ # Saved vector store (created after ingestion)
	├── index.faiss
	├── documents.pkl
	└── embeddings.pkl
	```

	## How It Works

	The chatbot uses Retrieval-Augmented Generation (RAG):

	1. Document Ingestion: PDFs and URLs are processed into chunks and embedded using sentence transformers
	2. Vector Search: When you ask a question, the system searches for the most relevant document chunks
	3. Answer Generation: The retrieved context is sent to Mistral-7B-Instruct via Inference API, which synthesizes a coherent answer based on the context

	This approach combines the accuracy of document retrieval with the natural language capabilities of a large language model.

	## Limitations

	- Vector store is stored locally (not persistent on Hugging Face Spaces by default)
	- Large documents may take time to process
	- Some URLs may be blocked or require authentication
	- Requires HF_TOKEN for Inference API access (free tier available)
	- If you change embedding model or chunk parameters, you must re-run ingestion

	## Troubleshooting

	### Out of Memory Errors
	- Use smaller models
	- Reduce chunk size in `ingestion.py`
	- Process fewer documents at once

	### URL Fetching Issues
	- Some websites block automated requests
	- Try different URLs or use PDF uploads instead

	### Inference API Issues
	- Verify your `HF_TOKEN` is set correctly
	- Check that the token has "Read" permissions
	- Ensure you have API access (free tier available)
	- If you get rate limit errors, you may need to upgrade your Hugging Face account

	### Ingestion Issues
	- If you changed embedding model or chunk parameters, re-run `ingest_documents.py`
	- Ensure you have enough disk space for the vector store
	- Large documents may take time to process

	## License

	This project is open source and available under the MIT License.