README.md

metadata

title: Api Rag
emoji: 🦁
colorFrom: red
colorTo: green
sdk: docker
pinned: false
license: mit

REST-API LLM RAG 🤖

📖 Overview

This Python application provides an API service that retrieves answers to user queries based on the given context.

• Context Management: Context is managed and stored using Astra DB.
• Chat Service Interfaces: The application supports two interfaces: Hugging Face and OpenAI, both requiring an API key for access.
• Supported Document Formats: The app can process documents in various formats, such as PDF and TXT.
• Core Libraries: The project uses FastAPI and LangChain as its core libraries.
• Docker Ready: The project is configured for containerization with Docker.

💡 For structured data types like CSV, XLSX, SQL, etc., it's recommended to utilize LLM Agents.

💡 If you want to use a Hugging Face models, make sure to choose a larger model, such as one with 70B parameters, to achieve better results.

📦 Requirements

• Python 3.8 or higher
• Docker
• Required libraries:
  • fastapi
  • hypercorn
  • werkzeug
  • langchain (openai, community, huggingface, astradb)
  • einops
  • python-multipart
  • unstructured

🛠️ Configuration

To set up the application, follow these steps:

1. Clone the repository and navigate to the project directory:

git clone http://iieg-app.jalisco.gob.mx/iieg/restapi-rag.git
cd restapi-rag

2. Set the API key environment variable in the rag/.env file:

• Hugging Face: Set the API key in the HF_TOKEN environment variable.
• OpenAI: Set the API key in the OPENAI_API_KEY environment variable.
• AstraDB: Set the API key in the ASTRA_DB_APPLICATION_TOKEN environment variable.
• AstraDB Endpoint: Set the Endpoint URL in the ASTRA_DB_API_ENDPOINT environment variable.

If both API keys, OpenAI or Hugging Face are configured, the OpenAI service will be used by default.

3. Set the LLM chat and embedding model environment variables:

• Embedding Model: Set the embedding model in the EMBEDDING_MODEL environment variable. Default nomic-ai/nomic-embed-text-v1.5.
• LLM Chat Model: Set the LLM chat model in the LLM_MODEL environment variable. Default meta-llama/Llama-3.2-1B-Instruct

💡 If you choose the Hugging Face LLM chat model, set the HF_HOME variable to models, meaning the models will be stored in the rag/models directory. This setup ensures that models used by the app are separated and ready for Docker containerization.

⚠️ If the chosen model is Meta Llama, you must request access to the repository. Simply go to the Hugging Face webpage and agree to their legal conditions.

4. Set commplementary environment variable for ntkl.

• NLTK Data: Specify the NLTK data path where the library will search for resources such as corpora and grammars. Default /api.

⚠️ If you run the API in local env, ensure to set the environment variable in the /home/.../restapi-rag/rag path.

⚙️ Installation

Docker

Follow these steps to build the Docker image and run the container:

1. Build the Docker image.

⚠️ Important: Before building the Docker image, make sure to run the app once to download the embeddings and/or LLM chat models in the directory specified by HF_HOME. The path of the environment variable must be models.

docker build -t [image name] rag/

2. Run the container.

docker run --rm --env-file rag/.env --name [container name] [image name]

Locally

To run the app locally, follow these steps:

1. Install the required libraries from the requirements.txt file.
2. Set the environment variables as defined in the Configuration section.
3. Start the app with the following command:

python -m uvicorn app.main:app

4. Navigate to http://127.0.0.1:8000/docs or http://127.0.0.1:8000/redoc to access the app's documentation.

🗂️ Documentation

You can access the API documentation once the application is running. To do so, follow these steps:

1. Run the container.
2. Retrieve the container's IP address:

docker inspect [container name] | grep IPAddress

3. Access the documentation via your web browser.

💡 General documentation: http://[IP]:8080/docs and endpoint documentation: http://[IP]:8080/redoc.

🤝 Contributing

Contributions are welcome! Please feel free to open an issue for any suggestions or improvements.

📜 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

• Hugging Face Transformers, LangChain, and Atra DB for providing the tools for working with LLMs.
• PyTorch for the deep learning framework.
• FastAPI for the web framework.

📝 TODO

• Create an endpoint that lists Astra DB collections.
• Implement an endpoint to update the collections.
• Implement an endpoint to delete a selected collection.
• Allow loading multiple files.
• Add endpoint for tabular data using Agents.
• Add endpoint for analyzing images.