| --- |
| title: Api Rag |
| emoji: 🦁 |
| colorFrom: red |
| colorTo: green |
| sdk: docker |
| pinned: false |
| license: mit |
| --- |
| |
| # REST-API LLM RAG 🤖 |
|
|
| <img src="logo.svg" width="512"/> |
|
|
|     |
|
|
| ## 📖 Overview |
|
|
| This Python application provides an API service that retrieves answers to user queries based on the given context. |
|
|
| - **Context Management**: Context is managed and stored using [**Astra DB**](https://www.datastax.com/products/datastax-astra). |
| - **Chat Service Interfaces**: The application supports two interfaces: [**Hugging Face**](https://huggingface.co/) and **OpenAI**, both requiring an API key for access. |
| - **Supported Document Formats**: The app can process documents in various formats, such as PDF and TXT. |
| - **Core Libraries**: The project uses [**FastAPI**](https://fastapi.tiangolo.com/) and [**LangChain**](https://www.langchain.com/) as its core libraries. |
| - **Docker Ready**: The project is configured for containerization with Docker. |
|
|
| > 💡 For structured data types like CSV, XLSX, SQL, etc., it's recommended to utilize LLM Agents. |
|
|
| > 💡 If you want to use a Hugging Face models, make sure to choose a larger model, such as one with 70B parameters, to achieve better results. |
|
|
| ## 📦 Requirements |
|
|
| - Python 3.8 or higher |
| - [Docker](https://www.docker.com/) |
| - Required libraries: |
| - `fastapi` |
| - `hypercorn` |
| - `werkzeug` |
| - `langchain` (openai, community, huggingface, astradb) |
| - `einops` |
| - `python-multipart` |
| - `unstructured` |
|
|
| ## 🛠️ Configuration |
|
|
| To set up the application, follow these steps: |
|
|
| 1. Clone the repository and navigate to the project directory: |
| ```bash |
| git clone http://iieg-app.jalisco.gob.mx/iieg/restapi-rag.git |
| cd restapi-rag |
| ``` |
|
|
| 2. Set the API key environment variable in the `rag/.env` file: |
|
|
| - **Hugging Face**: Set the API key in the `HF_TOKEN` environment variable. |
| - **OpenAI**: Set the API key in the `OPENAI_API_KEY` environment variable. |
| - **AstraDB**: Set the API key in the `ASTRA_DB_APPLICATION_TOKEN` environment variable. |
| - **AstraDB Endpoint**: Set the Endpoint URL in the `ASTRA_DB_API_ENDPOINT` environment variable. |
|
|
| If both API keys, OpenAI or Hugging Face are configured, the OpenAI service will be used by default. |
|
|
| 3. Set the LLM chat and embedding model environment variables: |
|
|
| - **Embedding Model**: Set the embedding model in the `EMBEDDING_MODEL` environment variable. Default `nomic-ai/nomic-embed-text-v1.5`. |
| - **LLM Chat Model**: Set the LLM chat model in the `LLM_MODEL` environment variable. Default `meta-llama/Llama-3.2-1B-Instruct` |
|
|
| > 💡 If you choose the Hugging Face LLM chat model, set the `HF_HOME` variable to `models`, meaning the models will be stored in the `rag/models` directory. This setup ensures that models used by the app are separated and ready for Docker containerization. |
| |
| > ⚠️ If the chosen model is Meta Llama, you must request access to the repository. Simply go to the Hugging Face webpage and agree to their legal conditions. |
| |
| 4. Set commplementary environment variable for `ntkl`. |
| - **NLTK Data**: Specify the NLTK data path where the library will search for resources such as corpora and grammars. Default `/api`. |
| |
| > ⚠️ If you run the API in local env, ensure to set the environment variable in the `/home/.../restapi-rag/rag` path. |
| |
| ## ⚙️ Installation |
| |
| ### Docker |
| |
| Follow these steps to build the Docker image and run the container: |
| |
| 1. Build the Docker image. |
| |
| > ⚠️ Important: Before building the Docker image, make sure to run the app once to download the embeddings and/or LLM chat models in the directory specified by `HF_HOME`. The path of the environment variable must be `models`. |
|
|
| ```bash |
| docker build -t [image name] rag/ |
| ``` |
|
|
| 2. Run the container. |
|
|
| ```bash |
| docker run --rm --env-file rag/.env --name [container name] [image name] |
| ``` |
|
|
| ### Locally |
|
|
| To run the app locally, follow these steps: |
|
|
| 1. Install the required libraries from the `requirements.txt` file. |
| 2. Set the environment variables as defined in the Configuration section. |
| 3. Start the app with the following command: |
|
|
| ```bash |
| python -m uvicorn app.main:app |
| ``` |
|
|
| 4. Navigate to `http://127.0.0.1:8000/docs` or `http://127.0.0.1:8000/redoc` to access the app's documentation. |
|
|
| ## 🗂️ Documentation |
|
|
| You can access the API documentation once the application is running. To do so, follow these steps: |
|
|
| 1. Run the container. |
| 2. Retrieve the container's IP address: |
|
|
| ```bash |
| docker inspect [container name] | grep IPAddress |
| ``` |
|
|
| 3. Access the documentation via your web browser. |
|
|
| > 💡 General documentation: `http://[IP]:8080/docs` and endpoint documentation: `http://[IP]:8080/redoc`. |
|
|
| ## 🤝 Contributing |
|
|
| Contributions are welcome! Please feel free to open an issue for any suggestions or improvements. |
|
|
| ## 📜 License |
|
|
| This project is licensed under the MIT License - see the [LICENSE](./LICENSE) file for details. |
|
|
| ## 🙏 Acknowledgments |
|
|
| - [Hugging Face Transformers](https://huggingface.co/docs/transformers/index), [LangChain](https://www.langchain.com/), and [Atra DB](https://www.datastax.com/products/datastax-astra) for providing the tools for working with LLMs. |
| - [PyTorch](https://pytorch.org/) for the deep learning framework. |
| - [FastAPI](https://fastapi.tiangolo.com/) for the web framework. |
|
|
| ## 📝 TODO |
|
|
| - [ ] Create an endpoint that lists Astra DB collections. |
| - [ ] Implement an endpoint to update the collections. |
| - [ ] Implement an endpoint to delete a selected collection. |
| - [ ] Allow loading multiple files. |
| - [ ] Add endpoint for tabular data using Agents. |
| - [ ] Add endpoint for analyzing images. |
|
|
