- Added CREATE_VECTOR_DB configuration option to config.py for vector database creation control.
- Updated RAG prompt template in prompts.py to reflect a more specialized assistant role, including detailed response guidelines and context management.
- Refactored rag.py to utilize a new chat_llm from chains.py for improved response handling.
- Introduced a new datetime tool in tools.py to provide current date and time information.
- Updated dependencies in pyproject.toml to include grandalf for enhanced functionality.
- Created chains.py to manage query tone checking and rude query responses using dedicated chains.
- Added a new Jupyter notebook (00_playground.ipynb) for interactive testing and demonstration of the assistant's features.
Remove unnecessary metadata from notebook cells for cleaner structure
- Implemented a RAG prompt template for generating answers based on provided context.
- Created a call LLM prompt template for concise answers with context.
- Added a query tone check prompt template to evaluate the tone of input queries.
Add test dataset for evaluating LLM applications with Ragas framework
- Introduced a new CSV file `testset_2.csv` containing user inputs, reference contexts, and synthesized responses.
- Included various queries related to the integration and evaluation of Large Language Models (LLMs) in modern applications, particularly focusing on the Ragas evaluation framework.
- Enhanced the dataset with examples covering topics such as healthcare applications, synthetic data generation, and observability best practices for AI systems.
Add Jupyter Notebook for Fine Tuning Dataset with Question Generation
- Created a new notebook `07_Fine_Tuning_Dataset.ipynb` for fine-tuning tasks.
- Implemented code to load blog posts, update metadata, and split documents.
- Integrated LangChain's ChatOpenAI for generating questions based on document context.
- Added functions for extracting questions from the generated responses.
- Included asynchronous processing for creating questions from multiple documents.
- Handled potential kernel crashes during execution.
Refactor code structure for improved readability and maintainability
- Introduced a new Jupyter notebook `update_blog_data.ipynb` for updating blog data and vector store with new posts.
- Created a Python script `pipeline.py` to handle the blog data update process, including command-line argument parsing and statistics saving.
- Added a shell script `build-vector-db.sh` to facilitate the execution of the pipeline script.
- Generated multiple JSON files in the `stats` directory to track blog statistics over time, including total documents, character counts, and length metrics.
