Spaces:
Runtime error
Runtime error
Commit History
-
Update embedding model to Google Generative AI and enhance vector store functionality
- Changed the embedding model from OpenAI to Google Generative AI in the EmbeddingManager class. - Updated the configuration to reflect the new embedding model path. - Modified validation checks to ensure the presence of the Google API key for RAG embeddings. - Added a new method to reset the vector store, allowing for a complete clear and recreation of the collection. - Enhanced logging to provide clearer feedback on embedding model initialization and vector store operations.4dfec96
-
Refactor LimitedEnsembleRetriever for improved compatibility and functionality
- Updated the `LimitedEnsembleRetriever` class to remove inheritance from `BaseRetriever`, simplifying its structure. - Changed the method names to align with the new invoke interface, replacing deprecated methods with `invoke` and `ainvoke`. - Added a compatibility method to handle both string input and other data types for the `invoke` method, enhancing usability. - Improved documentation within the class to clarify the purpose and functionality of methods.5da24ca -
Enhance vector store retrieval with limited results
- Introduced a new `LimitedEnsembleRetriever` class to limit the number of results returned by the ensemble retriever to a specified count (k). - Updated the `get_hybrid_retriever` method to return a `LimitedEnsembleRetriever` instance, ensuring that exactly k results are provided for improved performance and usability. - Enhanced logging to reflect the creation of the limited retriever with specified weights and result limits.9e9e9ff
-
Refactor document ingestion and chunking to support LaTeX content
- Updated `DocumentIngestionService` to generalize content ingestion, allowing both Markdown and LaTeX formats. - Introduced `LaTeXAwareChunker` for chunking LaTeX documents while preserving structures. - Enhanced `UnifiedDocumentChunker` to handle both Markdown and LaTeX content types seamlessly. - Modified `_process_latex_content` to return raw LaTeX for GOT-OCR without LLM conversion. - Improved UI rendering for LaTeX content, ensuring proper display using MathJax. - Added backward compatibility for existing Markdown ingestion methods.63279a9
-
Add advanced retrieval strategies and update dependencies for RAG implementation
- Introduced BM25Retriever and EnsembleRetriever for enhanced document retrieval methods. - Updated `app.py`, `requirements.txt`, and `setup.sh` to include new dependencies for BM25 and community retrievers. - Enhanced `RAGChatService` to support multiple retrieval methods: similarity, MMR, BM25, and hybrid. - Updated README to document new retrieval strategies and configuration options. - Added comprehensive tests for retrieval methods and implementation structure.21c909d
-
Add data clearing service and vector store management
- Introduced `DataClearingService` for managing the clearing of vector store and chat history data. - Implemented `clear_all_documents` method in `VectorStoreManager` to remove all documents from the vector store. - Enhanced UI with a button to clear all data, integrating the new service for user interaction. - Updated logging to provide detailed feedback during data clearing operations.f46dfbd -
Refactor document ingestion and output file handling
- Updated `create_document_hash` to `create_file_hash` for full SHA-256 hashing. - Introduced duplicate checking and deletion methods in `DocumentIngestionService`. - Enhanced `ingest_markdown_content` to utilize original file content for deduplication. - Modified `_create_output_file` to preserve original filename and handle conflicts. - Improved logging for document processing and ingestion status.3f1b4af -
Update .gitignore and enhance README with data management instructions
- Added log files to .gitignore to prevent unnecessary tracking. - Expanded README to include new commands for clearing data and running the application with fresh data. - Clarified what data gets cleared during testing, improving usability for developers.a4f1c9e
-
Enhance RAG (Retrieval-Augmented Generation) functionality and dependencies
- Added RAG configuration management and integrated support for LangChain, ChromaDB, and sentence-transformers. - Implemented document ingestion, chunking, and embedding management for RAG capabilities. - Updated application to check and install necessary RAG dependencies automatically. - Enhanced README to include RAG features and usage instructions. - Refactored environment checks to ensure RAG dependencies are available. - Improved UI to support document conversion and chat functionalities with RAG integration. - Updated requirements.txt to include new RAG-related packages.575f1c7