| --- |
| library_name: efficient-context |
| language: code |
| tags: |
| - context-optimization |
| - llm |
| - cpu-optimization |
| - resource-constrained |
| - memory-management |
| - python |
| license: mit |
| datasets: |
| - None |
| --- |
| |
| <!-- filepath: /Users/biswanath2.roul/Desktop/biswanath/office/poc/pypi/190525/efficient-context/README.md --> |
| # efficient-context |
|
|
| A Python library for optimizing LLM context handling in CPU-constrained environments. |
|
|
| ## Overview |
|
|
| `efficient-context` addresses the challenge of working with large language models (LLMs) on CPU-only and memory-limited systems by providing efficient context management strategies. The library focuses on: |
|
|
| - **Context Compression**: Reduce memory requirements while preserving information quality |
| - **Semantic Chunking**: Go beyond token-based approaches for more effective context management |
| - **Retrieval Optimization**: Minimize context size through intelligent retrieval strategies |
| - **Memory Management**: Handle large contexts on limited hardware resources |
|
|
| ## Installation |
|
|
| ```bash |
| pip install efficient-context |
| ``` |
|
|
| ## Quick Start |
|
|
| ```python |
| from efficient_context import ContextManager |
| from efficient_context.compression import SemanticDeduplicator |
| from efficient_context.chunking import SemanticChunker |
| from efficient_context.retrieval import CPUOptimizedRetriever |
| |
| # Initialize a context manager with custom strategies |
| context_manager = ContextManager( |
| compressor=SemanticDeduplicator(threshold=0.85), |
| chunker=SemanticChunker(chunk_size=256), |
| retriever=CPUOptimizedRetriever(embedding_model="lightweight") |
| ) |
| |
| # Add documents to your context |
| context_manager.add_documents(documents) |
| |
| # Generate optimized context for a query |
| optimized_context = context_manager.generate_context(query="Tell me about the climate impact of renewable energy") |
| |
| # Use the optimized context with your LLM |
| response = your_llm_model.generate(prompt=prompt, context=optimized_context) |
| ``` |
|
|
| ## Features |
|
|
| ### Context Compression |
| - Semantic deduplication to remove redundant information |
| - Importance-based pruning that keeps critical information |
| - Automatic summarization of less relevant sections |
|
|
| ### Advanced Chunking |
| - Semantic chunking that preserves logical units |
| - Adaptive chunk sizing based on content complexity |
| - Chunk relationships mapping for coherent retrieval |
|
|
| ### Retrieval Optimization |
| - Lightweight embedding models optimized for CPU |
| - Tiered retrieval strategies (local vs. remote) |
| - Query-aware context assembly |
|
|
| ### Memory Management |
| - Progressive loading/unloading of context |
| - Streaming context processing |
| - Memory-aware caching strategies |
|
|
| ## Maintainer |
|
|
| This project is maintained by [Biswanath Roul](https://github.com/biswanathroul) |
|
|
| ## License |
|
|
| MIT |
