Spaces:
Build error
Build error
|
Download docs/explanations/phase5/async_processing.md from AIenthusSP/RAG-Documind-AI: direct link, hf CLI and curl.
- Browser
- Download file 3.11 kB
-
https://huggingface.co/spaces/AIenthusSP/RAG-Documind-AI/resolve/main/docs/explanations/phase5/async_processing.md
- Command line
-
hf download hf://spaces/AIenthusSP/RAG-Documind-AI/docs/explanations/phase5/async_processing.md
-
curl -L -o async_processing.md https://huggingface.co/spaces/AIenthusSP/RAG-Documind-AI/resolve/main/docs/explanations/phase5/async_processing.md
3.11 kB
π Phase 5: Async & Batch Processing
π§ Why Async/Batch Processing?
Processing documents and embeddings one at a time is slow. Async/batch processing enables:
| Benefit | Explanation |
|---|---|
| Parallelism | Process multiple items simultaneously |
| Efficiency | Batch API calls to reduce overhead |
| Resilience | Retry failed operations automatically |
| Control | Rate limiting prevents API throttling |
π File: src/rag/async_utils.py
Classes Overview
| Class | Purpose |
|---|---|
BatchResult |
Dataclass for batch operation results |
AsyncBatchProcessor |
Generic batch processor with concurrency |
AsyncDocumentProcessor |
Document pipeline processor |
RateLimiter |
Token bucket rate limiter |
AsyncRetry |
Retry decorator with backoff |
π§ Class: BatchResult
@dataclass
class BatchResult(Generic[T]):
results: List[T] # All results (may include None)
errors: List[Optional[Exception]] # Errors for each item
success_count: int
error_count: int
@property
def all_succeeded(self) -> bool: ...
def get_successful(self) -> List[T]: ...
π§ Class: AsyncBatchProcessor
Constructor
def __init__(
self,
process_fn: Callable[[T], R], # Sync or async function
batch_size: int = 32,
max_concurrency: int = 4,
retry_count: int = 3,
retry_delay: float = 1.0,
):
Method: process()
async def process(
self,
items: List[T],
progress_callback: Optional[Callable[[int, int], None]] = None,
) -> BatchResult[R]:
Usage:
processor = AsyncBatchProcessor(
process_fn=embed_text,
batch_size=32,
max_concurrency=4,
)
# Async usage
results = await processor.process(texts)
# Sync usage
results = processor.process_sync(texts)
print(f"Processed {results.success_count} items")
π§ Class: RateLimiter
Token bucket algorithm for smooth rate limiting.
limiter = RateLimiter(
requests_per_second=10.0,
burst_size=20,
)
async with limiter:
await make_api_call()
π§ Class: AsyncRetry
Decorator for automatic retries with exponential backoff.
@AsyncRetry(max_retries=3, base_delay=1.0)
async def fetch_data():
response = await api.get("/data")
return response
π§ Helper Functions
gather_with_concurrency()
results = await gather_with_concurrency(
coros=[fetch(url) for url in urls],
max_concurrency=10,
)
run_async()
# Run async code from sync context
result = run_async(some_coroutine())
π Summary
| Component | Purpose |
|---|---|
BatchResult |
Structured batch output |
AsyncBatchProcessor |
Concurrent item processing |
AsyncDocumentProcessor |
Full document pipeline |
RateLimiter |
API rate control |
AsyncRetry |
Automatic retry with backoff |
gather_with_concurrency |
Limited parallel execution |
run_async |
Sync-async bridge |