File size: 3,113 Bytes
dcb67f2
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
# πŸš€ Phase 5: Async & Batch Processing

## 🧠 Why Async/Batch Processing?

Processing documents and embeddings one at a time is slow. Async/batch processing enables:

| Benefit | Explanation |
|---------|-------------|
| **Parallelism** | Process multiple items simultaneously |
| **Efficiency** | Batch API calls to reduce overhead |
| **Resilience** | Retry failed operations automatically |
| **Control** | Rate limiting prevents API throttling |

---

## πŸ“ File: `src/rag/async_utils.py`

### Classes Overview

| Class | Purpose |
|-------|---------|
| `BatchResult` | Dataclass for batch operation results |
| `AsyncBatchProcessor` | Generic batch processor with concurrency |
| `AsyncDocumentProcessor` | Document pipeline processor |
| `RateLimiter` | Token bucket rate limiter |
| `AsyncRetry` | Retry decorator with backoff |

---

## πŸ”§ Class: `BatchResult`

```python
@dataclass
class BatchResult(Generic[T]):
    results: List[T]                    # All results (may include None)
    errors: List[Optional[Exception]]   # Errors for each item
    success_count: int
    error_count: int
    
    @property
    def all_succeeded(self) -> bool: ...
    
    def get_successful(self) -> List[T]: ...
```

---

## πŸ”§ Class: `AsyncBatchProcessor`

### Constructor
```python
def __init__(
    self,
    process_fn: Callable[[T], R],  # Sync or async function
    batch_size: int = 32,
    max_concurrency: int = 4,
    retry_count: int = 3,
    retry_delay: float = 1.0,
):
```

### Method: `process()`
```python
async def process(
    self,
    items: List[T],
    progress_callback: Optional[Callable[[int, int], None]] = None,
) -> BatchResult[R]:
```

**Usage:**
```python
processor = AsyncBatchProcessor(
    process_fn=embed_text,
    batch_size=32,
    max_concurrency=4,
)

# Async usage
results = await processor.process(texts)

# Sync usage
results = processor.process_sync(texts)

print(f"Processed {results.success_count} items")
```

---

## πŸ”§ Class: `RateLimiter`

Token bucket algorithm for smooth rate limiting.

```python
limiter = RateLimiter(
    requests_per_second=10.0,
    burst_size=20,
)

async with limiter:
    await make_api_call()
```

---

## πŸ”§ Class: `AsyncRetry`

Decorator for automatic retries with exponential backoff.

```python
@AsyncRetry(max_retries=3, base_delay=1.0)
async def fetch_data():
    response = await api.get("/data")
    return response
```

---

## πŸ”§ Helper Functions

### `gather_with_concurrency()`
```python
results = await gather_with_concurrency(
    coros=[fetch(url) for url in urls],
    max_concurrency=10,
)
```

### `run_async()`
```python
# Run async code from sync context
result = run_async(some_coroutine())
```

---

## πŸ“‹ Summary

| Component | Purpose |
|-----------|---------|
| `BatchResult` | Structured batch output |
| `AsyncBatchProcessor` | Concurrent item processing |
| `AsyncDocumentProcessor` | Full document pipeline |
| `RateLimiter` | API rate control |
| `AsyncRetry` | Automatic retry with backoff |
| `gather_with_concurrency` | Limited parallel execution |
| `run_async` | Sync-async bridge |