AIenthusSP commited on
Commit
dcb67f2
·
1 Parent(s): 7dd3c91

feat: add async/batch processing utilities

Browse files

- Add AsyncBatchProcessor with concurrency control
- Add AsyncDocumentProcessor for document pipelines
- Add RateLimiter (token bucket algorithm)
- Add AsyncRetry decorator with exponential backoff
- Add gather_with_concurrency helper
- Add run_async sync-async bridge
- Add comprehensive unit tests (16 tests, all passing)
- Add documentation in docs/explanations/phase5/

docs/explanations/phase5/async_processing.md ADDED
@@ -0,0 +1,147 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # 🚀 Phase 5: Async & Batch Processing
2
+
3
+ ## 🧠 Why Async/Batch Processing?
4
+
5
+ Processing documents and embeddings one at a time is slow. Async/batch processing enables:
6
+
7
+ | Benefit | Explanation |
8
+ |---------|-------------|
9
+ | **Parallelism** | Process multiple items simultaneously |
10
+ | **Efficiency** | Batch API calls to reduce overhead |
11
+ | **Resilience** | Retry failed operations automatically |
12
+ | **Control** | Rate limiting prevents API throttling |
13
+
14
+ ---
15
+
16
+ ## 📁 File: `src/rag/async_utils.py`
17
+
18
+ ### Classes Overview
19
+
20
+ | Class | Purpose |
21
+ |-------|---------|
22
+ | `BatchResult` | Dataclass for batch operation results |
23
+ | `AsyncBatchProcessor` | Generic batch processor with concurrency |
24
+ | `AsyncDocumentProcessor` | Document pipeline processor |
25
+ | `RateLimiter` | Token bucket rate limiter |
26
+ | `AsyncRetry` | Retry decorator with backoff |
27
+
28
+ ---
29
+
30
+ ## 🔧 Class: `BatchResult`
31
+
32
+ ```python
33
+ @dataclass
34
+ class BatchResult(Generic[T]):
35
+ results: List[T] # All results (may include None)
36
+ errors: List[Optional[Exception]] # Errors for each item
37
+ success_count: int
38
+ error_count: int
39
+
40
+ @property
41
+ def all_succeeded(self) -> bool: ...
42
+
43
+ def get_successful(self) -> List[T]: ...
44
+ ```
45
+
46
+ ---
47
+
48
+ ## 🔧 Class: `AsyncBatchProcessor`
49
+
50
+ ### Constructor
51
+ ```python
52
+ def __init__(
53
+ self,
54
+ process_fn: Callable[[T], R], # Sync or async function
55
+ batch_size: int = 32,
56
+ max_concurrency: int = 4,
57
+ retry_count: int = 3,
58
+ retry_delay: float = 1.0,
59
+ ):
60
+ ```
61
+
62
+ ### Method: `process()`
63
+ ```python
64
+ async def process(
65
+ self,
66
+ items: List[T],
67
+ progress_callback: Optional[Callable[[int, int], None]] = None,
68
+ ) -> BatchResult[R]:
69
+ ```
70
+
71
+ **Usage:**
72
+ ```python
73
+ processor = AsyncBatchProcessor(
74
+ process_fn=embed_text,
75
+ batch_size=32,
76
+ max_concurrency=4,
77
+ )
78
+
79
+ # Async usage
80
+ results = await processor.process(texts)
81
+
82
+ # Sync usage
83
+ results = processor.process_sync(texts)
84
+
85
+ print(f"Processed {results.success_count} items")
86
+ ```
87
+
88
+ ---
89
+
90
+ ## 🔧 Class: `RateLimiter`
91
+
92
+ Token bucket algorithm for smooth rate limiting.
93
+
94
+ ```python
95
+ limiter = RateLimiter(
96
+ requests_per_second=10.0,
97
+ burst_size=20,
98
+ )
99
+
100
+ async with limiter:
101
+ await make_api_call()
102
+ ```
103
+
104
+ ---
105
+
106
+ ## 🔧 Class: `AsyncRetry`
107
+
108
+ Decorator for automatic retries with exponential backoff.
109
+
110
+ ```python
111
+ @AsyncRetry(max_retries=3, base_delay=1.0)
112
+ async def fetch_data():
113
+ response = await api.get("/data")
114
+ return response
115
+ ```
116
+
117
+ ---
118
+
119
+ ## 🔧 Helper Functions
120
+
121
+ ### `gather_with_concurrency()`
122
+ ```python
123
+ results = await gather_with_concurrency(
124
+ coros=[fetch(url) for url in urls],
125
+ max_concurrency=10,
126
+ )
127
+ ```
128
+
129
+ ### `run_async()`
130
+ ```python
131
+ # Run async code from sync context
132
+ result = run_async(some_coroutine())
133
+ ```
134
+
135
+ ---
136
+
137
+ ## 📋 Summary
138
+
139
+ | Component | Purpose |
140
+ |-----------|---------|
141
+ | `BatchResult` | Structured batch output |
142
+ | `AsyncBatchProcessor` | Concurrent item processing |
143
+ | `AsyncDocumentProcessor` | Full document pipeline |
144
+ | `RateLimiter` | API rate control |
145
+ | `AsyncRetry` | Automatic retry with backoff |
146
+ | `gather_with_concurrency` | Limited parallel execution |
147
+ | `run_async` | Sync-async bridge |
src/rag/async_utils.py ADDED
@@ -0,0 +1,460 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Async Utilities Module
3
+
4
+ Provides async/batch processing utilities for improved
5
+ performance in embedding generation, document processing,
6
+ and API requests.
7
+ """
8
+
9
+ import asyncio
10
+ import logging
11
+ from concurrent.futures import ThreadPoolExecutor
12
+ from dataclasses import dataclass
13
+ from typing import (
14
+ Any,
15
+ Callable,
16
+ Coroutine,
17
+ Dict,
18
+ Generic,
19
+ List,
20
+ Optional,
21
+ TypeVar,
22
+ Union,
23
+ )
24
+
25
+ logger = logging.getLogger(__name__)
26
+
27
+ T = TypeVar("T")
28
+ R = TypeVar("R")
29
+
30
+
31
+ @dataclass
32
+ class BatchResult(Generic[T]):
33
+ """Result of a batch operation."""
34
+ results: List[T]
35
+ errors: List[Optional[Exception]]
36
+ success_count: int
37
+ error_count: int
38
+
39
+ @property
40
+ def all_succeeded(self) -> bool:
41
+ """Check if all items succeeded."""
42
+ return self.error_count == 0
43
+
44
+ def get_successful(self) -> List[T]:
45
+ """Get only successful results."""
46
+ return [r for r, e in zip(self.results, self.errors) if e is None]
47
+
48
+
49
+ class AsyncBatchProcessor(Generic[T, R]):
50
+ """
51
+ Process items in batches with concurrency control.
52
+
53
+ Supports both sync and async processing functions,
54
+ with configurable batch size and max concurrency.
55
+
56
+ Example:
57
+ >>> processor = AsyncBatchProcessor(
58
+ ... process_fn=embed_text,
59
+ ... batch_size=32,
60
+ ... max_concurrency=4
61
+ ... )
62
+ >>> results = await processor.process(texts)
63
+ """
64
+
65
+ def __init__(
66
+ self,
67
+ process_fn: Callable[[T], R],
68
+ batch_size: int = 32,
69
+ max_concurrency: int = 4,
70
+ retry_count: int = 3,
71
+ retry_delay: float = 1.0,
72
+ ):
73
+ """
74
+ Initialize the batch processor.
75
+
76
+ Args:
77
+ process_fn: Function to process each item (sync or async).
78
+ batch_size: Number of items per batch.
79
+ max_concurrency: Maximum concurrent batches.
80
+ retry_count: Number of retries on failure.
81
+ retry_delay: Delay between retries (seconds).
82
+ """
83
+ self.process_fn = process_fn
84
+ self.batch_size = batch_size
85
+ self.max_concurrency = max_concurrency
86
+ self.retry_count = retry_count
87
+ self.retry_delay = retry_delay
88
+ self._semaphore: Optional[asyncio.Semaphore] = None
89
+
90
+ def _create_batches(self, items: List[T]) -> List[List[T]]:
91
+ """Split items into batches."""
92
+ return [
93
+ items[i:i + self.batch_size]
94
+ for i in range(0, len(items), self.batch_size)
95
+ ]
96
+
97
+ async def _process_with_retry(
98
+ self,
99
+ item: T,
100
+ executor: Optional[ThreadPoolExecutor] = None,
101
+ ) -> tuple[Optional[R], Optional[Exception]]:
102
+ """Process a single item with retry logic."""
103
+ last_error = None
104
+
105
+ for attempt in range(self.retry_count):
106
+ try:
107
+ if asyncio.iscoroutinefunction(self.process_fn):
108
+ result = await self.process_fn(item)
109
+ else:
110
+ # Run sync function in thread pool
111
+ loop = asyncio.get_event_loop()
112
+ result = await loop.run_in_executor(
113
+ executor,
114
+ self.process_fn,
115
+ item,
116
+ )
117
+ return result, None
118
+ except Exception as e:
119
+ last_error = e
120
+ if attempt < self.retry_count - 1:
121
+ await asyncio.sleep(self.retry_delay * (attempt + 1))
122
+ logger.warning(f"Retry {attempt + 1} for item: {e}")
123
+
124
+ return None, last_error
125
+
126
+ async def _process_batch(
127
+ self,
128
+ batch: List[T],
129
+ executor: Optional[ThreadPoolExecutor] = None,
130
+ ) -> List[tuple[Optional[R], Optional[Exception]]]:
131
+ """Process a single batch with concurrency control."""
132
+ async with self._semaphore:
133
+ tasks = [
134
+ self._process_with_retry(item, executor)
135
+ for item in batch
136
+ ]
137
+ return await asyncio.gather(*tasks)
138
+
139
+ async def process(
140
+ self,
141
+ items: List[T],
142
+ progress_callback: Optional[Callable[[int, int], None]] = None,
143
+ ) -> BatchResult[R]:
144
+ """
145
+ Process all items in batches.
146
+
147
+ Args:
148
+ items: Items to process.
149
+ progress_callback: Optional callback(completed, total).
150
+
151
+ Returns:
152
+ BatchResult with all results and errors.
153
+ """
154
+ if not items:
155
+ return BatchResult(
156
+ results=[],
157
+ errors=[],
158
+ success_count=0,
159
+ error_count=0,
160
+ )
161
+
162
+ self._semaphore = asyncio.Semaphore(self.max_concurrency)
163
+ batches = self._create_batches(items)
164
+
165
+ all_results: List[Optional[R]] = []
166
+ all_errors: List[Optional[Exception]] = []
167
+ completed = 0
168
+
169
+ with ThreadPoolExecutor(max_workers=self.max_concurrency) as executor:
170
+ for batch in batches:
171
+ batch_results = await self._process_batch(batch, executor)
172
+
173
+ for result, error in batch_results:
174
+ all_results.append(result)
175
+ all_errors.append(error)
176
+
177
+ completed += len(batch)
178
+ if progress_callback:
179
+ progress_callback(completed, len(items))
180
+
181
+ success_count = sum(1 for e in all_errors if e is None)
182
+ error_count = len(all_errors) - success_count
183
+
184
+ logger.info(
185
+ f"Batch processing complete: {success_count} succeeded, "
186
+ f"{error_count} failed"
187
+ )
188
+
189
+ return BatchResult(
190
+ results=all_results,
191
+ errors=all_errors,
192
+ success_count=success_count,
193
+ error_count=error_count,
194
+ )
195
+
196
+ def process_sync(
197
+ self,
198
+ items: List[T],
199
+ progress_callback: Optional[Callable[[int, int], None]] = None,
200
+ ) -> BatchResult[R]:
201
+ """Synchronous wrapper for process()."""
202
+ return asyncio.run(self.process(items, progress_callback))
203
+
204
+
205
+ class AsyncDocumentProcessor:
206
+ """
207
+ Async document processing pipeline.
208
+
209
+ Handles concurrent document loading, chunking, and embedding.
210
+ """
211
+
212
+ def __init__(
213
+ self,
214
+ loader_fn: Callable,
215
+ chunker_fn: Callable,
216
+ embedder_fn: Callable,
217
+ batch_size: int = 32,
218
+ max_concurrency: int = 4,
219
+ ):
220
+ self.loader_fn = loader_fn
221
+ self.chunker_fn = chunker_fn
222
+ self.embedder_fn = embedder_fn
223
+ self.batch_size = batch_size
224
+ self.max_concurrency = max_concurrency
225
+
226
+ async def process_files(
227
+ self,
228
+ file_paths: List[str],
229
+ progress_callback: Optional[Callable[[str, int, int], None]] = None,
230
+ ) -> Dict[str, Any]:
231
+ """
232
+ Process multiple files concurrently.
233
+
234
+ Args:
235
+ file_paths: Paths to files to process.
236
+ progress_callback: Optional callback(stage, completed, total).
237
+
238
+ Returns:
239
+ Dict with processed documents, chunks, and statistics.
240
+ """
241
+ semaphore = asyncio.Semaphore(self.max_concurrency)
242
+
243
+ async def load_file(path: str) -> tuple[str, Any, Optional[Exception]]:
244
+ async with semaphore:
245
+ try:
246
+ loop = asyncio.get_event_loop()
247
+ docs = await loop.run_in_executor(
248
+ None, self.loader_fn, path
249
+ )
250
+ return path, docs, None
251
+ except Exception as e:
252
+ return path, None, e
253
+
254
+ # Load all files concurrently
255
+ if progress_callback:
256
+ progress_callback("loading", 0, len(file_paths))
257
+
258
+ load_tasks = [load_file(path) for path in file_paths]
259
+ load_results = await asyncio.gather(*load_tasks)
260
+
261
+ all_documents = []
262
+ errors = []
263
+
264
+ for path, docs, error in load_results:
265
+ if error:
266
+ errors.append({"file": path, "error": str(error)})
267
+ logger.error(f"Failed to load {path}: {error}")
268
+ elif docs:
269
+ all_documents.extend(docs)
270
+
271
+ if progress_callback:
272
+ progress_callback("loading", len(file_paths), len(file_paths))
273
+
274
+ # Chunk documents
275
+ if progress_callback:
276
+ progress_callback("chunking", 0, len(all_documents))
277
+
278
+ loop = asyncio.get_event_loop()
279
+ all_chunks = await loop.run_in_executor(
280
+ None, self.chunker_fn, all_documents
281
+ )
282
+
283
+ if progress_callback:
284
+ progress_callback("chunking", len(all_documents), len(all_documents))
285
+
286
+ # Embed chunks in batches
287
+ if progress_callback:
288
+ progress_callback("embedding", 0, len(all_chunks))
289
+
290
+ chunk_texts = [chunk.page_content for chunk in all_chunks]
291
+
292
+ # Process embeddings in batches
293
+ processor = AsyncBatchProcessor(
294
+ process_fn=self.embedder_fn,
295
+ batch_size=self.batch_size,
296
+ max_concurrency=self.max_concurrency,
297
+ )
298
+
299
+ embed_result = await processor.process(
300
+ chunk_texts,
301
+ lambda c, t: progress_callback("embedding", c, t) if progress_callback else None,
302
+ )
303
+
304
+ return {
305
+ "documents": all_documents,
306
+ "chunks": all_chunks,
307
+ "embeddings": embed_result.get_successful(),
308
+ "errors": errors,
309
+ "stats": {
310
+ "files_processed": len(file_paths) - len(errors),
311
+ "files_failed": len(errors),
312
+ "documents": len(all_documents),
313
+ "chunks": len(all_chunks),
314
+ "embeddings": embed_result.success_count,
315
+ },
316
+ }
317
+
318
+
319
+ async def gather_with_concurrency(
320
+ coros: List[Coroutine],
321
+ max_concurrency: int = 10,
322
+ ) -> List[Any]:
323
+ """
324
+ Run coroutines with limited concurrency.
325
+
326
+ Args:
327
+ coros: List of coroutines to run.
328
+ max_concurrency: Maximum concurrent executions.
329
+
330
+ Returns:
331
+ List of results in order.
332
+ """
333
+ semaphore = asyncio.Semaphore(max_concurrency)
334
+
335
+ async def bounded(coro):
336
+ async with semaphore:
337
+ return await coro
338
+
339
+ return await asyncio.gather(*[bounded(c) for c in coros])
340
+
341
+
342
+ def run_async(coro: Coroutine) -> Any:
343
+ """
344
+ Run an async coroutine from sync code.
345
+
346
+ Handles event loop creation/reuse properly.
347
+ """
348
+ try:
349
+ loop = asyncio.get_running_loop()
350
+ # We're in an async context, can't use asyncio.run
351
+ import nest_asyncio
352
+ nest_asyncio.apply()
353
+ return loop.run_until_complete(coro)
354
+ except RuntimeError:
355
+ # No running loop, create one
356
+ return asyncio.run(coro)
357
+
358
+
359
+ class RateLimiter:
360
+ """
361
+ Async rate limiter for API calls.
362
+
363
+ Uses token bucket algorithm for smooth rate limiting.
364
+ """
365
+
366
+ def __init__(
367
+ self,
368
+ requests_per_second: float = 10.0,
369
+ burst_size: int = 20,
370
+ ):
371
+ """
372
+ Initialize rate limiter.
373
+
374
+ Args:
375
+ requests_per_second: Maximum sustained rate.
376
+ burst_size: Maximum burst size.
377
+ """
378
+ self.rate = requests_per_second
379
+ self.burst_size = burst_size
380
+ self.tokens = burst_size
381
+ self.last_update = asyncio.get_event_loop().time()
382
+ self._lock = asyncio.Lock()
383
+
384
+ async def acquire(self) -> None:
385
+ """Acquire permission to make a request."""
386
+ async with self._lock:
387
+ now = asyncio.get_event_loop().time()
388
+ elapsed = now - self.last_update
389
+ self.last_update = now
390
+
391
+ # Add tokens based on elapsed time
392
+ self.tokens = min(
393
+ self.burst_size,
394
+ self.tokens + elapsed * self.rate
395
+ )
396
+
397
+ if self.tokens < 1:
398
+ # Wait for a token
399
+ wait_time = (1 - self.tokens) / self.rate
400
+ await asyncio.sleep(wait_time)
401
+ self.tokens = 0
402
+ else:
403
+ self.tokens -= 1
404
+
405
+ async def __aenter__(self):
406
+ await self.acquire()
407
+ return self
408
+
409
+ async def __aexit__(self, *args):
410
+ pass
411
+
412
+
413
+ class AsyncRetry:
414
+ """
415
+ Async retry decorator with exponential backoff.
416
+
417
+ Usage:
418
+ @AsyncRetry(max_retries=3, base_delay=1.0)
419
+ async def fetch_data():
420
+ ...
421
+ """
422
+
423
+ def __init__(
424
+ self,
425
+ max_retries: int = 3,
426
+ base_delay: float = 1.0,
427
+ max_delay: float = 60.0,
428
+ exponential_base: float = 2.0,
429
+ exceptions: tuple = (Exception,),
430
+ ):
431
+ self.max_retries = max_retries
432
+ self.base_delay = base_delay
433
+ self.max_delay = max_delay
434
+ self.exponential_base = exponential_base
435
+ self.exceptions = exceptions
436
+
437
+ def __call__(self, func: Callable) -> Callable:
438
+ async def wrapper(*args, **kwargs):
439
+ last_error = None
440
+
441
+ for attempt in range(self.max_retries + 1):
442
+ try:
443
+ return await func(*args, **kwargs)
444
+ except self.exceptions as e:
445
+ last_error = e
446
+ if attempt < self.max_retries:
447
+ delay = min(
448
+ self.base_delay * (self.exponential_base ** attempt),
449
+ self.max_delay
450
+ )
451
+ logger.warning(
452
+ f"Retry {attempt + 1}/{self.max_retries} "
453
+ f"for {func.__name__}: {e}. "
454
+ f"Waiting {delay:.1f}s"
455
+ )
456
+ await asyncio.sleep(delay)
457
+
458
+ raise last_error
459
+
460
+ return wrapper
tests/unit/test_async_utils.py ADDED
@@ -0,0 +1,279 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Unit tests for the async_utils module.
3
+ """
4
+
5
+ import asyncio
6
+ import pytest
7
+ import importlib.util
8
+ import os
9
+
10
+ # Load module directly to avoid triggering full package init
11
+ spec = importlib.util.spec_from_file_location(
12
+ "async_utils",
13
+ os.path.join(os.path.dirname(__file__), "..", "..", "src", "rag", "async_utils.py")
14
+ )
15
+ async_utils = importlib.util.module_from_spec(spec)
16
+ spec.loader.exec_module(async_utils)
17
+
18
+ BatchResult = async_utils.BatchResult
19
+ AsyncBatchProcessor = async_utils.AsyncBatchProcessor
20
+ RateLimiter = async_utils.RateLimiter
21
+ AsyncRetry = async_utils.AsyncRetry
22
+ gather_with_concurrency = async_utils.gather_with_concurrency
23
+
24
+
25
+ class TestBatchResult:
26
+ """Tests for BatchResult dataclass."""
27
+
28
+ def test_batch_result_creation(self):
29
+ """Test creating a batch result."""
30
+ result = BatchResult(
31
+ results=[1, 2, 3],
32
+ errors=[None, None, None],
33
+ success_count=3,
34
+ error_count=0,
35
+ )
36
+
37
+ assert len(result.results) == 3
38
+ assert result.all_succeeded
39
+
40
+ def test_batch_result_with_errors(self):
41
+ """Test batch result with errors."""
42
+ result = BatchResult(
43
+ results=[1, None, 3],
44
+ errors=[None, ValueError("failed"), None],
45
+ success_count=2,
46
+ error_count=1,
47
+ )
48
+
49
+ assert not result.all_succeeded
50
+ assert result.success_count == 2
51
+ assert result.error_count == 1
52
+
53
+ def test_get_successful(self):
54
+ """Test getting only successful results."""
55
+ result = BatchResult(
56
+ results=["a", None, "c"],
57
+ errors=[None, ValueError("failed"), None],
58
+ success_count=2,
59
+ error_count=1,
60
+ )
61
+
62
+ successful = result.get_successful()
63
+ assert successful == ["a", "c"]
64
+
65
+
66
+ class TestAsyncBatchProcessor:
67
+ """Tests for AsyncBatchProcessor."""
68
+
69
+ @pytest.mark.asyncio
70
+ async def test_process_empty_items(self):
71
+ """Test processing empty list."""
72
+ processor = AsyncBatchProcessor(
73
+ process_fn=lambda x: x * 2,
74
+ batch_size=10,
75
+ )
76
+
77
+ result = await processor.process([])
78
+
79
+ assert result.results == []
80
+ assert result.success_count == 0
81
+
82
+ @pytest.mark.asyncio
83
+ async def test_process_sync_function(self):
84
+ """Test processing with sync function."""
85
+ def double(x):
86
+ return x * 2
87
+
88
+ processor = AsyncBatchProcessor(
89
+ process_fn=double,
90
+ batch_size=3,
91
+ max_concurrency=2,
92
+ )
93
+
94
+ result = await processor.process([1, 2, 3, 4, 5])
95
+
96
+ assert result.success_count == 5
97
+ assert result.results == [2, 4, 6, 8, 10]
98
+
99
+ @pytest.mark.asyncio
100
+ async def test_process_async_function(self):
101
+ """Test processing with async function."""
102
+ async def async_double(x):
103
+ await asyncio.sleep(0.01)
104
+ return x * 2
105
+
106
+ processor = AsyncBatchProcessor(
107
+ process_fn=async_double,
108
+ batch_size=3,
109
+ )
110
+
111
+ result = await processor.process([1, 2, 3])
112
+
113
+ assert result.success_count == 3
114
+ assert result.results == [2, 4, 6]
115
+
116
+ @pytest.mark.asyncio
117
+ async def test_process_with_errors(self):
118
+ """Test processing with some errors."""
119
+ def maybe_fail(x):
120
+ if x == 3:
121
+ raise ValueError("Failed on 3")
122
+ return x * 2
123
+
124
+ processor = AsyncBatchProcessor(
125
+ process_fn=maybe_fail,
126
+ batch_size=10,
127
+ retry_count=1,
128
+ )
129
+
130
+ result = await processor.process([1, 2, 3, 4])
131
+
132
+ assert result.success_count == 3
133
+ assert result.error_count == 1
134
+ assert result.results[0] == 2
135
+ assert result.results[2] is None # Failed item
136
+
137
+ @pytest.mark.asyncio
138
+ async def test_progress_callback(self):
139
+ """Test progress callback is called."""
140
+ progress_calls = []
141
+
142
+ def track_progress(completed, total):
143
+ progress_calls.append((completed, total))
144
+
145
+ processor = AsyncBatchProcessor(
146
+ process_fn=lambda x: x,
147
+ batch_size=2,
148
+ )
149
+
150
+ await processor.process([1, 2, 3, 4], progress_callback=track_progress)
151
+
152
+ assert len(progress_calls) > 0
153
+ assert progress_calls[-1] == (4, 4)
154
+
155
+ def test_process_sync_wrapper(self):
156
+ """Test synchronous wrapper."""
157
+ processor = AsyncBatchProcessor(
158
+ process_fn=lambda x: x * 2,
159
+ batch_size=2,
160
+ )
161
+
162
+ result = processor.process_sync([1, 2, 3])
163
+
164
+ assert result.success_count == 3
165
+ assert result.results == [2, 4, 6]
166
+
167
+
168
+ class TestRateLimiter:
169
+ """Tests for RateLimiter."""
170
+
171
+ @pytest.mark.asyncio
172
+ async def test_rate_limiter_allows_burst(self):
173
+ """Test that burst size is initially available."""
174
+ limiter = RateLimiter(requests_per_second=10, burst_size=5)
175
+
176
+ # Should allow 5 requests immediately
177
+ for _ in range(5):
178
+ await limiter.acquire()
179
+
180
+ @pytest.mark.asyncio
181
+ async def test_rate_limiter_context_manager(self):
182
+ """Test async context manager usage."""
183
+ limiter = RateLimiter(requests_per_second=100, burst_size=10)
184
+
185
+ async with limiter:
186
+ pass # Request allowed
187
+
188
+ # Should work multiple times
189
+ async with limiter:
190
+ pass
191
+
192
+
193
+ class TestAsyncRetry:
194
+ """Tests for AsyncRetry decorator."""
195
+
196
+ @pytest.mark.asyncio
197
+ async def test_retry_success_first_try(self):
198
+ """Test function that succeeds first time."""
199
+ call_count = 0
200
+
201
+ @AsyncRetry(max_retries=3)
202
+ async def always_succeeds():
203
+ nonlocal call_count
204
+ call_count += 1
205
+ return "success"
206
+
207
+ result = await always_succeeds()
208
+
209
+ assert result == "success"
210
+ assert call_count == 1
211
+
212
+ @pytest.mark.asyncio
213
+ async def test_retry_success_after_failures(self):
214
+ """Test function that succeeds after retry."""
215
+ call_count = 0
216
+
217
+ @AsyncRetry(max_retries=3, base_delay=0.01)
218
+ async def fails_twice():
219
+ nonlocal call_count
220
+ call_count += 1
221
+ if call_count < 3:
222
+ raise ValueError("Not yet")
223
+ return "success"
224
+
225
+ result = await fails_twice()
226
+
227
+ assert result == "success"
228
+ assert call_count == 3
229
+
230
+ @pytest.mark.asyncio
231
+ async def test_retry_exhausted(self):
232
+ """Test function that always fails."""
233
+ @AsyncRetry(max_retries=2, base_delay=0.01)
234
+ async def always_fails():
235
+ raise ValueError("Always fails")
236
+
237
+ with pytest.raises(ValueError, match="Always fails"):
238
+ await always_fails()
239
+
240
+
241
+ class TestGatherWithConcurrency:
242
+ """Tests for gather_with_concurrency."""
243
+
244
+ @pytest.mark.asyncio
245
+ async def test_gather_results_in_order(self):
246
+ """Test that results maintain order."""
247
+ async def delayed_return(x, delay):
248
+ await asyncio.sleep(delay)
249
+ return x
250
+
251
+ coros = [
252
+ delayed_return(1, 0.02),
253
+ delayed_return(2, 0.01),
254
+ delayed_return(3, 0.03),
255
+ ]
256
+
257
+ results = await gather_with_concurrency(coros, max_concurrency=3)
258
+
259
+ assert results == [1, 2, 3]
260
+
261
+ @pytest.mark.asyncio
262
+ async def test_gather_respects_concurrency(self):
263
+ """Test that concurrency limit is respected."""
264
+ max_concurrent = 0
265
+ current_concurrent = 0
266
+
267
+ async def track_concurrency(x):
268
+ nonlocal max_concurrent, current_concurrent
269
+ current_concurrent += 1
270
+ max_concurrent = max(max_concurrent, current_concurrent)
271
+ await asyncio.sleep(0.05)
272
+ current_concurrent -= 1
273
+ return x
274
+
275
+ coros = [track_concurrency(i) for i in range(10)]
276
+
277
+ await gather_with_concurrency(coros, max_concurrency=3)
278
+
279
+ assert max_concurrent <= 3