DenCT/test-storage / DSA Notes /Threading.md
DenCT's picture
|
download
raw
3.37 kB
Python threading enables concurrent execution of tasks within a single process, ideal for I/O-bound operations like network requests despite the Global Interpreter Lock (GIL) limiting true parallelism for CPU-bound work. The `threading` module builds on lower-level primitives, providing classes for threads, synchronization, and coordination.[1]
## Core Concepts
Threads represent lightweight units of execution sharing process memory, allowing overlap in I/O waits. Key types include daemon threads (background tasks that exit with the main program) and the main thread (initial execution flow). Synchronization primitives like Locks prevent race conditions on shared data, while Conditions, Semaphores, Events, and Barriers coordinate thread actions.[1]
## Creating Threads
Use `threading.Thread(target=func, args=(args,))` for functional style or subclass `Thread` overriding `run()`. Start with `start()` and synchronize via `join(timeout=None)` to wait for completion.[1]
**Basic Example (I/O Simulation):**
```python
import threading
import time
def crawl(link, delay=2):
print(f"crawl started for {link}")
time.sleep(delay) # Simulate network I/O
print(f"crawl ended for {link}")
links = ["https://python.org", "https://docs.python.org"]
threads = [threading.Thread(target=crawl, args=(link,)) for link in links]
for t in threads: t.start()
for t in threads: t.join()
```
This processes multiple links concurrently, reducing total wait time from sequential execution.[1]
**Subclass Example:**
```python
class WorkerThread(threading.Thread):
def __init__(self, name, delay):
super().__init__()
self.name = name
self.delay = delay
def run(self):
print(f"Starting {self.name}")
time.sleep(self.delay * 3)
print(f"{self.name} done")
t1 = WorkerThread("Worker1", 1)
t1.start(); t1.join()
```
Overrides behavior for custom logic.[1]
## Synchronization Primitives
**Locks (`Lock()` or `RLock()`):** Protect shared resources.
```python
import threading
counter = 0
lock = threading.Lock()
def increment():
global counter
with lock:
counter += 1 # Atomic under lock
threads = [threading.Thread(target=increment) for _ in range(1000)]
for t in threads: t.start()
for t in threads: t.join()
print(counter) # Reliable 1000, avoids race conditions [web:1]
```
`RLock` allows reentrancy for nested acquires by the same thread.[1]
**Events:** Signal between threads.
```python
event = threading.Event()
def waiter(): event.wait(); print("Event signaled!")
def signaler(): time.sleep(2); event.set()
threading.Thread(target=waiter).start()
threading.Thread(target=signaler).start()
```
Waiters block until `set()`.[1]
**Scenario:** Producer-consumer with `Condition` for bounded queues, or Semaphores for limited resources like database connections (e.g., `BoundedSemaphore(5)`).[1]
## Common Scenarios
- **I/O-Bound (Web Scraping/Downloads):** Threads excel as GIL releases during I/O; use `ThreadPoolExecutor` from `concurrent.futures` for pools. Sequential: 6s for 3x2s tasks; threaded: ~2s.[2][1]
- **Avoid CPU-Bound:** Use `multiprocessing` instead due to GIL.[1]
- **Daemon Threads:** Set `daemon=True` for non-critical tasks like logging.[1]
Utility functions: `active_count()` for live threads, `current_thread()` for identity, `enumerate()` for listing.[1]

Xet Storage Details

Size:
3.37 kB
·
Xet hash:
7b912498f1820c5a19885ba2b9bddb8136711334445d897b7fc6a7872aeed64d

Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.