DenCT/test-storage / DSA Notes /Threading.md
DenCT's picture
|
download
raw
3.37 kB

Python threading enables concurrent execution of tasks within a single process, ideal for I/O-bound operations like network requests despite the Global Interpreter Lock (GIL) limiting true parallelism for CPU-bound work. The threading module builds on lower-level primitives, providing classes for threads, synchronization, and coordination.[1]

Core Concepts

Threads represent lightweight units of execution sharing process memory, allowing overlap in I/O waits. Key types include daemon threads (background tasks that exit with the main program) and the main thread (initial execution flow). Synchronization primitives like Locks prevent race conditions on shared data, while Conditions, Semaphores, Events, and Barriers coordinate thread actions.[1]

Creating Threads

Use threading.Thread(target=func, args=(args,)) for functional style or subclass Thread overriding run(). Start with start() and synchronize via join(timeout=None) to wait for completion.[1]

Basic Example (I/O Simulation):

import threading
import time

def crawl(link, delay=2):
    print(f"crawl started for {link}")
    time.sleep(delay)  # Simulate network I/O
    print(f"crawl ended for {link}")

links = ["https://python.org", "https://docs.python.org"]
threads = [threading.Thread(target=crawl, args=(link,)) for link in links]
for t in threads: t.start()
for t in threads: t.join()

This processes multiple links concurrently, reducing total wait time from sequential execution.[1]

Subclass Example:

class WorkerThread(threading.Thread):
    def __init__(self, name, delay):
        super().__init__()
        self.name = name
        self.delay = delay
    def run(self):
        print(f"Starting {self.name}")
        time.sleep(self.delay * 3)
        print(f"{self.name} done")

t1 = WorkerThread("Worker1", 1)
t1.start(); t1.join()

Overrides behavior for custom logic.[1]

Synchronization Primitives

Locks (Lock() or RLock()): Protect shared resources.

import threading
counter = 0
lock = threading.Lock()

def increment():
    global counter
    with lock:
        counter += 1  # Atomic under lock

threads = [threading.Thread(target=increment) for _ in range(1000)]
for t in threads: t.start()
for t in threads: t.join()
print(counter)  # Reliable 1000, avoids race conditions [web:1]

RLock allows reentrancy for nested acquires by the same thread.[1]

Events: Signal between threads.

event = threading.Event()

def waiter(): event.wait(); print("Event signaled!")
def signaler(): time.sleep(2); event.set()

threading.Thread(target=waiter).start()
threading.Thread(target=signaler).start()

Waiters block until set().[1]

Scenario: Producer-consumer with Condition for bounded queues, or Semaphores for limited resources like database connections (e.g., BoundedSemaphore(5)).[1]

Common Scenarios

  • I/O-Bound (Web Scraping/Downloads): Threads excel as GIL releases during I/O; use ThreadPoolExecutor from concurrent.futures for pools. Sequential: 6s for 3x2s tasks; threaded: ~2s.[2][1]
  • Avoid CPU-Bound: Use multiprocessing instead due to GIL.[1]
  • Daemon Threads: Set daemon=True for non-critical tasks like logging.[1]

Utility functions: active_count() for live threads, current_thread() for identity, enumerate() for listing.[1]

Xet Storage Details

Size:
3.37 kB
·
Xet hash:
7b912498f1820c5a19885ba2b9bddb8136711334445d897b7fc6a7872aeed64d

Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.