Spaces:
Running
Running
Commit History
-
fix(F-7a): add per-key loading locks to ModelCache
get()/set() are each individually thread-safe, but the check-then-act pattern every consumer (clip/dire/own_embedding detectors) wraps around them is not: concurrent cold-start requests for the same key can each observe a cache miss and each independently perform a full model load. get_load_lock(key) returns a lock consumers should hold across the entire get-load-set sequence with a double-check after acquiring it. This commit only adds the mechanism; the three detectors are wired to it in the next commits.3282fe1
-
Add model caching infrastructure (#35)
Implemented singleton model cache with LRU eviction: 1. backend/core/model_cache.py: - Singleton pattern for global cache - Thread-safe operations - LRU eviction policy - Memory limit enforcement - Cache statistics tracking 2. backend/config/cache_config.py: - Configurable cache limits - Model size estimates - Enable/disable toggle 3. backend/tests/test_model_cache.py: - 7 comprehensive tests - Tests singleton, LRU, memory limits - Statistics validation Features: - Max 10 models cached - Max 8GB memory usage - Automatic LRU eviction - Thread-safe access - Real-time statistics Part 1 of 2 - Next: Integrate with detectors Part of #354bb6afa