Spaces:
Paused
Paused
Download api/scope_cache.py from aios-loopable/cfo-os: direct link, hf CLI and curl.
- Browser
- Download file 3.71 kB
-
https://huggingface.co/spaces/aios-loopable/cfo-os/resolve/main/api/scope_cache.py
- Command line
-
hf download hf://spaces/aios-loopable/cfo-os/api/scope_cache.py
-
curl -L -o scope_cache.py https://huggingface.co/spaces/aios-loopable/cfo-os/resolve/main/api/scope_cache.py
3.71 kB
| """scope_cache.py β stale-while-refresh for the scope-shaped caches (the owner cache rule). | |
| THE RULE, ported from the Streamlit host's cache architecture ([[ri-os-cache-architecture]]): | |
| once a copy exists, NO request ever blocks on a rebuild. A fresh hit returns. A STALE hit | |
| returns the stale copy immediately and refreshes ONCE in a background thread (per-key | |
| single-flight). Only a true miss β no copy at all β builds synchronously, and concurrent | |
| missers wait on the builder's lock instead of stampeding Odoo with N identical pulls. | |
| WHY THIS EXISTS. The wave-2 caches were expire-and-block: at TTL+1s the next visitor paid the | |
| full Odoo pool build (10β30s) in the request path, every 15 minutes, per scope β measured live | |
| as the "extremely long loads". Serving 15-minute-old figures for the seconds a refresh takes is | |
| the documented, owner-chosen trade; blocking a person on an Odoo pull is not. | |
| WHAT THIS DOES NOT CHANGE. The cache MAP and its entry shape stay exactly as the routes had | |
| them β `rt.pool_cache[key] = (ts, value)` β because `verify_api.py` asserts that map holds only | |
| scope-keyed raw values (the cross-user-leak gate). This module only decides WHEN `build()` runs | |
| and on which thread. | |
| β `build` runs with no request context on the refresh path β it must be scope-shaped only | |
| (team_id/agent/granularity), never session-shaped. That is already the routes' own leak rule. | |
| """ | |
| import threading | |
| import time | |
| #: (id(cache), key) -> Lock. Keyed by the cache object's identity so two tenants' runtimes can | |
| #: never share a flight; bounded by the caches' own eviction (a lock per live cache key). | |
| _FLIGHTS = {} | |
| _FLIGHTS_GUARD = threading.Lock() | |
| def _flight(cache, key): | |
| fk = (id(cache), key) | |
| with _FLIGHTS_GUARD: | |
| lk = _FLIGHTS.get(fk) | |
| if lk is None: | |
| lk = _FLIGHTS[fk] = threading.Lock() | |
| # The map only grows with distinct live keys; prune anything not currently held once it | |
| # gets silly. Cheap and rare β this is hygiene, not a hot path. | |
| if len(_FLIGHTS) > 512: | |
| for k in [k for k, v in _FLIGHTS.items() if not v.locked()][:256]: | |
| _FLIGHTS.pop(k, None) | |
| return lk | |
| def get(cache, key, ttl, build, evict=None): | |
| """The value for `key`, per the rule above. `evict` (optional callable) runs after a write | |
| to apply the caller's own bound β bounds differ per cache and stay owned by the caller.""" | |
| now = time.time() | |
| hit = cache.get(key) | |
| if hit and now - hit[0] < ttl: | |
| return hit[1] | |
| lk = _flight(cache, key) | |
| if hit: | |
| # STALE: serve it now; refresh once in the background. If a refresh is already in | |
| # flight, this request just rides the stale copy β that is the whole point. | |
| if lk.acquire(blocking=False): | |
| def _refresh(): | |
| try: | |
| cache[key] = (time.time(), build()) | |
| if evict: | |
| evict() | |
| except Exception: | |
| pass # stale stays servable; the next expiry retries | |
| finally: | |
| lk.release() | |
| threading.Thread(target=_refresh, daemon=True, | |
| name=f"scope-refresh:{key!r}").start() | |
| return hit[1] | |
| # MISS: nothing to serve β build in the request path, single-flight. Late arrivals block | |
| # on the lock and then reuse the winner's entry instead of rebuilding. | |
| with lk: | |
| hit = cache.get(key) | |
| if hit and time.time() - hit[0] < ttl: | |
| return hit[1] | |
| value = build() | |
| cache[key] = (time.time(), value) | |
| if evict: | |
| evict() | |
| return value | |