Buckets:
| """cwm3/common/pagecache.py — evict a file's pages from the OS page cache after it has been read into arrays (posix_fadvise DONTNEED). | |
| Why (2026-09-11): the host's low-memory guard reacts to FREE memory, not to MemAvailable; loading the multi-GB npz packs leaves a redundant | |
| cached copy that pushes free memory down while the arrays are already in RSS. Evicting the cached pages is non-destructive (a later reader | |
| re-reads from disk). Used by data/cwm3_dataset.load_split / load_bank and the paired-comparison scripts. Never fails: errors are swallowed.""" | |
| from __future__ import annotations | |
| import os | |
| from pathlib import Path | |
| def drop(path) -> bool: | |
| try: | |
| fd = os.open(str(Path(path)), os.O_RDONLY) | |
| try: | |
| os.posix_fadvise(fd, 0, 0, os.POSIX_FADV_DONTNEED) | |
| finally: | |
| os.close(fd) | |
| return True | |
| except Exception: # noqa: BLE001 (missing file / unsupported fs: nothing to evict) | |
| return False | |
| def drop_all(paths) -> int: | |
| return sum(drop(p) for p in paths if p is not None) | |
Xet Storage Details
- Size:
- 1.06 kB
- Xet hash:
- a8226782baa5c94bbed43015c251e2badbcb9a53a3a50237cc8bdfcc1c75272c
·
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.