compile-cache-v5e / README.md
avgrocketman's picture
Add chip-level cache index, runtime pins, and model manifest
48ab30c verified
|
Raw History Blame Contribute Delete
3.18 kB
# TPU v5e cold-start cache
Chip-scoped cache for Google Cloud TPU v5e-1 on Colab. It contains the pinned
Python 3.12 `vllm-tpu` virtualenv, `zstd`, the per-model XLA compile caches,
and a per-model manifest. It contains **no model weights**.
Do not use this repo on a GPU or on TPU v6e: the venv, XLA executables, and
`libtpu` are built for one chip and one engine version. Each chip gets its own
`compile-cache-<chip>` repo.
## Layout
```
manifest.json
venv312_cpu.tar.zst # fixed filename used by coldstart
zstd_bin # fixed filename used by coldstart
models/LFM/LFM2.5-24B-A2B/
manifest.json
vllm-tpu-0.32.0/
len100000__b16-8192__42f6059b.tar
len100000__b16-8192__noclip__0a680e5e.tar
archive/
lfm2.5-24b-a2b/ # superseded LFM caches
qwen3-4b/ # legacy Qwen3 caches + versions.json
```
The two root runtime files keep their historical names because
`coldstart/download.py` fetches `venv312_cpu.tar.zst` and `zstd_bin` by
default. Model caches are namespaced paths and are pinned explicitly in the
model TOML via `COLDSTART_CACHE_NAME`.
## Runtime
| File | Size | SHA-256 |
| :--- | ---: | :--- |
| `venv312_cpu.tar.zst` | 1,454,946,513 B | `d3af735580f7ac2e49ba65a394e40ffdd8d1736f1167b00d1c0e0a0405d0df40` |
| `zstd_bin` | 1,022,760 B | `7c5468b370f7c47eda07281e3437fafc568f95d10420051e3aa522709f9342c5` |
Pins recorded inside the venv: Python 3.12, `torch 2.10.0+cpu`,
`vllm-tpu 0.32.0`, `tpu-inference 0.32.0`, `jax 0.11.2`, `jaxlib 0.11.2`,
`libtpu 0.0.48`.
## Active XLA caches
| Variant | Path | Size | SHA-256 | Pinned by |
| :--- | :--- | ---: | :--- | :--- |
| LFM2.5 V1 | `models/LFM/LFM2.5-24B-A2B/vllm-tpu-0.32.0/len100000__b16-8192__42f6059b.tar` | 196,085,760 B | `bdbb4fb3872664a713d8517268dff984af4ee7ceabf9ad90700a2bfba27b41ef` | `config/v5e-w8a8-w4a16.toml` |
| LFM2.5 V2 `noclip` | `models/LFM/LFM2.5-24B-A2B/vllm-tpu-0.32.0/len100000__b16-8192__noclip__0a680e5e.tar` | 199,987,200 B | `32827c9cb9f736d7e66b643813541f9d3f06c1dc2a5cc9ba99a967e8f309f5e7` | `config/v5e-w8a8-w4a16-v2.toml` |
The old `clip99` archive was byte-identical to `noclip` (same SHA-256), so it
is not duplicated here. Superseded caches are under `archive/` and are never
auto-restored.
## Consuming it
```toml
[install]
repo = "avgrocketman/compile-cache-v5e"
filename = "venv312_cpu.tar.zst"
[env]
COLDSTART_CACHE_NAME = "models/LFM/LFM2.5-24B-A2B/vllm-tpu-0.32.0/len100000__b16-8192__42f6059b.tar"
```
A cache is valid only for the exact combination of chip, engine version, model,
`context_length`, sorted buckets, KV mode, and patch set. The filename hash and
the explicit pin enforce that; there is no fallback to another model's cache.
## History
Moved from `avgrocketman/tpu-compile-cache` on 2026-11-06. The old flat-name
caches and the Qwen3 caches are preserved under `archive/`; the old repo is kept
read-only as a redirect. The move is a Hub-side copy, so the archive bytes and
XLA keys are unchanged.
License: the venv contains open-source Python packages under their own licenses;
the cache archives contain compiled XLA programs, not model weights.