|
Download README.md from avgrocketman/compile-cache-t4: direct link, hf CLI and curl.
- Browser
- Download file 2.77 kB
-
https://huggingface.co/avgrocketman/compile-cache-t4/resolve/main/README.md
- Command line
-
hf download hf://avgrocketman/compile-cache-t4/README.md
-
curl -L -o README.md https://huggingface.co/avgrocketman/compile-cache-t4/resolve/main/README.md
2.77 kB
| # T4 cold-start cache | |
| Chip-scoped cache for one NVIDIA Tesla T4 (`sm_75`, CUDA 13.0) on Colab. | |
| It contains the **one** prebuilt `llama.cpp` server used by every T4 model | |
| home, plus a per-model manifest. It contains **no model weights**. | |
| Do not use this repo on L4 (`sm_89`), A100 (`sm_80`), or G4 (`sm_120`): a | |
| binary built for `sm_75` will not load there. Each chip gets its own | |
| `compile-cache-<chip>` repo. | |
| ## Layout | |
| ``` | |
| manifest.json | |
| llama.cpp/<engine-revision>/sm75/cuda13.0/llama-server-t4-sm75-cuda13.0.tar.gz | |
| models/NVIDIA/Dory1.0-4B/manifest.json | |
| models/XHToken/Spark-X2.6-4B/manifest.json | |
| ``` | |
| ## Artifact | |
| | Field | Value | | |
| | :--- | :--- | | |
| | File | `llama.cpp/98a74c97ea09ac72f6ceda1559808fc0a2f447ce/sm75/cuda13.0/llama-server-t4-sm75-cuda13.0.tar.gz` | | |
| | Size | 48,476,283 B | | |
| | SHA-256 | `d58c4433aabe5a500b800c1949498591e94f3e887c5f21e6729308633524e5d5` | | |
| | llama.cpp source | `https://github.com/brianchmielNV/llama.cpp` | | |
| | Revision | `98a74c97ea09ac72f6ceda1559808fc0a2f447ce` (PR #30306) | | |
| | Arch | `sm_75` only | | |
| | CUDA | 13.0 (Colab T4 image) | | |
| | Built on | Colab Tesla T4 `--high-mem` (8 vCPU), 2026-11-06 | | |
| | Reuse | `NVIDIA/Dory1.0-4B`, `XHToken/Spark-X2.6-4B` | | |
| The archive contains `llama-server`, the `libllama*` / `libggml*` shared | |
| objects, and `BUILD-INFO.txt`. It serves both models because the pinned | |
| revision contains both `src/models/dory.cpp` and `src/models/spark2-5.cpp`. | |
| CMake flags: | |
| ``` | |
| -DGGML_CUDA=ON | |
| -DCMAKE_CUDA_ARCHITECTURES=75 | |
| -DLLAMA_BUILD_SERVER=ON | |
| -DLLAMA_BUILD_TESTS=OFF | |
| -DLLAMA_BUILD_EXAMPLES=OFF | |
| -DLLAMA_BUILD_APP=OFF | |
| -DLLAMA_BUILD_UI=OFF | |
| -DLLAMA_USE_PREBUILT_UI=OFF | |
| -DGGML_NATIVE=OFF | |
| ``` | |
| ## Consuming it | |
| A model config pins the repo, path, revision, and SHA-256: | |
| ```toml | |
| binary_repo = "avgrocketman/compile-cache-t4" | |
| binary_file = "llama.cpp/98a74c97ea09ac72f6ceda1559808fc0a2f447ce/sm75/cuda13.0/llama-server-t4-sm75-cuda13.0.tar.gz" | |
| binary_revision = "a1a394aa84d03bd2c6779724655aa9aae4712cb4" # commit that added this file | |
| binary_sha256 = "d58c4433aabe5a500b800c1949498591e94f3e887c5f21e6729308633524e5d5" | |
| ``` | |
| The launcher verifies the SHA-256 before unpacking. A mismatch stops the run. | |
| It unpacks to `/content/llama`, not `/dev/shm` (tmpfs is `noexec` on Colab T4). | |
| ## Rebuild trigger | |
| Rebuild only when the llama.cpp revision, CUDA arch, or CMake feature set | |
| changes. Build on a CPU host with the CUDA 13 toolkit; never compile on a GPU | |
| runtime. Verify `--version`, `ldd`, the embedded SASS arch, and the SHA-256, | |
| then upload one new file and update `manifest.json` and the model manifests. | |
| ## History | |
| Moved from `avgrocketman/llamacpp-dory-t4` on 2026-11-06. The old repo is kept | |
| read-only as a redirect; new runs use this repo. | |
| License: llama.cpp is MIT-licensed; see the upstream repository. | |