File size: 2,766 Bytes
27df38e
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
# T4 cold-start cache

Chip-scoped cache for one NVIDIA Tesla T4 (`sm_75`, CUDA 13.0) on Colab.
It contains the **one** prebuilt `llama.cpp` server used by every T4 model
home, plus a per-model manifest. It contains **no model weights**.

Do not use this repo on L4 (`sm_89`), A100 (`sm_80`), or G4 (`sm_120`): a
binary built for `sm_75` will not load there. Each chip gets its own
`compile-cache-<chip>` repo.

## Layout

```
manifest.json
llama.cpp/<engine-revision>/sm75/cuda13.0/llama-server-t4-sm75-cuda13.0.tar.gz
models/NVIDIA/Dory1.0-4B/manifest.json
models/XHToken/Spark-X2.6-4B/manifest.json
```

## Artifact

| Field | Value |
| :--- | :--- |
| File | `llama.cpp/98a74c97ea09ac72f6ceda1559808fc0a2f447ce/sm75/cuda13.0/llama-server-t4-sm75-cuda13.0.tar.gz` |
| Size | 48,476,283 B |
| SHA-256 | `d58c4433aabe5a500b800c1949498591e94f3e887c5f21e6729308633524e5d5` |
| llama.cpp source | `https://github.com/brianchmielNV/llama.cpp` |
| Revision | `98a74c97ea09ac72f6ceda1559808fc0a2f447ce` (PR #30306) |
| Arch | `sm_75` only |
| CUDA | 13.0 (Colab T4 image) |
| Built on | Colab Tesla T4 `--high-mem` (8 vCPU), 2026-11-06 |
| Reuse | `NVIDIA/Dory1.0-4B`, `XHToken/Spark-X2.6-4B` |

The archive contains `llama-server`, the `libllama*` / `libggml*` shared
objects, and `BUILD-INFO.txt`. It serves both models because the pinned
revision contains both `src/models/dory.cpp` and `src/models/spark2-5.cpp`.

CMake flags:

```
-DGGML_CUDA=ON
-DCMAKE_CUDA_ARCHITECTURES=75
-DLLAMA_BUILD_SERVER=ON
-DLLAMA_BUILD_TESTS=OFF
-DLLAMA_BUILD_EXAMPLES=OFF
-DLLAMA_BUILD_APP=OFF
-DLLAMA_BUILD_UI=OFF
-DLLAMA_USE_PREBUILT_UI=OFF
-DGGML_NATIVE=OFF
```

## Consuming it

A model config pins the repo, path, revision, and SHA-256:

```toml
binary_repo = "avgrocketman/compile-cache-t4"
binary_file = "llama.cpp/98a74c97ea09ac72f6ceda1559808fc0a2f447ce/sm75/cuda13.0/llama-server-t4-sm75-cuda13.0.tar.gz"
binary_revision = "a1a394aa84d03bd2c6779724655aa9aae4712cb4"  # commit that added this file
binary_sha256 = "d58c4433aabe5a500b800c1949498591e94f3e887c5f21e6729308633524e5d5"
```

The launcher verifies the SHA-256 before unpacking. A mismatch stops the run.
It unpacks to `/content/llama`, not `/dev/shm` (tmpfs is `noexec` on Colab T4).

## Rebuild trigger

Rebuild only when the llama.cpp revision, CUDA arch, or CMake feature set
changes. Build on a CPU host with the CUDA 13 toolkit; never compile on a GPU
runtime. Verify `--version`, `ldd`, the embedded SASS arch, and the SHA-256,
then upload one new file and update `manifest.json` and the model manifests.

## History

Moved from `avgrocketman/llamacpp-dory-t4` on 2026-11-06. The old repo is kept
read-only as a redirect; new runs use this repo.

License: llama.cpp is MIT-licensed; see the upstream repository.