q-qwen2.5-1.5b-instruct-q4: Qwen2.5-1.5B-Instruct as one streamable .holo
This is Qwen/Qwen2.5-1.5B-Instruct-GGUF at revision
91cad51170dc346986eccefdc2dd33a9da36ead9, file qwen2.5-1.5b-instruct-q4_0.gguf (1,066,227,232 bytes),
re-laid for the Hologram engine so it runs in a browser on WebGPU.
What changed and what did not:
- 197 Q4_0 weight tensors pass through: the same 4-bit codes, with each 32-weight group's f16 scale widened to f32.
- 141 f32 tensors (norms and the q, k, v biases) pass through.
- One tensor is requantised: the output head, Q6_K in the source, is dequantised and requantised to the same 4-bit format (one f32 scale per 32 weights).
Experimental. It runs on the Hologram AI engine and on the original Q engine; tools that read GGUF do not read this format.
File
| File | What it is |
|---|---|
q-qwen2.5-1.5b-instruct-q4.v1.holo |
One file, 953,978,079 bytes: header, embedded tokenizer (the source GGUF header), then gzip'd tensor blocks in the order the engine reads them, each named by the BLAKE3 of its stored bytes. |
- sha256 of the file:
84b00c7b445488f27d4df0fafc9e74d4ccbd4e478390f243b59c9d765fe6eefc - manifest root:
sha256:2147533113fbe3634ae3e1fceaf2fe0f6d72dbd948e6ab78c6a9f6924fc1348b - shape: Qwen2, 28 layers, width 1536, 12 heads, 2 key-value heads, head size 128, FFN 8960, vocabulary 151,936, q/k/v biases, output head separate from the embedding, rotary base 1e6.
Format
The header manifest is holo-2bit/1 with mode: "q4", bits: 4, attn_bias: true, qk_norm: false. A q4 block is
[codes N*K/2 bytes][f32 scale per 32 weights]; a weight is scale * (code - 8).
Measured
One laptop (Radeon RDNA-3 integrated GPU, Chromium, WebGPU), file served locally, greedy decoding, every run cold:
| Hologram AI engine | Original Q engine | |
|---|---|---|
| First token, 41-token prompt | 156 ms | 545 ms |
| First token, ~430-token prompts | 1.85 s | 5.0 s |
| Tokens per second, ~430-token prompts | 70.7 | 73.6 |
Both engines produced the same tokens on every prompt measured.
Licence
Apache 2.0, as the source. LICENSE is copied unchanged from the source repository; NOTICE.txt states the
modifications.
Model tree for HOLOGRAMTECH/q-qwen2.5-1.5b-instruct-q4
Base model
Qwen/Qwen2.5-1.5B