feat: add Q4_0 GGUF for Snapdragon NPU (GenieX)

Adds a Q4_0 build quantized from the F16 GGUF with llama-quantize (no imatrix), and documents it in the model card with GenieX usage and context-window guidance.

.gitattributes

@@ -44,3 +44,4 @@ TwIL-LM3-Pro-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
 TwIL-LM3-Pro-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
 TwIL-LM3-Pro-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
 TwIL-LM3-Pro-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+TwIL-LM3-Pro-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -92,7 +92,7 @@ assistant — there is no safety or preference tuning here beyond what Granite 4
 | Language                  | English                                                                                        |
 | Tokenizer vocabulary size | 100,352                                                                                        |
 | Context window            | 131,072 tokens                                                                                 |
-| Checkpoint precision      | bfloat16 (6.82 GiB), plus Q4\_K\_M / Q5\_K\_M / Q6\_K / Q8\_0 / F16 GGUF builds              |
+| Checkpoint precision      | bfloat16 (6.82 GiB), plus Q4\_0 / Q4\_K\_M / Q5\_K\_M / Q6\_K / Q8\_0 / F16 GGUF builds      |
 | Post-training             | LoRA SFT → checkpoint fusion → WiSE-FT (α = 0.15) → MGPO reinforcement learning (β = 0.02, step 2580) |
 | Reasoning format          | Emits a `<think>…</think>` block before the answer (default chat template)                     |
 | Evaluated decoding        | Greedy, 2048 new tokens (one retry at 4096), `max_seq_len` 8192                                |
@@ -423,6 +423,7 @@ as EOS, so run it in conversation mode (`-cnv`).
 
 | file | quant | size | bits/weight | notes |
 |---|---|---:|---:|---|
+| `TwIL-LM3-Pro-Q4_0.gguf` | Q4_0 | 1.98 GiB | 4.65 | for the Snapdragon Hexagon NPU via GenieX (see below) |
 | `TwIL-LM3-Pro-Q4_K_M.gguf` | Q4_K_M | 2.09 GiB | 4.91 | recommended default; runs on CPU or 4 GB of VRAM |
 | `TwIL-LM3-Pro-Q5_K_M.gguf` | Q5_K_M | 2.43 GiB | 5.71 | a little more headroom than Q4_K_M |
 | `TwIL-LM3-Pro-Q6_K.gguf` | Q6_K | 2.80 GiB | 6.57 | close to Q8_0 quality at about three-quarters the size |
@@ -439,14 +440,38 @@ budget large — 2048 tokens or more — since the model emits a `<think>` block
 and a short budget truncates it, which costs far more accuracy than the quantization does.
 
 F16 and Q8_0 were produced directly by `convert_hf_to_gguf.py` from the released bf16 weights; the
-K-quants (Q4_K_M, Q5_K_M, Q6_K) were quantized from the F16 build with `llama-quantize`, without
-an importance matrix. F16 is not bit-identical to the released weights: bf16 and f16 carry the
+K-quants (Q4_K_M, Q5_K_M, Q6_K) and Q4_0 were quantized from the F16 build with `llama-quantize`,
+without an importance matrix. F16 is not bit-identical to the released weights: bf16 and f16 carry the
 same 16 bits but trade exponent range against mantissa precision, so the conversion is a
 narrowing one, in practice negligible for inference.
 
 The published Track A and Track B numbers were measured on the **bf16** weights through vLLM, not
-on any of these GGUF builds, so expect small deviations — most likely at Q4_K_M — that have not
-been quantified here.
+on any of these GGUF builds, so expect small deviations — most likely at Q4_K_M and Q4_0 — that
+have not been quantified here.
+
+### Snapdragon NPU / GenieX
+
+The Q4_0 build is the one to use with [GenieX](https://github.com/qualcomm/GenieX) on Snapdragon
+devices (for example Snapdragon X Elite / X2 Elite), since Q4_0 has the best Hexagon NPU support in
+its llama.cpp runtime.
+
+```bash
+geniex infer webAI-Official/TwIL-LM3-Pro:Q4_0 --nctx 16384
+```
+
+```python
+from geniex import AutoModelForCausalLM
+
+model = AutoModelForCausalLM.from_pretrained(
+    "webAI-Official/TwIL-LM3-Pro", precision="Q4_0", n_ctx=16384
+)
+```
+
+Set the context explicitly: GenieX defaults to a 4,096-token window for GGUF models, which is too
+short for this model's `<think>` blocks (1,902 tokens on average on Track A). The GGUF carries the
+full 131,072-token context, and on the NPU GenieX stores the KV cache in Q8_0, at about 42 KiB per
+token — roughly 0.7 GiB at 16k and 1.3 GiB at 32k on top of the weights. As above, scores were
+measured inside an 8,192-token window, and this build has not yet been benchmarked on device.
 
 ## How it was built
 

TwIL-LM3-Pro-Q4_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:db09635d997275fc07d0df12ea3ab43aa7fe03fa612e061d904581583e9dbbcf
+size 2129118656