GGUF kernels: dequantize + fused gemv over packed blocks

.pytest_cache/v/cache/lastfailed

@@ -1 +1,3 @@
-{}
+{
+  "tests/test_gguf_kernels.py": true
+}

README.md

@@ -60,3 +60,8 @@ nix run .#build-and-copy -L    # every variant, into build/
 nix build .#ci                 # just this system's variant
 pytest tests/                  # against a built variant on PYTHONPATH
 ```
+
+`flake.nix` tracks the builder's default branch, so CI builds the current variant matrix. The
+`build/` directory here was produced against the builder release matching the `kernels` client it was
+verified with (torch 2.11/2.12): both ops tested against `gguf.quants.dequantize` for Q4_K/Q5_K/Q6_K/
+Q8_0 in f32/f16/bf16, the gemv against a dense matmul, and one `fullgraph=True` compile.

SKILL.md

@@ -57,8 +57,9 @@ dispatch on the tensor's device as usual.
 ## 3. Which builder, and why it decides everything
 
 Use `github:huggingface/kernels` (the old `kernel-builder` repo is stale) with
-`lib.genKernelFlakeOutputs`. **Pin it to the same version as the `kernels` client you load with**,
-because the builder version decides two things at once:
+`lib.genKernelFlakeOutputs`. Leave it unpinned for a repo whose variants CI builds — that keeps the
+matrix current. Pin it to the `kernels` client version only when *you* build the artifacts you will
+load locally, because the builder version decides two things at once:
 
 | | |
 | --- | --- |
@@ -66,7 +67,8 @@ because the builder version decides two things at once:
 | the `metadata.json` format | older builders omit `name`, and a current client refuses to load that |
 
 A mismatch surfaces as `Cannot parse metadata from ...: missing field 'name'`, or as a build with no
-variant your interpreter can import. Check what came out before debugging anything else:
+variant your interpreter can import — which is what unpinned costs locally: `main` targets torch
+2.12/2.13, so testing a local build needs a matching interpreter. Check what came out before debugging anything else:
 
 ```bash
 ls build/ && head -6 build/*/metadata.json
@@ -137,9 +139,13 @@ reproducible. Then, in increasing order of surprise:
 2. **Trust.** `get_kernel` loads unattended only from a publisher the Hub marks as a trusted kernel
    publisher (e.g. `kernels-community`). A personal repo needs `trust_remote_code=True`, i.e. the
    caller opting into running downloaded code — so a library should not pass it silently.
-3. **Indexing.** The client resolves through `https://huggingface.co/api/kernels/<repo>/...`, which
-   401s for a repo the Hub has not indexed as a kernel. A correct repo can still be unfetchable;
-   `kernels-community` repos work, so compare against one before suspecting your own build.
+3. **The repo has to be under an organization.** The client resolves through
+   `https://huggingface.co/api/kernels/<repo>/...`, which 401s (404s with a token) for a repo in a
+   *user* namespace — no tag, metadata or `kernel` tag changes that. The trust check gives the same
+   hint: it calls `get_organization_overview(publisher)`, which a username has no answer for. So a
+   correct, fully published repo can still be unfetchable by id. Compare against a
+   `kernels-community` repo (`get_kernel("kernels-community/relu", version=1)`) before suspecting
+   your own build, and use the override below until the kernels live in an org.
 
 Test without the Hub at all:
 
@@ -160,3 +166,17 @@ _MY_OP = module.my_op          # NOT self.kernel.my_op
 Reaching an op through an attribute on an object makes dynamo graph-break at every call, which costs
 more than the kernel saves — the same reason the fakes matter. Treat "no kernel" as a normal outcome:
 return `None`, log why, and fall back to the dense path.
+
+## 8. Sanity numbers
+
+From this build, so you know what "working" looks like:
+
+| | |
+| --- | --- |
+| compile, one variant | ~150 s (5 translation units, 5 SM archs) |
+| all variants | ~8 min for 6 |
+| the whole exercise | dominated by rebuilds after each build.toml/builder discovery, not by compiling |
+
+End state worth reproducing: `pytest tests/` green against a built variant, `fullgraph=True` compile
+green, and the calling library loading the same artifact through `get_kernel` — for GGUF that was a
+4B Q4_K_M model at 3.07 GB and 187 tok/s instead of 8.4 GB dequantized.

flake.nix

@@ -1,7 +1,7 @@
 {
   description = "GGUF kernels: compute on the packed blocks of a GGUF checkpoint";
   inputs = {
-    kernel-builder.url = "github:huggingface/kernels/v0.15.2";
+    kernel-builder.url = "github:huggingface/kernels";
   };
   outputs = { self, kernel-builder }:
     kernel-builder.lib.genKernelFlakeOutputs {