| ---
|
| license: apache-2.0
|
| base_model: Cloudflare/clef
|
| base_model_relation: quantized
|
| library_name: gguf
|
| quantized_by: PollardWeights
|
| pipeline_tag: text-classification
|
| language:
|
| - en
|
| tags:
|
| - pollard-weights
|
| - pollard
|
| - gguf
|
| - llama.cpp
|
| - imatrix
|
| - qwen3_5
|
| - quantized
|
| - mixed-precision
|
| - measured-allocation
|
| - decision-model
|
| - systemone
|
| - structured-output
|
| ---
|
|
|
| # clef -- Pollard
|
|
|
| > ### Pollard shrank this model: **54.05 GB (f16) -> 9.44 GB** -- **83% smaller, 5.7x down**.
|
| > The smallest rung here; larger, higher-fidelity rungs are listed below.
|
| >
|
| > | format | this model's size |
|
| > |---|---:|
|
| > | f16 | 54.05 GB |
|
| > | Q8_0 | ~28.65 GB |
|
| > | Q6_K | 22.19 GB |
|
| > | Q4_K_M | ~15.67 GB |
|
| > | **PollardMix (this repo's IQ2_XXS)** | **9.44 GB** |
|
|
|
| Pollard builds of [Cloudflare/clef](https://huggingface.co/Cloudflare/clef) made with [Pollard Weights](https://github.com/WestWaters/pollard-weights) -- a ladder of **measured-allocation** quants (bits placed by per-layer sensitivity, not a uniform crush).
|
|
|
| **Standard GGUF, but you need a recent llama.cpp.** This model's architecture (`clef`) is implemented upstream, so any build new enough to carry it runs these files -- llama.cpp itself, and Ollama or LM Studio once they ship a runtime with it. An older build will refuse them with `unknown model architecture`. The quants are ordinary K-quants.
|
|
|
| ## Model details
|
|
|
| | | |
|
| |---|---|
|
| | Parameter count | ~27.0B |
|
| | Architecture | `clef` |
|
| | Input support | text, image |
|
| | imatrix | **yes** -- see [calibration](#imatrix-calibration) |
|
| | Measured | **decision fidelity vs Q8_0** through `/v1/systemone` -- [table below](#decision-fidelity) (perplexity does not apply: a decision model answers with probabilities, not text) |
|
|
|
| ## Which file should I choose?
|
|
|
| Every rung is the **same weights**, sized to a different RAM budget by the measured allocation. Pick the largest one that fits your machine with room for context:
|
|
|
| - **~24.2 GB RAM / VRAM** -> **`Q6_K`** (22.19 GB). *(needs a current llama.cpp)*
|
| - **~18.2 GB RAM / VRAM** -> **`IQ4_XS`** (16.19 GB). *(needs a current llama.cpp)*
|
| - **~15.3 GB RAM / VRAM** -> **`IQ3_S`** (13.25 GB). *(needs a current llama.cpp)*
|
| - **~11.4 GB RAM / VRAM** -> **`IQ2_XXS`** (9.44 GB). *(needs a current llama.cpp)*
|
|
|
| ## Available files
|
|
|
| | file | size | agrees with Q8_0 | option KL | notes |
|
| |---|---:|---:|---:|---|
|
| | `clef-Pollard-IQ2_XXS.gguf` | 9.44 GB | 100% | 0.0038 | |
|
| | `clef-Pollard-IQ3_S.gguf` | 13.25 GB | 100% | 0.00049 | |
|
| | `clef-Pollard-IQ4_XS.gguf` | 16.19 GB | 100% | 9.5e-05 | |
|
| | `clef-Pollard-Q6_K.gguf` | 22.19 GB | 100% | 2.9e-05 | |
|
|
|
| ## Decision fidelity
|
|
|
| This is a **decision model**: it answers typed questions (choice / score / yes-no) with a probability per option, so it is measured on its decisions, not on text. Each rung and the Q8_0 answered the same 20 typed questions through llama-server's `/v1/systemone`, read the same way ([pollard-decision](https://github.com/WestWaters/pollard-weights)).
|
|
|
| | file | size | agrees with Q8_0 | option KL vs Q8_0 | mean prob. drift | accuracy |
|
| |---|---:|---:|---:|---:|---:|
|
| | `clef-Pollard-Q6_K.gguf` | 22.19 GB | 100% | 2.9e-05 | 0.0003 | 100% |
|
| | `clef-Pollard-IQ4_XS.gguf` | 16.19 GB | 100% | 9.5e-05 | 0.0007 | 100% |
|
| | `clef-Pollard-IQ3_S.gguf` | 13.25 GB | 100% | 0.00049 | 0.0014 | 100% |
|
| | `clef-Pollard-IQ2_XXS.gguf` | 9.44 GB | 100% | 0.0038 | 0.0066 | 100% |
|
|
|
| _Agrees = the rung picks the same option as the Q8_0. Option KL = how far its option probabilities moved from the Q8_0's (0 = identical); it is the number that separates the rungs._
|
|
|
| ## Build notes
|
|
|
| - **Measured allocation.** The sensitivity profile was derived from the imatrix and the f16 weights one tensor at a time (`pollard-probe --from-imatrix`), covering all 64 layers, including the 48 Gated DeltaNet linear-attention layers. Profile included: `clef-Pollard.sensitivity.json`. imatrix over the full Calib 3.0 corpus (1937 chunks), computed on a Q8_0 copy: `clef-Pollard.imatrix`.
|
| - **Reference.** The decision board compares each rung with **Q8_0**: the 54 GB f16 does not fit the build machine's RAM + VRAM for serving.
|
| - **Decision head precision.** The decision head (`dec.*`) is held at Q6_K in every rung. Rebuilding the IQ2_XXS rung with the head at BF16 (same plan otherwise) gave option KL 0.00377 vs 0.00385 with the Q6_K head: the head at Q6_K does not collapse the probabilities, and the drift that remains comes from the body.
|
| - **Images.** The projector carries `clip.vision.image_min_pixels = 65536` from the model's processor, so small images get the same number of tokens as the original processor on current llama.cpp.
|
|
|
| ## Multimodal
|
|
|
| Vision needs the projector shipped alongside: **`mmproj-clef-BF16.gguf`** -- download it too and pass it with `--mmproj`. It is **not quantized**; it is small and the text ladder is where the size lives.
|
|
|
| ```bash
|
| llama-server -m clef-Pollard-IQ2_XXS.gguf --mmproj mmproj-clef-BF16.gguf -ngl 99
|
| ```
|
|
|
| ## Download a specific file
|
|
|
| ```bash
|
| pip install -U "huggingface_hub[cli]"
|
| hf download PollardWeights/clef-Pollard \
|
| --include "clef-Pollard-IQ2_XXS.gguf" --local-dir ./
|
| ```
|
|
|
| ## How to run
|
|
|
| This model's architecture (`clef`) needs a llama.cpp new enough to carry it. A decision model is served on **`/v1/systemone`** (no chat or completions):
|
|
|
| ```bash
|
| llama-server -m clef-Pollard-IQ2_XXS.gguf -ngl 99 --port 8080
|
| curl http://127.0.0.1:8080/v1/systemone -H "Content-Type: application/json" -d '{
|
| "state": "Customer message: I was charged twice for my order last week and nobody has replied.",
|
| "questions": {
|
| "route": {"type": "choice", "instructions": "Which team should handle this?",
|
| "criteria": {"billing": null, "shipping": null, "technical": null}},
|
| "angry": {"type": "noul", "instructions": "Is the customer angry?"},
|
| "urgency": {"type": "score", "instructions": "How urgent is this?",
|
| "criteria": ["can wait", "this week", "today", "right now"]}
|
| }
|
| }'
|
| ```
|
|
|
| Each answer carries the probability of every option. Some decision models evaluate the whole prompt in one batch, so a long `state` may need a larger `--ubatch-size`.
|
|
|
| ## imatrix (calibration)
|
|
|
| The importance matrix (`clef-Pollard.imatrix`, included) was computed on a mixed-domain corpus so the matrix sees every register the model serves.
|
|
|
| ## ARM / AVX
|
|
|
| llama.cpp repacks weights into an interleaved layout at load time for faster inference on ARM and AVX machines -- no special file needed, online repacking covers these quants. The old `Q4_0_4_4/4_8/8_8` variants are not required.
|
|
|
| ## Errata
|
|
|
| - `general.architecture` is `clef`, which upstream llama.cpp added recently. A build older than that support refuses these files with `unknown model architecture` -- update llama.cpp rather than looking for a different quant. Checked with `pollard-ggufcheck`, which reads the architecture and the tensor types out of the header and asks upstream what it implements.
|
| - Measured allocation places bits by per-layer sensitivity under a size budget.
|
| - Single machine; replication invited.
|
|
|
| ## Credits & license
|
|
|
| - Base model: [`Cloudflare/clef`](https://huggingface.co/Cloudflare/clef)
|
| - Quantization tooling: [llama.cpp](https://github.com/ggml-org/llama.cpp) (ggml-org)
|
| - Method + tooling: [Pollard Weights](https://github.com/WestWaters/pollard-weights) -- *measure first, no claim before a number.*
|
| - License: `apache-2.0`, inherited from the base model.
|
|
|
| *Built with [Pollard Weights](https://github.com/WestWaters/pollard-weights) -- frontier models, small hardware, no compromise.*
|
|
|
