NANI-Nithin/clef-flash-GGUF

Clef-Flash — GGUF quants (language tower)

Cloudflare/clef-flash quantized for llama.cpp. The source is a 9.4B-parameter model (architecture Qwen3_5ForConditionalGeneration, 32 layers, hidden size 4096, vocab 248,320), post-trained by Cloudflare from Qwen/Qwen3.5-9B and released under Apache-2.0. This repo has 30 verified files: 29 GGUF quants plus the full-precision BF16 source.

⚠️ What these files do and don't contain

Clef-Flash is a decision model: it reads a state (text, JSON, images, or video) plus a schema of typed questions and returns a probability for every allowed option of every question in a single forward pass — no free-form text generation, no output parsing. That behavior comes from a backbone plus a separate joint schema head (and optionally the vision encoder).

This repo contains the language tower only.

• The vision encoder is not included — there is no --mmproj file here, so image and video inputs are not available from these GGUFs.
• The joint schema head is not included (joint_head.safetensors is a separate component in the source release). Per-question probabilities and the Jev / SystemOne API behavior therefore cannot be reproduced from these files.

For the full decision-model behavior, use Cloudflare/clef-flash directly with transformers (5.10.2) and torch (2.11) per its README. What you get from this repo in llama.cpp is the post-trained Qwen3.5-9B language tower.

About Clef-Flash

From the Cloudflare announcement: Clef-Flash turns a state plus a schema of typed questions into decisions. Each question is one of three types — noul (true/false), choice (named options), or score (ordered options) — and the model's joint schema head reads the backbone's final hidden states, routes evidence from the state to each question, and scores all options of all questions jointly. The Clef-Flash API is compatible with Jev and SystemOne. Clef-Flash is the smaller sibling of Cloudflare/clef.

Files

All 30 files below are verified present on the Hub.

File	Size (GB)	Notes
clef-flash-BF16.gguf	16.69	Full precision source. Every quant below is cut from this file.
clef-flash-Q8_0.gguf	8.87	Effectively lossless. Use when disk and RAM are not the constraint.
clef-flash-Q6_K.gguf	6.85	Near-lossless; the last stop before quality becomes measurable.
clef-flash-Q5_1.gguf	6.33	Legacy. Prefer Q5_K_M.
clef-flash-Q5_K_M.gguf	6.02	Very good quality, noticeably smaller than Q6_K.
clef-flash-Q5_K_S.gguf	5.87	Slightly smaller than Q5_K_M for a slight quality cost.
clef-flash-Q5_0.gguf	5.87	Legacy. Prefer Q5_K_M.
clef-flash-Q4_K_M.gguf	5.24	The usual default. Best quality-per-byte for most people.
clef-flash-Q4_1.gguf	5.41	Legacy. Prefer Q4_K_M.
clef-flash-IQ4_NL.gguf	5.05	Non-linear 4-bit; good on hardware without fast K-quant kernels.
clef-flash-Q4_K_S.gguf	4.98	A little smaller than Q4_K_M, a little worse.
clef-flash-Q4_0.gguf	4.95	Legacy round-to-nearest. Prefer Q4_K_M unless a runtime needs this.
clef-flash-IQ4_XS.gguf	4.84	Best sub-4.5bpw option; usually beats Q4_K_S at a smaller size.
clef-flash-Q3_K_L.gguf	4.59	Small, with real quality loss. Usable when RAM is tight.
clef-flash-Q3_K_M.gguf	4.31	Smaller again; noticeable degradation.
clef-flash-IQ3_M.gguf	4.11	Strong at ~3.7bpw, clearly better than Q3_K_M.
clef-flash-IQ3_S.gguf	4.07	Slightly smaller than IQ3_M.
clef-flash-Q3_K_S.gguf	3.97	Aggressive. Prefer IQ3_M at a similar size.
clef-flash-IQ3_XS.gguf	3.95	Aggressive but coherent.
clef-flash-IQ3_XXS.gguf	3.67	Very aggressive; the last coherent step down.
clef-flash-Q2_K.gguf	3.56	Very small, heavily degraded. For experimentation.
clef-flash-Q2_K_S.gguf	3.44	Smaller than Q2_K, at a further quality cost.
clef-flash-IQ2_M.gguf	3.36	The smallest size most people find usable.
clef-flash-IQ2_S.gguf	3.19	Below the usual usability line.
clef-flash-Q2_0.gguf	3.14	Extreme, group-64. Included for completeness.
clef-flash-IQ2_XS.gguf	3.06	Experimental.
clef-flash-IQ2_XXS.gguf	2.89	Experimental.
clef-flash-IQ1_M.gguf	2.68	Extreme. Expect substantial degradation.
clef-flash-IQ1_S.gguf	2.55	Extreme. Expect substantial degradation.
clef-flash-Q1_0.gguf	1.83	Extreme. Included for completeness.

Quantized 2026-10-04. The importance matrix was computed on the full BF16 weights, not on a lower-precision surrogate.

Usage

# run directly from the Hub
llama-cli -hf NANI-Nithin/clef-flash-GGUF:Q4_K_M -p "Hello"

# or serve it
llama-server -hf NANI-Nithin/clef-flash-GGUF:Q4_K_M

# or download a single file
huggingface-cli download NANI-Nithin/clef-flash-GGUF clef-flash-Q4_K_M.gguf --local-dir .
llama-cli -m clef-flash-Q4_K_M.gguf -p "Hello"

These files are llama.cpp GGUF. They are not loaded by vLLM, SGLang or transformers the way the source repo is — for that, use Cloudflare/clef-flash.

Which file?

• Q4_K_M (5.24 GB) — the default choice; best quality-per-byte for most setups.
• Q8_0 (8.87 GB) — effectively lossless, when resources allow.
• Q6_K (6.85 GB) — near-lossless middle ground.
• IQ4_XS (4.84 GB) — the best pick under ~4.5 bits per weight.
• IQ2_M (3.36 GB) — the smallest size most people find usable.
• Everything smaller than IQ2_M is experimental or extreme; expect substantial degradation. The lowest-bit files exist for completeness only.

Results

Cloudflare publishes decision-model benchmark results in the source README and on the Decision Index leaderboard. In Cloudflare's internal run of the Decision Index 0.2.1 suite, Clef-flash posts the best score among the compared decision models in the majority of listed rows — for example BFCL 98.8 (case exact accuracy), MMLU 91.8, HellaSwag 98.6, WinoGrande 97.5, and a median request latency of 38.8 ms — with ForecastBench reported as a Brier score (10.6), where lower is better. These numbers describe the source model with its joint head on Cloudflare's own harness; the quantized files in this repo were not re-evaluated.

Licence

Apache-2.0, following the base model Qwen/Qwen3.5-9B. Clef-Flash was created by Cloudflare; these quants are an independent llama.cpp conversion of the published weights.