Instructions to use lanestp/clef-flash-cut-L24-4bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use lanestp/clef-flash-cut-L24-4bit with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("lanestp/clef-flash-cut-L24-4bit", device_map="auto") - MLX
How to use lanestp/clef-flash-cut-L24-4bit with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] hf download lanestp/clef-flash-cut-L24-4bit --local-dir clef-flash-cut-L24-4bit
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
Clef-flash at 24 of 32 layers
Cloudflare's Clef-flash is a decision model: it reads a piece of text and a question and returns a probability for every allowed answer, without generating text. This repository is the same model with layers 24β31, the text head, the vision encoder and Cloudflare's 122M-parameter decision head removed, and the answer read from layer 24 with a small linear head instead.
On tasks where the answer is in the text (reading comprehension, entailment, picking an intent label) it matches the full model or beats it. On tasks that need specialised knowledge (MMLU-Pro, some legal-doctrine questions) it is clearly worse. It needs about a quarter less compute, and about 4 GB of memory at 4-bit on a Mac.
Credit where it is due: the underlying functionality is Cloudflare's. The same recipe on the base model Clef-flash was trained from (Qwen3.5-9B) loses 13β53 points (see below). What this repository shows is that, after their training, the decision is already made by layer 24.
Results
Against the full model, same questions
Decision Index public 0.3 suite (kit @ f5a6e4f), fixed random samples of 500 questions per benchmark (seed 2026), the full model run with Cloudflare's own released code and head. The numbers are accuracy.
| Benchmark | Full Clef-flash | This cut | This cut, 4-bit |
|---|---|---|---|
| WinoGrande | 97.2% | 96.6% | 96.0% |
| BBH (fixed-option) | 68.8% | 68.8% | 67.6% |
| HellaSwag | 98.2% | 98.4% | 97.4% |
| MuSR | 87.0% | 85.6% | 84.2% |
| MMLU-Pro | 64.4% | 57.4% | 56.0% |
| BANKING77 | 90.4% | 93.4% | 93.2% |
| CLINC150 + out-of-scope | 67.6% | 91.8% | 90.8% |
On WinoGrande the two gave the same answer on 491 of 500 questions. On CLINC150 the full model, with Cloudflare's own head, answers "out of scope" for 231 of 500 questions when only 95 are genuinely out of scope; at layer-24 it does not have that bias.
Where the MMLU-Pro gap comes from. The same head read from the full model at layer 32 scores 60.4%, against 57.4% at layer 24 and 64.4% for Cloudflare's head. So about 3 of the 7 points live in layers 25β32, and about 4 are Cloudflare's head beating a single linear vector at the same depth.
Reading vs recall. Across 29 tasks (these seven, our nine yes/no families, fourteen LegalBench tasks), labelled "reading" or "recall" in advance by an independent annotator who never saw the results: on reading tasks the cut never loses more than 2 points (mean +2.8); on recall tasks it averages β4.5, driven by MMLU-Pro and two legal-doctrine tasks, while plain factual recall (which country a city is in) is unaffected. The split is significant on the pre-registered permutation test (p = 0.015) but not on a rank test (p = 0.054).
Yes/no questions, our own held-out tasks
2,700 questions across nine task families none of which were used for training (true/false statements about cities, companies, translations, numbers, common claims and counterfactuals; reading comprehension; inference; sentiment).
| Accuracy | AUROC | Log loss | Calibration error (ECE) | |
|---|---|---|---|---|
| Full Clef-flash with its own head | 91.4% | 0.960 | 0.221 | 0.062 |
| This cut, full precision | 91.3% | 0.959 | 0.220 | 0.058 |
| This cut, 4-bit (MLX) | 91.1% | 0.955 | 0.226 | 0.059 |
Refitting the head on different samples of its training data moves log loss by about Β±0.008, so the three rows are within noise of each other.
The base-model control
The same heads trained the same way on base Qwen3.5-9B (layers 0β23), on the same questions:
| Benchmark | Clef-flash cut | Base Qwen3.5-9B cut |
|---|---|---|
| WinoGrande | 96.8% | 68.5% |
| MuSR | 84.5% | 52.0% |
| HellaSwag (2,000-question sample) | 98.6% | 78.7% |
| BANKING77 (1,000-question sample) | 93.6% | 40.5% |
| BBH | 67.3% | 56.1% |
| MMLU-Pro | 58.2% | 43.9% |
Cloudflare's training data is not public and may include the training splits of some of these benchmarks.
Size and speed
| Memory (short prompts) | Download | Speed per question (Apple M5 Max) | |
|---|---|---|---|
| Full Clef-flash backbone | 18.5 GB | 19.1 GB | 62 ms |
| This cut, PyTorch bf16 (NVIDIA L4) | 12.8 GB | 12.4 GB | 68 ms (L4) |
| This cut, PyTorch 4-bit NF4 (NVIDIA L4) | 5.0 GB weights | 12.4 GB, quantised on load | 63 ms (L4) |
| This cut, MLX full precision | 12.8 GB | 12.4 GB | 40 ms |
| This cut, MLX 4-bit | 4.2 GB | 3.5 GB | 37 ms |
Lower than 4-bit with plain affine quantization costs real accuracy (3-bit: 88.5% on the yes/no tasks). Several community builds of the full model reach a similar size with careful mixed-precision quantization and keep every feature; this cut is a different trade, not a smaller one.
Use
There are two versions of the model, PyTorch and Apple's MLX. Scoring the same on our held-out tests.
PyTorch (NVIDIA, CPU, Mac): pytorch/
from reader_torch import Reader
r = Reader.load("pytorch", device="cuda") # or "cpu" / "mps"; quantize="nf4" for 4-bit on NVIDIA
r.predict("The city of Kagoshima is in Japan.", "Is this statement true?") # P(yes) β 0.98
r.choose("Which planet is largest?", "Which option answers the question?",
{"A": "Mars", "B": "Jupiter", "C": "Venus"}) # {"B": 0.9997, ...}
MLX (Apple silicon, 4-bit): mlx/, same predict and choose calls.
The two calls cover yes/no and multiple-choice questions. Clef's "score" (ordered) questions are not supported.
How it was built and measured
- The cut: layers 0β23 are copied from Cloudflare/clef-flash, and none of them were retrained.
- Yes/no head: logistic regression on the layer-24 activations at three positions (end of the text, end of the question, last prompt token): 12,288 weights. Trained on 7,623 items from SNLI, WANLI, FEVER, StrategyQA, ARC, CommonsenseQA, HellaSwag, PAWS and DBpedia-14.
- Multiple-choice head: each option's tokens are averaged at layer 24 and scored by one shared 4,096-number vector, softmax across options. Trained on ARC and CommonsenseQA training splits.
- No Decision Index data was used for training. Every benchmark was asked with the same prompt wording, and no input was cut short to fit.
- Checks: shuffling the order of the options leaves the scores unchanged (e.g. WinoGrande 96.8% β 97.4%).
- Caveats: samples rather than the full suite; one training run per head; one machine. The Decision Index board requires the full suite, so these numbers are not a board submission. HellaSwag's training split was used for the yes/no head (not the multiple-choice head that produced the HellaSwag score above).
Limits
- It answers yes/no and multiple-choice questions only. It cannot generate free text, answer ordered "score" questions, or read images.
- Knowledge-heavy multiple choice (MMLU-Pro) is clearly below the full model.
- Date arithmetic is at chance (54.5% on deadline-window questions); use a solver.
- Negated questions ("Is this statement false?") are read correctly about 80% of the time.
- It was only tested in English. Memory use on long documents has not been measured for the 4-bit version.
Prior art
Reading a classifier off a middle layer and dropping the layers above is not new: see Chip-Tuning (Zhu et al., 2024), "Why Lift so Heavy?" (Yuan et al., 2024), Layer Enhanced Classification (Sawtell et al., 2024), and work on redundant deep layers (Gromov et al., 2024; ShortGPT, 2024). This repository applies it to a model post-trained for decisions and compares it against that model's own purpose-built head.
Licence
Apache-2.0, as the models it is built from: Cloudflare/clef-flash, post-trained from Qwen/Qwen3.5-9B. Head training data: permissive licences only (non-commercial and undeclared-licence datasets excluded). Decision Index benchmark data was built locally for evaluation and is not redistributed here.
Quantized