Xylolabs

Clairic Flash GGUF

Base model Formats License Code

GGUF quants of Cloudflare/clairic-flash (9B, a fine-tune of Qwen/Qwen3.5-9B for structured extraction) for llama.cpp, on CUDA, Metal or CPU. Stock llama.cpp runs it, with image input through the mmproj file.

  • Files: Q8_0, Q6_K, Q5_K_M, Q4_K_M, Q4_K_S, IQ4_XS, Q3_K_M, IQ3_M and Q2_K (each one file with the tokenizer and chat template inside), the vision projector mmproj-clairic-flash-BF16.gguf, and the importance matrix.
  • Q4_K_M: KL 0.0468 from the f32 reference, top-1 93.47 %.
  • Cloudflare's description: below, unchanged.

Files

  • clairic-flash-<QUANT>.gguf is one file per quant: the language model.
  • mmproj-clairic-flash-BF16.gguf is the vision encoder, for image input.
  • clairic-flash.imatrix.gguf is the importance matrix.
Quant Size PPL Mean KL vs f32 reference Same top-1 vs f32 reference
Q8_0 9.53 GB 6.0278 0.0072 97.95%
Q6_K 7.37 GB 6.0361 0.0132 96.94%
Q5_K_M 6.50 GB 5.9699 0.0203 95.98%
Q4_K_M 5.69 GB 5.9742 0.0468 93.47%
Q4_K_S 5.37 GB 5.9592 0.0507 93.13%
IQ4_XS 5.27 GB 6.2223 0.0562 92.61%
Q3_K_M 4.69 GB 5.7594 0.1636 87.62%
IQ3_M 4.48 GB 6.7828 0.1510 87.25%
Q2_K 3.93 GB 5.3667 0.3112 82.22%

How we measured:

  • Reference: the model in transformers with f32 compute on Cloudflare's BF16 weights (PPL 6.0592).
  • KL, top-1, PPL: 12 chunks of 4,096 tokens, scoring positions 2,048 to 4,095 of each (24,564 tokens): 9 held-out chat-formatted chunks (English, Korean, code, math, JSON extraction from a schema, and Clairic's own field-extraction prompts) and 3 from wikitext-2.

Mean KL and same top-1 against the f32 reference by file size

The extraction head

Cloudflare's typed extraction head (clair_head.safetensors) and their Python runtime need the original weights and are not in these files. These files hold the language model and the vision encoder.

How to run

llama-server -m clairic-flash-Q4_K_M.gguf --mmproj mmproj-clairic-flash-BF16.gguf \
  --jinja -ngl 99 -c 32768 --chat-template-kwargs '{"enable_thinking":false}'

Run it with thinking off, as above, the way Cloudflare's runtime does. For JSON that follows a schema, pass the schema in the request (response_format with json_schema in llama-server, or --json-schema in llama-cli); llama.cpp turns it into a grammar.

How these were made

  1. Convert Cloudflare's BF16 checkpoint with llama.cpp's convert_hf_to_gguf.py --no-mtp (commit e4b620af19f02960dea7a3a81de7c5805aa3a333), and the vision encoder with --mmproj (BF16). The checkpoint's multi-token-prediction block is left out, as Cloudflare's config turns it off.
  2. Build the importance matrix with llama-imatrix: 600 chunks of 4,096 tokens of chat-formatted text rendered with the model's template, thinking off (databricks-dolly-15k in English, KoAlpaca-v1.1a in Korean, python_code_instructions_18k_alpaca for code, gsm8k for math, MasterControlAIML/JSON-Unstructured-Structured for schema-guided JSON, and paraloq/json_data_extraction as Clairic's field-extraction prompts), with --process-output.
  3. Make each quant with llama-quantize --imatrix IM --tensor-type ssm_beta_alpha=f16 <bf16 gguf> <out> <QUANT> (Q8_0 without the importance matrix): every weight matrix gets llama.cpp's own mix for that type; the Gated DeltaNet beta/alpha projection stays F16 (all but 0.035 % of its BF16 values are exact in F16, the largest difference 3e-8); norms, ssm_a, ssm_dt and the conv1d kernels stay F32.

The model: 8.95B parameters in the language model and 0.46B in the vision encoder; 32 layers (24 Gated DeltaNet linear-attention layers, 8 full-attention layers), 16 query and 4 KV heads of dimension 256, hidden size 4,096, context 262,144, vocabulary 248,320.

License and notices

  • License: Apache-2.0, copied in LICENSE (Cloudflare's text).
  • What we changed: we quantized the weights and converted them to GGUF.
  • Notices: NOTICE names the source model and its copyright holders.
  • Xylolabs is not affiliated with or endorsed by Cloudflare.

Cloudflare's original model description

This section is Cloudflare's model card, copied from Cloudflare/clairic-flash (Apache-2.0).

  • It describes Cloudflare's original model, not these files. Its usage code needs the original weights and Cloudflare's runtime; for the GGUF files, see "How to run" above.
  • We changed only the heading levels (two deeper).

Clairic Flash

Clairic Flash is a 9B multimodal model that turns a state and a schema of typed fields into structured answers. It supports text and image inputs, extraction, numerical estimation, and forecasting. Each field has its own value and scoring information; the response also reports input truncation and token usage.

See Clairic for the other model in the Clairic family.

Model

  • Base model: Qwen3.5-9B.
  • Inputs: a text or JSON state, optional images, and a mapping of field names to specifications.
  • Outputs: typed values, lists, nulls, estimates, and forecast paths.
  • Scoring: extraction confidence and null probabilities, plus optional quantiles for estimates.
  • Runtime: PyTorch and Transformers, with local Python request helpers.

Usage

Use Python 3.11 or later and install the packages listed in requirements.txt. Inference uses PyTorch and Transformers; xgrammar supplies schema constraints through a Transformers logits processor.

Download the repository, load the model and processor, and call systemone with a System One: Extract request.

import sys

from huggingface_hub import snapshot_download

path = snapshot_download("Cloudflare/clairic-flash")
sys.path.insert(0, path)
from clairic_model import load_release_model, systemone

model, processor = load_release_model(path, device="cuda")

response = systemone(
    model,
    processor,
    {
        "state": (
            "Invoice from Northwind Supply. Issued 2026-09-02. "
            "Items: Pens; Paper. Total: $1080.00."
        ),
        "fields": {
            "vendor": {"type": "text", "instructions": "The invoice vendor."},
            "total": {"type": "number", "decimals": 2},
            "issued": {"type": "date"},
            "items": {
                "type": "list",
                "items": "text",
                "instructions": "The purchased item names, in order.",
            },
        },
    },
)
print(response["answers"])
Images

Pass PIL images alongside the state. Embedded base64 image inputs are also accepted.

from PIL import Image

response = systemone(
    model,
    processor,
    {
        "state": "Read the attached receipt.",
        "images": [Image.open("receipt.png")],
        "fields": {
            "merchant": {"type": "text", "instructions": "The merchant name."},
            "total": {"type": "number", "decimals": 2},
        },
    },
)
print(response["answers"])
Estimates and forecasts

Declare a range for estimated values. Forecasts also specify a horizon and interval. Set include_quantiles to return the available quantiles.

response = systemone(
    model,
    processor,
    {
        "state": "Daily demand for the last seven days: 41, 45, 43, 47, 49, 46, 48.",
        "fields": {
            "next_day": {"type": "estimate", "min": 0, "max": 100},
            "next_three_days": {
                "type": "forecast",
                "min": 0,
                "max": 100,
                "horizon": 3,
                "interval": "day",
            },
        },
        "options": {"include_quantiles": True},
    },
)
print(response["answers"])

Input format

Field Description
model Optional clairic-flash or @cf/cloudflare/clairic-flash
state Source text; the Python helper serializes JSON values to text
images Optional PIL images or embedded base64 images
documents Optional embedded PDF data URIs or content_type / base64 objects
fields Mapping of output field names to typed specifications
reference_date Optional ISO date for relative date instructions
options.include_quantiles Include available estimate and forecast quantiles
options.return_state_text Include the retained source text in the response

Field specifications support text, number, date, time, datetime, duration, boolean, enum, object, list, compute, estimate, and forecast. instructions describes what to return. Numeric fields can specify units and rounding; lists declare their item schema; estimates and forecasts require bounds. nullable defaults to true. source_only: true selects literal printed numbers, dates, times, datetimes, or number/date lists. knowledge: true permits knowledge answers for text, numbers, and supported list children. compute selects the typed numeric head. Unsupported options and invalid combinations are rejected.

Embedded inputs are limited to 4 MiB each and 8 MiB combined. Images and rendered PDF pages share a four-image/page limit. Image rotation accepts 0, 90, 180, or 270 clockwise degrees after EXIF orientation. Remote media URLs are not accepted.

Response

systemone returns a dictionary with model, answers, truncated, and usage. Answers are keyed by the requested field names. Generated scalar extraction answers retain valid, complete, full-sequence confidence, and independently scored p_null. Uniquely matched text evidence includes character spans in state; image-only or ambiguous evidence has null spans. Lists preserve item order and duplicates. Estimates and forecasts include their typed values and, when requested, quantiles.

Scores are model probabilities and should be evaluated for the application's own acceptance thresholds. Inspect truncated and any answer completeness indicators before relying on a result.

Usage includes retries and supplemental work. The response model ID is clairic-flash. The Python API raises validation or request errors for invalid inputs, a queue-full error for simultaneous requests, and an inference error for failed generation. Use await model.predict_async(request) in an async application; cancellation drains owned work before admitting another request.

Files

File Purpose
model*.safetensors, model.safetensors.index.json, config.json Model weights and configuration
clair_head.safetensors, clair_head_config.json Typed output components
clairic_model.py and supporting Python modules Transformers loading and request handling
Tokenizer and processor files Text and image preprocessing
requirements.txt Python dependencies
LICENSE Apache-2.0 license

License

Released under the Apache-2.0 license.

Downloads last month
-
GGUF
Model size
9B params
Architecture
qwen35
Hardware compatibility
Log In to add your hardware

2-bit

3-bit

4-bit

5-bit

6-bit

8-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for Xylolabs/clairic-flash-GGUF

Finetuned
Qwen/Qwen3.5-9B
Quantized
(2)
this model

Collection including Xylolabs/clairic-flash-GGUF