Instructions to use Xylolabs/clairic-flash-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use Xylolabs/clairic-flash-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf Xylolabs/clairic-flash-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf Xylolabs/clairic-flash-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf Xylolabs/clairic-flash-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf Xylolabs/clairic-flash-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf Xylolabs/clairic-flash-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf Xylolabs/clairic-flash-GGUF:Q4_K_M
Use Docker
docker model run hf.co/Xylolabs/clairic-flash-GGUF:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use Xylolabs/clairic-flash-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Xylolabs/clairic-flash-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Xylolabs/clairic-flash-GGUF", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/Xylolabs/clairic-flash-GGUF:Q4_K_M
- Ollama
How to use Xylolabs/clairic-flash-GGUF with Ollama:
ollama run hf.co/Xylolabs/clairic-flash-GGUF:Q4_K_M
- Unsloth Desktop
- Pi
How to use Xylolabs/clairic-flash-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf Xylolabs/clairic-flash-GGUF:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "Xylolabs/clairic-flash-GGUF:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use Xylolabs/clairic-flash-GGUF with Docker Model Runner:
docker model run hf.co/Xylolabs/clairic-flash-GGUF:Q4_K_M
- Lemonade
How to use Xylolabs/clairic-flash-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull Xylolabs/clairic-flash-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.clairic-flash-GGUF-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use Xylolabs/clairic-flash-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf Xylolabs/clairic-flash-GGUF:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default Xylolabs/clairic-flash-GGUF:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use Xylolabs/clairic-flash-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf Xylolabs/clairic-flash-GGUF:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "Xylolabs/clairic-flash-GGUF:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
GGUF quants of Cloudflare/clairic-flash (9B, a fine-tune of Qwen/Qwen3.5-9B for structured extraction) for llama.cpp, on CUDA, Metal or CPU. Stock llama.cpp runs it, with image input through the mmproj file.
- Files: Q8_0, Q6_K, Q5_K_M, Q4_K_M, Q4_K_S, IQ4_XS, Q3_K_M, IQ3_M and Q2_K (each one file with the tokenizer and chat template inside), the vision projector
mmproj-clairic-flash-BF16.gguf, and the importance matrix. - Q4_K_M: KL 0.0468 from the f32 reference, top-1 93.47 %.
- Cloudflare's description: below, unchanged.
Files
clairic-flash-<QUANT>.ggufis one file per quant: the language model.mmproj-clairic-flash-BF16.ggufis the vision encoder, for image input.clairic-flash.imatrix.ggufis the importance matrix.
| Quant | Size | PPL | Mean KL vs f32 reference | Same top-1 vs f32 reference |
|---|---|---|---|---|
| Q8_0 | 9.53 GB | 6.0278 | 0.0072 | 97.95% |
| Q6_K | 7.37 GB | 6.0361 | 0.0132 | 96.94% |
| Q5_K_M | 6.50 GB | 5.9699 | 0.0203 | 95.98% |
| Q4_K_M | 5.69 GB | 5.9742 | 0.0468 | 93.47% |
| Q4_K_S | 5.37 GB | 5.9592 | 0.0507 | 93.13% |
| IQ4_XS | 5.27 GB | 6.2223 | 0.0562 | 92.61% |
| Q3_K_M | 4.69 GB | 5.7594 | 0.1636 | 87.62% |
| IQ3_M | 4.48 GB | 6.7828 | 0.1510 | 87.25% |
| Q2_K | 3.93 GB | 5.3667 | 0.3112 | 82.22% |
How we measured:
- Reference: the model in transformers with f32 compute on Cloudflare's BF16 weights (PPL 6.0592).
- KL, top-1, PPL: 12 chunks of 4,096 tokens, scoring positions 2,048 to 4,095 of each (24,564 tokens): 9 held-out chat-formatted chunks (English, Korean, code, math, JSON extraction from a schema, and Clairic's own field-extraction prompts) and 3 from wikitext-2.
The extraction head
Cloudflare's typed extraction head (clair_head.safetensors) and their Python runtime need the original weights and are not in these files. These files hold the language model and the vision encoder.
How to run
llama-server -m clairic-flash-Q4_K_M.gguf --mmproj mmproj-clairic-flash-BF16.gguf \
--jinja -ngl 99 -c 32768 --chat-template-kwargs '{"enable_thinking":false}'
Run it with thinking off, as above, the way Cloudflare's runtime does. For JSON that follows a schema, pass the schema in the request (response_format with json_schema in llama-server, or --json-schema in llama-cli); llama.cpp turns it into a grammar.
How these were made
- Convert Cloudflare's BF16 checkpoint with llama.cpp's
convert_hf_to_gguf.py --no-mtp(commite4b620af19f02960dea7a3a81de7c5805aa3a333), and the vision encoder with--mmproj(BF16). The checkpoint's multi-token-prediction block is left out, as Cloudflare's config turns it off. - Build the importance matrix with
llama-imatrix: 600 chunks of 4,096 tokens of chat-formatted text rendered with the model's template, thinking off (databricks-dolly-15k in English, KoAlpaca-v1.1a in Korean, python_code_instructions_18k_alpaca for code, gsm8k for math, MasterControlAIML/JSON-Unstructured-Structured for schema-guided JSON, and paraloq/json_data_extraction as Clairic's field-extraction prompts), with--process-output. - Make each quant with
llama-quantize --imatrix IM --tensor-type ssm_beta_alpha=f16 <bf16 gguf> <out> <QUANT>(Q8_0 without the importance matrix): every weight matrix gets llama.cpp's own mix for that type; the Gated DeltaNet beta/alpha projection stays F16 (all but 0.035 % of its BF16 values are exact in F16, the largest difference 3e-8); norms,ssm_a,ssm_dtand the conv1d kernels stay F32.
The model: 8.95B parameters in the language model and 0.46B in the vision encoder; 32 layers (24 Gated DeltaNet linear-attention layers, 8 full-attention layers), 16 query and 4 KV heads of dimension 256, hidden size 4,096, context 262,144, vocabulary 248,320.
License and notices
- License: Apache-2.0, copied in
LICENSE(Cloudflare's text). - What we changed: we quantized the weights and converted them to GGUF.
- Notices:
NOTICEnames the source model and its copyright holders. - Xylolabs is not affiliated with or endorsed by Cloudflare.
Cloudflare's original model description
This section is Cloudflare's model card, copied from Cloudflare/clairic-flash (Apache-2.0).
- It describes Cloudflare's original model, not these files. Its usage code needs the original weights and Cloudflare's runtime; for the GGUF files, see "How to run" above.
- We changed only the heading levels (two deeper).
Clairic Flash
Clairic Flash is a 9B multimodal model that turns a state and a schema of typed fields into structured answers. It supports text and image inputs, extraction, numerical estimation, and forecasting. Each field has its own value and scoring information; the response also reports input truncation and token usage.
See Clairic for the other model in the Clairic family.
Model
- Base model: Qwen3.5-9B.
- Inputs: a text or JSON state, optional images, and a mapping of field names to specifications.
- Outputs: typed values, lists, nulls, estimates, and forecast paths.
- Scoring: extraction confidence and null probabilities, plus optional quantiles for estimates.
- Runtime: PyTorch and Transformers, with local Python request helpers.
Usage
Use Python 3.11 or later and install the packages listed in requirements.txt. Inference uses PyTorch and
Transformers; xgrammar supplies schema constraints through a Transformers logits
processor.
Download the repository, load the model and processor, and call systemone with a
System One: Extract request.
import sys
from huggingface_hub import snapshot_download
path = snapshot_download("Cloudflare/clairic-flash")
sys.path.insert(0, path)
from clairic_model import load_release_model, systemone
model, processor = load_release_model(path, device="cuda")
response = systemone(
model,
processor,
{
"state": (
"Invoice from Northwind Supply. Issued 2026-09-02. "
"Items: Pens; Paper. Total: $1080.00."
),
"fields": {
"vendor": {"type": "text", "instructions": "The invoice vendor."},
"total": {"type": "number", "decimals": 2},
"issued": {"type": "date"},
"items": {
"type": "list",
"items": "text",
"instructions": "The purchased item names, in order.",
},
},
},
)
print(response["answers"])
Images
Pass PIL images alongside the state. Embedded base64 image inputs are also accepted.
from PIL import Image
response = systemone(
model,
processor,
{
"state": "Read the attached receipt.",
"images": [Image.open("receipt.png")],
"fields": {
"merchant": {"type": "text", "instructions": "The merchant name."},
"total": {"type": "number", "decimals": 2},
},
},
)
print(response["answers"])
Estimates and forecasts
Declare a range for estimated values. Forecasts also specify a horizon and interval.
Set include_quantiles to return the available quantiles.
response = systemone(
model,
processor,
{
"state": "Daily demand for the last seven days: 41, 45, 43, 47, 49, 46, 48.",
"fields": {
"next_day": {"type": "estimate", "min": 0, "max": 100},
"next_three_days": {
"type": "forecast",
"min": 0,
"max": 100,
"horizon": 3,
"interval": "day",
},
},
"options": {"include_quantiles": True},
},
)
print(response["answers"])
Input format
| Field | Description |
|---|---|
model |
Optional clairic-flash or @cf/cloudflare/clairic-flash |
state |
Source text; the Python helper serializes JSON values to text |
images |
Optional PIL images or embedded base64 images |
documents |
Optional embedded PDF data URIs or content_type / base64 objects |
fields |
Mapping of output field names to typed specifications |
reference_date |
Optional ISO date for relative date instructions |
options.include_quantiles |
Include available estimate and forecast quantiles |
options.return_state_text |
Include the retained source text in the response |
Field specifications support text, number, date, time, datetime, duration,
boolean, enum, object, list, compute, estimate, and forecast.
instructions describes what to return. Numeric fields can specify units and
rounding; lists declare their item schema; estimates and forecasts require bounds.
nullable defaults to true. source_only: true selects literal printed numbers,
dates, times, datetimes, or number/date lists. knowledge: true permits knowledge
answers for text, numbers, and supported list children. compute selects the typed
numeric head. Unsupported options and invalid combinations are rejected.
Embedded inputs are limited to 4 MiB each and 8 MiB combined. Images and rendered
PDF pages share a four-image/page limit. Image rotation accepts 0, 90, 180, or 270
clockwise degrees after EXIF orientation. Remote media URLs are not accepted.
Response
systemone returns a dictionary with model, answers, truncated, and usage.
Answers are keyed by the requested field names. Generated scalar extraction answers
retain valid, complete, full-sequence confidence, and independently scored p_null.
Uniquely matched text evidence includes character spans in state; image-only or
ambiguous evidence has null spans. Lists preserve item order and duplicates.
Estimates and forecasts include their typed values and, when requested, quantiles.
Scores are model probabilities and should be evaluated for the application's own
acceptance thresholds. Inspect truncated and any answer completeness indicators
before relying on a result.
Usage includes retries and supplemental work. The response model ID is clairic-flash.
The Python API raises validation or request errors for invalid inputs, a queue-full
error for simultaneous requests, and an inference error for failed generation.
Use await model.predict_async(request) in an async application; cancellation
drains owned work before admitting another request.
Files
| File | Purpose |
|---|---|
model*.safetensors, model.safetensors.index.json, config.json |
Model weights and configuration |
clair_head.safetensors, clair_head_config.json |
Typed output components |
clairic_model.py and supporting Python modules |
Transformers loading and request handling |
| Tokenizer and processor files | Text and image preprocessing |
requirements.txt |
Python dependencies |
LICENSE |
Apache-2.0 license |
License
Released under the Apache-2.0 license.
- Downloads last month
- -

docker model run hf.co/Xylolabs/clairic-flash-GGUF: