Image-Text-to-Text
MLX
Safetensors
qwen3_5
clairic
cloudflare
systemone
structured-output
information-extraction
multimodal
custom-code
conversational
4-bit precision
Instructions to use mlx-community/clairic-flash-4bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use mlx-community/clairic-flash-4bit with MLX:
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("mlx-community/clairic-flash-4bit") config = load_config("mlx-community/clairic-flash-4bit") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use mlx-community/clairic-flash-4bit with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "mlx-community/clairic-flash-4bit"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "mlx-community/clairic-flash-4bit" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent
How to use mlx-community/clairic-flash-4bit with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "mlx-community/clairic-flash-4bit"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default mlx-community/clairic-flash-4bit
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use mlx-community/clairic-flash-4bit with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "mlx-community/clairic-flash-4bit"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "mlx-community/clairic-flash-4bit" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
File size: 15,395 Bytes
ae97c5d | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 | ---
license: apache-2.0
base_model: Cloudflare/clairic-flash
base_model_relation: quantized
library_name: mlx
pipeline_tag: image-text-to-text
tags:
- mlx
- clairic
- cloudflare
- systemone
- structured-output
- information-extraction
- multimodal
- custom-code
---
# mlx-community/clairic-flash-4bit
An [MLX](https://github.com/ml-explore/mlx) 4-bit conversion of
[`Cloudflare/clairic-flash`](https://huggingface.co/Cloudflare/clairic-flash) for Apple Silicon, with the runtime it
needs and a small HTTP server. The larger, more accurate model of the family is at [`mlx-community/clairic-4bit`](https://huggingface.co/mlx-community/clairic-4bit).
**Clairic is a structured-extraction model, not a chat model.** Give it a *state* (text or JSON, plus optional
images or PDFs) and a schema of typed *fields*; it returns one typed value per field with a confidence and a
null probability. It also does numerical estimates and forecasts.
| Field types | Answered by |
|---|---|
| `text`, `number`, `boolean`, `enum`, `object`, and `list` items of those | grammar-constrained decoding, one short prompt per field or list item |
| `date`, `time`, `datetime`, `duration`, `compute`, `estimate`, `forecast`, date lists | a typed head that reads one forward pass |
## Why it needs the included code
`mlx_lm.load` / `mlx_vlm.load` can load the backbone, but they **will not reproduce Clairic**:
- Answers are produced by Cloudflare's request pipeline, not by chatting: it rewrites each field into its own
prompt, constrains decoding with a grammar built from the field's schema (and, for values copied from the
document, from the document itself), scores `null`, retries, and assembles lists and tables.
- Dates, times, durations, computed numbers, estimates and forecasts come from a separate **typed head**
(`clair_head.safetensors`) applied to the backbone's hidden states.
- The upstream runtime is PyTorch with CUDA kernels. This repository keeps Cloudflare's pipeline
(`clairic_runtime/`, Apache-2.0) and replaces its engine with an MLX one (`clairic_runtime/mlx_engine.py`).
Requests and responses have the same format as upstream.
## Quick start
Requires an Apple Silicon Mac and Python 3.11 or later (tested with 3.12). The model uses 6 GB at rest and up to 11 GB on the requests we timed (see [Speed and memory](#speed-and-memory)), so a 16 GB Mac is the practical minimum; 24 GB gives more room.
```bash
hf download mlx-community/clairic-flash-4bit --local-dir clairic-flash-4bit
cd clairic-flash-4bit
pip install -r requirements.txt
```
`xgrammar` (the grammar engine upstream uses) depends on PyTorch, so `pip` installs `torch`; the model itself
runs entirely in MLX.
```python
from clairic_mlx import load, systemone
model = load(".") # or load("mlx-community/clairic-flash-4bit")
response = systemone(model, {
"state": "Invoice from Northwind Supply. Issued 2026-09-02. Items: Pens; Paper. Total: $1080.00.",
"fields": {
"vendor": {"type": "text", "instructions": "The invoice vendor."},
"total": {"type": "number", "decimals": 2},
"issued": {"type": "date"},
"items": {"type": "list", "items": "text", "instructions": "The purchased item names, in order."},
},
})
print({name: answer["value"] for name, answer in response["answers"].items()})
# {'vendor': 'Northwind Supply', 'total': 1080.0, 'issued': '2026-09-02', 'items': ['Pens', 'Paper']}
```
Images and PDFs (up to four images or pages per request) can be PIL images, file paths or base64 data URIs:
```python
response = systemone(model, {
"state": "Read the attached receipt.",
"images": ["receipt.png"],
"fields": {"merchant": {"type": "text"}, "total": {"type": "number", "decimals": 2}, "date": {"type": "date"}},
})
response = systemone(model, {"documents": ["invoice.pdf"], "fields": {"invoice_number": {"type": "text"}}})
```
Estimates and forecasts return a distribution; `include_quantiles` adds the quantiles:
```python
response = systemone(model, {
"state": "Daily demand for the last seven days: 41, 45, 43, 47, 49, 46, 48.",
"fields": {
"next_day": {"type": "estimate", "min": 0, "max": 100},
"next_three_days": {"type": "forecast", "min": 0, "max": 100, "horizon": 3, "interval": "day"},
},
"options": {"include_quantiles": True},
})
```
Command line and server:
```bash
python clairic_mlx.py --state "Gate B14 closes at 18:20." --fields '{"gate": {"type": "text"}, "closes": {"type": "time"}}'
python clairic_mlx.py --image receipt.png --fields '{"total": {"type": "number", "decimals": 2}}'
python server.py --port 8093
curl localhost:8093/v1/systemone -H 'content-type: application/json' -d '{
"state": "Gate B14 closes at 18:20.",
"fields": {"gate": {"type": "text"}, "closes": {"type": "time"}}}'
```
The server answers one request at a time (a request already fans out into many batched prompts); concurrent
requests wait their turn. Set `CLAIRIC_API_KEY` to require a bearer token. In an async application use
`await model.predict_async(request)`.
The request and response format, the field options (`instructions`, `nullable`, `decimals`, `unit`, `enum`,
`source_only`, `knowledge`, `reference_date`, ...) and the limits (65k-token context, four images or PDF pages,
4 MiB per embedded file) are those of the
[upstream model card](https://huggingface.co/Cloudflare/clairic-flash).
## Fidelity
Tested on 81 requests with known answers: invoices, support tickets, dates and times, computed values, lists and tables, every field option, four languages, documents from 1k to 39k tokens, 13 image requests and 3 PDF requests. They contain 262 fields with exact values, plus estimates and forecasts. The reference is the official Cloudflare runtime on an NVIDIA H100 in its bfloat16 serving configuration, recorded call by call (every prompt, its output tokens and log-probabilities, and every typed-head read).
| Build | Fields with the same value | Prompts decoded to the same tokens | Typed-head reads with the same values | Mean confidence difference | Correct against known answers |
|---|---|---|---|---|---|
| Official runtime, H100 | reference | reference | reference | | 238 / 256 |
| MLX, unquantized bf16 (same code as this repo) | 261 / 262 | 1256 / 1258 | 54 / 55 | 0.005 | 238 / 256 |
| **This repo, 4-bit** | 247 / 262 | 1243 / 1258 | 51 / 55 | 0.031 | 231 / 256 |
The unquantized MLX build reproduces the official run almost exactly. **4-bit costs this 9B model some accuracy:** 15 of 262 fields differ from the official run, and 7 fewer answers are correct. The typical error is a misread character in a number or identifier, often one read from an image or PDF (`2231.5` for `2231.25`, `KR-2026-187` for `KR-2026-1187`). Estimate and forecast summaries differ from the official run by 7.8% on average. [`mlx-community/clairic-4bit`](https://huggingface.co/mlx-community/clairic-4bit), the 27B model, loses almost nothing at 4-bit.
If you have the memory, a higher-precision build of Flash is closer to the original. On the 77 requests of the suite without the longest documents:
| Build (size) | Fields with the same value | Prompts decoded to the same tokens | Typed-head reads with the same values | Mean confidence difference | Correct against known answers |
|---|---|---|---|---|---|
| Official runtime, H100 | reference | reference | reference | | 223 / 240 |
| This repo, 4-bit (6.3 GB) | 231 / 246 | 971 / 986 | 48 / 52 | 0.031 | 216 / 240 |
| Mixed: first 20 layers 6-bit, last 12 layers 4-bit (7.3 GB) | 241 / 246 | 978 / 986 | 50 / 52 | 0.016 | 223 / 240 |
| 8-bit (10.8 GB) | 246 / 246 | 985 / 986 | 51 / 52 | 0.006 | 223 / 240 |
Both can be built from the upstream checkpoint with the included script and run with the same code:
```bash
python convert.py <upstream_checkpoint> <output_dir> --bits 4 --high 'layers\.([0-9]|1[0-9])\.' --high-bits 6 # mixed
python convert.py <upstream_checkpoint> <output_dir> --bits 8 # 8-bit
```
The mixed recipe comes from a sensitivity test: quantizing only the last 12 of the 32 layers to 4 bits changes the output distribution no more than noise, while the first 20 account for the damage. Leaving the vocabulary head or the embeddings unquantized, or raising only the output projections to 8 bits, did not help.
**The port itself is exact.** With unquantized float32 weights on the CPU, the MLX engine agrees with the official PyTorch runtime to float32 rounding on Clairic Flash (confidences within 4e-06, token log-probabilities within 2e-06, same tokens and values on all 27 prompts and 36 fields checked). On the GPU, where MLX multiplies matrices at reduced precision, the float32 build gives the same value on 246 of 246 fields and the same tokens on 987 of 987 prompts of a 77-request run, images and PDFs included.
Confidence is a product of token probabilities, so it moves more than the answers do: 4-bit confidences differ from the official ones by 0.031 on average and by up to 0.36. Calibrate thresholds on this build rather than reusing ones tuned on the official runtime.
## Speed and memory
MacBook Pro, M5 Max, 128 GB, macOS 27.2, mlx 0.32.4. The model loads in about 2 s and uses 6.2 GB at rest. Times are for a warm model (the first request of a new shape compiles kernels and is slower).
| Request | Fields | Prompt tokens run | Time | Peak memory |
|---|---|---|---|---|
| short text, 1 generated field | 1 | 131 | 0.1 s | 6.7 GB |
| short text, 1 typed field (date) | 1 | 207 | 0.1 s | 6.7 GB |
| short text, 8 fields | 8 | 1,733 | 1.4 s | 8.1 GB |
| 1000-token document, 1 field | 1 | 1,155 | 0.7 s | 7.6 GB |
| 1000-token document, 8 fields | 8 | 11,743 | 6.0 s | 8.4 GB |
| 4000-token document, 1 field | 1 | 4,155 | 2.1 s | 7.8 GB |
| 4000-token document, 8 fields | 8 | 41,743 | 20.5 s | 7.8 GB |
| 8000-token document, 1 field | 1 | 8,155 | 4.1 s | 8.1 GB |
| 8000-token document, 8 fields | 8 | 81,743 | 40.5 s | 8.2 GB |
| 16000-token document, 1 field | 1 | 16,157 | 8.6 s | 9.4 GB |
| 32000-token document, 1 field | 1 | 32,156 | 19.2 s | 10.6 GB |
| receipt image, 3 fields | 3 | 2,698 | 2.1 s | 7.6 GB |
| invoice image, 8 fields incl. line items | 8 | 42,541 | 8.7 s | 9.1 GB |
| two images, 4 fields | 4 | 5,163 | 1.4 s | 7.7 GB |
| 2-page PDF, 3 fields | 3 | 3,629 | 1.2 s | 7.6 GB |
| CSV table, 5 rows x 3 columns | 1 | 26,955 | 14.1 s | 8.4 GB |
| 18-row table, 3 columns | 1 | 145,102 | 71.1 s | 8.5 GB |
| forecast (3 steps) + estimate | 2 | 421 | 0.2 s | 7.0 GB |
"Prompt tokens run" is the total over all of a request's prompts: each generated field or list item is its own prompt and re-reads the document, so a request costs roughly *fields x document length*. Typed fields (dates, times, estimates, forecasts) share one pass. Images are read once per request and reused by every field. Inputs longer than 1,024 tokens are processed in segments, so memory grows slowly with length: the 32k-token document peaks at 10.6 GB.
`load(path, memory_limit_gb=...)` sets a ceiling (default: 75% of the machine's memory) above which a request fails with an error instead of exhausting memory. `batch_size` (8) and `batch_token_budget` (8192) control how many prompts run together; larger values were not faster in our tests.
## How it was converted
`convert.py` (included) reproduces these weights from the upstream checkpoint
(revision `36ad68b999560f56b57ef16a22180f8cb83c6d77`):
```bash
python convert.py <upstream_checkpoint> <output_dir> --bits 4 --keep in_proj_a,in_proj_b
```
- Language model: linear layers, token embedding and vocabulary head quantized to 4 bits (affine, group size 64), 4.5 bits per weight.
- Left in bf16: the vision tower, the Gated DeltaNet gate projections (`in_proj_a`, `in_proj_b`: 0.07% of the weights), convolutions and decay parameters.
- RMSNorm weights are stored as exact float32 `1 + w` and evaluated in float32.
- The typed head is the upstream float32 file, with its weights rounded through bfloat16 at load as upstream's serving path does.
- The unused `mtp.*` draft-head weights are dropped.
## Differences from the upstream runtime
- **Engine.** MLX instead of PyTorch/CUDA. The CUDA-only accelerations (Triton kernels, CUDA graphs,
FlashAttention-3) are not needed and not included.
- **Image position encoding.** Upstream has two code paths that treat image tokens differently: in bfloat16
(its serving configuration, and what it does by default on a GPU) every token of an image gets the same
rotary position, while in float32 it uses the stock Qwen 3-axis positions. This port defaults to the first
(`load(path, rope="temporal")`) and also offers the second (`rope="mrope"`). Text-only requests are
unaffected. Upstream's two paths, which also differ in numeric precision, give the same value on 250 of 252 fields of Clairic Flash on an H100, and this port reproduces each of them with the matching setting.
- **Images are read once per request.** Upstream re-reads the image tokens for every field's prompt. Attention
is causal and the linear layers are recurrent, so the engine runs the shared image prefix once and continues
each prompt from that state. Results are the same up to floating-point rounding; requests with images and
many fields are much faster.
- **Image resizing** uses PIL instead of torchvision; pixel values differ from upstream by at most about 2/255.
- **No PyTorch in the model path.** The typed head runs in MLX and its decoding in NumPy (ported from PyTorch
and checked against it; see `NOTICE`).
## Limitations
- 4-bit quantization costs this model some accuracy, mainly on numbers and identifiers read from images (see Fidelity), and shifts confidences. Use the 27B model or a higher-precision build when that matters.
- Speed scales with fields x document length: every generated field re-reads the document, as upstream does.
Long documents with many fields take minutes (see the table above).
- On the M5 Max we tested (mlx 0.32.4), MLX's GPU matrix multiply is accurate to about 1e-3 even for float32
inputs, so results are not bit-identical to PyTorch (see Fidelity). `load(path, device="cpu")` is accurate to float32
rounding and very slow; it exists for verification.
- Scores are model probabilities. Check `p_null`, `confidence`, `valid`, `complete` and `truncated` before
relying on an answer, as the upstream card advises.
- Not tested: requests close to the 65k-token limit.
## Files
| File | Purpose |
|---|---|
| `model-*.safetensors`, `model.safetensors.index.json`, `config.json` | MLX weights (quantized language model, bf16 vision tower) and configuration |
| `clair_head.safetensors`, `clair_head_config.json` | The typed head, unchanged from upstream (fp32) |
| `clairic_mlx.py` | `load`, `systemone`, and the command line |
| `clairic_runtime/` | Cloudflare's request pipeline with the MLX engine (`mlx_engine.py`, `processor.py`) |
| `server.py` | HTTP server: `POST /v1/systemone`, `GET /v1/models`, `/health`, `/stats` |
| `convert.py` | The conversion script |
| tokenizer and processor files, `requirements.txt`, `LICENSE`, `NOTICE` | As named |
## License
Apache-2.0, as upstream. `clairic_runtime/` is Cloudflare's code with the changes listed in `NOTICE`. The
backbone uses [mlx-vlm](https://github.com/Blaizzy/mlx-vlm)'s `qwen3_5` model definition.
|