Image-Text-to-Text
MLX
Safetensors
qwen3_5
clairic
cloudflare
systemone
structured-output
information-extraction
multimodal
custom-code
conversational
4-bit precision
Instructions to use mlx-community/clairic-flash-4bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use mlx-community/clairic-flash-4bit with MLX:
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("mlx-community/clairic-flash-4bit") config = load_config("mlx-community/clairic-flash-4bit") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use mlx-community/clairic-flash-4bit with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "mlx-community/clairic-flash-4bit"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "mlx-community/clairic-flash-4bit" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent
How to use mlx-community/clairic-flash-4bit with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "mlx-community/clairic-flash-4bit"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default mlx-community/clairic-flash-4bit
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use mlx-community/clairic-flash-4bit with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "mlx-community/clairic-flash-4bit"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "mlx-community/clairic-flash-4bit" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
|
Download README.md from mlx-community/clairic-flash-4bit: direct link, hf CLI and curl.
- Browser
- Download file 15.4 kB
-
https://huggingface.co/mlx-community/clairic-flash-4bit/resolve/main/README.md
- Command line
-
hf download hf://mlx-community/clairic-flash-4bit/README.md
-
curl -L -o README.md https://huggingface.co/mlx-community/clairic-flash-4bit/resolve/main/README.md
15.4 kB
| license: apache-2.0 | |
| base_model: Cloudflare/clairic-flash | |
| base_model_relation: quantized | |
| library_name: mlx | |
| pipeline_tag: image-text-to-text | |
| tags: | |
| - mlx | |
| - clairic | |
| - cloudflare | |
| - systemone | |
| - structured-output | |
| - information-extraction | |
| - multimodal | |
| - custom-code | |
| # mlx-community/clairic-flash-4bit | |
| An [MLX](https://github.com/ml-explore/mlx) 4-bit conversion of | |
| [`Cloudflare/clairic-flash`](https://huggingface.co/Cloudflare/clairic-flash) for Apple Silicon, with the runtime it | |
| needs and a small HTTP server. The larger, more accurate model of the family is at [`mlx-community/clairic-4bit`](https://huggingface.co/mlx-community/clairic-4bit). | |
| **Clairic is a structured-extraction model, not a chat model.** Give it a *state* (text or JSON, plus optional | |
| images or PDFs) and a schema of typed *fields*; it returns one typed value per field with a confidence and a | |
| null probability. It also does numerical estimates and forecasts. | |
| | Field types | Answered by | | |
| |---|---| | |
| | `text`, `number`, `boolean`, `enum`, `object`, and `list` items of those | grammar-constrained decoding, one short prompt per field or list item | | |
| | `date`, `time`, `datetime`, `duration`, `compute`, `estimate`, `forecast`, date lists | a typed head that reads one forward pass | | |
| ## Why it needs the included code | |
| `mlx_lm.load` / `mlx_vlm.load` can load the backbone, but they **will not reproduce Clairic**: | |
| - Answers are produced by Cloudflare's request pipeline, not by chatting: it rewrites each field into its own | |
| prompt, constrains decoding with a grammar built from the field's schema (and, for values copied from the | |
| document, from the document itself), scores `null`, retries, and assembles lists and tables. | |
| - Dates, times, durations, computed numbers, estimates and forecasts come from a separate **typed head** | |
| (`clair_head.safetensors`) applied to the backbone's hidden states. | |
| - The upstream runtime is PyTorch with CUDA kernels. This repository keeps Cloudflare's pipeline | |
| (`clairic_runtime/`, Apache-2.0) and replaces its engine with an MLX one (`clairic_runtime/mlx_engine.py`). | |
| Requests and responses have the same format as upstream. | |
| ## Quick start | |
| Requires an Apple Silicon Mac and Python 3.11 or later (tested with 3.12). The model uses 6 GB at rest and up to 11 GB on the requests we timed (see [Speed and memory](#speed-and-memory)), so a 16 GB Mac is the practical minimum; 24 GB gives more room. | |
| ```bash | |
| hf download mlx-community/clairic-flash-4bit --local-dir clairic-flash-4bit | |
| cd clairic-flash-4bit | |
| pip install -r requirements.txt | |
| ``` | |
| `xgrammar` (the grammar engine upstream uses) depends on PyTorch, so `pip` installs `torch`; the model itself | |
| runs entirely in MLX. | |
| ```python | |
| from clairic_mlx import load, systemone | |
| model = load(".") # or load("mlx-community/clairic-flash-4bit") | |
| response = systemone(model, { | |
| "state": "Invoice from Northwind Supply. Issued 2026-09-02. Items: Pens; Paper. Total: $1080.00.", | |
| "fields": { | |
| "vendor": {"type": "text", "instructions": "The invoice vendor."}, | |
| "total": {"type": "number", "decimals": 2}, | |
| "issued": {"type": "date"}, | |
| "items": {"type": "list", "items": "text", "instructions": "The purchased item names, in order."}, | |
| }, | |
| }) | |
| print({name: answer["value"] for name, answer in response["answers"].items()}) | |
| # {'vendor': 'Northwind Supply', 'total': 1080.0, 'issued': '2026-09-02', 'items': ['Pens', 'Paper']} | |
| ``` | |
| Images and PDFs (up to four images or pages per request) can be PIL images, file paths or base64 data URIs: | |
| ```python | |
| response = systemone(model, { | |
| "state": "Read the attached receipt.", | |
| "images": ["receipt.png"], | |
| "fields": {"merchant": {"type": "text"}, "total": {"type": "number", "decimals": 2}, "date": {"type": "date"}}, | |
| }) | |
| response = systemone(model, {"documents": ["invoice.pdf"], "fields": {"invoice_number": {"type": "text"}}}) | |
| ``` | |
| Estimates and forecasts return a distribution; `include_quantiles` adds the quantiles: | |
| ```python | |
| response = systemone(model, { | |
| "state": "Daily demand for the last seven days: 41, 45, 43, 47, 49, 46, 48.", | |
| "fields": { | |
| "next_day": {"type": "estimate", "min": 0, "max": 100}, | |
| "next_three_days": {"type": "forecast", "min": 0, "max": 100, "horizon": 3, "interval": "day"}, | |
| }, | |
| "options": {"include_quantiles": True}, | |
| }) | |
| ``` | |
| Command line and server: | |
| ```bash | |
| python clairic_mlx.py --state "Gate B14 closes at 18:20." --fields '{"gate": {"type": "text"}, "closes": {"type": "time"}}' | |
| python clairic_mlx.py --image receipt.png --fields '{"total": {"type": "number", "decimals": 2}}' | |
| python server.py --port 8093 | |
| curl localhost:8093/v1/systemone -H 'content-type: application/json' -d '{ | |
| "state": "Gate B14 closes at 18:20.", | |
| "fields": {"gate": {"type": "text"}, "closes": {"type": "time"}}}' | |
| ``` | |
| The server answers one request at a time (a request already fans out into many batched prompts); concurrent | |
| requests wait their turn. Set `CLAIRIC_API_KEY` to require a bearer token. In an async application use | |
| `await model.predict_async(request)`. | |
| The request and response format, the field options (`instructions`, `nullable`, `decimals`, `unit`, `enum`, | |
| `source_only`, `knowledge`, `reference_date`, ...) and the limits (65k-token context, four images or PDF pages, | |
| 4 MiB per embedded file) are those of the | |
| [upstream model card](https://huggingface.co/Cloudflare/clairic-flash). | |
| ## Fidelity | |
| Tested on 81 requests with known answers: invoices, support tickets, dates and times, computed values, lists and tables, every field option, four languages, documents from 1k to 39k tokens, 13 image requests and 3 PDF requests. They contain 262 fields with exact values, plus estimates and forecasts. The reference is the official Cloudflare runtime on an NVIDIA H100 in its bfloat16 serving configuration, recorded call by call (every prompt, its output tokens and log-probabilities, and every typed-head read). | |
| | Build | Fields with the same value | Prompts decoded to the same tokens | Typed-head reads with the same values | Mean confidence difference | Correct against known answers | | |
| |---|---|---|---|---|---| | |
| | Official runtime, H100 | reference | reference | reference | | 238 / 256 | | |
| | MLX, unquantized bf16 (same code as this repo) | 261 / 262 | 1256 / 1258 | 54 / 55 | 0.005 | 238 / 256 | | |
| | **This repo, 4-bit** | 247 / 262 | 1243 / 1258 | 51 / 55 | 0.031 | 231 / 256 | | |
| The unquantized MLX build reproduces the official run almost exactly. **4-bit costs this 9B model some accuracy:** 15 of 262 fields differ from the official run, and 7 fewer answers are correct. The typical error is a misread character in a number or identifier, often one read from an image or PDF (`2231.5` for `2231.25`, `KR-2026-187` for `KR-2026-1187`). Estimate and forecast summaries differ from the official run by 7.8% on average. [`mlx-community/clairic-4bit`](https://huggingface.co/mlx-community/clairic-4bit), the 27B model, loses almost nothing at 4-bit. | |
| If you have the memory, a higher-precision build of Flash is closer to the original. On the 77 requests of the suite without the longest documents: | |
| | Build (size) | Fields with the same value | Prompts decoded to the same tokens | Typed-head reads with the same values | Mean confidence difference | Correct against known answers | | |
| |---|---|---|---|---|---| | |
| | Official runtime, H100 | reference | reference | reference | | 223 / 240 | | |
| | This repo, 4-bit (6.3 GB) | 231 / 246 | 971 / 986 | 48 / 52 | 0.031 | 216 / 240 | | |
| | Mixed: first 20 layers 6-bit, last 12 layers 4-bit (7.3 GB) | 241 / 246 | 978 / 986 | 50 / 52 | 0.016 | 223 / 240 | | |
| | 8-bit (10.8 GB) | 246 / 246 | 985 / 986 | 51 / 52 | 0.006 | 223 / 240 | | |
| Both can be built from the upstream checkpoint with the included script and run with the same code: | |
| ```bash | |
| python convert.py <upstream_checkpoint> <output_dir> --bits 4 --high 'layers\.([0-9]|1[0-9])\.' --high-bits 6 # mixed | |
| python convert.py <upstream_checkpoint> <output_dir> --bits 8 # 8-bit | |
| ``` | |
| The mixed recipe comes from a sensitivity test: quantizing only the last 12 of the 32 layers to 4 bits changes the output distribution no more than noise, while the first 20 account for the damage. Leaving the vocabulary head or the embeddings unquantized, or raising only the output projections to 8 bits, did not help. | |
| **The port itself is exact.** With unquantized float32 weights on the CPU, the MLX engine agrees with the official PyTorch runtime to float32 rounding on Clairic Flash (confidences within 4e-06, token log-probabilities within 2e-06, same tokens and values on all 27 prompts and 36 fields checked). On the GPU, where MLX multiplies matrices at reduced precision, the float32 build gives the same value on 246 of 246 fields and the same tokens on 987 of 987 prompts of a 77-request run, images and PDFs included. | |
| Confidence is a product of token probabilities, so it moves more than the answers do: 4-bit confidences differ from the official ones by 0.031 on average and by up to 0.36. Calibrate thresholds on this build rather than reusing ones tuned on the official runtime. | |
| ## Speed and memory | |
| MacBook Pro, M5 Max, 128 GB, macOS 27.2, mlx 0.32.4. The model loads in about 2 s and uses 6.2 GB at rest. Times are for a warm model (the first request of a new shape compiles kernels and is slower). | |
| | Request | Fields | Prompt tokens run | Time | Peak memory | | |
| |---|---|---|---|---| | |
| | short text, 1 generated field | 1 | 131 | 0.1 s | 6.7 GB | | |
| | short text, 1 typed field (date) | 1 | 207 | 0.1 s | 6.7 GB | | |
| | short text, 8 fields | 8 | 1,733 | 1.4 s | 8.1 GB | | |
| | 1000-token document, 1 field | 1 | 1,155 | 0.7 s | 7.6 GB | | |
| | 1000-token document, 8 fields | 8 | 11,743 | 6.0 s | 8.4 GB | | |
| | 4000-token document, 1 field | 1 | 4,155 | 2.1 s | 7.8 GB | | |
| | 4000-token document, 8 fields | 8 | 41,743 | 20.5 s | 7.8 GB | | |
| | 8000-token document, 1 field | 1 | 8,155 | 4.1 s | 8.1 GB | | |
| | 8000-token document, 8 fields | 8 | 81,743 | 40.5 s | 8.2 GB | | |
| | 16000-token document, 1 field | 1 | 16,157 | 8.6 s | 9.4 GB | | |
| | 32000-token document, 1 field | 1 | 32,156 | 19.2 s | 10.6 GB | | |
| | receipt image, 3 fields | 3 | 2,698 | 2.1 s | 7.6 GB | | |
| | invoice image, 8 fields incl. line items | 8 | 42,541 | 8.7 s | 9.1 GB | | |
| | two images, 4 fields | 4 | 5,163 | 1.4 s | 7.7 GB | | |
| | 2-page PDF, 3 fields | 3 | 3,629 | 1.2 s | 7.6 GB | | |
| | CSV table, 5 rows x 3 columns | 1 | 26,955 | 14.1 s | 8.4 GB | | |
| | 18-row table, 3 columns | 1 | 145,102 | 71.1 s | 8.5 GB | | |
| | forecast (3 steps) + estimate | 2 | 421 | 0.2 s | 7.0 GB | | |
| "Prompt tokens run" is the total over all of a request's prompts: each generated field or list item is its own prompt and re-reads the document, so a request costs roughly *fields x document length*. Typed fields (dates, times, estimates, forecasts) share one pass. Images are read once per request and reused by every field. Inputs longer than 1,024 tokens are processed in segments, so memory grows slowly with length: the 32k-token document peaks at 10.6 GB. | |
| `load(path, memory_limit_gb=...)` sets a ceiling (default: 75% of the machine's memory) above which a request fails with an error instead of exhausting memory. `batch_size` (8) and `batch_token_budget` (8192) control how many prompts run together; larger values were not faster in our tests. | |
| ## How it was converted | |
| `convert.py` (included) reproduces these weights from the upstream checkpoint | |
| (revision `36ad68b999560f56b57ef16a22180f8cb83c6d77`): | |
| ```bash | |
| python convert.py <upstream_checkpoint> <output_dir> --bits 4 --keep in_proj_a,in_proj_b | |
| ``` | |
| - Language model: linear layers, token embedding and vocabulary head quantized to 4 bits (affine, group size 64), 4.5 bits per weight. | |
| - Left in bf16: the vision tower, the Gated DeltaNet gate projections (`in_proj_a`, `in_proj_b`: 0.07% of the weights), convolutions and decay parameters. | |
| - RMSNorm weights are stored as exact float32 `1 + w` and evaluated in float32. | |
| - The typed head is the upstream float32 file, with its weights rounded through bfloat16 at load as upstream's serving path does. | |
| - The unused `mtp.*` draft-head weights are dropped. | |
| ## Differences from the upstream runtime | |
| - **Engine.** MLX instead of PyTorch/CUDA. The CUDA-only accelerations (Triton kernels, CUDA graphs, | |
| FlashAttention-3) are not needed and not included. | |
| - **Image position encoding.** Upstream has two code paths that treat image tokens differently: in bfloat16 | |
| (its serving configuration, and what it does by default on a GPU) every token of an image gets the same | |
| rotary position, while in float32 it uses the stock Qwen 3-axis positions. This port defaults to the first | |
| (`load(path, rope="temporal")`) and also offers the second (`rope="mrope"`). Text-only requests are | |
| unaffected. Upstream's two paths, which also differ in numeric precision, give the same value on 250 of 252 fields of Clairic Flash on an H100, and this port reproduces each of them with the matching setting. | |
| - **Images are read once per request.** Upstream re-reads the image tokens for every field's prompt. Attention | |
| is causal and the linear layers are recurrent, so the engine runs the shared image prefix once and continues | |
| each prompt from that state. Results are the same up to floating-point rounding; requests with images and | |
| many fields are much faster. | |
| - **Image resizing** uses PIL instead of torchvision; pixel values differ from upstream by at most about 2/255. | |
| - **No PyTorch in the model path.** The typed head runs in MLX and its decoding in NumPy (ported from PyTorch | |
| and checked against it; see `NOTICE`). | |
| ## Limitations | |
| - 4-bit quantization costs this model some accuracy, mainly on numbers and identifiers read from images (see Fidelity), and shifts confidences. Use the 27B model or a higher-precision build when that matters. | |
| - Speed scales with fields x document length: every generated field re-reads the document, as upstream does. | |
| Long documents with many fields take minutes (see the table above). | |
| - On the M5 Max we tested (mlx 0.32.4), MLX's GPU matrix multiply is accurate to about 1e-3 even for float32 | |
| inputs, so results are not bit-identical to PyTorch (see Fidelity). `load(path, device="cpu")` is accurate to float32 | |
| rounding and very slow; it exists for verification. | |
| - Scores are model probabilities. Check `p_null`, `confidence`, `valid`, `complete` and `truncated` before | |
| relying on an answer, as the upstream card advises. | |
| - Not tested: requests close to the 65k-token limit. | |
| ## Files | |
| | File | Purpose | | |
| |---|---| | |
| | `model-*.safetensors`, `model.safetensors.index.json`, `config.json` | MLX weights (quantized language model, bf16 vision tower) and configuration | | |
| | `clair_head.safetensors`, `clair_head_config.json` | The typed head, unchanged from upstream (fp32) | | |
| | `clairic_mlx.py` | `load`, `systemone`, and the command line | | |
| | `clairic_runtime/` | Cloudflare's request pipeline with the MLX engine (`mlx_engine.py`, `processor.py`) | | |
| | `server.py` | HTTP server: `POST /v1/systemone`, `GET /v1/models`, `/health`, `/stats` | | |
| | `convert.py` | The conversion script | | |
| | tokenizer and processor files, `requirements.txt`, `LICENSE`, `NOTICE` | As named | | |
| ## License | |
| Apache-2.0, as upstream. `clairic_runtime/` is Cloudflare's code with the changes listed in `NOTICE`. The | |
| backbone uses [mlx-vlm](https://github.com/Blaizzy/mlx-vlm)'s `qwen3_5` model definition. | |