How to use from
OpenClaw
Start the MLX server
# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "mlx-community/clairic-4bit"
Configure OpenClaw
# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "mlx-community/clairic-4bit" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Quick Links

mlx-community/clairic-4bit

An MLX 4-bit conversion of Cloudflare/clairic for Apple Silicon, with the runtime it needs and a small HTTP server. The smaller model of the family is at mlx-community/clairic-flash-4bit.

Clairic is a structured-extraction model, not a chat model. Give it a state (text or JSON, plus optional images or PDFs) and a schema of typed fields; it returns one typed value per field with a confidence and a null probability. It also does numerical estimates and forecasts.

Field types Answered by
text, number, boolean, enum, object, and list items of those grammar-constrained decoding, one short prompt per field or list item
date, time, datetime, duration, compute, estimate, forecast, date lists a typed head that reads one forward pass

Why it needs the included code

mlx_lm.load / mlx_vlm.load can load the backbone, but they will not reproduce Clairic:

  • Answers are produced by Cloudflare's request pipeline, not by chatting: it rewrites each field into its own prompt, constrains decoding with a grammar built from the field's schema (and, for values copied from the document, from the document itself), scores null, retries, and assembles lists and tables.
  • Dates, times, durations, computed numbers, estimates and forecasts come from a separate typed head (clair_head.safetensors) applied to the backbone's hidden states.
  • The upstream runtime is PyTorch with CUDA kernels. This repository keeps Cloudflare's pipeline (clairic_runtime/, Apache-2.0) and replaces its engine with an MLX one (clairic_runtime/mlx_engine.py). Requests and responses have the same format as upstream.

Quick start

Requires an Apple Silicon Mac and Python 3.11 or later (tested with 3.12). The model uses 16 GB at rest and up to 24 GB on the requests we timed (see Speed and memory), so a 32 GB Mac is the practical minimum; 48 GB is comfortable.

hf download mlx-community/clairic-4bit --local-dir clairic-4bit
cd clairic-4bit
pip install -r requirements.txt

xgrammar (the grammar engine upstream uses) depends on PyTorch, so pip installs torch; the model itself runs entirely in MLX.

from clairic_mlx import load, systemone

model = load(".")          # or load("mlx-community/clairic-4bit")
response = systemone(model, {
    "state": "Invoice from Northwind Supply. Issued 2026-09-02. Items: Pens; Paper. Total: $1080.00.",
    "fields": {
        "vendor": {"type": "text", "instructions": "The invoice vendor."},
        "total": {"type": "number", "decimals": 2},
        "issued": {"type": "date"},
        "items": {"type": "list", "items": "text", "instructions": "The purchased item names, in order."},
    },
})
print({name: answer["value"] for name, answer in response["answers"].items()})
# {'vendor': 'Northwind Supply', 'total': 1080.0, 'issued': '2026-09-02', 'items': ['Pens', 'Paper']}

Images and PDFs (up to four images or pages per request) can be PIL images, file paths or base64 data URIs:

response = systemone(model, {
    "state": "Read the attached receipt.",
    "images": ["receipt.png"],
    "fields": {"merchant": {"type": "text"}, "total": {"type": "number", "decimals": 2}, "date": {"type": "date"}},
})
response = systemone(model, {"documents": ["invoice.pdf"], "fields": {"invoice_number": {"type": "text"}}})

Estimates and forecasts return a distribution; include_quantiles adds the quantiles:

response = systemone(model, {
    "state": "Daily demand for the last seven days: 41, 45, 43, 47, 49, 46, 48.",
    "fields": {
        "next_day": {"type": "estimate", "min": 0, "max": 100},
        "next_three_days": {"type": "forecast", "min": 0, "max": 100, "horizon": 3, "interval": "day"},
    },
    "options": {"include_quantiles": True},
})

Command line and server:

python clairic_mlx.py --state "Gate B14 closes at 18:20." --fields '{"gate": {"type": "text"}, "closes": {"type": "time"}}'
python clairic_mlx.py --image receipt.png --fields '{"total": {"type": "number", "decimals": 2}}'

python server.py --port 8093
curl localhost:8093/v1/systemone -H 'content-type: application/json' -d '{
  "state": "Gate B14 closes at 18:20.",
  "fields": {"gate": {"type": "text"}, "closes": {"type": "time"}}}'

The server answers one request at a time (a request already fans out into many batched prompts); concurrent requests wait their turn. Set CLAIRIC_API_KEY to require a bearer token. In an async application use await model.predict_async(request).

The request and response format, the field options (instructions, nullable, decimals, unit, enum, source_only, knowledge, reference_date, ...) and the limits (65k-token context, four images or PDF pages, 4 MiB per embedded file) are those of the upstream model card.

Fidelity

Tested on 81 requests with known answers: invoices, support tickets, dates and times, computed values, lists and tables, every field option, four languages, documents from 1k to 39k tokens, 13 image requests and 3 PDF requests. They contain 262 fields with exact values, plus estimates and forecasts. The reference is the official Cloudflare runtime on an NVIDIA H100 in its bfloat16 serving configuration, recorded call by call (every prompt, its output tokens and log-probabilities, and every typed-head read).

Build Fields with the same value Prompts decoded to the same tokens Typed-head reads with the same values Mean confidence difference Correct against known answers
Official runtime, H100 reference reference reference 242 / 256
MLX, unquantized bf16 (same code as this repo) 262 / 262 640 / 640 55 / 55 0.002 242 / 256
This repo, 4-bit 257 / 262 628 / 639 53 / 55 0.024 245 / 256

The unquantized MLX build reproduces the official run on every field and every prompt. At 4-bit, 5 of 262 fields differ (for example "Room 12" against "12", or a time's seconds), and accuracy against the known answers is within noise of the official run. All 52 image fields and 9 PDF fields match. Estimate and forecast summaries (mean, median, 10th and 90th percentile) differ from the official run by 1.7% on average.

The port itself is exact. With unquantized float32 weights on the CPU, the MLX engine agrees with the official PyTorch runtime to float32 rounding on Clairic Flash (confidences within 4e-06, token log-probabilities within 2e-06, same tokens and values on all 27 prompts and 36 fields checked). On the GPU, where MLX multiplies matrices at reduced precision, the float32 build gives the same value on 246 of 246 fields and the same tokens on 987 of 987 prompts of a 77-request run, images and PDFs included.

Confidence is a product of token probabilities, so it moves more than the answers do: 4-bit confidences differ from the official ones by 0.024 on average and by up to 0.68. Calibrate thresholds on this build rather than reusing ones tuned on the official runtime.

Speed and memory

MacBook Pro, M5 Max, 128 GB, macOS 27.2, mlx 0.32.4. The model loads in about 2 s and uses 16.4 GB at rest. Times are for a warm model (the first request of a new shape compiles kernels and is slower).

Request Fields Prompt tokens run Time Peak memory
short text, 1 generated field 1 131 0.4 s 17.0 GB
short text, 1 typed field (date) 1 179 0.3 s 16.8 GB
short text, 8 fields 8 1,703 4.8 s 19.7 GB
1000-token document, 1 field 1 1,155 2.7 s 18.4 GB
1000-token document, 8 fields 8 11,713 20.7 s 20.2 GB
4000-token document, 1 field 1 4,155 7.2 s 18.7 GB
4000-token document, 8 fields 8 41,713 70.7 s 18.7 GB
8000-token document, 1 field 1 8,155 14.3 s 20.9 GB
8000-token document, 8 fields 8 81,713 141.5 s 20.9 GB
16000-token document, 1 field 1 16,157 29.9 s 21.9 GB
32000-token document, 1 field 1 32,156 65.5 s 24.3 GB
receipt image, 3 fields 3 1,526 2.5 s 18.0 GB
invoice image, 8 fields incl. line items 8 17,410 16.9 s 21.6 GB
two images, 4 fields 4 5,249 4.0 s 18.9 GB
2-page PDF, 3 fields 3 3,629 3.4 s 18.4 GB
CSV table, 5 rows x 3 columns 1 2,648 7.2 s 20.7 GB
18-row table, 3 columns 1 145,102 255.2 s 20.9 GB
forecast (3 steps) + estimate 2 421 0.7 s 17.1 GB

"Prompt tokens run" is the total over all of a request's prompts: each generated field or list item is its own prompt and re-reads the document, so a request costs roughly fields x document length. Typed fields (dates, times, estimates, forecasts) share one pass. Images are read once per request and reused by every field. Inputs longer than 1,024 tokens are processed in segments, so memory grows slowly with length: the 32k-token document peaks at 24.3 GB.

load(path, memory_limit_gb=...) sets a ceiling (default: 75% of the machine's memory) above which a request fails with an error instead of exhausting memory. batch_size (8) and batch_token_budget (8192) control how many prompts run together; larger values were not faster in our tests.

How it was converted

convert.py (included) reproduces these weights from the upstream checkpoint (revision 2c161be5cc71bacd6605b99a8c07547e58cfa434):

python convert.py <upstream_checkpoint> <output_dir> --bits 4 --keep in_proj_a,in_proj_b
  • Language model: linear layers, token embedding and vocabulary head quantized to 4 bits (affine, group size 64), 4.5 bits per weight.
  • Left in bf16: the vision tower, the Gated DeltaNet gate projections (in_proj_a, in_proj_b: 0.06% of the weights, and the decay gate is sensitive), convolutions and decay parameters.
  • RMSNorm weights are stored as exact float32 1 + w and evaluated in float32.
  • The typed head is the upstream float32 file, with its weights rounded through bfloat16 at load as upstream's serving path does.
  • The unused mtp.* draft-head weights are dropped.

Differences from the upstream runtime

  • Engine. MLX instead of PyTorch/CUDA. The CUDA-only accelerations (Triton kernels, CUDA graphs, FlashAttention-3) are not needed and not included.
  • Image position encoding. Upstream has two code paths that treat image tokens differently: in bfloat16 (its serving configuration, and what it does by default on a GPU) every token of an image gets the same rotary position, while in float32 it uses the stock Qwen 3-axis positions. This port defaults to the first (load(path, rope="temporal")) and also offers the second (rope="mrope"). Text-only requests are unaffected. Upstream's two paths, which also differ in numeric precision, give the same value on 250 of 252 fields of Clairic Flash on an H100, and this port reproduces each of them with the matching setting.
  • Images are read once per request. Upstream re-reads the image tokens for every field's prompt. Attention is causal and the linear layers are recurrent, so the engine runs the shared image prefix once and continues each prompt from that state. Results are the same up to floating-point rounding; requests with images and many fields are much faster.
  • Image resizing uses PIL instead of torchvision; pixel values differ from upstream by at most about 2/255.
  • No PyTorch in the model path. The typed head runs in MLX and its decoding in NumPy (ported from PyTorch and checked against it; see NOTICE).

Limitations

  • 4-bit quantization changes a small share of answers (see Fidelity) and shifts confidences.
  • Speed scales with fields x document length: every generated field re-reads the document, as upstream does. Long documents with many fields take minutes (see the table above).
  • On the M5 Max we tested (mlx 0.32.4), MLX's GPU matrix multiply is accurate to about 1e-3 even for float32 inputs, so results are not bit-identical to PyTorch (see Fidelity). load(path, device="cpu") is accurate to float32 rounding and very slow; it exists for verification.
  • Scores are model probabilities. Check p_null, confidence, valid, complete and truncated before relying on an answer, as the upstream card advises.
  • Not tested: requests close to the 65k-token limit.

Files

File Purpose
model-*.safetensors, model.safetensors.index.json, config.json MLX weights (quantized language model, bf16 vision tower) and configuration
clair_head.safetensors, clair_head_config.json The typed head, unchanged from upstream (fp32)
clairic_mlx.py load, systemone, and the command line
clairic_runtime/ Cloudflare's request pipeline with the MLX engine (mlx_engine.py, processor.py)
server.py HTTP server: POST /v1/systemone, GET /v1/models, /health, /stats
convert.py The conversion script
tokenizer and processor files, requirements.txt, LICENSE, NOTICE As named

License

Apache-2.0, as upstream. clairic_runtime/ is Cloudflare's code with the changes listed in NOTICE. The backbone uses mlx-vlm's qwen3_5 model definition.

Downloads last month
-
Safetensors
Model size
5B params
Tensor type
U32
ยท
F32
ยท
BF16
ยท
MLX
Hardware compatibility
Log In to add your hardware

4-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for mlx-community/clairic-4bit

Base model

Qwen/Qwen3.8-27B
Quantized
(1)
this model