clairic-4bit / NOTICE
lucataco's picture
Clairic, MLX 4-bit, with runtime, server and conversion script
1a48d6b verified
Raw History Blame Contribute Delete
1.73 kB
Clairic for MLX
This repository is a derivative of Cloudflare's Clairic release
(https://huggingface.co/Cloudflare/clairic and https://huggingface.co/Cloudflare/clairic-flash),
Copyright 2026 Cloudflare, Inc., licensed under the Apache License, Version 2.0 (see LICENSE).
Clairic is fine-tuned from Qwen models by the Qwen Team, also released under Apache-2.0.
Changes made for this MLX port:
- The model weights were converted to the MLX format and the language model's linear layers, token embedding
and vocabulary head were quantized (see README.md for the exact recipe). The typed head
(`clair_head.safetensors`) is the upstream file, unchanged.
- `clairic_runtime/` is the upstream request pipeline with its PyTorch engine replaced:
- removed: `native.py`, `math.py`, `readout.py`, `normalization.py`, `kernels.py`, `gdn/`, `gdn_pool.py`,
`decoder_capture.py`, `decoder_graph.py`, `vision_cache.py`, `attention_metadata.py`,
`recurrent_metadata.py` (PyTorch, Triton and CUDA code);
- added: `mlx_engine.py` (MLX backbone loop, typed head, grammar-constrained decoding) and `processor.py`
(a torch-free processor);
- modified: `model.py` (typed head moved to `mlx_engine.py`; tensor code ported from PyTorch to NumPy),
`backend.py` (`media_payload` and `load`) and `__init__.py` (docstring). Each modified file says so
at its top. All other files in `clairic_runtime/` are unchanged.
- `clairic_mlx.py` replaces `clairic_model.py`; `server.py` and `convert.py` are new.
The MLX backbone uses mlx-vlm's `qwen3_5` model definition (https://github.com/Blaizzy/mlx-vlm, MIT License,
Copyright (c) Prince Canuma); the linear-attention layer loop in `mlx_engine.py` follows that implementation.