Agnostic

Quantized weights, built for the metal.

Website: lisa-ml.com

Agnostic publishes MLX-format, affine-quantized builds of recent language and multimodal models, shaped for fast local inference on Apple Silicon and packed for lisa — a pure-Rust Metal inference engine with no Python, no MLX runtime, and no tensor framework at load time. Built from scratch on the Metal C API, with an OpenAI-compatible server.

Every checkpoint here is a faithful re-pack of the public original, not a fine-tune: same weights, same tokenizer, same chat template, reduced precision. Quantization is deliberately boring — one bf16 scale/bias pair per group, packed u32 payloads, group 32/64 as required by the kernels that consume them.

Models

What's in the box

• Affine quantization (mx.quantize): packed u32 weights plus one bf16 scale and bias per group along the input dimension.
• Group size is fixed by the checkpoint and must be honored by the loader. Group 32 is required by lisa's fused Flash-Next decode kernels; other builds use group 64 unless noted.
• Only linear layers are quantized. Embeddings, norms, convolutions and the vision patch embedder keep original precision.
• MTP heads ride along where the base model has one, for lossless speculative decoding: mtp/ holds the multi-token-prediction sidecar.
• Transformers are reworked to the engine's expected layout at publish time (prefix renames, MoE switch_mlp packing, conv1d transposes, RMSNorm fold) — loading is the contract test, not a separate transform pass.

Usage

With lisa, --model takes a repo id directly:

lisa serve --model agnosticeng/Qwen3.8-Flash-Next-4bit --addr 127.0.0.1:8000

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"Hello"}],"max_tokens":64}'

With MLX / mlx-vlm these are ordinary quantized checkpoints:

from mlx_vlm import load, generate
model, processor = load("agnosticeng/MiMo-V2.6-Distill-Qwen-9B-4bit")

Greedy decoding is bit-exact against the reference engine's goldens (256/256, serial and MTP depths 1–6 on the public long-copy gate).

Conversion

Built from the public originals by streaming the checkpoint shard by shard (download → quantize → delete), so no host ever holds the full-precision model twice. Scripts and the exact transforms are documented on each model card.

Links

• Website — lisa-ml.com
• Engine — lisa

Licensing

Weights carry the base model's license — see each card and its LICENSE file. We claim no rights over the upstream models; please credit and comply with the original terms.