nativ-community/LFM3-1.7B-5bit

LFM3-1.7B · 5bit

Native MLX text generation.

Converted from lq-preview/LFM3-1.7B at c74a294117b6c4e0371af0d74be7d9cc0e90bfe9. Conversion implementation: 5d5ff5af600ec99ce01a833697a4aa1417844ce3 (recorded in conversion.json). Public inference code: Blaizzy/mlx-vlm.

Weights: affine 5-bit, group size 64. Multimodal towers/projectors remain BF16. Activations and KV caches are not quantized.

Usage

Install the pinned public implementation:

pip install 'mlx-vlm @ git+https://github.com/Blaizzy/mlx-vlm.git@3070dcc31948f48f9d8639b606af7bc019494fac'

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

model, processor = load("nativ-community/LFM3-1.7B-5bit", trust_remote_code=False)
prompt = apply_chat_template(processor, model.config, "What is 17 + 25?", enable_thinking=False)
print(generate(model, processor, prompt, max_tokens=128, temperature=0).text)

Validation

Strict native reload on Apple M3 Ultra, MLX 0.32.4; no remote Python code is executed. These are short capability smoke checks, not a general quality benchmark. Exact inputs, outputs, expectations and BF16 comparisons are in validation.json.

Check	Result
finite_logits	completed; expected result matched
arithmetic	17 + 25 = 42 42
capital	The capital of Australia is Canberra.

Fixture mismatches: none on these fixtures. The recorded results are retained even when a quantized answer differs.

The native helper defaults to thinking off; pass enable_thinking=True to use the checkpoint template's thinking behavior. Low-bit quantization can change reasoning and tool-use quality. These short checks do not establish long-context or broader benchmark parity.

The source revision, implementation commit, precision, and tensor audit are in conversion.json; SHA256SUMS covers the release files. The original checkpoint terms apply: see LICENSE and ORIGINAL_MODEL_CARD.md. No new weight license is assigned.