klcharizardqwq/flint-nvfp4-mlx

Flint NVFP4 (MLX)

NVFP4-quantized MLX checkpoint of RecursiveSI/flint (revision 2a247be), Recursive's 39.8B-total / ~2.5B-active coding-agent MoE. The checkpoint is 25.2 GB instead of 79.6 GB BF16 and runs on Apple Silicon with MLX.

What is quantized

Same module selection as FutureMLS-Lab/flint-NVFP4:

• The 12,032 routed experts of MoE layers 2–48 (gate_proj, up_proj, down_proj) are NVFP4: FP4 E2M1 weights in 16-element blocks, FP8 E4M3 block scales, FP32 per-expert global scale. Weights are packed in MLX layout (uint32 packed weights, uint8 scales, float32 global_scale) under mlp.switch_mlp.{gate,up,down}_proj.
• Every other tensor is the unchanged BF16 source: attention and its learned sinks, routers, shared experts, dense layers 0–1, norms, embeddings and lm_head.

Weights were quantized with round-to-nearest (MLX mx.quantize, mode nvfp4), not GPTQ. The reference checkpoint used Hessian-based GPTQ with calibration; this checkpoint is the straightforward RTN equivalent and will show slightly higher error on the same modules.

Layout

MLX-lm layout with a bundled architecture file (flint.py, adapted from the Apache-2.0 recursive-oss/rec engine). Expert weights are stacked per layer as model.layers.N.mlp.switch_mlp.{gate,up,down}_proj.{weight,scales,global_scale} so they load into QuantizedSwitchLinear directly.

"quantization": {"group_size": 16, "bits": 4, "mode": "nvfp4"},
"model_file": "flint.py"

Use with mlx-lm

from mlx_lm import load, generate

model, tokenizer = load(
    "klcharizardqwq/flint-nvfp4-mlx",
    tokenizer_config={"trust_remote_code": True},
    trust_remote_code=True,
)
messages = [{"role": "user", "content": "Write a Python function that merges overlapping intervals."}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
print(generate(model, tokenizer, prompt=prompt, max_tokens=1024))

trust_remote_code=True is required so mlx-lm picks up flint.py (the flint model type is not in mlx-lm's built-in registry). Generation must use the checkpoint's chat template; Flint emits reasoning in <think> blocks before the answer.

Reference

	BF16 source	This checkpoint	FutureMLS-Lab/flint-NVFP4
Size	79.6 GB	25.2 GB	23.5 GB
Quantized modules	—	routed experts L2–48	routed experts L2–48
Method	—	RTN NVFP4	GPTQ NVFP4
Runtime	SGLang	MLX (Apple Silicon)	SGLang / ModelOpt (Blackwell)

The BF16 source weights were also the calibration source for the GPTQ reference; both quantize the same modules.

License

Apache License 2.0, inherited from the source checkpoint. flint.py is adapted from recursive-oss/rec (Apache-2.0).