Instructions to use balllvin/Flint-NVFP4-MLX with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use balllvin/Flint-NVFP4-MLX with MLX:

# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm

# Generate text with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("balllvin/Flint-NVFP4-MLX")

prompt = "Write a story about Einstein"
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True
)

text = generate(model, tokenizer, prompt=prompt, verbose=True)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use balllvin/Flint-NVFP4-MLX with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "balllvin/Flint-NVFP4-MLX"

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "balllvin/Flint-NVFP4-MLX"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

MLX LM

How to use balllvin/Flint-NVFP4-MLX with MLX LM:

Generate or start a chat session

# Install MLX LM
uv tool install mlx-lm
# Interactive chat REPL
mlx_lm.chat --model "balllvin/Flint-NVFP4-MLX"

Run an OpenAI-compatible server

# Install MLX LM
uv tool install mlx-lm
# Start the server
mlx_lm.server --model "balllvin/Flint-NVFP4-MLX"
# Calling the OpenAI-compatible server with curl
curl -X POST "http://localhost:8000/v1/chat/completions" \
   -H "Content-Type: application/json" \
   --data '{
     "model": "balllvin/Flint-NVFP4-MLX",
     "messages": [
       {"role": "user", "content": "Hello"}
     ]
   }'

Hermes Agent

How to use balllvin/Flint-NVFP4-MLX with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "balllvin/Flint-NVFP4-MLX"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default balllvin/Flint-NVFP4-MLX

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use balllvin/Flint-NVFP4-MLX with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "balllvin/Flint-NVFP4-MLX"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "balllvin/Flint-NVFP4-MLX" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

You need to agree to share your Hugging Face information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Terms of access and use

By requesting access to, downloading or using this model, including its weights, code and anything it generates, you agree to these terms.

Your responsibility. You alone are responsible for how you use this model and for everything it produces for you. You accept full liability for that use and for those outputs, including any decision you make, action you take or content you share because of them.
No warranty. The model is provided "as is" and "as available", without warranties of any kind, express or implied, including merchantability, fitness for a particular purpose, accuracy, title and non-infringement.
No liability. To the fullest extent the law allows, the authors and publishers of this repository are not liable for any damages, claims, losses or costs, whether direct, indirect, incidental, special, consequential or punitive, that arise from or relate to your access to or use of the model or its outputs, however caused.
Indemnity. You will indemnify, defend and hold harmless the authors and publishers against any claim, damage, loss, liability or cost, including reasonable legal fees, that arises from your use of the model or its outputs or from your breach of these terms.
Laws and licences. You will follow every law that applies to you, the licence of this repository and the licence of every model or dataset it is built on. These terms add to those licences and do not take away any right they grant you.
Agreement. Accessing or downloading this repository means you agree to these terms. If you do not agree, do not access, download or use it. If any part of these terms cannot be enforced, the rest still applies.

Flint, MLX for Apple Silicon

This repo holds the NVFP4-MLX build of Flint by Recursive. It keeps the 4-bit codes and block scales of RecursiveSI/flint-nvfp4-gguf byte for byte, laid out for MLX; the tensor scales stay fp32. The weights take 22.9 GB.

Which file do I use?

Your hardware	Build	Repo
Apple Silicon Mac (this one)	NVFP4 codes and scales in MLX's layout	balllvin/Flint-NVFP4-MLX
NVIDIA GPU (CUDA)	NVFP4: 4-bit weights for NVIDIA GPUs, by Recursive	RecursiveSI/flint-nvfp4-gguf
Any, full precision	The original weights	RecursiveSI/flint

Set it up with a coding agent

Paste this into a coding agent such as Claude Code or OpenCode. Edit the first line first.

Use it in: [Insert application you will use it in, for example OpenCode]

Set up Flint (NVFP4-MLX build) on this computer. Do every step yourself.
1. Check the tools: Python 3.12, uv or pip, git, the Hugging Face CLI (`hf`). If the HF CLI is missing, tell me and show me how to install it; do not install it yourself.
2. Check the hardware. This build needs an Apple Silicon Mac with at least 32 GB of memory. Anything else: tell me it will not run here, and suggest the original model's own runtimes.
3. Ask me to open https://huggingface.co/balllvin/Flint-NVFP4-MLX and accept the terms at the top of the page, then run `hf auth login` (or set HF_TOKEN). Download the repo with the HF CLI and check every file against the repo's file list.
4. Install the runtime in a fresh virtual environment: `uv pip install mlx==0.32.4 numpy transformers`.
5. Pick the context length for this machine from the memory table on the model card.
6. Run `flint_mlx.py` from the repo on one short question and check the answer makes sense. Report the speed. If anything fails, fix it and try again.
7. `flint_mlx.py` is not a server, so stop after the test and say so.
8. Report what you set up, the server address if there is one, and one command to start it again.

Quick check

Every 4-bit code and scale byte in this repo equals the official NVFP4 file (a byte check of every tensor). We did not generate with this build on our Mac: the build machine keeps model jobs under 12 GB while it runs other work, and this build needs more.

Memory

GB means 10^9 bytes; K means 1,024 tokens. The weights take 22.9 GB (22,857,768,320 bytes, measured on the files in this repo).

The model has 49 layers: 14 full attention and 35 sliding window (513 tokens). The KV cache, computed from config.json in bf16, takes 28,672 bytes per token on the full-attention layers (4 KV heads of 128) plus 36.8 MB per request on the sliding-window layers.

Machine	Usable	Left for KV	Longest context, 1 request	4 requests	8 requests
Mac with 64 GB	51.8 GB	24.9 GB	256K	211K	104K

Usable is the GPU working set Metal recommends on a 64 GB M5 Pro; Left for KV subtracts the weights and 4 GB of runtime room. The table is computed, not measured; contexts stop at the model's maximum of 256K.

Run it

Before the first download, accept the terms at the top of this page, then log in with the Hugging Face CLI (hf auth login) or set HF_TOKEN to a read token.

Thinking is always on. flint_mlx.py is a small greedy runner, not a server; it applies each tensor scale after its 4-bit matmul.

hf download balllvin/Flint-NVFP4-MLX --local-dir flint-mlx
uv run --with mlx==0.32.4 --with numpy --with transformers python flint-mlx/flint_mlx.py --model flint-mlx --dtype bf16 --tokens 512 --prompt "Explain how vaccines train the immune system."

How it was made

Source: RecursiveSI/flint-nvfp4-gguf (revision 6ec43e19), the NVFP4 build by Recursive, an NVFP4 GGUF file.
Our Flint converter (brainspa_flint.convert to-modelopt, then brainspa_nvfp4mlx flint-convert) moved every code and scale byte into MLX's layout; flint_mlx.py applies each tensor scale after its matmul.
Check: every code and scale byte of every tensor equals the source file.
In the source, the linears that NVIDIA runtimes fuse (q, k and v; gate and up) share one tensor scale per group, so the fused and the separate layouts decode the same values.
MLX 0.32.4 on an M5 Pro Mac. No training, no change to what the model learned.

Credits

Recursive, for Flint.
llama.cpp's GGUF format, which the official NVFP4 file uses.
MLX and mlx-lm, which run this build.

Licence

The weights are under the Apache License 2.0, the licence of the original model; this repo carries its licence files unchanged and NOTICE.md with the origin of every file and what we changed. This is an independent community build, unaffiliated with Recursive.

Terms of use

Access to this repository requires agreeing to the terms in the access form on this page. In short: you alone are responsible and liable for your use of this model and for everything that comes from it, it is provided as is with no warranty, and its authors are not liable for any result of that use. Full text: TERMS.md.

To download after you agree, log in with the Hugging Face CLI (hf auth login) or set HF_TOKEN.

Downloads last month: -

Safetensors

Model size

40B params

Tensor type

F32

U32

BF16

MLX

Hardware compatibility

4-bit

Model tree for balllvin/Flint-NVFP4-MLX

Base model

RecursiveSI/flint

Quantized

(112)

this model