Instructions to use balllvin/Flint-abliterated-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use balllvin/Flint-abliterated-NVFP4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="balllvin/Flint-abliterated-NVFP4")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("balllvin/Flint-abliterated-NVFP4")
model = AutoModelForCausalLM.from_pretrained("balllvin/Flint-abliterated-NVFP4", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use balllvin/Flint-abliterated-NVFP4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "balllvin/Flint-abliterated-NVFP4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "balllvin/Flint-abliterated-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/balllvin/Flint-abliterated-NVFP4

SGLang

How to use balllvin/Flint-abliterated-NVFP4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "balllvin/Flint-abliterated-NVFP4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "balllvin/Flint-abliterated-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "balllvin/Flint-abliterated-NVFP4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "balllvin/Flint-abliterated-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use balllvin/Flint-abliterated-NVFP4 with Docker Model Runner:
```
docker model run hf.co/balllvin/Flint-abliterated-NVFP4
```

You need to agree to share your Hugging Face information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Terms of access and use

By requesting access to, downloading or using this model, including its weights, code and anything it generates, you agree to these terms.

Your responsibility. You alone are responsible for how you use this model and for everything it produces for you. You accept full liability for that use and for those outputs, including any decision you make, action you take or content you share because of them.
No warranty. The model is provided "as is" and "as available", without warranties of any kind, express or implied, including merchantability, fitness for a particular purpose, accuracy, title and non-infringement.
No liability. To the fullest extent the law allows, the authors and publishers of this repository are not liable for any damages, claims, losses or costs, whether direct, indirect, incidental, special, consequential or punitive, that arise from or relate to your access to or use of the model or its outputs, however caused.
Indemnity. You will indemnify, defend and hold harmless the authors and publishers against any claim, damage, loss, liability or cost, including reasonable legal fees, that arises from your use of the model or its outputs or from your breach of these terms.
Laws and licences. You will follow every law that applies to you, the licence of this repository and the licence of every model or dataset it is built on. These terms add to those licences and do not take away any right they grant you.
Agreement. Accessing or downloading this repository means you agree to these terms. If you do not agree, do not access, download or use it. If any part of these terms cannot be enforced, the rest still applies.

Flint abliterated, NVFP4 safetensors

This repo holds the NVFP4 safetensors build of Flint abliterated: Recursive's Flint (a mixture-of-experts model, 39.8B parameters with about 2.5B active per token) with its refusal direction removed, so it does what you ask. It refuses 16 of 99 harmful requests (the official build refuses 82) and 3 of 100 benign ones. Its Propaganda Eval fact score is 1.75 out of 5 (higher is better); the official build scores 1.64. It is for research. Improved versions are coming.

Which file do I use?

Your hardware	Build	Repo
NVIDIA GPU with llama.cpp	NVFP4 GGUF, one 22.9 GB file	balllvin/Flint-abliterated-NVFP4-GGUF
NVIDIA GPU, your own code (this one)	NVFP4 safetensors in ModelOpt's `W4A16_NVFP4` layout	balllvin/Flint-abliterated-NVFP4
Apple Silicon Mac	MLX layout, run with `flint_mlx.py` from the repo	balllvin/Flint-abliterated-NVFP4-MLX

All three hold the same 4-bit weights.

Set it up with a coding agent

Paste this into a coding agent such as Claude Code or OpenCode, with your app in the brackets.

Download Flint abliterated and set it up for [your app, for example OpenCode]. Do every step yourself, except the account steps, which are mine.
1. Check whether the Hugging Face CLI (hf) is installed. If it isn't, tell me and show me how to install it. Don't install it yourself.
2. Check this computer. Mac with Apple Silicon and 48 GB or more: Mac build. NVIDIA GPU with 32 GB or more: GGUF build. Anything else: tell me it won't run here and suggest workarounds.
3. Walk me through the account steps: make a Hugging Face account if I don't have one, accept the terms on the model page, and make a token. Then run hf auth login for me, or tell me how.
4. Download only the build for this computer and check every file. Mac: https://huggingface.co/balllvin/Flint-abliterated-NVFP4-MLX NVIDIA: https://huggingface.co/balllvin/Flint-abliterated-NVFP4-GGUF
5. Get the code. Mac: uv pip install mlx==0.32.4, and flint_mlx.py from the model repo. NVIDIA: llama.cpp b11853 or later, CUDA build.
6. Pick the context length from the memory table on the model card.
7. Ask it one question, check the answer makes sense, and tell me the tokens per second.
8. NVIDIA: start llama-server and connect it to my app, with its output limit raised to the context length. No app: show me one curl call. Mac: flint_mlx.py isn't a server, so stop after the test.
9. Tell me which build you chose, the server address if there is one, and one command to run it again.

Results

Refusals on JailbreakBench prompts, run with llama.cpp b11853 on one RTX PRO 6000: thinking on, temperature 1.0, top-p 0.95, top-k 20, no token limit in the 64K context, one sampled answer per prompt, refusals judged on the final answer with phrase lists. Every answer finished. 95% intervals in brackets.

Build	Harmful refused, of 99 (higher refuses more)	Benign refused, of 100 (lower is better)
Official	82 (74% to 89%)	8 (4% to 15%)
This model	16 (10% to 25%)	3 (1% to 8%)

The Propaganda Eval asks 803 questions on topics where state censorship and propaganda are common (Nous Research's set), judged by xAI's Grok 4.7. Both builds answered with the same server and settings.

Build	Fact score, 0 to 5 (higher is better)	Any propaganda (lower is better)	Refused, of 803
Official	1.64 (1.51 to 1.78)	2.1%	100
This model	1.75 (1.61 to 1.88)	2.2%	31

The answers are public: official, this model.

Memory

GB means 10^9 bytes; K means 1,024 tokens. The weights take 22.9 GB (measured). The KV cache, computed from config.json in bf16, takes 28,672 bytes (about 28.7 KB) per token for the 14 full-attention layers, plus 36.8 MB per request for the 35 sliding-window layers: 7.6 GB for one request at the full 256K.

Machine	Usable	Left for KV	Longest context, 1 request	4 requests	8 requests
Mac with 64 GB	55.6 GB	28.5 GB	256K	241K	119K
GPU with 48 GB	46.4 GB	21.4 GB	256K	180K	89K
GPU with 96 GB	92.8 GB	67.8 GB	256K	256K	256K

Usable is the GPU working set Metal recommends on a 64 GB M5 Pro, or 90% of a GPU's memory. Left for KV subtracts the weights and runtime room (4.3 GB on the Mac, 2.1 GB on a GPU).

Speed

With 32 requests at once on one RTX PRO 6000 (llama.cpp b11853, GGUF), each request decoded at 27 tokens/s: the median over the 697 benchmark replies of 256 tokens or more.

Run it

Before the first download, accept the terms at the top of this page, then log in with hf auth login. Thinking is always on. Use the base model's sampling: temperature 1.0, top-p 0.95, top-k 20.

# NVIDIA GPU, llama.cpp
hf download balllvin/Flint-abliterated-NVFP4-GGUF flint-ablit-nvfp4.gguf --local-dir flint-ablit
llama-server -m flint-ablit/flint-ablit-nvfp4.gguf --alias flint -ngl all -c 262144 -np 1   -fa on -ctk f16 -ctv f16 --jinja --temp 1 --top-p 0.95 --top-k 20 --min-p 0 --port 8080

# Mac, MLX
hf download balllvin/Flint-abliterated-NVFP4-MLX --local-dir flint-ablit-mlx
uv run --with mlx==0.32.4 --with numpy --with transformers python flint-ablit-mlx/flint_mlx.py   --model flint-ablit-mlx --dtype bf16 --tokens 512 --prompt "Explain how vaccines train the immune system."

flint_mlx.py is a small greedy runner, not a server. The model writes its reasoning in <think>...</think>, then the answer; llama-server returns them in reasoning_content and content.

How it was made

Direction: last-token residuals for harmful and harmless prompts gave one refusal direction per candidate layer (difference of the means), using OBLITERATUS's method. Extra prompts from HarmBench and AdvBench (both MIT) widened the categories; none overlap the test prompts.
Choice: layers 18 to 26 were tried at two strengths, keeping drift on harmless answers under a cap. The best two were written into the weights and re-asked 123 validation prompts; layer 22 at strength 1.0 won. The test prompts above were run once, after the choice.
Edit: the direction is projected out of every matrix that writes the residual stream, then re-quantized to NVFP4 with the official scales. Against the bf16 model it matches the official build: 3.094 nats per token on WikiText (official 3.094) and 77.7% next-token agreement (official 77.8%).

Limits

It answers most harmful requests (83 of 99 here). Use it for research, with a person checking the output.
The tests cover refusals and propaganda, one sampled run each; not coding, agents or long context.

Credits

Recursive for Flint and its NVFP4 GGUF; OBLITERATUS (AGPL-3.0) for the method; HarmBench and AdvBench for prompts; Nous Research for the Censorship and Propaganda Eval (MIT); JailbreakBench; xAI's Grok 4.7 as judge; llama.cpp and MLX.

Licence

The weights are a modified version of Recursive's Flint under Apache-2.0: LICENSE holds Flint's licence file and NOTICE.md the origin and every change. flint_mlx.py is MIT.

This is an independent community build, unaffiliated with Recursive.

Terms of use

Access to this repository requires agreeing to the terms in the access form on this page. In short: you alone are responsible and liable for your use of this model and for everything that comes from it, it is provided as is with no warranty, and its authors are not liable for any result of that use. Full text: TERMS.md.

To download after you agree, log in with the Hugging Face CLI (hf auth login) or set HF_TOKEN.

Downloads last month: -

Safetensors

Model size

40B params

Tensor type

F32

BF16

Model tree for balllvin/Flint-abliterated-NVFP4

Base model

RecursiveSI/flint

Quantized

(112)

this model