MiniCPM-V-4.6 - Q4NX for FastFlowLM (AMD Ryzen AI XDNA2)

MiniCPM-V-4.6 is converted to Q4NX for hardware-accelerated inference with FastFlowLM on AMD Ryzen AI NPUs.

What is Q4NX?

Q4NX is FastFlowLM's native packed-quantization format - a rearranged Q4_1 layout tuned for the NPU matrix engine's tile sizes and memory access patterns. It is not a GGUF file and it does not run on llama.cpp or Ollama; it is meant exclusively for the FastFlowLM engine on AMD Ryzen AI NPUs.

Requirements

FastFlowLM (flm CLI)
AMD Ryzen AI processor with XDNA2 (NPU2) - Strix Point / Ryzen AI 300 series or later
Linux with the XRT NPU stack installed
~8 GB of unified system memory (Q4NX weights + activations + KV cache)

Files

File	Purpose
model.q4nx	Quantized Q4NX text weights
config.json	FastFlowLM model configuration
tokenizer.json	Tokenizer
tokenizer_config.json	Special tokens and chat template
chat_template.jinja	Chat template (optional)
README.md
flm-add.py	Installer script - registers this model with FastFlowLM

Install and run

This repository works with flm-add, a small installer that copies the model into the FastFlowLM user directory and registers the tag minicpm4.6:0.8b. It never modifies the system FastFlowLM install.

pip install flm-add or uv tool install flm-add

uv tool install flm-add
flm-add Atomic-Germ/MiniCPM-V-4.6-NPU2 --family qwen3.5 --tag minicpm-4.6:0.8b
FLM_CONFIG_PATH="$HOME/.config/flm/model_list.json" FLM_XCLBIN_PATH="$HOME/.config/flm" flm run minicpm-4.6:0.8b

Model

Registry tag: minicpm4.6:0.8b
Engine family: qwen3.5
Kernel source: official qwen3.5:0.8b (Qwen3.5-0.8B-NPU2)
Context length: 262,144 tokens (from config)
Hidden size: 1024
Layers: 24
Intermediate size: 3584
Vocabulary: 248094
model.q4nx size: 1.03 GB
Base model: openbmb/MiniCPM-V-4.6-gguf
License: other

GhostWriter Influence Test (Arbitrary but repeatable benchmark)

Tested on an AMD Ryzen AI 340 Framework 13 laptop.

Metric	Value
Prompt Tokens	9,210
Completion Tokens	631
Total Tokens	9,841
Active KV Tokens	9,841
Max KV Token Capacity	32,768
KV Token Occupancy	30.03%
Load Duration	0.000000842 seconds
Prefill Duration (TTFT)	7.58 ms
Decoding Duration	21.44 ms
Prefill Speed	1214.89 tokens/sec
Decoding Speed	29.44 tokens/sec