Card: FastFlowLM 1.0.11 or later

23e3cec verified 1 day ago

2.58 kB

license: other
license_name: lfm1.0
license_link: https://huggingface.co/LiquidAI/LFM3-1.7B/blob/main/LICENSE
base_model:
  - LiquidAI/LFM3-1.7B
pipeline_tag: text-generation
library_name: fastflowlm
tags:
  - npu
  - amd
  - ryzen-ai
  - fastflowlm
  - lfm3
  - liquid

LFM3-1.7B-NPU2 (FastFlowLM)

LiquidAI/LFM3-1.7B packaged for the AMD Ryzen™ AI NPU (XDNA2) with FastFlowLM. The weights are the vendor's quantization-aware-distilled Q4_0 build (LFM3-1.7B-GGUF, QAD-Q4_0); the tied embedding / LM head is stored as Q4_1. The package holds the converted weights (model.q4nx), the tokenizer and the chat template; it runs only with FastFlowLM (version 1.0.11 or later).

flm run lfm3:1.7b
flm serve lfm3:1.7b        # OpenAI-compatible server

Model

24 layers: 13 gated short-convolution (width 7), 5 sliding-window attention (window 256), 6 full attention
hidden 2048, FFN 7680, 16 query / 4 KV heads, head dim 128, per-head sigmoid attention gate, sandwich RMSNorms, vocab 128000
ChatML prompt format; thinking is on by default and can be switched off; tool calls use the Qwen-style XML format
up to 32k tokens of context in FastFlowLM (the model supports 256k)
FastFlowLM: 1.0.11 or later
LiquidAI's recommended sampling is the default: temperature 0.1, top_k 50, repetition penalty 1.1

Performance (FastFlowLM on an AMD Ryzen™ AI MAX+ PRO 395, NPU only)

context	1k	2k	4k	8k	16k	32k
decode, tokens/s	41	40	39	36	31	24
prefill, tokens/s	1092	1543	1467	1397	1346	1223

Accuracy

The NPU computes in bf16, so its logits differ slightly from a PyTorch reference run on the same quantized weights.

test	PyTorch reference	FastFlowLM on NPU
WikiText-2 perplexity (6.1k tokens)	19.23	20.16
GSM8K, first 150 test questions (greedy, no thinking)	n/a	81.3%

Greedy output on the project's golden prompts is identical to the reference. Running the attention layers on the CPU (LFM3_DECODE=conv) raises GSM8K to 84.0% at a lower decoding speed.

License

This package contains a conversion of LiquidAI's LFM3-1.7B and is distributed under the LFM Open License v1.0 (see LICENSE). In short: free for research and for commercial use by organizations below the revenue threshold stated in the license; read it before deploying. FastFlowLM's own runtime is licensed separately by the FastFlowLM project.