| --- |
| license: other |
| license_name: lfm1.0 |
| license_link: https://huggingface.co/LiquidAI/LFM3-1.7B/blob/main/LICENSE |
| base_model: |
| - LiquidAI/LFM3-1.7B |
| pipeline_tag: text-generation |
| library_name: fastflowlm |
| tags: |
| - npu |
| - amd |
| - ryzen-ai |
| - fastflowlm |
| - lfm3 |
| - liquid |
| --- |
| |
| # LFM3-1.7B-NPU2 (FastFlowLM) |
|
|
| [LiquidAI/LFM3-1.7B](https://huggingface.co/LiquidAI/LFM3-1.7B) packaged for the **AMD Ryzen™ AI NPU** (XDNA2) with |
| [FastFlowLM](https://github.com/FastFlowLM/FastFlowLM). The weights are the vendor's quantization-aware-distilled **Q4_0** build |
| (`LFM3-1.7B-GGUF`, `QAD-Q4_0`); the tied embedding / LM head is stored as Q4_1. The package holds the converted weights |
| (`model.q4nx`), the tokenizer and the chat template; it runs only with FastFlowLM (version 1.0.11 or later). |
| |
| ```shell |
| flm run lfm3:1.7b |
| flm serve lfm3:1.7b # OpenAI-compatible server |
| ``` |
| |
| ## Model |
| |
| - 24 layers: 13 gated short-convolution (width 7), 5 sliding-window attention (window 256), 6 full attention |
| - hidden 2048, FFN 7680, 16 query / 4 KV heads, head dim 128, per-head sigmoid attention gate, sandwich RMSNorms, vocab 128000 |
| - ChatML prompt format; **thinking is on by default** and can be switched off; tool calls use the Qwen-style XML format |
| - up to 32k tokens of context in FastFlowLM (the model supports 256k) |
| - **FastFlowLM:** 1.0.11 or later |
| - LiquidAI's recommended sampling is the default: temperature 0.1, top_k 50, repetition penalty 1.1 |
| |
| ## Performance (FastFlowLM on an AMD Ryzen™ AI MAX+ PRO 395, NPU only) |
| |
| | context | 1k | 2k | 4k | 8k | 16k | 32k | |
| |---|---:|---:|---:|---:|---:|---:| |
| | decode, tokens/s | 41 | 40 | 39 | 36 | 31 | 24 | |
| | prefill, tokens/s | 1092 | 1543 | 1467 | 1397 | 1346 | 1223 | |
| |
| ## Accuracy |
| |
| The NPU computes in bf16, so its logits differ slightly from a PyTorch reference run on the same quantized weights. |
| |
| | test | PyTorch reference | FastFlowLM on NPU | |
| |---|---:|---:| |
| | WikiText-2 perplexity (6.1k tokens) | 19.23 | 20.16 | |
| | GSM8K, first 150 test questions (greedy, no thinking) | n/a | 81.3% | |
| |
| Greedy output on the project's golden prompts is identical to the reference. Running the attention layers on the CPU |
| (`LFM3_DECODE=conv`) raises GSM8K to 84.0% at a lower decoding speed. |
| |
| ## License |
| |
| This package contains a conversion of LiquidAI's LFM3-1.7B and is distributed under the **LFM Open License v1.0** (see `LICENSE`). |
| In short: free for research and for commercial use by organizations below the revenue threshold stated in the license; read it |
| before deploying. FastFlowLM's own runtime is licensed separately by the FastFlowLM project. |
| |
