---
license: other
license_name: lfm1.0
license_link: LICENSE
base_model: LiquidAI/LFM3-1.7B-Base
base_model_relation: quantized
pipeline_tag: text-generation
library_name: gguf
tags: [gguf, rocmfpx, amd, rocm, vulkan, strix-halo, quantized, liquid, lfm3, edge]
---

# LFM3-1.7B-Base, ROCmFPX GGUF

[LiquidAI/LFM3-1.7B-Base](https://huggingface.co/LiquidAI/LFM3-1.7B-Base), the pretrained base model of LFM3-1.7B, without chat tuning. It is meant for fine-tuning and plain text completion, not for chat, quantized for the
[ROCmFPX](https://github.com/HarshaVardhan3002/ROCmFPX) llama.cpp fork (low-bit weight types for AMD GPUs, Vulkan and
HIP/ROCm).

> The recommended 6-bit file needs [HarshaVardhan3002/ROCmFPX](https://github.com/HarshaVardhan3002/ROCmFPX). Stock
> llama.cpp, LM Studio, Ollama and other ROCmFPX builds may not load it. The Q8_0 and Q4_K_M files use stock types and
> load in any llama.cpp build with LFM3 support. **These files were checked on the CPU and have not been tested on AMD
> hardware yet.**
>
> Not an official release of Liquid AI. All credit for the model goes to its authors.

## Files

| File | Size | Weights | KL divergence vs BF16 (lower is better) | Same top token as BF16 |
|---|---:|---|---:|---:|
| `LFM3-1.7B-Base-Q6_K_ROCMFPX.gguf` **(recommended)** | 1.43 GB | ROCmFPX 6-bit (`Q6_0_ROCMFPX` with tensor scales) | 0.0031 | 96.9% |
| `LFM3-1.7B-Base-Q8_0.gguf` | 1.85 GB | stock llama.cpp Q8_0 | 0.0006 | 98.7% |
| `LFM3-1.7B-Base-Q4_K_M.gguf` | 1.10 GB | stock llama.cpp Q4_K_M | 0.0267 | 92.0% |
| stock Q6_K, for comparison (not uploaded) | 1.43 GB | stock llama.cpp Q6_K | 0.0039 | 96.6% |

- **Measured on:** 40 chunks of 512 tokens of WikiText-2 (test split), against the BF16 GGUF, CPU.
- **The ROCmFPX 6-bit file beats stock Q6_K** at the same size. Its file name says `Q6_K_ROCMFPX` so that Hugging Face
  lists it as a 6-bit quant; the weights are the ROCmFPX `Q6_0_ROCMFPX` type, not a stock `Q6_K`.
- **No ROCmFPX 4-bit file:** our 4-bit types came out clearly worse than stock Q4_K_M on this model (KL divergence 1.5 to
  1.7 times higher), so we ship Q4_K_M instead.
- Every file passed a short greedy answer test before upload.

## How these were made

1. BF16 GGUF converted with the fork's `convert_hf_to_gguf.py`.
2. Importance matrix from the BF16 file over a general calibration text (`calibration_datav3`), 512-token chunks.
3. `llama-quantize --imatrix` from the BF16 file for every file, stock ones included. The 6-bit file was made with `LLAMA_ROCMFPX_TENSOR_SCALE=1`, which
   stores a power-of-two scale per weight tensor (the runtime applies it; no setting is needed to run the file), with
   `--token-embedding-type q6_K`.

## Run

```bash
git clone https://github.com/HarshaVardhan3002/ROCmFPX && cd ROCmFPX
cmake -B build -DGGML_VULKAN=ON          # or -DGGML_HIP=ON -DAMDGPU_TARGETS=gfx1151 (gfx1100 for RX 7900)
cmake --build build -j
build/bin/llama-completion -m LFM3-1.7B-Base-Q6_K_ROCMFPX.gguf -ngl 99 -p "The capital of France is" -n 64
```

This is a base model: give it text to continue, or fine-tune it. For chat use LFM3-1.7B instead.

LFM3 support in the fork comes from llama.cpp pull request #31252. LFM3-Omni and LFM3-Audio need further upstream
llama.cpp work and will follow later.

## License

[LFM Open License v1.0](LICENSE), same as the base model. Commercial use is allowed for companies below 10 million US
dollars of annual revenue; above that it is not covered by the license. Read `LICENSE` before you use these files.