Add README.md

README.md

@@ -0,0 +1,66 @@
+---
+license: other
+license_name: lfm1.0
+license_link: LICENSE
+base_model: LiquidAI/LFM3-1.7B-Base
+base_model_relation: quantized
+pipeline_tag: text-generation
+library_name: gguf
+tags: [gguf, rocmfpx, amd, rocm, vulkan, strix-halo, quantized, liquid, lfm3, edge]
+---
+
+# LFM3-1.7B-Base, ROCmFPX GGUF
+
+[LiquidAI/LFM3-1.7B-Base](https://huggingface.co/LiquidAI/LFM3-1.7B-Base), the pretrained base model of LFM3-1.7B, without chat tuning. It is meant for fine-tuning and plain text completion, not for chat, quantized for the
+[ROCmFPX](https://github.com/HarshaVardhan3002/ROCmFPX) llama.cpp fork (low-bit weight types for AMD GPUs, Vulkan and
+HIP/ROCm).
+
+> The recommended 6-bit file needs [HarshaVardhan3002/ROCmFPX](https://github.com/HarshaVardhan3002/ROCmFPX). Stock
+> llama.cpp, LM Studio, Ollama and other ROCmFPX builds may not load it. The Q8_0 and Q4_K_M files use stock types and
+> load in any llama.cpp build with LFM3 support. **These files were checked on the CPU and have not been tested on AMD
+> hardware yet.**
+>
+> Not an official release of Liquid AI. All credit for the model goes to its authors.
+
+## Files
+
+| File | Size | Weights | KL divergence vs BF16 (lower is better) | Same top token as BF16 |
+|---|---:|---|---:|---:|
+| `LFM3-1.7B-Base-Q6_K_ROCMFPX.gguf` **(recommended)** | 1.43 GB | ROCmFPX 6-bit (`Q6_0_ROCMFPX` with tensor scales) | 0.0031 | 96.9% |
+| `LFM3-1.7B-Base-Q8_0.gguf` | 1.85 GB | stock llama.cpp Q8_0 | 0.0006 | 98.7% |
+| `LFM3-1.7B-Base-Q4_K_M.gguf` | 1.10 GB | stock llama.cpp Q4_K_M | 0.0267 | 92.0% |
+| stock Q6_K, for comparison (not uploaded) | 1.43 GB | stock llama.cpp Q6_K | 0.0039 | 96.6% |
+
+- **Measured on:** 40 chunks of 512 tokens of WikiText-2 (test split), against the BF16 GGUF, CPU.
+- **The ROCmFPX 6-bit file beats stock Q6_K** at the same size. Its file name says `Q6_K_ROCMFPX` so that Hugging Face
+  lists it as a 6-bit quant; the weights are the ROCmFPX `Q6_0_ROCMFPX` type, not a stock `Q6_K`.
+- **No ROCmFPX 4-bit file:** our 4-bit types came out clearly worse than stock Q4_K_M on this model (KL divergence 1.5 to
+  1.7 times higher), so we ship Q4_K_M instead.
+- Every file passed a short greedy answer test before upload.
+
+## How these were made
+
+1. BF16 GGUF converted with the fork's `convert_hf_to_gguf.py`.
+2. Importance matrix from the BF16 file over a general calibration text (`calibration_datav3`), 512-token chunks.
+3. `llama-quantize --imatrix` from the BF16 file for every file, stock ones included. The 6-bit file was made with `LLAMA_ROCMFPX_TENSOR_SCALE=1`, which
+   stores a power-of-two scale per weight tensor (the runtime applies it; no setting is needed to run the file), with
+   `--token-embedding-type q6_K`.
+
+## Run
+
+```bash
+git clone https://github.com/HarshaVardhan3002/ROCmFPX && cd ROCmFPX
+cmake -B build -DGGML_VULKAN=ON          # or -DGGML_HIP=ON -DAMDGPU_TARGETS=gfx1151 (gfx1100 for RX 7900)
+cmake --build build -j
+build/bin/llama-completion -m LFM3-1.7B-Base-Q6_K_ROCMFPX.gguf -ngl 99 -p "The capital of France is" -n 64
+```
+
+This is a base model: give it text to continue, or fine-tune it. For chat use LFM3-1.7B instead.
+
+LFM3 support in the fork comes from llama.cpp pull request #31252. LFM3-Omni and LFM3-Audio need further upstream
+llama.cpp work and will follow later.
+
+## License
+
+[LFM Open License v1.0](LICENSE), same as the base model. Commercial use is allowed for companies below 10 million US
+dollars of annual revenue; above that it is not covered by the license. Read `LICENSE` before you use these files.