| # llama.cpp — K2-FAST2 branch | |
| This repository publishes branch **`k2-fast2`**: MBZUAI-IFM's llama.cpp | |
| `model/K2Horizon` support (base commit `42adf01`) plus the FAST2 sliding-window | |
| attention override for `k2-horizon` GGUFs (51 lines in `src/models/k2-horizon.cpp`). | |
| - Activates `k2-horizon.attention.sliding_window` GGUF metadata (e.g. the | |
| FAST2-SWA8K model) or `--override-kv k2-horizon.attention.sliding_window=int:W` | |
| - Measured with K2-Horizon-MoVA-36B-A4B APEX-Mini @61K ctx on RTX 2080 Ti: | |
| decode 16.5 -> 50.7 tok/s (3.1x), prefill 692 -> 1482 tok/s (2.1x), | |
| ppl inside window identical to full attention | |
| - Stock behavior is unchanged when no window key is present | |
| Build: | |
| ```bash | |
| cmake -B build -DGGML_CUDA=ON -DGGML_CUDA_FA_ALL_QUANTS=ON -DLLAMA_CURL=OFF \ | |
| -DCMAKE_CUDA_ARCHITECTURES=<your-sm> -DCMAKE_BUILD_TYPE=Release | |
| cmake --build build --target llama-server -j | |
| ``` | |
| The same change as a standalone patch: `k2-fast2-swa.patch` in the FAST2 model repo. | |