# llama.cpp — K2-FAST2 branch

This repository publishes branch **`k2-fast2`**: MBZUAI-IFM's llama.cpp
`model/K2Horizon` support (base commit `42adf01`) plus the FAST2 sliding-window
attention override for `k2-horizon` GGUFs (51 lines in `src/models/k2-horizon.cpp`).

- Activates `k2-horizon.attention.sliding_window` GGUF metadata (e.g. the
  FAST2-SWA8K model) or `--override-kv k2-horizon.attention.sliding_window=int:W`
- Measured with K2-Horizon-MoVA-36B-A4B APEX-Mini @61K ctx on RTX 2080 Ti:
  decode 16.5 -> 50.7 tok/s (3.1x), prefill 692 -> 1482 tok/s (2.1x),
  ppl inside window identical to full attention
- Stock behavior is unchanged when no window key is present

Build:
```bash
cmake -B build -DGGML_CUDA=ON -DGGML_CUDA_FA_ALL_QUANTS=ON -DLLAMA_CURL=OFF \
      -DCMAKE_CUDA_ARCHITECTURES=<your-sm> -DCMAKE_BUILD_TYPE=Release
cmake --build build --target llama-server -j
```

The same change as a standalone patch: `k2-fast2-swa.patch` in the FAST2 model repo.