Upload README.md with huggingface_hub

README.md

@@ -1,94 +1,57 @@
----
-language:
-- en
-tags:
-- llama.cpp
-- llama-cpp
-- cuda
-- nvidia
-- k2horizon
-- gguf
-- inference
-license: mit
----
+# SageAttention 2.2.0 — Pre-built Wheel
 
-# K2Horizon llama.cpp CUDA Binary
+Pre-compiled SageAttention 2.2.0 wheel for **Linux x86_64** with CUDA 13 and C++20.
 
-Prebuilt Linux x86_64 CUDA binaries for the model/K2Horizon branch of MBZUAI-IFM llama.cpp.
+## Compatibility
 
-This package is already built. You do not need to compile llama.cpp again.
+| Component | Version |
+|-----------|---------|
+| Python | 3.12 |
+| PyTorch | 2.15.0 nightly (cu132) |
+| CUDA Toolkit | 13.3 |
+| GPU | Ada Lovelace (sm_89) — RTX 4050/4060/4070/4080/4090 |
+| SageAttention | 2.2.0 |
 
-## Build Information
+## Install
 
-- Repository: https://github.com/MBZUAI-IFM/llama.cpp
-- Branch: model/K2Horizon
-- Platform: Linux x86_64
-- CUDA: Enabled
-- Version: 0.3.0-dev
-- Build: 10671
-- Commit: 35999d101
+```bash
+pip install sageattention-2.2.0-cp312-cp312-linux_x86_64.whl --no-deps
+```
 
-## Important
+## Build from Source (if needed)
 
-The package includes its own shared libraries inside bin/.
+### Requirements
+- CUDA Toolkit (nvcc)
+- PyTorch with CUDA support
+- 16GB+ RAM or adequate swap
 
-Before running llama-server or llama-cli, set:
+### Steps
 
-`export LD_LIBRARY_PATH="$PWD/bin:$LD_LIBRARY_PATH"`
+```bash
+git clone --depth 1 https://github.com/thu-ml/SageAttention.git /tmp/SageAttention
+cd /tmp/SageAttention
 
-You can also add the binaries to PATH:
+# Patch: PyTorch 2.15 nightly requires C++20
+sed -i 's/-std=c++17/-std=c++20/g' setup.py
 
-`export PATH="$PWD/bin:$PATH"`
+# Build
+CUDA_HOME=/usr/local/cuda-13.3 \
+TORCH_CUDA_ARCH_LIST="8.9" \
+MAX_JOBS=1 \
+NVCC_APPEND_FLAGS="--threads 8" \
+python setup.py install
+```
 
-## Quick Start
+### Create Wheel
 
-Extract the archive:
-
-`tar -xzf llama-k2horizon-cuda-linux-x64.tar.gz`
-
-Enter the directory:
-
-`cd llama-k2horizon-cuda-linux-x64`
-
-Set the runtime library path:
-
-`export LD_LIBRARY_PATH="$PWD/bin:$LD_LIBRARY_PATH"`
-
-Optional PATH:
-
-`export PATH="$PWD/bin:$PATH"`
-
-Check the binary:
-
-`llama-server --version`
-
-Check CUDA devices:
-
-`llama-server --list-devices`
-
-## Run a GGUF Model
-
-`llama-server -m /path/to/model.gguf -ngl 999`
-
-## Run CLI
-
-`llama-cli -m /path/to/model.gguf -ngl 999 -p "Hello"`
-
-## Included
-
-The bin directory contains llama-server, llama-cli, CUDA GGML libraries, CPU GGML libraries, libllama shared libraries, and other utilities from the existing build.
-
-## Requirements
-
-- Linux x86_64
-- NVIDIA GPU
-- NVIDIA driver
-- Compatible CUDA driver support
-
-Check GPU availability with:
-
-`nvidia-smi`
+```bash
+python setup.py bdist_wheel
+cp dist/*.whl /path/to/wheels/
+```
 
 ## Notes
 
-This package contains prebuilt binaries and runtime libraries. The source code and CMake build directory are not required for normal inference.
+- Built on Ubuntu 24.04 with 14GB RAM + 19GB swap
+- `MAX_JOBS=1` used to avoid OOM during CUDA kernel compilation
+- PyTorch 2.15 nightly requires C++20; upstream SageAttention ships with C++17 which must be patched
+- Wheel includes sm_80 (Ampere) and sm_89 (Ada) kernels