File size: 1,732 Bytes
7230ed6 a2d1b9b 7230ed6 a2d1b9b | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 | ---
title: README
emoji: 🔧
colorFrom: indigo
colorTo: purple
sdk: static
pinned: false
---
# SlyBase
**Fast LLM inference on consumer AMD GPUs.**
We run large models on a single AMD Radeon AI PRO R9700 (RDNA4, 32 GB) and publish what it takes to make
them fast: kernels, serving configs and ready-to-run quantizations — measured, not guessed.
### 🚀 [vllm-sly-radiance](https://github.com/SlyBase/vllm-sly-radiance)
A vLLM fork for RDNA4 (gfx1201). Hand-written W4A8 MXFP4 kernels on the fp8 matrix cores, fused
norm/quant, tuned decode and drafter attention, DFlash2 speculative decoding and int4 heads — Qwen3.8-27B at
~130 tokens/s decode on one card, 262k context. One Docker image, every change gated on a benchmark and a
quality check.
### 📦 Models
Quantizations built for hardware with native MXFP4 (RDNA4, AMD Instinct MI350), each with a model card
that says what it costs in quality and speed — including when you are better off with another build.
- [ThinkingCap-Qwen3.8-27B-MXFP4](https://huggingface.co/slybase/ThinkingCap-Qwen3.8-27B-MXFP4) —
BottleCap AI's thinking-efficient Qwen3.8-27B finetune, GPTQ-quantized to MXFP4.
### 📏 How we measure
Decode by step time and speculative acceptance (not single noisy runs), quality by deterministic
prompt-logprob comparison plus GSM8K, all on the same card and power cap. Numbers and scripts live in the
fork's [docs](https://github.com/SlyBase/vllm-sly-radiance/tree/main/docs).
---
Credit where it's due: this work stands on [vLLM](https://github.com/vllm-project/vllm),
[StillDeadcode/vllm-radiance](https://github.com/StillDeadcode/vllm-radiance) and ggz14's RDNA4 kernels.
Model weights belong to their authors and keep their licenses.
|