Quantized model · Apache-2.0
Qwen3.8-27B-FAFO-W4A16-G32-B70
1,798 tokens per second total throughput from one Intel Arc Pro B70 (32 GB). Qwen3.8-27B reduced from 55.6 GB of BF16 weights to a 20.6 GB W4A16 checkpoint, validated only on the B70 with Intel's XPU build of vLLM.
Open the model card- Total throughput
- 1,798 tok/s
- Weights on disk
- 20.6 GB
- Aggregate output
- ~296 tok/s
- Longest prompt tested
- 110,000 tokens
in + out, 32 clients, prefix caching on
from 55.6 GB BF16
generated tokens, sustained, 64 clients
single card, no failures
LetsFAFO