Commit History
-
ggml-flash-attn: ggml's flash attention as a standalone kernels repo
Split out of ggml-attn-kernels so that a repo mapped to a transformers attn_implementation contains only an attention implementation. The gated delta net, l2_norm, swiglu and ssm_conv ops stay behind: they have no library-native extension point yet. Carries a fix the original did not have. ggml_flash_attn_ext takes no is_causal argument -- in ggml the mask is the causality, and a null mask means attend to everything -- while transformers drops the mask whenever sdpa's own is_causal flag would carry it, and that flag never reaches an attention function. So flash_attn_forward received None for a 2..19 token prefill and attended to future positions. It now rebuilds the mask. The failure was silent: on Qwen3.5-4B it moved the last-position logits by 1.44 and left the greedy top-5 unchanged, which is why the original 73 tests passed with the bug present. Six new cases assert both directions -- close to causal and far from bidirectional -- and fail without the fix.8732af0