Instructions to use aymous/lfm2-8b-a1b-short-conv with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Kernels
How to use aymous/lfm2-8b-a1b-short-conv with Kernels:
# !pip install kernels from kernels import get_kernel # a version (or an explicit revision) is required; see the "Files and versions" tab for the available ones kernel = get_kernel("aymous/lfm2-8b-a1b-short-conv", version=1) - Notebooks
- Google Colab
- Kaggle
LFM2.5-8B-A1B gated short-conv
Fused gated short-conv decode step for LiquidAI/LFM2.5-8B-A1B (Lfm2MoeShortConv, the LFM2 token-mixer on 18 of 24 layers, L_cache=3). Collapses the eager decode chain โ chunk โ Bยทx โ 3-tap depthwise conv โ Cยทconv_out โ conv-state roll โ into a single launch per layer (the fallback path when causal_conv1d is absent). The in_proj/out_proj GEMVs stay cuBLAS, outside the op.
Signature: lfm2_short_conv(BCx, conv_state, weight) -> (y, new_conv_state), where BCx = in_proj(x) laid out [B(H) | C(H) | x(H)] and conv_state is the [H, 3] cache; returns the gated output and the rolled state.
Part of a 3-kernel pack (with lfm2-8b-a1b-moe-experts and lfm2-8b-a1b-rms-norm) that makes the whole model 1.90ร faster end-to-end on GB10.
Usage
from kernels import get_kernel
k = get_kernel("aymous/lfm2-8b-a1b-short-conv", version=1, trust_remote_code=True)
y, new_state = k.lfm2_short_conv(BCx, conv_state, conv_weight) # single-token decode, bf16
aarch64, torch 2.11/2.12 ร CUDA 12.6/12.8/13.0/13.2, archs 7.5โ9.0 + 10.0/11.0/12.0/12.1+PTX. Benchmarked on GB10 (sm_121).
- Downloads last month
- -