Embed Mamba2
Collection
Text embedding models based on Mamba2 with linear-time and constant-memory inference through vertical chunking. β’ 5 items β’ Updated β’ 3
How to use dynatrace-oss/chunkable-mamba2 with Transformers:
# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("dynatrace-oss/chunkable-mamba2", device_map="auto")The fused chunked path exposed by this repo is deprecated in favor of the upstream split chunked path introduced in transformers#46084.
Custom Mamba2 model and configuration classes for π€ Transformers that add support for vertically chunked inference, which processes input sequences in fixed-size vertical chunks through all model layers with constant memory usage, regardless of sequence length.
ChunkableMamba2Config: extends Mamba2Config with a use_mem_eff_path option for the memory-efficient CUDA kernel path.ChunkableMamba2Model: extends Mamba2Model with a chunkable mixer and cache that correctly propagate the recurrent states across vertical chunks.chunkable_mamba_split_conv1d_scan_combined: modified mamba_split_conv1d_scan_combined kernel wrapper that passes cache parameters through the SSD scan so that conv and SSM states are properly initialized and exported during chunked inference.This repository is designed to be referenced directly from Hugging Face model configs via auto_map, so that models can be loaded with trust_remote_code=True without any local installation:
"auto_map": {
"AutoConfig": "dynatrace-oss/chunkable-mamba2--configuration_chunkable_mamba2.ChunkableMamba2Config",
"AutoModel": "dynatrace-oss/chunkable-mamba2--modeling_chunkable_mamba2.ChunkableMamba2Model"
}
This code was created for the following embedding models:
pip install transformers==5.21.0 kernels==0.17.2 einops
Apache-2.0