jmarquex/nexusquant
Updated
KV Cache Compression Methods
Methods for compressing the KV cache in transformer inference. E8 lattice, KVarN, TurboQuant, KIVI.
Methods for compressing the KV cache in transformer inference. E8 lattice, KVarN, TurboQuant, KIVI.