Hugging Face's logo

harshithsaiv
/
kv-cache-compression

English
quantization
kv-cache
llm-inference
cuda
triton
memory-efficient
mitral
llama
inference-optimization
4-bit precision
mixed-precision
Model card Files
xet
 Community
kv-cache-compression / kernel

Commit History

  1. feat: complete honest 4-method benchmark both models⁠

    harshithsaiv
    5e16ca3

  2. feat: true Triton 4-bit kernel with real bit packing⁠

    harshithsaiv
    35feffe

  3. feat: Complete working pytorch version of kernel

    harshithsaiv
    4b2bdf2

  4. feat: Testing pure pytorch

    harshithsaiv
    e23db09

  5. chore: clean the quant_cache

    harshithsaiv
    6a962fc

  6. chore: libdevice not present in the current version

    harshithsaiv
    bc4bbbe

  7. feat: Initial Triton kernel

    harshithsaiv
    91c163e