Hugging Face's logo

pankajroark
/
llama-fp16-engine

Model card Files
xet
 Community
llama-fp16-engine

Commit History

  1. update engine with inflight options for 7b-sq-int8kv-tp8

    pankajroark
    ad987a1

  2. inflight batching engine for 7b-sq-int8kv-tp1

    pankajroark
    2319002

  1. update non-quant engine again, this time with one that is locally tested

    pankajroark
    18b067e

  2. update no-quant engine

    pankajroark
    bd472e5

  1. tp8 checkpoint

    pankajroark
    a3806a1

  2. sq version

    pankajroark
    4b63086

  3. rename

    pankajroark
    3098727

  4. checkpoint

    pankajroark
    ac5b847

  5. initial commit

    pankajroark
    6f63e60