Mehfuz Hossain

We are building a local inference / fine-tuning solution / app to be used as a GPT alternate. I was wondering if can I take gemma-e2b model and quantize it to 1-bit and then inference it instead of using llama.cpp, using this framework directly