How to use Fischerboot/InternLM2-ToxicRP-QLORA-8Bit with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("intervitens/internlm2-limarp-chat-20b") model = PeftModel.from_pretrained(base_model, "Fischerboot/InternLM2-ToxicRP-QLORA-8Bit")
