Spaces:

sourabht
/

testingFinetuning

Running on Zero

sourabht commited on 5 days ago

Commit

2fedcc0

verified ·

1 Parent(s): 526256d

Update app.py

Files changed (1) hide show

app.py CHANGED Viewed

@@ -18,7 +18,12 @@ base = AutoModelForCausalLM.from_pretrained(
     MODEL, revision=REVISION, dtype=torch.float32,
     attn_implementation="eager",
 )
-model = PeftModel.from_pretrained(base, "adapter").eval()
 def reply(text):
     messages = [

     MODEL, revision=REVISION, dtype=torch.float32,
     attn_implementation="eager",
 )
+model = PeftModel.from_pretrained(
+    base,
+    "adapter",
+    torch_device="cpu",
+    is_trainable=False,
+).eval()
 def reply(text):
     messages = [