Spaces:

pystoff
/

tilly

Sleeping

pystoff commited on 24 days ago

Commit

0337439

verified ·

1 Parent(s): c06bc72

Update app.py

Files changed (1) hide show

app.py CHANGED Viewed

@@ -16,7 +16,7 @@ tokenizer = AutoTokenizer.from_pretrained(
 # -----------------------------
 model = AutoModelForCausalLM.from_pretrained(
     "Gryphe/MythoMax-L2-13B",
-    torch_dtype=torch.float16,
     device_map="auto"
 )

 # -----------------------------
 model = AutoModelForCausalLM.from_pretrained(
     "Gryphe/MythoMax-L2-13B",
+    load_in_8bit=True,
     device_map="auto"
 )