Instructions to use GenAI-CoolCats/WLU-Phi2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use GenAI-CoolCats/WLU-Phi2 with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("microsoft/phi-2") model = PeftModel.from_pretrained(base_model, "GenAI-CoolCats/WLU-Phi2") - Notebooks
- Google Colab
- Kaggle
Download load_from_checkpoint.py from GenAI-CoolCats/WLU-Phi2: direct link, hf CLI and curl.
- Browser
- Download file 1.02 kB
-
https://huggingface.co/GenAI-CoolCats/WLU-Phi2/resolve/main/load_from_checkpoint.py
- Command line
-
hf download hf://GenAI-CoolCats/WLU-Phi2/load_from_checkpoint.py
-
curl -L -o load_from_checkpoint.py https://huggingface.co/GenAI-CoolCats/WLU-Phi2/resolve/main/load_from_checkpoint.py
1.02 kB
| from transformers import AutoModelForCausalLM, AutoTokenizer | |
| save_dir = "checkpoint-1750" | |
| print("Loading model from checkpoint...") | |
| model = AutoModelForCausalLM.from_pretrained(save_dir, load_in_8bit=True) | |
| print("Attaching adapter...") | |
| model.load_adapter(save_dir, adapter_name="Adapter1") | |
| print("Loading tokenizer...") | |
| tokenizer = AutoTokenizer.from_pretrained(save_dir) | |
| while True: | |
| text = input(">>> ") | |
| if text == "exit": | |
| break | |
| model_inputs = tokenizer([text], return_tensors="pt", max_length=256).to("cuda") | |
| generated_ids = model.generate(**model_inputs, | |
| max_length=1024, | |
| #truncation=True, | |
| temperature=0.1, | |
| do_sample=True, | |
| pad_token_id=tokenizer.eos_token_id) | |
| response=tokenizer.batch_decode(generated_ids, | |
| skip_special_tokens=True)[0] | |
| # remove repeat of the question | |
| if '?' in response: | |
| to_q = response.index('?') | |
| if len(text)-1 <= to_q and response[:to_q] == text[:to_q]: | |
| response = response[to_q+1:] | |
| print(f"\n\t<<< {response} >>>\n") |