Instructions to use maxspin/medichat with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use maxspin/medichat with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("NousResearch/Llama-2-7b-chat-hf") model = PeftModel.from_pretrained(base_model, "maxspin/medichat") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -18,3 +18,26 @@ The following `bitsandbytes` quantization config was used during training:
|
|
| 18 |
|
| 19 |
|
| 20 |
- PEFT 0.4.0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 18 |
|
| 19 |
|
| 20 |
- PEFT 0.4.0
|
| 21 |
+
|
| 22 |
+
- Here is the code to load the load along with llama2 chat model:
|
| 23 |
+
|
| 24 |
+
!pip install -q accelerate==0.21.0 peft==0.4.0 bitsandbytes==0.40.2 transformers==4.31.0 trl==0.4.7
|
| 25 |
+
|
| 26 |
+
import os, torch, logging
|
| 27 |
+
from datasets import load_dataset
|
| 28 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig, HfArgumentParser, TrainingArguments, pipeline
|
| 29 |
+
from peft import LoraConfig, PeftModel
|
| 30 |
+
from trl import SFTTrainer
|
| 31 |
+
import torch
|
| 32 |
+
from peft import PeftModel, PeftConfig
|
| 33 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 34 |
+
|
| 35 |
+
config = PeftConfig.from_pretrained(path_to_directory_containing adapter_config.json)
|
| 36 |
+
model = AutoModelForCausalLM.from_pretrained("NousResearch/Llama-2-7b-chat-hf", return_dict=True, load_in_8bit=True, device_map='auto')
|
| 37 |
+
tokenizer = AutoTokenizer.from_pretrained("NousResearch/Llama-2-7b-chat-hf")
|
| 38 |
+
|
| 39 |
+
model = PeftModel.from_pretrained(model, "/content/llama-2-7b-medichat")
|
| 40 |
+
batch = tokenizer("My friend has been feeling a little dizzy these days.What could he pissibely be suffering from ?", return_tensors='pt')
|
| 41 |
+
batch = batch.to('cuda')
|
| 42 |
+
output_tokens = model.generate(**batch, max_new_tokens=300) ### You can adjust the max_new_tokens
|
| 43 |
+
output_response = tokenizer.decode(output_tokens[0], skip_special_tokens=True)
|