Instructions to use Arcpolar/Ubuntu_Llama_Chat_7B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Arcpolar/Ubuntu_Llama_Chat_7B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Arcpolar/Ubuntu_Llama_Chat_7B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Arcpolar/Ubuntu_Llama_Chat_7B")
model = AutoModelForCausalLM.from_pretrained("Arcpolar/Ubuntu_Llama_Chat_7B", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Arcpolar/Ubuntu_Llama_Chat_7B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Arcpolar/Ubuntu_Llama_Chat_7B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Arcpolar/Ubuntu_Llama_Chat_7B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Arcpolar/Ubuntu_Llama_Chat_7B

SGLang

How to use Arcpolar/Ubuntu_Llama_Chat_7B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Arcpolar/Ubuntu_Llama_Chat_7B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Arcpolar/Ubuntu_Llama_Chat_7B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Arcpolar/Ubuntu_Llama_Chat_7B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Arcpolar/Ubuntu_Llama_Chat_7B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Arcpolar/Ubuntu_Llama_Chat_7B with Docker Model Runner:
```
docker model run hf.co/Arcpolar/Ubuntu_Llama_Chat_7B
```

Arcpolar commited on Oct 12, 2023

Commit

77ea2a7

1 Parent(s): 9f73289

Update README.md

Browse files

Files changed (1) hide show

README.md +4 -8

README.md CHANGED Viewed

@@ -9,16 +9,12 @@ Ubuntu_Llama_Chat_7B is a fine-tuned model based on Llama 2 Chat 7b base model a
 ## Acknowledgments
-### Base Model: Llama2
 - We utilized the Llama2 model as the base model for our project. The model was obtained from [source/repository](URL_to_source).
 - Special thanks to [authors/creators of Llama2](URL_to_author_profile) for providing the model and the community around it for the support.
 - License: Mention the license under which Llama2 model is distributed, and adhere to any attribution requirements.
 ### Fine-Tune Dataset
-- The fine-tuning was performed on [Name of the Dataset](URL_to_dataset) dataset, which was crucial for achieving the results.
-- The dataset is provided under [License Name](URL_to_license) license. We thank [Dataset Creators/Providers](URL_to_creator_profile) for making the dataset publicly available.
-- Any additional acknowledgments or information regarding the dataset.
-<b>Acknowledgements:</b><br>
-<br> <b>Fine-Tuned Dataset</b><br>
-Ubuntu Dialogue Corpus dataset was collected by Ryan Lowe, Nissan Pow , Iulian V. Serban† and Joelle Pineau. It is made available here under the Apache License, 2.0. with following citation:
-Ryan Lowe, Nissan Pow, Iulian V. Serban and Joelle Pineau, "The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems", SIGDial 2015. URL: http://www.sigdial.org/workshops/conference16/proceedings/pdf/SIGDIAL40.pdf

 ## Acknowledgments
+### Base Model: Llama2_Chat_7b
 - We utilized the Llama2 model as the base model for our project. The model was obtained from [source/repository](URL_to_source).
 - Special thanks to [authors/creators of Llama2](URL_to_author_profile) for providing the model and the community around it for the support.
 - License: Mention the license under which Llama2 model is distributed, and adhere to any attribution requirements.
 ### Fine-Tune Dataset
+- The fine-tuning was performed on [Ubuntu Dialogue Corpus](https://www.kaggle.com/datasets/rtatman/ubuntu-dialogue-corpus) dataset, which was crucial for achieving the results.
+- The dataset is provided under [Apache License, 2.0](https://www.apache.org/licenses/LICENSE-2.0) license. We thank [Ryan Lowe, Nissan Pow , Iulian V. Serban† and Joelle Pineau](http://www.sigdial.org/workshops/conference16/proceedings/pdf/SIGDIAL40.pdf) for making the dataset publicly available.
+- Ryan Lowe, Nissan Pow, Iulian V. Serban and Joelle Pineau, "The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems", SIGDial 2015. URL: http://www.sigdial.org/workshops/conference16/proceedings/pdf/SIGDIAL40.pdf