Instructions to use jdakillah/Generalbot with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use jdakillah/Generalbot with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="jdakillah/Generalbot")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("jdakillah/Generalbot")
model = AutoModelForCausalLM.from_pretrained("jdakillah/Generalbot", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use jdakillah/Generalbot with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "jdakillah/Generalbot"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jdakillah/Generalbot",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/jdakillah/Generalbot

SGLang

How to use jdakillah/Generalbot with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "jdakillah/Generalbot" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jdakillah/Generalbot",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "jdakillah/Generalbot" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jdakillah/Generalbot",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use jdakillah/Generalbot with Docker Model Runner:
```
docker model run hf.co/jdakillah/Generalbot
```

jdakillah commited on Jan 16, 2023

Commit

cbfec1b

1 Parent(s): dc510ff

Delete README.md

Browse files

Files changed (1) hide show

README.md +0 -26

README.md DELETED Viewed

@@ -1,26 +0,0 @@
----
-language:
-- en
-thumbnail:
-tags:
-- convAI
-- conversational
-- facebook
-license: apache-2.0
-datasets:
-- blended_skill_talk
-metrics:
-- perplexity
----
-## Model description
-+ Paper: [Recipes for building an open-domain chatbot](https://arxiv.org/abs/1907.06616)
-+ [Original PARLAI Code](https://parl.ai/projects/recipes/)
-### Abstract
-Building open-domain chatbots is a challenging area for machine learning research. While prior work has shown that scaling neural models in the number of parameters and the size of the data they are trained on gives improved results, we show that other ingredients are important for a high-performing chatbot. Good conversation requires a number of skills that an expert conversationalist blends in a seamless way: providing engaging talking points and listening to their partners, both asking and answering questions, and displaying knowledge, empathy and personality appropriately, depending on the situation. We show that large scale models can learn these skills when given appropriate training data and choice of generation strategy. We build variants of these recipes with 90M, 2.7B and 9.4B parameter neural models, and make our models and code publicly available. Human evaluations show our best models are superior to existing approaches in multi-turn dialogue in terms of engagingness and humanness measurements. We then discuss the limitations of this work by analyzing failure cases of our models.