Instructions to use research-backup/bart-large-squad-qg-default with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use research-backup/bart-large-squad-qg-default with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="research-backup/bart-large-squad-qg-default")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("research-backup/bart-large-squad-qg-default")
model = AutoModelForSeq2SeqLM.from_pretrained("research-backup/bart-large-squad-qg-default", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use research-backup/bart-large-squad-qg-default with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "research-backup/bart-large-squad-qg-default"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "research-backup/bart-large-squad-qg-default",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/research-backup/bart-large-squad-qg-default

SGLang

How to use research-backup/bart-large-squad-qg-default with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "research-backup/bart-large-squad-qg-default" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "research-backup/bart-large-squad-qg-default",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "research-backup/bart-large-squad-qg-default" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "research-backup/bart-large-squad-qg-default",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use research-backup/bart-large-squad-qg-default with Docker Model Runner:
```
docker model run hf.co/research-backup/bart-large-squad-qg-default
```

asahi417 commited on Feb 15, 2022

Commit

3bd5052

1 Parent(s): f9dee64

model update

Browse files

Files changed (6) hide show

eval/metric.first.answer.sentence_answer.question.json +1 -0
eval/metric.first.sentence.sentence_answer.question.json +1 -0
eval/metric.last.sentence.sentence_answer.question.json +1 -0
eval/metric.long.sentence.sentence_answer.question.json +1 -0
eval/metric.middle.sentence.sentence_answer.question.json +1 -0
eval/metric.short.sentence.sentence_answer.question.json +1 -0

eval/metric.first.answer.sentence_answer.question.json ADDED Viewed

	@@ -0,0 +1 @@


1	+ {"validation": {"Bleu_1": 0.5030155141448129, "Bleu_2": 0.35237363844212877, "Bleu_3": 0.26814856359231537, "Bleu_4": 0.21107749039635343, "METEOR": 0.241590165538654, "ROUGE_L": 0.49772692273801433}, "test": {"Bleu_1": 0.472241609361628, "Bleu_2": 0.3234940373498494, "Bleu_3": 0.24090221049606247, "Bleu_4": 0.18508475292468374, "METEOR": 0.22837861592098846, "ROUGE_L": 0.47664750426052543}}

eval/metric.first.sentence.sentence_answer.question.json ADDED Viewed

	@@ -0,0 +1 @@


1	+ {"validation": {"Bleu_1": 0.5488413887163694, "Bleu_2": 0.3914319469080578, "Bleu_3": 0.3015711885265142, "Bleu_4": 0.24000479239072373, "METEOR": 0.254528451474518, "ROUGE_L": 0.5183547355633444}, "test": {"Bleu_1": 0.5415351539785386, "Bleu_2": 0.3823233214267522, "Bleu_3": 0.28886709980119873, "Bleu_4": 0.2240821126708425, "METEOR": 0.24516805242731088, "ROUGE_L": 0.5077593215086487}}

eval/metric.last.sentence.sentence_answer.question.json ADDED Viewed

	@@ -0,0 +1 @@


1	+ {"validation": {"Bleu_1": 0.5440035897494925, "Bleu_2": 0.3838775107253782, "Bleu_3": 0.2927600517381518, "Bleu_4": 0.2304805078196388, "METEOR": 0.24863247633209454, "ROUGE_L": 0.5086532380978919}, "test": {"Bleu_1": 0.5373712085457809, "Bleu_2": 0.37529670319993563, "Bleu_3": 0.28113501935334984, "Bleu_4": 0.21654068523889092, "METEOR": 0.24206911694349129, "ROUGE_L": 0.5016783813158281}}

eval/metric.long.sentence.sentence_answer.question.json ADDED Viewed

	@@ -0,0 +1 @@


1	+ {"validation": {"Bleu_1": 0.5541338008035492, "Bleu_2": 0.39125545830403613, "Bleu_3": 0.2982771596359661, "Bleu_4": 0.23496725429610837, "METEOR": 0.25395021043348387, "ROUGE_L": 0.5084489925300315}, "test": {"Bleu_1": 0.549722194745834, "Bleu_2": 0.38462521024775664, "Bleu_3": 0.28863886468588945, "Bleu_4": 0.22275250253248094, "METEOR": 0.24734021439517329, "ROUGE_L": 0.500746652056343}}

eval/metric.middle.sentence.sentence_answer.question.json ADDED Viewed

	@@ -0,0 +1 @@


1	+ {"validation": {"Bleu_1": 0.5527062036283785, "Bleu_2": 0.39121003345465194, "Bleu_3": 0.29954300516157606, "Bleu_4": 0.23674890022977294, "METEOR": 0.25345759718611466, "ROUGE_L": 0.5120443389486449}, "test": {"Bleu_1": 0.5481581542169782, "Bleu_2": 0.3861467258384913, "Bleu_3": 0.2915139185789507, "Bleu_4": 0.22587063099316915, "METEOR": 0.2469544403734596, "ROUGE_L": 0.5058276030744512}}

eval/metric.short.sentence.sentence_answer.question.json ADDED Viewed

	@@ -0,0 +1 @@


1	+ {"validation": {"Bleu_1": 0.5361802362093184, "Bleu_2": 0.38181905487068873, "Bleu_3": 0.29393218446746866, "Bleu_4": 0.23377332365665837, "METEOR": 0.24839006923173318, "ROUGE_L": 0.5169650762849358}, "test": {"Bleu_1": 0.5240954951370504, "Bleu_2": 0.36699333862652767, "Bleu_3": 0.27522613350456676, "Bleu_4": 0.2124422038874024, "METEOR": 0.23838651699107208, "ROUGE_L": 0.5042520333554934}}