Instructions to use AlanRobotics/opt-assisted-generation with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use AlanRobotics/opt-assisted-generation with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="AlanRobotics/opt-assisted-generation")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("AlanRobotics/opt-assisted-generation") model = AutoModelForCausalLM.from_pretrained("AlanRobotics/opt-assisted-generation", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use AlanRobotics/opt-assisted-generation with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "AlanRobotics/opt-assisted-generation" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "AlanRobotics/opt-assisted-generation", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/AlanRobotics/opt-assisted-generation
- SGLang
How to use AlanRobotics/opt-assisted-generation with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "AlanRobotics/opt-assisted-generation" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "AlanRobotics/opt-assisted-generation", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "AlanRobotics/opt-assisted-generation" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "AlanRobotics/opt-assisted-generation", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use AlanRobotics/opt-assisted-generation with Docker Model Runner:
docker model run hf.co/AlanRobotics/opt-assisted-generation
Commit History
-
Training in progress, step 2650
01c5de0 -
Training in progress, step 2600
a4647a6 -
Training in progress, step 2500
7b70f75 -
Training in progress, step 2450
1c6f6a7 -
Training in progress, step 2400
b246ce8 -
Training in progress, step 2350
e9733c8 -
Training in progress, step 2300
c792a28 -
Training in progress, step 2200
0afa02f -
Training in progress, step 2150
7824121 -
Training in progress, step 2100
ce9f860 -
Training in progress, step 2050
8b65c30 -
Training in progress, step 2000
35185a4 -
Training in progress, step 1950
d6a61ae -
Training in progress, step 1900
a447118 -
Training in progress, step 1850
b398ae0 -
Training in progress, step 1800
96aa0e4 -
Training in progress, step 1750
5f16257 -
Training in progress, step 1700
6e38b06 -
Training in progress, step 1650
28abb84 -
Training in progress, step 1550
99a1503 -
Training in progress, step 1500
c3042d8 -
Training in progress, step 1450
fb945fb -
Training in progress, step 1400
9838c00 -
Training in progress, step 1300
fc9a009 -
Training in progress, step 1250
93fa980 -
Training in progress, step 1200
313d6cc -
Training in progress, step 1150
1aac11e -
Training in progress, step 1100
a368e20 -
Training in progress, step 1050
c303d8c -
Training in progress, step 1000
83579c5 -
Training in progress, step 950
5b348f1 -
Training in progress, step 900
3c886b2 -
Training in progress, step 850
b3e4b2d -
Training in progress, step 800
7ab4a0e -
Training in progress, step 750
1f0700d -
Training in progress, step 700
e3f6079 -
Training in progress, step 650
a0d99a9 -
Training in progress, step 600
14f0a71 -
Training in progress, step 500
178338a -
Training in progress, step 450
d98a4f8 -
Training in progress, step 400
372fea7 -
Training in progress, step 350
62e060a -
Training in progress, step 300
354ce82 -
Training in progress, step 250
e1449e6 -
Training in progress, step 200
2e76e12 -
Training in progress, step 150
7684d1a -
Training in progress, step 100
eecbcc5 -
Training in progress, step 50
1cd63d4 -
initial commit
ebd50bb