Instructions to use osiria/primo with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use osiria/primo with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="osiria/primo")# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("osiria/primo", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use osiria/primo with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "osiria/primo" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "osiria/primo", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/osiria/primo
- SGLang
How to use osiria/primo with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "osiria/primo" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "osiria/primo", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "osiria/primo" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "osiria/primo", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use osiria/primo with Docker Model Runner:
docker model run hf.co/osiria/primo
Update README.md
Browse files
README.md
CHANGED
|
@@ -55,7 +55,7 @@ In order to use the model for inference, it's strongly recommended to build a Gr
|
|
| 55 |
|
| 56 |
<h3>Limitations</h3>
|
| 57 |
|
| 58 |
-
The model might behave erratically when presented with prompts which are too far away from its pre-training or fine-tuning and, because of the probabilistic nature of its generation, it might occasionally produce biased or offensive content with respect to gender, race, ideologies, and political or religious beliefs.
|
| 59 |
These limitations imply that the model and its outputs should be used with caution, and should not be involved in situations that require the generated text to be fair or true.
|
| 60 |
|
| 61 |
<h3>References</h3>
|
|
|
|
| 55 |
|
| 56 |
<h3>Limitations</h3>
|
| 57 |
|
| 58 |
+
The model might behave erratically when presented with prompts which are too far away from its pre-training or fine-tuning and, because of the probabilistic nature of its generation mechanism, it might occasionally produce biased or offensive content with respect to gender, race, ideologies, and political or religious beliefs.
|
| 59 |
These limitations imply that the model and its outputs should be used with caution, and should not be involved in situations that require the generated text to be fair or true.
|
| 60 |
|
| 61 |
<h3>References</h3>
|