Image-Text-to-Text
Transformers
Safetensors
premonition_diffusion_gemma
vllm
diffusion-gemma
multimodal
audio
video
custom_code
conversational
Instructions to use prem-research/Premonition-Exp with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use prem-research/Premonition-Exp with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="prem-research/Premonition-Exp", trust_remote_code=True) messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForImageTextToText model = AutoModelForImageTextToText.from_pretrained("prem-research/Premonition-Exp", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use prem-research/Premonition-Exp with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "prem-research/Premonition-Exp" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "prem-research/Premonition-Exp", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/prem-research/Premonition-Exp
- SGLang
How to use prem-research/Premonition-Exp with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "prem-research/Premonition-Exp" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "prem-research/Premonition-Exp", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "prem-research/Premonition-Exp" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "prem-research/Premonition-Exp", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use prem-research/Premonition-Exp with Docker Model Runner:
docker model run hf.co/prem-research/Premonition-Exp
File size: 1,865 Bytes
b965204 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 | """Send the recorded fictional scenarios to a live Premonition decision server."""
import argparse
import base64
import json
from pathlib import Path
import httpx
def request(case, root):
media = []
for name in case["files"]:
path = root / "demo" / "assets" / name
mime, kind = {
".png": ("image/png", "image_url"),
".mp4": ("video/mp4", "video_url"),
".wav": ("audio/wav", "audio_url"),
}[path.suffix]
encoded = base64.b64encode(path.read_bytes()).decode()
media.append({"type": kind, kind: {"url": f"data:{mime};base64,{encoded}"}})
return {
"model": "Premonition-Exp",
"state": case["state"],
"questions": case["questions"],
"media": media,
"samples": 1,
"seed": 42,
}
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--url", default="http://127.0.0.1:8011")
parser.add_argument("--case", default="omni-review")
parser.add_argument("--all", action="store_true")
parser.add_argument("--root", type=Path, default=Path(__file__).resolve().parents[1])
args = parser.parse_args()
rows = json.loads((args.root / "demo/cases.json").read_text())
selected = rows if args.all else [c for c in rows if c["id"] == args.case]
if not selected:
parser.error("Unknown case. Choose from: " + ", ".join(c["id"] for c in rows))
with httpx.Client(timeout=300) as client:
for case in selected:
response = client.post(
args.url.rstrip("/") + "/v1/systemone", json=request(case, args.root)
)
response.raise_for_status()
print(
json.dumps({"case": case["id"], "response": response.json()}, indent=2), flush=True
)
if __name__ == "__main__":
main()
|