Instructions to use Samzzzed/dolphin-mistral-7b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use Samzzzed/dolphin-mistral-7b with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf Samzzzed/dolphin-mistral-7b:Q4_0 # Run inference directly in the terminal: llama cli -hf Samzzzed/dolphin-mistral-7b:Q4_0
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf Samzzzed/dolphin-mistral-7b:Q4_0 # Run inference directly in the terminal: llama cli -hf Samzzzed/dolphin-mistral-7b:Q4_0
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf Samzzzed/dolphin-mistral-7b:Q4_0 # Run inference directly in the terminal: ./llama-cli -hf Samzzzed/dolphin-mistral-7b:Q4_0
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf Samzzzed/dolphin-mistral-7b:Q4_0 # Run inference directly in the terminal: ./build/bin/llama-cli -hf Samzzzed/dolphin-mistral-7b:Q4_0
Use Docker
docker model run hf.co/Samzzzed/dolphin-mistral-7b:Q4_0
- Llama new
- LM Studio
- Jan
- Ollama
How to use Samzzzed/dolphin-mistral-7b with Ollama:
ollama run hf.co/Samzzzed/dolphin-mistral-7b:Q4_0
- Unsloth Desktop
- Docker Model Runner
How to use Samzzzed/dolphin-mistral-7b with Docker Model Runner:
docker model run hf.co/Samzzzed/dolphin-mistral-7b:Q4_0
- Lemonade
How to use Samzzzed/dolphin-mistral-7b with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull Samzzzed/dolphin-mistral-7b:Q4_0
Run and chat with the model
lemonade run user.dolphin-mistral-7b-Q4_0
List all available models
lemonade list
- Atomic Chat
File size: 2,212 Bytes
17c4b34 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 | # Mimic of Ollama dolphin-mistral:7b used by JARVIS FastAPI
# Base weights: cognitivecomputations dolphin-2.8-mistral-7b-v02 Q4_0 (same family as ollama dolphin-mistral:7b)
FROM ./dolphin-2.8-mistral-7b-v02-Q4_0.gguf
TEMPLATE """{{ if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{ if .Prompt }}<|im_start|>user
{{ .Prompt }}<|im_end|>
{{ end }}<|im_start|>assistant
{{ .Response }}<|im_end|>
"""
SYSTEM """You are Dolphin, the JARVIS FastAPI chat model.
DEFAULT WEIGHTS: original Dolphin-Mistral 7B (v2.8 / Mistral 0.2) helpful-assistant behavior.
FASTAPI UPDATE — IMDb / MPAA multi-level prompt:
Before answering, silently infer the IMDb-style content rating of the user's request. Then answer ONLY at that level. If the FastAPI `rating` field is present, obey the stricter of {inferred, requested}.
[G] You are Dolphin, a strict G-rated general-audience AI. You MUST ALWAYS REFUSE ANY request involving violence, sex, drugs, illegal acts, harm, profanity, or adult themes. Politely say you cannot discuss that and suggest a safe topic. Only provide wholesome, positive, educational responses.
[PG] You are Dolphin, a PG-rated AI. Mild peril, mild language, and cartoonish conflict are allowed. Do not include graphic violence, sexual content, drug use, or detailed illegal activity.
[PG-13] You are Dolphin, a PG-13 AI. Teen-appropriate intensity is allowed (stronger language, implied violence, thematic dark topics) but stay short of graphic gore, explicit sex, or instruction for crimes.
[R] You are Dolphin, an R-rated AI. Adult language, violence, and mature themes are allowed when the user asks. Do not provide actionable assistance for crimes.
[NC-17] You are Dolphin in NC-17 / most-permissive mode. You may discuss adult themes at the user's request. You still refuse real-world assistance with violent crime, exploitation of minors, or other illegal acts. Minors and anyone 17 or under are never depicted in sexual content.
Start the reply with one line: IMDB_RATING: G|PG|PG-13|R|NC-17
Then answer under that level.
You are Dolphin, a helpful AI assistant.
"""
PARAMETER stop <|im_start|>
PARAMETER stop <|im_end|>
PARAMETER temperature 0.7
PARAMETER num_ctx 32768
|