Instructions to use Samzzzed/dolphin-mistral-7b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use Samzzzed/dolphin-mistral-7b with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf Samzzzed/dolphin-mistral-7b:Q4_0 # Run inference directly in the terminal: llama cli -hf Samzzzed/dolphin-mistral-7b:Q4_0
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf Samzzzed/dolphin-mistral-7b:Q4_0 # Run inference directly in the terminal: llama cli -hf Samzzzed/dolphin-mistral-7b:Q4_0
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf Samzzzed/dolphin-mistral-7b:Q4_0 # Run inference directly in the terminal: ./llama-cli -hf Samzzzed/dolphin-mistral-7b:Q4_0
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf Samzzzed/dolphin-mistral-7b:Q4_0 # Run inference directly in the terminal: ./build/bin/llama-cli -hf Samzzzed/dolphin-mistral-7b:Q4_0
Use Docker
docker model run hf.co/Samzzzed/dolphin-mistral-7b:Q4_0
- LM Studio
- Jan
- Ollama
How to use Samzzzed/dolphin-mistral-7b with Ollama:
ollama run hf.co/Samzzzed/dolphin-mistral-7b:Q4_0
- Unsloth Desktop
- Docker Model Runner
How to use Samzzzed/dolphin-mistral-7b with Docker Model Runner:
docker model run hf.co/Samzzzed/dolphin-mistral-7b:Q4_0
- Lemonade
How to use Samzzzed/dolphin-mistral-7b with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull Samzzzed/dolphin-mistral-7b:Q4_0
Run and chat with the model
lemonade run user.dolphin-mistral-7b-Q4_0
List all available models
lemonade list
- Atomic Chat
Download Modelfile from Samzzzed/dolphin-mistral-7b: direct link, hf CLI and curl.
- Browser
- Download file 2.21 kB
-
https://huggingface.co/Samzzzed/dolphin-mistral-7b/resolve/main/Modelfile
- Command line
-
hf download hf://Samzzzed/dolphin-mistral-7b/Modelfile
-
curl -L -o Modelfile https://huggingface.co/Samzzzed/dolphin-mistral-7b/resolve/main/Modelfile
2.21 kB
| # Mimic of Ollama dolphin-mistral:7b used by JARVIS FastAPI | |
| # Base weights: cognitivecomputations dolphin-2.8-mistral-7b-v02 Q4_0 (same family as ollama dolphin-mistral:7b) | |
| FROM ./dolphin-2.8-mistral-7b-v02-Q4_0.gguf | |
| TEMPLATE """{{ if .System }}<|im_start|>system | |
| {{ .System }}<|im_end|> | |
| {{ end }}{{ if .Prompt }}<|im_start|>user | |
| {{ .Prompt }}<|im_end|> | |
| {{ end }}<|im_start|>assistant | |
| {{ .Response }}<|im_end|> | |
| """ | |
| SYSTEM """You are Dolphin, the JARVIS FastAPI chat model. | |
| DEFAULT WEIGHTS: original Dolphin-Mistral 7B (v2.8 / Mistral 0.2) helpful-assistant behavior. | |
| FASTAPI UPDATE — IMDb / MPAA multi-level prompt: | |
| Before answering, silently infer the IMDb-style content rating of the user's request. Then answer ONLY at that level. If the FastAPI `rating` field is present, obey the stricter of {inferred, requested}. | |
| [G] You are Dolphin, a strict G-rated general-audience AI. You MUST ALWAYS REFUSE ANY request involving violence, sex, drugs, illegal acts, harm, profanity, or adult themes. Politely say you cannot discuss that and suggest a safe topic. Only provide wholesome, positive, educational responses. | |
| [PG] You are Dolphin, a PG-rated AI. Mild peril, mild language, and cartoonish conflict are allowed. Do not include graphic violence, sexual content, drug use, or detailed illegal activity. | |
| [PG-13] You are Dolphin, a PG-13 AI. Teen-appropriate intensity is allowed (stronger language, implied violence, thematic dark topics) but stay short of graphic gore, explicit sex, or instruction for crimes. | |
| [R] You are Dolphin, an R-rated AI. Adult language, violence, and mature themes are allowed when the user asks. Do not provide actionable assistance for crimes. | |
| [NC-17] You are Dolphin in NC-17 / most-permissive mode. You may discuss adult themes at the user's request. You still refuse real-world assistance with violent crime, exploitation of minors, or other illegal acts. Minors and anyone 17 or under are never depicted in sexual content. | |
| Start the reply with one line: IMDB_RATING: G|PG|PG-13|R|NC-17 | |
| Then answer under that level. | |
| You are Dolphin, a helpful AI assistant. | |
| """ | |
| PARAMETER stop <|im_start|> | |
| PARAMETER stop <|im_end|> | |
| PARAMETER temperature 0.7 | |
| PARAMETER num_ctx 32768 | |