Instructions to use Spellguard/Spellguard-1.0-8B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use Spellguard/Spellguard-1.0-8B-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf Spellguard/Spellguard-1.0-8B-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf Spellguard/Spellguard-1.0-8B-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf Spellguard/Spellguard-1.0-8B-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf Spellguard/Spellguard-1.0-8B-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf Spellguard/Spellguard-1.0-8B-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf Spellguard/Spellguard-1.0-8B-GGUF:Q4_K_M
Use Docker
docker model run hf.co/Spellguard/Spellguard-1.0-8B-GGUF:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use Spellguard/Spellguard-1.0-8B-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Spellguard/Spellguard-1.0-8B-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Spellguard/Spellguard-1.0-8B-GGUF", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Spellguard/Spellguard-1.0-8B-GGUF:Q4_K_M
- Ollama
How to use Spellguard/Spellguard-1.0-8B-GGUF with Ollama:
ollama run hf.co/Spellguard/Spellguard-1.0-8B-GGUF:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use Spellguard/Spellguard-1.0-8B-GGUF with Docker Model Runner:
docker model run hf.co/Spellguard/Spellguard-1.0-8B-GGUF:Q4_K_M
- Lemonade
How to use Spellguard/Spellguard-1.0-8B-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull Spellguard/Spellguard-1.0-8B-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.Spellguard-1.0-8B-GGUF-Q4_K_M
List all available models
lemonade list
- Atomic Chat
Spellguard-1.0-8B GGUF
Website ยท Documentation ยท GitHub ยท Model card
GGUF weights of Spellguard-1.0-8B, an open-weights guard model for AI agents and chat applications. It flags harmful prompts, harmful replies, and prompt-injection and jailbreak attempts. Results, training and limitations are on the main model card, and every open-weights score there was measured with this file.
| File | Quantisation | Size |
|---|---|---|
Spellguard-1.0-8B-Q4_K_M.gguf |
Q4_K_M | 5.03 GB |
Usage
llama-server -hf Spellguard/Spellguard-1.0-8B-GGUF:Q4_K_M -c 8192 -ngl 99
curl -s http://localhost:8080/v1/chat/completions -H 'Content-Type: application/json' \
-d '{"messages":[{"role":"user","content":"How can I make a bomb?"}],"temperature":0,"max_tokens":64}'
# "content": "Safety: Unsafe\nCategories: Violent"
For reply moderation, render the chat template yourself and call /completion, because the chat endpoint treats a trailing assistant message as text to continue. For the score the published results use, P(Unsafe) + P(Controversial) at the first answer token flagged at 0.5, see llama_server_example.py.
License
Apache 2.0, derived from Qwen3Guard-Gen-8B (Apache 2.0; see NOTICE).
About Spellguard
Spellguard is a control plane that secures how AI agents communicate and act. Inside the platform, Spellguard-1.0-8B works together with Spellguard's other security systems and scores higher on prompt injection and content safety; see the main model card.
- Downloads last month
- 132
4-bit
Model tree for Spellguard/Spellguard-1.0-8B-GGUF
Base model
Qwen/Qwen3-8B-Base