Zen Embeddings
Collection
Embedding model family for RAG and semantic search. โข 7 items โข Updated
How to use zenlm/zen-embedding-8B-GGUF with llama.cpp:
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible embeddings server: llama serve -hf zenlm/zen-embedding-8B-GGUF:Q4_K_M --embeddings
winget install llama.cpp # Start a local OpenAI-compatible embeddings server: llama serve -hf zenlm/zen-embedding-8B-GGUF:Q4_K_M --embeddings
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible embeddings server: ./build/bin/llama serve -hf zenlm/zen-embedding-8B-GGUF:Q4_K_M --embeddings
How to use zenlm/zen-embedding-8B-GGUF with Ollama:
ollama run hf.co/zenlm/zen-embedding-8B-GGUF:Q4_K_M
How to use zenlm/zen-embedding-8B-GGUF with Docker Model Runner:
docker model run hf.co/zenlm/zen-embedding-8B-GGUF:Q4_K_M
How to use zenlm/zen-embedding-8B-GGUF with Lemonade:
# Download Lemonade from https://lemonade-server.ai/ lemonade pull zenlm/zen-embedding-8B-GGUF:Q4_K_M
lemonade run user.zen-embedding-8B-GGUF-Q4_K_M
lemonade list
8B Zen embedding model, quantized to GGUF (Q4_K_M) for CPU and mixed CPU/GPU inference with llama.cpp and compatible runtimes.
Repackaged from Qwen/Qwen3-Embedding-8B (apache-2.0, Alibaba Qwen), quantized to GGUF. Not trained from scratch โ a permissively-licensed redistribution for the OSS-clean Zen model line.
| File | Format |
|---|---|
zen-embedding-8B-Q4_K_M.gguf |
GGUF Q4_K_M |
from llama_cpp import Llama
llm = Llama.from_pretrained(
repo_id="zenlm/zen-embedding-8B-GGUF",
filename="*Q4_K_M.gguf",
embedding=True,
)
print(llm.create_embedding("Hello!"))
Full-precision safetensors: zenlm/zen-embedding-8B.
apache-2.0. Upstream: Qwen/Qwen3-Embedding-8B by Alibaba Qwen. This repository redistributes a quantized derivative under the same license.
4-bit