Feature Extraction
GGUF
llama.cpp
embeddings
multimodal
vision
audio
image-feature-extraction
audio-feature-extraction
Instructions to use MirrorNeuronLab/embeddinggemma-2-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use MirrorNeuronLab/embeddinggemma-2-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible embeddings server: llama serve -hf MirrorNeuronLab/embeddinggemma-2-GGUF:Q4_K_M --embeddings
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible embeddings server: llama serve -hf MirrorNeuronLab/embeddinggemma-2-GGUF:Q4_K_M --embeddings
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible embeddings server: ./build/bin/llama serve -hf MirrorNeuronLab/embeddinggemma-2-GGUF:Q4_K_M --embeddings
- Llama new
- LM Studio
- Jan
- Ollama
How to use MirrorNeuronLab/embeddinggemma-2-GGUF with Ollama:
ollama run hf.co/MirrorNeuronLab/embeddinggemma-2-GGUF:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use MirrorNeuronLab/embeddinggemma-2-GGUF with Docker Model Runner:
docker model run hf.co/MirrorNeuronLab/embeddinggemma-2-GGUF:Q4_K_M
- Lemonade
How to use MirrorNeuronLab/embeddinggemma-2-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull MirrorNeuronLab/embeddinggemma-2-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.embeddinggemma-2-GGUF-Q4_K_M
List all available models
lemonade list
- Atomic Chat
File size: 3,250 Bytes
336c50f | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 | import base64
import json
import math
import pathlib
import struct
import time
import urllib.error
import urllib.request
import wave
import zlib
ROOT = pathlib.Path(__file__).resolve().parents[1]
TEXTS = [
"task: search result | query: What causes the northern lights?",
"title: none | text: The northern lights are caused by charged particles from the Sun interacting with Earth's atmosphere.",
"title: none | text: To bake a chocolate cake, mix flour, eggs, cocoa powder, and sugar, then bake in the oven.",
"task: sentence similarity | query: Bonjour, le soleil brille aujourd'hui.",
]
def post(url, payload, log_name):
(ROOT / f"logs/{log_name}.request.json").write_text(json.dumps(payload, indent=2) + "\n")
start = time.monotonic()
request = urllib.request.Request(url, data=json.dumps(payload).encode(), headers={"Content-Type": "application/json"})
try:
with urllib.request.urlopen(request, timeout=240) as response:
status = response.status
body = response.read()
except urllib.error.HTTPError as error:
(ROOT / f"logs/{log_name}.error.txt").write_bytes(error.read())
raise
(ROOT / f"logs/{log_name}.response.json").write_bytes(body)
result = json.loads(body)
vectors = [item["embedding"] for item in sorted(result["data"], key=lambda x: x["index"])]
assert len(vectors) == len(payload["input"]), result
for vector in vectors:
assert len(vector) == 768, len(vector)
assert all(math.isfinite(x) for x in vector)
assert abs(norm(vector) - 1.0) < 1e-4, norm(vector)
return vectors, dict(http_status=status, elapsed_seconds=time.monotonic()-start,
vectors=len(vectors), dimensions=768, finite=True,
l2_norms=[norm(v) for v in vectors])
def norm(vector):
return math.sqrt(sum(x*x for x in vector))
def cosine(a, b):
return sum(x*y for x, y in zip(a, b)) / (norm(a)*norm(b))
def fixtures():
folder = ROOT / "work/fixtures"
folder.mkdir(parents=True, exist_ok=True)
def chunk(kind, data):
return struct.pack(">I", len(data)) + kind + data + struct.pack(">I", zlib.crc32(kind+data) & 0xffffffff)
# A simple deterministic raster fixture, not an evaluation dataset.
png = b"\x89PNG\r\n\x1a\n" + chunk(b"IHDR", struct.pack(">IIBBBBB", 224, 224, 8, 2, 0, 0, 0))
png += chunk(b"IDAT", zlib.compress((b"\x00" + bytes([230, 20, 20])*224)*224)) + chunk(b"IEND", b"")
image = folder / "red-square.png"
image.write_bytes(png)
audio = folder / "tone-440hz.wav"
samples = [int(8000*math.sin(2*math.pi*440*i/16000)) for i in range(32000)]
with wave.open(str(audio), "wb") as output:
output.setnchannels(1)
output.setsampwidth(2)
output.setframerate(16000)
output.writeframes(struct.pack("<"+"h"*len(samples), *samples))
return image, audio
def content_image(path):
return {"content": [{"type": "image_url", "image_url": {"url": "data:image/png;base64," + base64.b64encode(path.read_bytes()).decode()}}]}
def content_audio(path):
return {"content": [{"type": "input_audio", "input_audio": {"data": base64.b64encode(path.read_bytes()).decode(), "format": "wav"}}]}
|