File size: 2,758 Bytes
8e05986 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 | ---
license: apache-2.0
base_model:
- google/embeddinggemma-2
pipeline_tag: feature-extraction
tags:
- radiance
- rocm
- amd
- rdna4
- bf16
- embedding
- multimodal-embedding
- sentence-similarity
---
# EmbeddingGemma 2 BF16 — radiance container
[google/embeddinggemma-2](https://huggingface.co/google/embeddinggemma-2) in bf16, exactly as the
checkpoint ships it, as a single `.rad` container for the
[radiance](https://codeberg.org/StillDeadcode/radiance) inference engine. It embeds text, images, video
and audio on an AMD card (RDNA4, ROCm) or on the CPU.
| | |
|---|---|
| File | `embeddinggemma2-bf16.rad` — 1.40 GiB |
| Weights | every weight of the model, the vision and audio towers included, bf16 as the checkpoint ships it |
| Output | 768-dimensional vectors, mean-pooled and normalised as the model's sentence-transformers configuration states |
| Modalities | text, image, video, audio |
| Needs | radiance 1.5.0 or newer |
## Serve
```sh
radiance --model embeddinggemma2-bf16.rad --host 0.0.0.0 --port 8000
```
With no card visible, radiance serves it on the CPU. The Docker image is `stilldeadcode/radiance`.
Requests go to `/v1/embeddings`. `input` is a string, a list of strings, or content parts:
```json
{"input": [{"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}},
{"type": "text", "text": "a red square"}]}
```
Video is a `video_url` part, and audio an `input_audio` part (`{"data": <base64 wav>, "format": "wav"}`).
`prompt_name` applies one of the model's named prompts (`query`, `document`, `Retrieval-query`, ...),
`dimensions` truncates the vector and normalises it again, and `image_tokens` (70, 140, 280, 560 or 1120) sets an image's
token budget.
## Accuracy
Cosine similarity against the Hugging Face transformers reference in bf16, served by radiance 1.5.0:
| input | R9700 | CPU |
|---|---|---|
| text, 4 to 6,704 tokens | 0.99992 – 0.99997 | 0.99990 – 0.99998 |
| images | 0.99998 – 0.99999 | 0.99998 – 0.99999 |
| video | 0.99997 | 0.99997 |
| audio (7 s, 30 s) | 0.99992 – 0.99993 | 0.99974 (7 s) |
| text + image | 0.99997 | 0.99996 |
A text that itself begins with the literal string `<bos>` scores 0.998: radiance does not add a second
`<bos>` in front of it, where the reference does.
## Speed
On one Radeon AI PRO R9700 (gfx1201): about 148,000 text tokens a second in batches; 27 images a second
one at a time and 52 at eight in parallel; a 7 s audio clip in 20 ms and a 30 s clip in 49 ms. On a
Ryzen 9 9950X3D: about 12,000 text tokens a second, 2.3 images a second, and a 30 s audio clip in
265 ms.
## How this file was made
```sh
rad-convert google/embeddinggemma-2 -o embeddinggemma2-bf16.rad
```
No recipe: every weight is the checkpoint's own.
|