File size: 8,018 Bytes

2616e2f

---
base_model:
- google/embeddinggemma-2
base_model_relation: quantized
license: apache-2.0
pipeline_tag: feature-extraction
library_name: coreml
tags:
- embeddings
- multimodal
- sentence-similarity
- retrieval
- matryoshka
- apple-silicon
- on-device
- coreml
- ios
- macos
- thestage-apple-sdk
---

# EmbeddingGemma 2 (Apple Silicon)

![On-Device Apple SDK](assets/on-device-apple-sdk-banner.png)


[TheStage Apple SDK](https://github.com/TheStageAI/AppleSDK)

**Original model:** [google/embeddinggemma-2](https://huggingface.co/google/embeddinggemma-2) (Google)

On-device **multimodal embeddings** compressed and packaged by **TheStage AI** for Apple Silicon (CoreML / Neural Engine). Text, images, audio, video, and mixed inputs (an image with its caption) map into **one 768-d, L2-normalized space**, so a typed query can find a paragraph, a photo, a sound or a clip. Matryoshka-trained: the first 512, 256 or 128 values are themselves embeddings.

| | |
| --- | --- |
| Runtime | TheStage Apple SDK (`EmbeddingPipeline`) |
| Embedding dim | **768** (L2-normalized); Matryoshka **512 / 256 / 128** |
| Inputs | text, image, audio, video, mixed |
| Default device | **NPU** (Neural Engine) |
| SDK | **1.7.0+** |
| HF engines | `TheStageAI/embeddinggemma-2` (tag `v1.7`) |


## Overview

---

This repo hosts **CoreML engine bundles** for the TheStage Apple SDK. Point `engines_path` at the repo id; the SDK downloads, caches, and loads the engines automatically. Task prompts, token limits and Matryoshka sizes all come from the bundle. Open only the inputs you use (`modalities:`) to shorten the first load.

## System Requirements

---

| **Property** | **Value** |
| --- | --- |
| **Hardware** | Apple Silicon Mac, or physical iPhone / iPad |
| **macOS** | 15.0+ |
| **iOS** | 18.0+ |
| **Xcode** | 16.0+ |
| **Swift** | 6.0+ |
| **Flutter** (optional) | 3.24+ |

> Simulator is **not** supported — run on real Apple Silicon hardware.


## TheStage AI Access Token

---

Create a token at [app.thestage.ai](https://app.thestage.ai) and pass it to the SDK:

```swift
try await TheStageAI.shared.initialize(api_token: "th_…")
```

```dart
await TheStageFlutterSDK.initialize(api_token: 'th_…');
```

Token is checked online in `initialize` (once per app process when reachable). Inference runs fully on-device. Offline initialize fails — reconnect and call `initialize` again.


## TheStage Apple SDK

---

Docs: [TheStage Apple SDK · Embeddings](https://docs.thestage.ai) · Repo: [TheStage Apple SDK](https://github.com/TheStageAI/AppleSDK)


### Installation (SwiftPM)

In Xcode: **File → Add Package Dependencies…**, paste `https://github.com/TheStageAI/AppleSDK.git`, and add the `TheStageSDK` product. Or in `Package.swift`:

```swift
.package(
    url: "https://github.com/TheStageAI/AppleSDK.git",
    exact: Version(1, 7, 0)
)
```

### Installation (Flutter / iOS)

```yaml
# pubspec.yaml
dependencies:
  thestage_apple_sdk:
    git:
      url: https://github.com/TheStageAI/AppleSDK.git
      path: plugin/thestage_apple_sdk
      ref: 1.7.0
```


### Swift

```swift
import TheStageSDK

try await TheStageAI.shared.initialize(api_token: "th_…")

let embedder = try await EmbeddingPipeline(
    engines_path: "TheStageAI/embeddinggemma-2",
    modalities: [.TEXT, .IMAGE]          // nil = every input kind
)

// Search: queries and documents use different task prompts
let query: EmbeddingResult = try await embedder.infer(text: "how to reset the router", task: .QUERY)
let doc: EmbeddingResult = try await embedder.infer(text: doc_text, task: .DOCUMENT)
let score: Float = try EmbeddingPipeline.similarity(query, doc)

// Other inputs — same space
let photo: EmbeddingResult = try await embedder.infer(image: cg_image)
let product: EmbeddingResult = try await embedder.infer(inputs: [.image(cg_image), .text("red sneakers")])
// with modalities [.AUDIO] / [.VIDEO]:
//   try await embedder.infer(audio: samples, sample_rate: 44_100)
//   try await embedder.infer(video: file_url, max_frames: 8)

// Smaller vectors
let small: EmbeddingResult = try await embedder.infer(text: "how to reset the router", task: .QUERY, dimension: 256)
```

### Flutter (iOS)

```dart
await TheStageFlutterSDK.start_model(
  model_name: 'embed',
  engines_path: 'TheStageAI/embeddinggemma-2',
);

final rows = await TheStageFlutterSDK.infer(
  model_name: 'embed',
  input_json: {'text': 'how to reset the router', 'task': 'query'},
);
final embedding = (rows[0]['embedding'] as List).cast<double>();

// Images, audio, video: pass a file path
final photo = await TheStageFlutterSDK.infer(
  model_name: 'embed',
  input_json: {'image_path': '/path/to/photo.jpg', 'dimension': 256},
);
```

### Inputs / outputs

| Direction | Swift | Flutter key | Notes |
| --- | --- | --- | --- |
| in | `infer(text:)` | `text` | Up to 2048 tokens; longer text is cut (`truncated`) |
| in | `infer(image:)` | `image_path` | Any size; 256–280 tokens per image |
| in | `infer(audio:sample_rate:)` | `audio_path` | Mono, any rate (resampled); 25 tokens/s, first 30 s (`truncated` past that) |
| in | `infer(video:max_frames:)` | `video_path`, `max_frames` | 1 frame/s, up to 14 frames (default), up to 140 tokens per frame |
| in | `infer(inputs:)` | several keys | One embedding for mixed parts (Flutter order: text, image, audio, video) |
| in | `task:` | `task` | `query`, `document`, `STS`, `Classification`, `Clustering`, `CodeRetrieval`, `QuestionAnswering`, `FactChecking` |
| in | `dimension:` | `dimension` | 768 (default), 512, 256, 128 |
| out | `embedding` | `embedding` | Unit-length vector |
| out | `dimension`, `embedding_space_id` | same | Compare only results with equal `embedding_space_id` |
| out | `token_count`, `truncated` | same | Input length read; whether anything was cut |

### Matryoshka sizes

Reference quality of the base model (Google model card, bf16):

| Size | MTEB eng v2 | MTEB Code | MIEB lite | MAEB | MMEB v2 |
| --- | --- | --- | --- | --- | --- |
| 768 | 68.46 | 78.68 | 65.67 | 49.87 | 59.01 |
| 512 | 68.41 | 77.24 | 65.34 | 49.76 | 58.38 |
| 256 | 67.78 | 76.18 | 64.16 | 49.13 | 56.24 |
| 128 | 65.68 | 71.41 | 59.97 | 47.28 | 45.65 |

256 is near-lossless for text; 128 costs noticeably more on images, audio and video. For large indexes, store 768-d vectors, shortlist on the first 256 values renormalized, and re-rank the shortlist on 768.

### On-device latency

Release build, Neural Engine, warmed, median of 5 (guidance, not an SLA):

| Input | Tokens | Apple M-series Mac | iPhone 13 |
| --- | --- | --- | --- |
| Text query | 16 | **8.1 ms** | **12.6 ms** |
| Text document | 368 | **47.6 ms** | **69.7 ms** |
| Image 768x768 | 260 | **439.5 ms** | **538.0 ms** |
| Audio 3 s | 79 | **137.7 ms** | **227.7 ms** |
| Audio 30 s | 754 | **236.6 ms** | **382.7 ms** |
| Video, 4 frames | 494 | **607.1 ms** | **757.8 ms** |
| Image + caption | 271 | **439.8 ms** | **538.4 ms** |

The first load on a device prepares every engine (about two minutes on a Mac, under four on an iPhone 13, with all four inputs); later loads take seconds (7.9 s on an iPhone 13). Memory stays under 200 MB once loaded on an iPhone 13. Always measure release builds on device.

## Acknowledgments

---

This work builds on **EmbeddingGemma 2** by Google: [google/embeddinggemma-2](https://huggingface.co/google/embeddinggemma-2).

The base model [google/embeddinggemma-2](https://huggingface.co/google/embeddinggemma-2) is released under Apache-2.0. This Apple Silicon package is produced by TheStage AI; it is not an official Google release.

## Links

---


* __Original model__: [google/embeddinggemma-2](https://huggingface.co/google/embeddinggemma-2)
* __Platform__: [app.thestage.ai](https://app.thestage.ai)
* __TheStage Apple SDK__: [github.com/TheStageAI/AppleSDK](https://github.com/TheStageAI/AppleSDK)
* __Docs__: [docs.thestage.ai](https://docs.thestage.ai)
* __Subscribe for updates__: [TheStageAI X](https://x.com/TheStageAI)
* __Contact email__: contact@thestage.ai