Mask Generation
LiteRT
LiteRT
mediapipe
on-device
android
ios
web
python
flutter
interactive-segmentation
promptable-segmentation
magictouch
int8
Instructions to use litert-community/MediaPipe-InteractiveSegmenter-MagicTouch with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- LiteRT
How to use litert-community/MediaPipe-InteractiveSegmenter-MagicTouch with LiteRT:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
File size: 9,904 Bytes
61de3ef | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 | ---
license: apache-2.0
library_name: mediapipe
pipeline_tag: mask-generation
tags:
- mediapipe
- litert
- tflite
- mask-generation
- on-device
- android
- ios
- web
- python
- flutter
- interactive-segmentation
- promptable-segmentation
- magictouch
- int8
---
# MediaPipe MagicTouch (Interactive Segmenter)
Stateful point/stroke-prompted interactive segmentation bundle (`interactive_segmentation.task`, 30.5 MB, INT8 v2 encoder/decoder at `768 x 768`) that encodes an image once and iteratively segments user-indicated objects via positive/negative strokes in the [MediaPipe Interactive Segmenter (`InteractiveSegmenter`)](https://developers.google.com/edge/mediapipe/solutions/vision/interactive_segmenter) task.
<div style="border: 1px solid rgba(59, 130, 246, 0.28); background-color: rgba(59, 130, 246, 0.06); border-radius: 8px; padding: 12px 16px; margin: 16px 0;">
<p style="margin: 0 0 6px 0;"><strong>Documentation:</strong> <a href="https://developers.google.com/edge/mediapipe/solutions/vision/interactive_segmenter">MediaPipe Interactive Segmenter Docs</a></p>
<p style="margin: 0 0 6px 0;"><strong>Samples:</strong> <a href="https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/interactive_segmentation/android">Android</a> · <a href="https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/interactive_segmentation/ios">iOS</a> · <a href="https://colab.research.google.com/github/google-ai-edge/mediapipe-samples/blob/main/examples/interactive_segmentation/python/interactive_segmenter.ipynb">Python Colab</a> · <a href="https://github.com/google-ai-edge/mediapipe-samples-web/blob/main/src/tasks/interactive-segmenter.ts">Web</a> · <a href="https://github.com/google/flutter-mediapipe/tree/main/samples/vision/interactive_segmenter">Flutter</a></p>
<p style="margin: 0;"><strong>Try live:</strong> <a href="https://google-ai-edge.github.io/mediapipe-samples-web/#/vision/interactive_segmenter">Web Demo</a></p>
</div>
`interactive_segmentation.task` (`interactive_segmenter_v2`) separates image feature extraction from interactive decoding for low-latency refinement:
1. **`interactive_segmentation_encoder.int8.tflite` (28.3 MB):** Runs once in `set_image()`, encoding `[-1, 768, 768, 3]` RGB images into 5 multi-scale feature maps (`384×384×16` .. `24×24×192`).
2. **`interactive_segmentation_decoder.int8.tflite` (2.3 MB):** Runs per user stroke call in `segment(strokes)`, combining the prompt canvas (`[1, 768, 768, 3]`) and cached encoder features into a `[1, 768, 768, 1]` mask.
**Bundle version note:** The bundled [`Model Card MagicTouch.pdf`](./Model%20Card%20MagicTouch.pdf) (March 13, 2023) documents the earlier v1 single-graph `512 x 512 x 4` (≈6 MB) FP model (≈30 FPS on Pixel 7 GPU), whereas this `.task` bundle packages the upgraded **768×768 INT8 two-stage v2** encoder/decoder.
## Files
| File | Precision | Size | SHA-256 |
|---|---|---:|---|
| [`interactive_segmentation.task`](https://huggingface.co/litert-community/MediaPipe-InteractiveSegmenter-MagicTouch/blob/main/interactive_segmentation.task) | `int8` | 30.53 MB (30,525,312 B) | `38431bc66b883404e8397f74c3579404315b9b52b04a46c6346fe906a7309b03` |
### Companion Files in Repo Root
| File | Role | Size | SHA-256 |
|---|---|---:|---|
| [`Model Card MagicTouch.pdf`](./Model%20Card%20MagicTouch.pdf) | Upstream Google model card (PDF) | 301.8 KB (301,841 B) | `b4accf5f7be991903646e6dd9765faa6974cdf69c5a9e84841684dffa27df0f1` |
### Download
```bash
pip install -U huggingface_hub
hf download litert-community/MediaPipe-InteractiveSegmenter-MagicTouch interactive_segmentation.task --local-dir .
```
## Inputs, Outputs, and Model Specifications
### Bundle Contents (`interactive_segmentation.task`)
| Bundled Member | Uncompressed Size | SHA-256 (prefix) |
|---|---:|---|
| `interactive_segmentation_decoder.int8.tflite` | 2.27 MB (2,268,000 B) | `140d36971403c38b…` |
| `interactive_segmentation_encoder.int8.tflite` | 28.26 MB (28,256,696 B) | `0edbceec02cb1570…` |
| `manifest.pb` | 0.1 KB (50 B) | `c18392128d95dc4d…` |
| `metadata.pb` | 0.0 KB (12 B) | `d4945e22f0e49ea2…` |
### Sub-model `interactive_segmentation_decoder.int8.tflite`
| Direction | Tensor | Shape | Type | Notes |
|---|---|---|---|---|
| Input | `input_21` | `[1, 768, 768, 3]` | `float32` | — |
| Input | `input_22` | `[1, 384, 384, 16]` | `float32` | — |
| Input | `input_23` | `[1, 192, 192, 24]` | `float32` | — |
| Input | `input_24` | `[1, 96, 96, 40]` | `float32` | — |
| Input | `input_25` | `[1, 48, 48, 80]` | `float32` | — |
| Input | `input_26` | `[1, 24, 24, 192]` | `float32` | — |
| Output | `Identity` | `[1, 768, 768, 1]` | `float32` | — |
### Sub-model `interactive_segmentation_encoder.int8.tflite`
| Direction | Tensor | Shape | Type | Notes |
|---|---|---|---|---|
| Input | `input_1` | `[-1, 768, 768, 3]` | `float32` | — |
| Output | `Identity` | `[-1, 384, 384, 16]` | `float32` | — |
| Output | `Identity_1` | `[-1, 192, 192, 24]` | `float32` | — |
| Output | `Identity_2` | `[-1, 96, 96, 40]` | `float32` | — |
| Output | `Identity_3` | `[-1, 48, 48, 80]` | `float32` | — |
| Output | `Identity_4` | `[-1, 24, 24, 192]` | `float32` | — |
## Use with MediaPipe Tasks
### Python Quickstart
```bash
pip install -U mediapipe huggingface_hub
```
```python
import mediapipe as mp
from huggingface_hub import hf_hub_download
model_path = hf_hub_download(repo_id="litert-community/MediaPipe-InteractiveSegmenter-MagicTouch", filename="interactive_segmentation.task")
BaseOptions = mp.tasks.BaseOptions
InteractiveSegmenter = mp.tasks.vision.InteractiveSegmenter
InteractiveSegmenterOptions = mp.tasks.vision.InteractiveSegmenterOptions
BrushMode = mp.tasks.vision.InteractiveSegmenterBrushMode
Stroke = mp.tasks.vision.InteractiveSegmenterStroke
StrokePoint = mp.tasks.vision.InteractiveSegmenterStrokePoint
options = InteractiveSegmenterOptions(
base_options=BaseOptions(model_asset_path=model_path)
)
with InteractiveSegmenter.create_from_options(options) as segmenter:
segmenter.set_image(mp.Image.create_from_file("input.jpg"))
strokes = [
Stroke(brush_mode=BrushMode.POSITIVE, points=[StrokePoint(x=0.5, y=0.5)], is_completed=True)
]
mask = segmenter.segment(strokes)
print("Segmented mask size:", mask.width, "x", mask.height)
```
### Cross-Platform Guides & Samples
- **Task overview & options:** [Interactive Segmenter documentation](https://developers.google.com/edge/mediapipe/solutions/vision/interactive_segmenter)
- **Platform setup guides:** [Python](https://developers.google.com/edge/mediapipe/solutions/setup_python) · [Web / JavaScript](https://developers.google.com/edge/mediapipe/solutions/setup_web) · [Android (Kotlin / Java)](https://developers.google.com/edge/mediapipe/solutions/setup_android) · [iOS (Swift)](https://developers.google.com/edge/mediapipe/solutions/setup_ios) · [Flutter (Dart)](https://developers.google.com/edge/mediapipe/solutions/setup_flutter)
- **Runnable sample apps:** [Web Demo](https://google-ai-edge.github.io/mediapipe-samples-web/#/vision/interactive_segmenter) · [Web Source](https://github.com/google-ai-edge/mediapipe-samples-web/blob/main/src/tasks/interactive-segmenter.ts) · [Android](https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/interactive_segmentation/android) · [iOS](https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/interactive_segmentation/ios) · [Python Notebook](https://colab.research.google.com/github/google-ai-edge/mediapipe-samples/blob/main/examples/interactive_segmentation/python/interactive_segmenter.ipynb) · [Flutter](https://github.com/google/flutter-mediapipe/tree/main/samples/vision/interactive_segmenter)
## Latency Benchmarks
End-to-end MediaPipe Task pipeline average latency reported by the [Interactive Segmenter documentation](https://developers.google.com/edge/mediapipe/solutions/vision/interactive_segmenter#benchmarks):
| Model Variant | Device | CPU Latency | GPU Latency |
|---|---|---:|---:|
| MagicTouch (v2 int8) | Pixel 10 | 208.2 ms | 580.6 ms |
## Intended Use, Limitations, and Fairness
Summarized from the bundled [`Model Card MagicTouch.pdf`](./Model%20Card%20MagicTouch.pdf) (Valentin Bazarevsky & Ben Hahn, Google, March 13, 2023):
- **Intended applications:** Interactive photo editing and semi-automatic object annotation (objects, animals, humans) from user clicks or brush strokes.
- **Out-of-scope uses:**
- **Any form of surveillance or identity recognition is explicitly out of scope and not enabled by this technology.**
- Human segmentation not consented to by the subject being segmented.
- Human life-critical decisions, or small objects / people > 4 meters (14 ft) from the camera.
- **Fairness evaluation (v1 MagicTouch PDF summary, 1,870 smartphone AR images vs. 98.74% human inter-annotator IoU):**
- **Monk Skin Tones 1–10:** average IoU **87.7%** (range **85.3%** Tones 8–10 to **89.4%** Tone 5; max discrepancy 4.1%).
- **Perceived gender:** average IoU **87.7%** (Male **87.3%**, Female **88.0%**; max discrepancy 0.7%).
## Provenance
Every model binary in this repository is byte-identical (unmodified weights, metadata, and `.task` bundle archive bytes) to the official artifact served by Google from `storage.googleapis.com/mediapipe-models`.
| Repo File | Upstream URL | GCS Generation | Upstream `Last-Modified` | Pinned Bucket Version |
|---|---|---|---|---|
| `interactive_segmentation.task` | [`interactive_segmentation.task`](https://storage.googleapis.com/mediapipe-models/interactive_segmenter_v2/magic_touch/int8/latest/interactive_segmentation.task) | `1781216228896365` | Thu, 11 Jun 2026 22:17:08 GMT | `/1/` |
## License and Citation
- License: `apache-2.0` — Stated as Apache License, Version 2.0 in the bundled upstream Google model-card PDF.
|