File size: 9,904 Bytes
61de3ef
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
---
license: apache-2.0
library_name: mediapipe
pipeline_tag: mask-generation
tags:
- mediapipe
- litert
- tflite
- mask-generation
- on-device
- android
- ios
- web
- python
- flutter
- interactive-segmentation
- promptable-segmentation
- magictouch
- int8
---

# MediaPipe MagicTouch (Interactive Segmenter)

Stateful point/stroke-prompted interactive segmentation bundle (`interactive_segmentation.task`, 30.5 MB, INT8 v2 encoder/decoder at `768 x 768`) that encodes an image once and iteratively segments user-indicated objects via positive/negative strokes in the [MediaPipe Interactive Segmenter (`InteractiveSegmenter`)](https://developers.google.com/edge/mediapipe/solutions/vision/interactive_segmenter) task.

<div style="border: 1px solid rgba(59, 130, 246, 0.28); background-color: rgba(59, 130, 246, 0.06); border-radius: 8px; padding: 12px 16px; margin: 16px 0;">
  <p style="margin: 0 0 6px 0;"><strong>Documentation:</strong> <a href="https://developers.google.com/edge/mediapipe/solutions/vision/interactive_segmenter">MediaPipe Interactive Segmenter Docs</a></p>
  <p style="margin: 0 0 6px 0;"><strong>Samples:</strong> <a href="https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/interactive_segmentation/android">Android</a> · <a href="https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/interactive_segmentation/ios">iOS</a> · <a href="https://colab.research.google.com/github/google-ai-edge/mediapipe-samples/blob/main/examples/interactive_segmentation/python/interactive_segmenter.ipynb">Python Colab</a> · <a href="https://github.com/google-ai-edge/mediapipe-samples-web/blob/main/src/tasks/interactive-segmenter.ts">Web</a> · <a href="https://github.com/google/flutter-mediapipe/tree/main/samples/vision/interactive_segmenter">Flutter</a></p>
  <p style="margin: 0;"><strong>Try live:</strong> <a href="https://google-ai-edge.github.io/mediapipe-samples-web/#/vision/interactive_segmenter">Web Demo</a></p>
</div>

`interactive_segmentation.task` (`interactive_segmenter_v2`) separates image feature extraction from interactive decoding for low-latency refinement:

1. **`interactive_segmentation_encoder.int8.tflite` (28.3 MB):** Runs once in `set_image()`, encoding `[-1, 768, 768, 3]` RGB images into 5 multi-scale feature maps (`384×384×16` .. `24×24×192`).
2. **`interactive_segmentation_decoder.int8.tflite` (2.3 MB):** Runs per user stroke call in `segment(strokes)`, combining the prompt canvas (`[1, 768, 768, 3]`) and cached encoder features into a `[1, 768, 768, 1]` mask.

**Bundle version note:** The bundled [`Model Card MagicTouch.pdf`](./Model%20Card%20MagicTouch.pdf) (March 13, 2023) documents the earlier v1 single-graph `512 x 512 x 4` (≈6 MB) FP model (≈30 FPS on Pixel 7 GPU), whereas this `.task` bundle packages the upgraded **768×768 INT8 two-stage v2** encoder/decoder.

## Files

| File | Precision | Size | SHA-256 |
|---|---|---:|---|
| [`interactive_segmentation.task`](https://huggingface.co/litert-community/MediaPipe-InteractiveSegmenter-MagicTouch/blob/main/interactive_segmentation.task) | `int8` | 30.53 MB (30,525,312 B) | `38431bc66b883404e8397f74c3579404315b9b52b04a46c6346fe906a7309b03` |

### Companion Files in Repo Root

| File | Role | Size | SHA-256 |
|---|---|---:|---|
| [`Model Card MagicTouch.pdf`](./Model%20Card%20MagicTouch.pdf) | Upstream Google model card (PDF) | 301.8 KB (301,841 B) | `b4accf5f7be991903646e6dd9765faa6974cdf69c5a9e84841684dffa27df0f1` |

### Download

```bash
pip install -U huggingface_hub
hf download litert-community/MediaPipe-InteractiveSegmenter-MagicTouch interactive_segmentation.task --local-dir .
```

## Inputs, Outputs, and Model Specifications

### Bundle Contents (`interactive_segmentation.task`)

| Bundled Member | Uncompressed Size | SHA-256 (prefix) |
|---|---:|---|
| `interactive_segmentation_decoder.int8.tflite` | 2.27 MB (2,268,000 B) | `140d36971403c38b…` |
| `interactive_segmentation_encoder.int8.tflite` | 28.26 MB (28,256,696 B) | `0edbceec02cb1570…` |
| `manifest.pb` | 0.1 KB (50 B) | `c18392128d95dc4d…` |
| `metadata.pb` | 0.0 KB (12 B) | `d4945e22f0e49ea2…` |

### Sub-model `interactive_segmentation_decoder.int8.tflite`

| Direction | Tensor | Shape | Type | Notes |
|---|---|---|---|---|
| Input | `input_21` | `[1, 768, 768, 3]` | `float32` | — |
| Input | `input_22` | `[1, 384, 384, 16]` | `float32` | — |
| Input | `input_23` | `[1, 192, 192, 24]` | `float32` | — |
| Input | `input_24` | `[1, 96, 96, 40]` | `float32` | — |
| Input | `input_25` | `[1, 48, 48, 80]` | `float32` | — |
| Input | `input_26` | `[1, 24, 24, 192]` | `float32` | — |
| Output | `Identity` | `[1, 768, 768, 1]` | `float32` | — |

### Sub-model `interactive_segmentation_encoder.int8.tflite`

| Direction | Tensor | Shape | Type | Notes |
|---|---|---|---|---|
| Input | `input_1` | `[-1, 768, 768, 3]` | `float32` | — |
| Output | `Identity` | `[-1, 384, 384, 16]` | `float32` | — |
| Output | `Identity_1` | `[-1, 192, 192, 24]` | `float32` | — |
| Output | `Identity_2` | `[-1, 96, 96, 40]` | `float32` | — |
| Output | `Identity_3` | `[-1, 48, 48, 80]` | `float32` | — |
| Output | `Identity_4` | `[-1, 24, 24, 192]` | `float32` | — |

## Use with MediaPipe Tasks

### Python Quickstart

```bash
pip install -U mediapipe huggingface_hub
```

```python
import mediapipe as mp
from huggingface_hub import hf_hub_download

model_path = hf_hub_download(repo_id="litert-community/MediaPipe-InteractiveSegmenter-MagicTouch", filename="interactive_segmentation.task")

BaseOptions = mp.tasks.BaseOptions
InteractiveSegmenter = mp.tasks.vision.InteractiveSegmenter
InteractiveSegmenterOptions = mp.tasks.vision.InteractiveSegmenterOptions
BrushMode = mp.tasks.vision.InteractiveSegmenterBrushMode
Stroke = mp.tasks.vision.InteractiveSegmenterStroke
StrokePoint = mp.tasks.vision.InteractiveSegmenterStrokePoint

options = InteractiveSegmenterOptions(
    base_options=BaseOptions(model_asset_path=model_path)
)

with InteractiveSegmenter.create_from_options(options) as segmenter:
    segmenter.set_image(mp.Image.create_from_file("input.jpg"))
    strokes = [
        Stroke(brush_mode=BrushMode.POSITIVE, points=[StrokePoint(x=0.5, y=0.5)], is_completed=True)
    ]
    mask = segmenter.segment(strokes)
    print("Segmented mask size:", mask.width, "x", mask.height)
```

### Cross-Platform Guides & Samples

- **Task overview & options:** [Interactive Segmenter documentation](https://developers.google.com/edge/mediapipe/solutions/vision/interactive_segmenter)
- **Platform setup guides:** [Python](https://developers.google.com/edge/mediapipe/solutions/setup_python) · [Web / JavaScript](https://developers.google.com/edge/mediapipe/solutions/setup_web) · [Android (Kotlin / Java)](https://developers.google.com/edge/mediapipe/solutions/setup_android) · [iOS (Swift)](https://developers.google.com/edge/mediapipe/solutions/setup_ios) · [Flutter (Dart)](https://developers.google.com/edge/mediapipe/solutions/setup_flutter)
- **Runnable sample apps:** [Web Demo](https://google-ai-edge.github.io/mediapipe-samples-web/#/vision/interactive_segmenter) · [Web Source](https://github.com/google-ai-edge/mediapipe-samples-web/blob/main/src/tasks/interactive-segmenter.ts) · [Android](https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/interactive_segmentation/android) · [iOS](https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/interactive_segmentation/ios) · [Python Notebook](https://colab.research.google.com/github/google-ai-edge/mediapipe-samples/blob/main/examples/interactive_segmentation/python/interactive_segmenter.ipynb) · [Flutter](https://github.com/google/flutter-mediapipe/tree/main/samples/vision/interactive_segmenter)

## Latency Benchmarks

End-to-end MediaPipe Task pipeline average latency reported by the [Interactive Segmenter documentation](https://developers.google.com/edge/mediapipe/solutions/vision/interactive_segmenter#benchmarks):

| Model Variant | Device | CPU Latency | GPU Latency |
|---|---|---:|---:|
| MagicTouch (v2 int8) | Pixel 10 | 208.2 ms | 580.6 ms |

## Intended Use, Limitations, and Fairness

Summarized from the bundled [`Model Card MagicTouch.pdf`](./Model%20Card%20MagicTouch.pdf) (Valentin Bazarevsky & Ben Hahn, Google, March 13, 2023):

- **Intended applications:** Interactive photo editing and semi-automatic object annotation (objects, animals, humans) from user clicks or brush strokes.
- **Out-of-scope uses:**
  - **Any form of surveillance or identity recognition is explicitly out of scope and not enabled by this technology.**
  - Human segmentation not consented to by the subject being segmented.
  - Human life-critical decisions, or small objects / people > 4 meters (14 ft) from the camera.
- **Fairness evaluation (v1 MagicTouch PDF summary, 1,870 smartphone AR images vs. 98.74% human inter-annotator IoU):**
  - **Monk Skin Tones 1–10:** average IoU **87.7%** (range **85.3%** Tones 8–10 to **89.4%** Tone 5; max discrepancy 4.1%).
  - **Perceived gender:** average IoU **87.7%** (Male **87.3%**, Female **88.0%**; max discrepancy 0.7%).

## Provenance

Every model binary in this repository is byte-identical (unmodified weights, metadata, and `.task` bundle archive bytes) to the official artifact served by Google from `storage.googleapis.com/mediapipe-models`.

| Repo File | Upstream URL | GCS Generation | Upstream `Last-Modified` | Pinned Bucket Version |
|---|---|---|---|---|
| `interactive_segmentation.task` | [`interactive_segmentation.task`](https://storage.googleapis.com/mediapipe-models/interactive_segmenter_v2/magic_touch/int8/latest/interactive_segmentation.task) | `1781216228896365` | Thu, 11 Jun 2026 22:17:08 GMT | `/1/` |

## License and Citation

- License: `apache-2.0` — Stated as Apache License, Version 2.0 in the bundled upstream Google model-card PDF.