Image Feature Extraction
LiteRT
LiteRT
mediapipe
on-device
android
ios
web
python
flutter
vision
mobilenet-v3
image-embeddings
fp32
Instructions to use litert-community/MediaPipe-ImageEmbedder-MobileNet-V3-Large with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- LiteRT
How to use litert-community/MediaPipe-ImageEmbedder-MobileNet-V3-Large with LiteRT:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Commit ·
c9f46d7
0
Parent(s):
Initial upload of MediaPipe-ImageEmbedder-MobileNet-V3-Large from MediaPipe Solutions
Browse files- .gitattributes +38 -0
- README.md +133 -0
- mobilenet_v3_large.tflite +3 -0
.gitattributes
ADDED
|
@@ -0,0 +1,38 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
*.7z filter=lfs diff=lfs merge=lfs -text
|
| 2 |
+
*.arrow filter=lfs diff=lfs merge=lfs -text
|
| 3 |
+
*.bin filter=lfs diff=lfs merge=lfs -text
|
| 4 |
+
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
| 5 |
+
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
| 6 |
+
*.ftz filter=lfs diff=lfs merge=lfs -text
|
| 7 |
+
*.gz filter=lfs diff=lfs merge=lfs -text
|
| 8 |
+
*.h5 filter=lfs diff=lfs merge=lfs -text
|
| 9 |
+
*.joblib filter=lfs diff=lfs merge=lfs -text
|
| 10 |
+
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
| 11 |
+
*.litertlm filter=lfs diff=lfs merge=lfs -text
|
| 12 |
+
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
| 13 |
+
*.model filter=lfs diff=lfs merge=lfs -text
|
| 14 |
+
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
| 15 |
+
*.npy filter=lfs diff=lfs merge=lfs -text
|
| 16 |
+
*.npz filter=lfs diff=lfs merge=lfs -text
|
| 17 |
+
*.onnx filter=lfs diff=lfs merge=lfs -text
|
| 18 |
+
*.ot filter=lfs diff=lfs merge=lfs -text
|
| 19 |
+
*.parquet filter=lfs diff=lfs merge=lfs -text
|
| 20 |
+
*.pb filter=lfs diff=lfs merge=lfs -text
|
| 21 |
+
*.pdf filter=lfs diff=lfs merge=lfs -text
|
| 22 |
+
*.pickle filter=lfs diff=lfs merge=lfs -text
|
| 23 |
+
*.pkl filter=lfs diff=lfs merge=lfs -text
|
| 24 |
+
*.pt filter=lfs diff=lfs merge=lfs -text
|
| 25 |
+
*.pth filter=lfs diff=lfs merge=lfs -text
|
| 26 |
+
*.rar filter=lfs diff=lfs merge=lfs -text
|
| 27 |
+
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
| 28 |
+
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
| 29 |
+
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
| 30 |
+
*.tar filter=lfs diff=lfs merge=lfs -text
|
| 31 |
+
*.task filter=lfs diff=lfs merge=lfs -text
|
| 32 |
+
*.tflite filter=lfs diff=lfs merge=lfs -text
|
| 33 |
+
*.tgz filter=lfs diff=lfs merge=lfs -text
|
| 34 |
+
*.wasm filter=lfs diff=lfs merge=lfs -text
|
| 35 |
+
*.xz filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 38 |
+
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
README.md
ADDED
|
@@ -0,0 +1,133 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
library_name: litert
|
| 4 |
+
pipeline_tag: image-feature-extraction
|
| 5 |
+
datasets:
|
| 6 |
+
- imagenet-1k
|
| 7 |
+
tags:
|
| 8 |
+
- mediapipe
|
| 9 |
+
- litert
|
| 10 |
+
- tflite
|
| 11 |
+
- image-feature-extraction
|
| 12 |
+
- on-device
|
| 13 |
+
- android
|
| 14 |
+
- ios
|
| 15 |
+
- web
|
| 16 |
+
- python
|
| 17 |
+
- flutter
|
| 18 |
+
- vision
|
| 19 |
+
- mobilenet-v3
|
| 20 |
+
- image-embeddings
|
| 21 |
+
- fp32
|
| 22 |
+
- arxiv:1905.02244
|
| 23 |
+
---
|
| 24 |
+
|
| 25 |
+
# MediaPipe MobileNet-V3 (Large) Image Embedder
|
| 26 |
+
|
| 27 |
+
MobileNet-V3 Large image feature extractor (`mobilenet_v3_large.tflite`, 10.9 MB, 0.75 depth multiplier, FP32 weights) producing **1280-dimensional** embedding vectors for visual similarity search and deduplication in the [MediaPipe Image Embedder (`ImageEmbedder`)](https://developers.google.com/edge/mediapipe/solutions/vision/image_embedder) task.
|
| 28 |
+
|
| 29 |
+
<div style="border: 1px solid rgba(59, 130, 246, 0.28); background-color: rgba(59, 130, 246, 0.06); border-radius: 8px; padding: 12px 16px; margin: 16px 0;">
|
| 30 |
+
<p style="margin: 0 0 6px 0;"><strong>Documentation:</strong> <a href="https://developers.google.com/edge/mediapipe/solutions/vision/image_embedder">MediaPipe Image Embedder Docs</a></p>
|
| 31 |
+
<p style="margin: 0 0 6px 0;"><strong>Samples:</strong> <a href="https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/image_embedder/android">Android</a> · <a href="https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/image_embedder/ios">iOS</a> · <a href="https://colab.research.google.com/github/google-ai-edge/mediapipe-samples/blob/main/examples/image_embedder/python/image_embedder.ipynb">Python Colab</a> · <a href="https://github.com/google-ai-edge/mediapipe-samples-web/blob/main/src/tasks/image-embedder.ts">Web</a> · <a href="https://github.com/google/flutter-mediapipe/tree/main/samples/vision/image_embedder">Flutter</a></p>
|
| 32 |
+
<p style="margin: 0;"><strong>Try live:</strong> <a href="https://google-ai-edge.github.io/mediapipe-samples-web/#/vision/image_embedder">Web Demo</a></p>
|
| 33 |
+
</div>
|
| 34 |
+
|
| 35 |
+
Trained on ImageNet with a `0.75` channel depth multiplier to balance accuracy and edge latency. Takes normalized `224 x 224` RGB images and outputs a `1280`-element feature vector (`StatefulPartitionedCall:0`), optionally L2-normalized (`l2_normalize=True`) and scalar-quantized (`quantize=True`) by MediaPipe `ImageEmbedder`.
|
| 36 |
+
|
| 37 |
+
## Files
|
| 38 |
+
|
| 39 |
+
| File | Precision | Size | SHA-256 |
|
| 40 |
+
|---|---|---:|---|
|
| 41 |
+
| [`mobilenet_v3_large.tflite`](https://huggingface.co/litert-community/MediaPipe-ImageEmbedder-MobileNet-V3-Large/blob/main/mobilenet_v3_large.tflite) | `float32` | 10.89 MB (10,889,458 B) | `11af3c560dfeed7737cb4c03c23bf52a8403020784192d4dea0b74862a12828d` |
|
| 42 |
+
|
| 43 |
+
### Download
|
| 44 |
+
|
| 45 |
+
```bash
|
| 46 |
+
pip install -U huggingface_hub
|
| 47 |
+
hf download litert-community/MediaPipe-ImageEmbedder-MobileNet-V3-Large mobilenet_v3_large.tflite --local-dir .
|
| 48 |
+
```
|
| 49 |
+
|
| 50 |
+
## Inputs, Outputs, and Model Specifications
|
| 51 |
+
|
| 52 |
+
### `mobilenet_v3_large.tflite`
|
| 53 |
+
|
| 54 |
+
- Embedded `TFLITE_METADATA`: name `MobileNetV3 image classifier` · version `1` · author `MediaPipe` · packed associated files: `labels.txt`
|
| 55 |
+
|
| 56 |
+
| Direction | Tensor | Shape | Type | Notes |
|
| 57 |
+
|---|---|---|---|---|
|
| 58 |
+
| Input | `serving_default_inputs:0` | `[-1, 224, 224, 3]` | `float32` | image — Input image to be embedded. The expected image is 224 x 224, with three channels (red, blue, and green) per pixel. Each element in the tensor is a value between min and max, where (per-channel) min is [0.0] and max is [1.0]. — Normalization: `mean=[0.0], std=[255.0]` |
|
| 59 |
+
| Output | `StatefulPartitionedCall:0` | `[-1, 1280]` | `float32` | feature — Feature vector of the input image. |
|
| 60 |
+
|
| 61 |
+
## Use with MediaPipe Tasks
|
| 62 |
+
|
| 63 |
+
### Python Quickstart
|
| 64 |
+
|
| 65 |
+
```bash
|
| 66 |
+
pip install -U mediapipe huggingface_hub
|
| 67 |
+
```
|
| 68 |
+
|
| 69 |
+
```python
|
| 70 |
+
import mediapipe as mp
|
| 71 |
+
from huggingface_hub import hf_hub_download
|
| 72 |
+
|
| 73 |
+
model_path = hf_hub_download(repo_id="litert-community/MediaPipe-ImageEmbedder-MobileNet-V3-Large", filename="mobilenet_v3_large.tflite")
|
| 74 |
+
|
| 75 |
+
BaseOptions = mp.tasks.BaseOptions
|
| 76 |
+
ImageEmbedder = mp.tasks.vision.ImageEmbedder
|
| 77 |
+
ImageEmbedderOptions = mp.tasks.vision.ImageEmbedderOptions
|
| 78 |
+
VisionRunningMode = mp.tasks.vision.RunningMode
|
| 79 |
+
|
| 80 |
+
options = ImageEmbedderOptions(
|
| 81 |
+
base_options=BaseOptions(model_asset_path=model_path),
|
| 82 |
+
running_mode=VisionRunningMode.IMAGE,
|
| 83 |
+
l2_normalize=True,
|
| 84 |
+
)
|
| 85 |
+
|
| 86 |
+
with ImageEmbedder.create_from_options(options) as embedder:
|
| 87 |
+
e1 = embedder.embed(mp.Image.create_from_file("input.jpg")).embeddings[0]
|
| 88 |
+
e2 = embedder.embed(mp.Image.create_from_file("other.jpg")).embeddings[0]
|
| 89 |
+
print("Cosine similarity:", ImageEmbedder.cosine_similarity(e1, e2))
|
| 90 |
+
```
|
| 91 |
+
|
| 92 |
+
### Cross-Platform Guides & Samples
|
| 93 |
+
|
| 94 |
+
- **Task overview & options:** [Image Embedder documentation](https://developers.google.com/edge/mediapipe/solutions/vision/image_embedder)
|
| 95 |
+
- **Platform setup guides:** [Python](https://developers.google.com/edge/mediapipe/solutions/setup_python) · [Web / JavaScript](https://developers.google.com/edge/mediapipe/solutions/setup_web) · [Android (Kotlin / Java)](https://developers.google.com/edge/mediapipe/solutions/setup_android) · [iOS (Swift)](https://developers.google.com/edge/mediapipe/solutions/setup_ios) · [Flutter (Dart)](https://developers.google.com/edge/mediapipe/solutions/setup_flutter)
|
| 96 |
+
- **Runnable sample apps:** [Web Demo](https://google-ai-edge.github.io/mediapipe-samples-web/#/vision/image_embedder) · [Web Source](https://github.com/google-ai-edge/mediapipe-samples-web/blob/main/src/tasks/image-embedder.ts) · [Android](https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/image_embedder/android) · [iOS](https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/image_embedder/ios) · [Python Notebook](https://colab.research.google.com/github/google-ai-edge/mediapipe-samples/blob/main/examples/image_embedder/python/image_embedder.ipynb) · [Flutter](https://github.com/google/flutter-mediapipe/tree/main/samples/vision/image_embedder)
|
| 97 |
+
|
| 98 |
+
## Latency Benchmarks
|
| 99 |
+
|
| 100 |
+
End-to-end MediaPipe Task pipeline average latency reported by the [Image Embedder documentation](https://developers.google.com/edge/mediapipe/solutions/vision/image_embedder#benchmarks):
|
| 101 |
+
|
| 102 |
+
| Model Variant | Device | CPU Latency | GPU Latency |
|
| 103 |
+
|---|---|---:|---:|
|
| 104 |
+
| MobileNet-V3 (large) | Pixel 6 | 9.75 ms | 9.08 ms |
|
| 105 |
+
|
| 106 |
+
## Intended Use, Limitations, and Fairness
|
| 107 |
+
|
| 108 |
+
- **Intended applications:** Generating dense feature vectors for on-device image similarity comparison, visual search, clustering, and near-duplicate detection via cosine similarity.
|
| 109 |
+
- **Limitations & considerations:** Pretrained on ImageNet-1K natural photographs; fine-grained domains (medical scans, OCR, specialized industrial parts) or biometric matching are outside its training distribution. No standalone Google fairness PDF is published for this model on the MediaPipe documentation page.
|
| 110 |
+
|
| 111 |
+
## Provenance
|
| 112 |
+
|
| 113 |
+
Every model binary in this repository is byte-identical (unmodified FlatBuffer weights and embedded `TFLITE_METADATA` bytes) to the official artifact served by Google from `storage.googleapis.com/mediapipe-models`.
|
| 114 |
+
|
| 115 |
+
| Repo File | Upstream URL | GCS Generation | Upstream `Last-Modified` | Pinned Bucket Version |
|
| 116 |
+
|---|---|---|---|---|
|
| 117 |
+
| `mobilenet_v3_large.tflite` | [`mobilenet_v3_large.tflite`](https://storage.googleapis.com/mediapipe-models/image_embedder/mobilenet_v3_large/float32/latest/mobilenet_v3_large.tflite) | `1682480014487254` | Wed, 26 Apr 2023 03:33:34 GMT | `/1/` |
|
| 118 |
+
|
| 119 |
+
## License and Citation
|
| 120 |
+
|
| 121 |
+
- License: `apache-2.0` — Stated as Apache License, Version 2.0 (`http://www.apache.org/licenses/LICENSE-2.0`) in the embedded `TFLITE_METADATA` block of the `.tflite` model file(s).
|
| 122 |
+
|
| 123 |
+
### Citation
|
| 124 |
+
|
| 125 |
+
```bibtex
|
| 126 |
+
@inproceedings{howard2019searching,
|
| 127 |
+
title = {Searching for MobileNetV3},
|
| 128 |
+
author = {Howard, Andrew and Sandler, Mark and Chu, Grace and Chen, Liang-Chieh and Chen, Bo and Tan, Mingxing and Wang, Weijun and Zhu, Yukun and Pang, Ruoming and Vasudevan, Vijay and Le, Quoc V. and Adam, Hartwig},
|
| 129 |
+
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
|
| 130 |
+
year = {2019},
|
| 131 |
+
url = {https://arxiv.org/abs/1905.02244}
|
| 132 |
+
}
|
| 133 |
+
```
|
mobilenet_v3_large.tflite
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:11af3c560dfeed7737cb4c03c23bf52a8403020784192d4dea0b74862a12828d
|
| 3 |
+
size 10889458
|