schmidt-sebastian commited on
Commit
c9f46d7
·
0 Parent(s):

Initial upload of MediaPipe-ImageEmbedder-MobileNet-V3-Large from MediaPipe Solutions

Browse files
Files changed (3) hide show
  1. .gitattributes +38 -0
  2. README.md +133 -0
  3. mobilenet_v3_large.tflite +3 -0
.gitattributes ADDED
@@ -0,0 +1,38 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ *.7z filter=lfs diff=lfs merge=lfs -text
2
+ *.arrow filter=lfs diff=lfs merge=lfs -text
3
+ *.bin filter=lfs diff=lfs merge=lfs -text
4
+ *.bz2 filter=lfs diff=lfs merge=lfs -text
5
+ *.ckpt filter=lfs diff=lfs merge=lfs -text
6
+ *.ftz filter=lfs diff=lfs merge=lfs -text
7
+ *.gz filter=lfs diff=lfs merge=lfs -text
8
+ *.h5 filter=lfs diff=lfs merge=lfs -text
9
+ *.joblib filter=lfs diff=lfs merge=lfs -text
10
+ *.lfs.* filter=lfs diff=lfs merge=lfs -text
11
+ *.litertlm filter=lfs diff=lfs merge=lfs -text
12
+ *.mlmodel filter=lfs diff=lfs merge=lfs -text
13
+ *.model filter=lfs diff=lfs merge=lfs -text
14
+ *.msgpack filter=lfs diff=lfs merge=lfs -text
15
+ *.npy filter=lfs diff=lfs merge=lfs -text
16
+ *.npz filter=lfs diff=lfs merge=lfs -text
17
+ *.onnx filter=lfs diff=lfs merge=lfs -text
18
+ *.ot filter=lfs diff=lfs merge=lfs -text
19
+ *.parquet filter=lfs diff=lfs merge=lfs -text
20
+ *.pb filter=lfs diff=lfs merge=lfs -text
21
+ *.pdf filter=lfs diff=lfs merge=lfs -text
22
+ *.pickle filter=lfs diff=lfs merge=lfs -text
23
+ *.pkl filter=lfs diff=lfs merge=lfs -text
24
+ *.pt filter=lfs diff=lfs merge=lfs -text
25
+ *.pth filter=lfs diff=lfs merge=lfs -text
26
+ *.rar filter=lfs diff=lfs merge=lfs -text
27
+ *.safetensors filter=lfs diff=lfs merge=lfs -text
28
+ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
29
+ *.tar.* filter=lfs diff=lfs merge=lfs -text
30
+ *.tar filter=lfs diff=lfs merge=lfs -text
31
+ *.task filter=lfs diff=lfs merge=lfs -text
32
+ *.tflite filter=lfs diff=lfs merge=lfs -text
33
+ *.tgz filter=lfs diff=lfs merge=lfs -text
34
+ *.wasm filter=lfs diff=lfs merge=lfs -text
35
+ *.xz filter=lfs diff=lfs merge=lfs -text
36
+ *.zip filter=lfs diff=lfs merge=lfs -text
37
+ *.zst filter=lfs diff=lfs merge=lfs -text
38
+ *tfevents* filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,133 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ library_name: litert
4
+ pipeline_tag: image-feature-extraction
5
+ datasets:
6
+ - imagenet-1k
7
+ tags:
8
+ - mediapipe
9
+ - litert
10
+ - tflite
11
+ - image-feature-extraction
12
+ - on-device
13
+ - android
14
+ - ios
15
+ - web
16
+ - python
17
+ - flutter
18
+ - vision
19
+ - mobilenet-v3
20
+ - image-embeddings
21
+ - fp32
22
+ - arxiv:1905.02244
23
+ ---
24
+
25
+ # MediaPipe MobileNet-V3 (Large) Image Embedder
26
+
27
+ MobileNet-V3 Large image feature extractor (`mobilenet_v3_large.tflite`, 10.9 MB, 0.75 depth multiplier, FP32 weights) producing **1280-dimensional** embedding vectors for visual similarity search and deduplication in the [MediaPipe Image Embedder (`ImageEmbedder`)](https://developers.google.com/edge/mediapipe/solutions/vision/image_embedder) task.
28
+
29
+ <div style="border: 1px solid rgba(59, 130, 246, 0.28); background-color: rgba(59, 130, 246, 0.06); border-radius: 8px; padding: 12px 16px; margin: 16px 0;">
30
+ <p style="margin: 0 0 6px 0;"><strong>Documentation:</strong> <a href="https://developers.google.com/edge/mediapipe/solutions/vision/image_embedder">MediaPipe Image Embedder Docs</a></p>
31
+ <p style="margin: 0 0 6px 0;"><strong>Samples:</strong> <a href="https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/image_embedder/android">Android</a> · <a href="https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/image_embedder/ios">iOS</a> · <a href="https://colab.research.google.com/github/google-ai-edge/mediapipe-samples/blob/main/examples/image_embedder/python/image_embedder.ipynb">Python Colab</a> · <a href="https://github.com/google-ai-edge/mediapipe-samples-web/blob/main/src/tasks/image-embedder.ts">Web</a> · <a href="https://github.com/google/flutter-mediapipe/tree/main/samples/vision/image_embedder">Flutter</a></p>
32
+ <p style="margin: 0;"><strong>Try live:</strong> <a href="https://google-ai-edge.github.io/mediapipe-samples-web/#/vision/image_embedder">Web Demo</a></p>
33
+ </div>
34
+
35
+ Trained on ImageNet with a `0.75` channel depth multiplier to balance accuracy and edge latency. Takes normalized `224 x 224` RGB images and outputs a `1280`-element feature vector (`StatefulPartitionedCall:0`), optionally L2-normalized (`l2_normalize=True`) and scalar-quantized (`quantize=True`) by MediaPipe `ImageEmbedder`.
36
+
37
+ ## Files
38
+
39
+ | File | Precision | Size | SHA-256 |
40
+ |---|---|---:|---|
41
+ | [`mobilenet_v3_large.tflite`](https://huggingface.co/litert-community/MediaPipe-ImageEmbedder-MobileNet-V3-Large/blob/main/mobilenet_v3_large.tflite) | `float32` | 10.89 MB (10,889,458 B) | `11af3c560dfeed7737cb4c03c23bf52a8403020784192d4dea0b74862a12828d` |
42
+
43
+ ### Download
44
+
45
+ ```bash
46
+ pip install -U huggingface_hub
47
+ hf download litert-community/MediaPipe-ImageEmbedder-MobileNet-V3-Large mobilenet_v3_large.tflite --local-dir .
48
+ ```
49
+
50
+ ## Inputs, Outputs, and Model Specifications
51
+
52
+ ### `mobilenet_v3_large.tflite`
53
+
54
+ - Embedded `TFLITE_METADATA`: name `MobileNetV3 image classifier` · version `1` · author `MediaPipe` · packed associated files: `labels.txt`
55
+
56
+ | Direction | Tensor | Shape | Type | Notes |
57
+ |---|---|---|---|---|
58
+ | Input | `serving_default_inputs:0` | `[-1, 224, 224, 3]` | `float32` | image — Input image to be embedded. The expected image is 224 x 224, with three channels (red, blue, and green) per pixel. Each element in the tensor is a value between min and max, where (per-channel) min is [0.0] and max is [1.0]. — Normalization: `mean=[0.0], std=[255.0]` |
59
+ | Output | `StatefulPartitionedCall:0` | `[-1, 1280]` | `float32` | feature — Feature vector of the input image. |
60
+
61
+ ## Use with MediaPipe Tasks
62
+
63
+ ### Python Quickstart
64
+
65
+ ```bash
66
+ pip install -U mediapipe huggingface_hub
67
+ ```
68
+
69
+ ```python
70
+ import mediapipe as mp
71
+ from huggingface_hub import hf_hub_download
72
+
73
+ model_path = hf_hub_download(repo_id="litert-community/MediaPipe-ImageEmbedder-MobileNet-V3-Large", filename="mobilenet_v3_large.tflite")
74
+
75
+ BaseOptions = mp.tasks.BaseOptions
76
+ ImageEmbedder = mp.tasks.vision.ImageEmbedder
77
+ ImageEmbedderOptions = mp.tasks.vision.ImageEmbedderOptions
78
+ VisionRunningMode = mp.tasks.vision.RunningMode
79
+
80
+ options = ImageEmbedderOptions(
81
+ base_options=BaseOptions(model_asset_path=model_path),
82
+ running_mode=VisionRunningMode.IMAGE,
83
+ l2_normalize=True,
84
+ )
85
+
86
+ with ImageEmbedder.create_from_options(options) as embedder:
87
+ e1 = embedder.embed(mp.Image.create_from_file("input.jpg")).embeddings[0]
88
+ e2 = embedder.embed(mp.Image.create_from_file("other.jpg")).embeddings[0]
89
+ print("Cosine similarity:", ImageEmbedder.cosine_similarity(e1, e2))
90
+ ```
91
+
92
+ ### Cross-Platform Guides & Samples
93
+
94
+ - **Task overview & options:** [Image Embedder documentation](https://developers.google.com/edge/mediapipe/solutions/vision/image_embedder)
95
+ - **Platform setup guides:** [Python](https://developers.google.com/edge/mediapipe/solutions/setup_python) · [Web / JavaScript](https://developers.google.com/edge/mediapipe/solutions/setup_web) · [Android (Kotlin / Java)](https://developers.google.com/edge/mediapipe/solutions/setup_android) · [iOS (Swift)](https://developers.google.com/edge/mediapipe/solutions/setup_ios) · [Flutter (Dart)](https://developers.google.com/edge/mediapipe/solutions/setup_flutter)
96
+ - **Runnable sample apps:** [Web Demo](https://google-ai-edge.github.io/mediapipe-samples-web/#/vision/image_embedder) · [Web Source](https://github.com/google-ai-edge/mediapipe-samples-web/blob/main/src/tasks/image-embedder.ts) · [Android](https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/image_embedder/android) · [iOS](https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/image_embedder/ios) · [Python Notebook](https://colab.research.google.com/github/google-ai-edge/mediapipe-samples/blob/main/examples/image_embedder/python/image_embedder.ipynb) · [Flutter](https://github.com/google/flutter-mediapipe/tree/main/samples/vision/image_embedder)
97
+
98
+ ## Latency Benchmarks
99
+
100
+ End-to-end MediaPipe Task pipeline average latency reported by the [Image Embedder documentation](https://developers.google.com/edge/mediapipe/solutions/vision/image_embedder#benchmarks):
101
+
102
+ | Model Variant | Device | CPU Latency | GPU Latency |
103
+ |---|---|---:|---:|
104
+ | MobileNet-V3 (large) | Pixel 6 | 9.75 ms | 9.08 ms |
105
+
106
+ ## Intended Use, Limitations, and Fairness
107
+
108
+ - **Intended applications:** Generating dense feature vectors for on-device image similarity comparison, visual search, clustering, and near-duplicate detection via cosine similarity.
109
+ - **Limitations & considerations:** Pretrained on ImageNet-1K natural photographs; fine-grained domains (medical scans, OCR, specialized industrial parts) or biometric matching are outside its training distribution. No standalone Google fairness PDF is published for this model on the MediaPipe documentation page.
110
+
111
+ ## Provenance
112
+
113
+ Every model binary in this repository is byte-identical (unmodified FlatBuffer weights and embedded `TFLITE_METADATA` bytes) to the official artifact served by Google from `storage.googleapis.com/mediapipe-models`.
114
+
115
+ | Repo File | Upstream URL | GCS Generation | Upstream `Last-Modified` | Pinned Bucket Version |
116
+ |---|---|---|---|---|
117
+ | `mobilenet_v3_large.tflite` | [`mobilenet_v3_large.tflite`](https://storage.googleapis.com/mediapipe-models/image_embedder/mobilenet_v3_large/float32/latest/mobilenet_v3_large.tflite) | `1682480014487254` | Wed, 26 Apr 2023 03:33:34 GMT | `/1/` |
118
+
119
+ ## License and Citation
120
+
121
+ - License: `apache-2.0` — Stated as Apache License, Version 2.0 (`http://www.apache.org/licenses/LICENSE-2.0`) in the embedded `TFLITE_METADATA` block of the `.tflite` model file(s).
122
+
123
+ ### Citation
124
+
125
+ ```bibtex
126
+ @inproceedings{howard2019searching,
127
+ title = {Searching for MobileNetV3},
128
+ author = {Howard, Andrew and Sandler, Mark and Chu, Grace and Chen, Liang-Chieh and Chen, Bo and Tan, Mingxing and Wang, Weijun and Zhu, Yukun and Pang, Ruoming and Vasudevan, Vijay and Le, Quoc V. and Adam, Hartwig},
129
+ booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
130
+ year = {2019},
131
+ url = {https://arxiv.org/abs/1905.02244}
132
+ }
133
+ ```
mobilenet_v3_large.tflite ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:11af3c560dfeed7737cb4c03c23bf52a8403020784192d4dea0b74862a12828d
3
+ size 10889458