Feature Extraction
LiteRT
LiteRT
on-device
edge
encoder
bidirectional
embeddings
fill-mask
liquid
lfm2
lfm2.5
Instructions to use litert-community/LFM2.5-Encoder-350M with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- LiteRT
How to use litert-community/LFM2.5-Encoder-350M with LiteRT:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Add measured Galaxy S26 NPU/GPU section
Browse filesNumbers from the S26 classic sweep. Primary logs:
litertlm-convert/community_accel_work/s2_npu_sweep/results.jsonl (on-device
JIT) and portal_work/bench_all.txt + bench_redo.txt (the 2026-08-23 AOT
sweep); every row states which of the two produced it. Rows carry device,
SoC, runtime, N and thermal state, and join to this repo by exact .tflite
file name. Rows taken outside the sweep's thermal window are not reported.
README.md
CHANGED
|
@@ -178,7 +178,22 @@ Android figures use the standard TFLite [`benchmark_model`](https://ai.google.de
|
|
| 178 |
| `encode_512` | 1287 ms | **581 ms** |
|
| 179 |
| `mlm_128` | 431 ms | **135 ms** |
|
| 180 |
|
| 181 |
-
**GPU works as of the 2026-08-13 re-export β via the single-signature file.** This re-export respells the one idiom mobile GPU delegates refuse β transformers' rank-5 `repeat_kv` expand β into an equivalent rank-4 matmul (outputs bitwise-identical on CPU), which makes the graph fully GPU-delegable. The 5-signature file
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 182 |
|
| 183 |
## License
|
| 184 |
|
|
|
|
| 178 |
| `encode_512` | 1287 ms | **581 ms** |
|
| 179 |
| `mlm_128` | 431 ms | **135 ms** |
|
| 180 |
|
| 181 |
+
**GPU works as of the 2026-08-13 re-export β via the single-signature file.** This re-export respells the one idiom mobile GPU delegates refuse β transformers' rank-5 `repeat_kv` expand β into an equivalent rank-4 matmul (outputs bitwise-identical on CPU), which makes the graph fully GPU-delegable. The 5-signature file does not compile on the Pixel 8a OpenCL or iPhone 17 Pro Metal delegates: each signature subgraph materializes its own fp32 weight copy (~7 GB total for this model), which exceeds those phones' memory. It does compile and run on the Adreno GPU of a Galaxy S26 under LiteRT `CompiledModel` 2.2.0 β 82.04 ms β but it pays a 17.3 s load there against 4.07 s for the single-signature file (see Snapdragon NPU (Hexagon) below). The new **`LFM2.5-Encoder-350M_wi8fc_single-sig.tflite`** (`encode_512` only, 359 MB) is the phone-GPU artifact: Pixel 8a OpenCL full delegation, `encode_512` **12.2 ms**, cosine **0.9994** vs the fp32 desktop reference (LiteRT CompiledModel API, fp32 GPU precision, best of 3 warm runs); iPhone 17 Pro Metal 171 ms, cosine 0.9994. The 5-signature file remains the CPU artifact (bit-exact on device; iPhone `encode_512` 144β158 ms). Set the GPU precision to **fp32** β at fp16 GPU precision this family's norm reductions overflow and every output is NaN. These CompiledModel timings are not comparable to the classic-delegate `benchmark_model` timings above (different GPU runtime).
|
| 182 |
+
|
| 183 |
+
## Snapdragon NPU (Hexagon)
|
| 184 |
+
|
| 185 |
+
- `LFM2.5-Encoder-350M_fp16.tflite` β neither accelerator produced a usable row on the S26. Both ended the same way: `LiteRtException: Failed to compile model`.
|
| 186 |
+
- `LFM2.5-Encoder-350M_wi8fc.tflite` β the GPU runs it at 82.04 ms. The NPU does not β `LiteRtException: Failed to compile model`.
|
| 187 |
+
- `LFM2.5-Encoder-350M_wi8fc_single-sig.tflite` β the GPU runs it at 82.24 ms. The NPU does not β the graph compiles and then fails to run (`LiteRtException: Failed to invoke the compiled model`).
|
| 188 |
+
|
| 189 |
+
| file | backend | compiled | inference (median / min) | load |
|
| 190 |
+
|---|---|---|---:|---:|
|
| 191 |
+
| `LFM2.5-Encoder-350M_wi8fc.tflite` | GPU (Adreno) | β | 82.04 ms / 81.14 ms | 17349 ms |
|
| 192 |
+
| `LFM2.5-Encoder-350M_wi8fc_single-sig.tflite` | GPU (Adreno) | β | 82.24 ms / 80.41 ms | 4074 ms |
|
| 193 |
+
|
| 194 |
+
Measured on a **Samsung Galaxy S26** (Snapdragon 8 Elite Gen 5 / SM8850, Hexagon v81, Android 16) with LiteRT `CompiledModel` 2.2.0, one accelerator per process, 5 warm-up runs then N=50 timed runs, median reported. Every run held thermal status `NONE` throughout. Headroom 0.72β0.78, where 1.0 is the throttling threshold.
|
| 195 |
+
|
| 196 |
+
GPU wiring: [GPU guide](https://github.com/john-rocky/hf-to-litertlm/blob/main/docs/android-gpu.md). NPU recipe: [NPU guide](https://github.com/john-rocky/hf-to-litertlm/blob/main/docs/android-npu.md).
|
| 197 |
|
| 198 |
## License
|
| 199 |
|