Image Classification
LiteRT
LiteRT
vit
vision
File size: 12,662 Bytes
2c83b43
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
35989ed
2c83b43
 
 
 
 
 
 
 
 
35989ed
2c83b43
 
 
 
 
 
 
 
 
 
 
 
35989ed
 
2c83b43
35989ed
2c83b43
 
 
 
 
f10f70a
 
 
 
 
 
 
2c83b43
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
35989ed
 
 
 
 
 
2c83b43
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
35989ed
2c83b43
 
 
 
 
35989ed
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2c83b43
 
35989ed
 
2c83b43
 
35989ed
 
 
 
 
2c83b43
 
35989ed
 
 
 
2c83b43
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
---
library_name: litert
pipeline_tag: image-classification
license: apache-2.0
base_model: google/vit-base-patch16-224
tags:
  - vision
  - image-classification
  - vit
  - tflite
datasets:
  - imagenet-1k
  - imagenet-21k
---

# Google ViT Base Patch16 224 — LiteRT

LiteRT/TFLite variants of
[`google/vit-base-patch16-224`](https://huggingface.co/google/vit-base-patch16-224),
pinned to source revision `3f49326eb077187dfe1c2a2bb15fbd74e6ab91e3`.

The package provides FP32, A16W8 and mixed A8W8 models for image classification.
All have FP32 input and output and require external image preprocessing.
The `.tflite` files are portable source artifacts for target-specific NPU
compilation; they do not contain device-specific compiled NPU programs.

## Model files

| File | Description |
| --- | --- |
| `model_fp32.tflite` | Floating-point reference with FP32 weights. |
| `model_a16w8.tflite` | INT16 transformer activations and INT8 fully connected weights; the primary quantized option for compatible NPUs. |
| `model_a16w8_qualcomm.tflite` | Dedicated Qualcomm NPU compilation input for A16W8 using UINT16 transformer activations and LayerNorm interfaces. |
| `model_a8w8.tflite` | Experimental mixed precision: INT8 fully connected layers with INT16 attention, residual and LayerNorm boundaries. |
| `config.json` | Original source configuration and class mapping. |
| `preprocessor_config.json` | Source image preprocessing configuration. |
| `labels.json` | Class index to label mapping. |
| `model_metadata.json` | Source identity, tensor interfaces, quantization scope and model hashes. |
| `run_image.py` | CPU image-classification example. |
| `requirements.txt` | Runtime and preprocessing package versions. |

## Choosing a variant

- **FP32** provides the floating-point reference and is the example's default.
- **A16W8** offers the best observed accuracy/speed balance on the tested NPUs.
  Its fully connected layers use INT16 inputs and outputs (or UINT16 in
  `model_a16w8_qualcomm.tflite` for Qualcomm HTP compilation), INT8 weights and
  INT64 biases. CPU fallback can be slow, and the tested LiteRT GPU IR importer
  does not support its 16-bit integer quantization.
- **Mixed A8W8** is retained as a reference that may suit devices limited to
  32-bit integer accumulation in their fully connected kernels. It has a
  substantial accuracy tradeoff and retains INT16 regions, so backend support
  must be verified.

Extra notes: Apple ANE and QC NPUs support FP16 natively and the performances are
usually very good. Deloying a FP16 model is much easier since it does not require
calibration steps. Note that the A16W8 and A8W8 assets provided here were calibrated
from a particular dataset and should not used in production directly. A FP16 model
file is not provided here is because it is easy to be generated from the fp32 asset. 
It is actually a top choice. 

## TorchAO quantization recipe

Both variants use post-training static quantization, without QAT or fine-tuning.
The run used PyTorch `2.14.1+cu130`, TorchAO `0.18.0+cu130` and Transformers
`5.21.0`. To reproduce the quantization approach:

- Load the pinned source checkpoint in FP32 evaluation mode. Calibrate with
  500 ImageNet-1k validation images, batch size one, using the included PIL
  preprocessing configuration. Keep calibration separate from evaluation.
- Observe inputs to all 73 `torch.nn.Linear` layers, including the classifier,
  with `AffineQuantizedMinMaxObserver`: asymmetric INT8 `[-128, 127]`,
  `PerTensor()` granularity, FP32 scales and INT32 zero points.
- Quantize the original FP32 weights with symmetric `PerRow()` granularity
  (one scale per output channel), zero point zero and the explicit narrow
  range `[-127, 127]`. Use TorchAO's `choose_qparams_affine` and
  `quantize_affine` with these bounds; this weight-range customization is
  additional to the activation configuration below.

After calibrating each layer's input observer, its static configuration is:

```python
from torchao.quantization import Int8StaticActivationInt8WeightConfig
from torchao.quantization.granularity import PerRow, PerTensor
from torchao.quantization.quant_primitives import MappingType

scale, zero_point = observer.calculate_qparams()
config = Int8StaticActivationInt8WeightConfig(
    act_quant_scale=scale,
    act_quant_zero_point=zero_point,
    granularity=[PerTensor(), PerRow()],
    act_mapping_type=MappingType.ASYMMETRIC,
    set_inductor_config=False,
)
```

The final TFLite export also needs calibrated linear-output and transformer
activation ranges, collected using the same 500 images. **A16W8** uses symmetric
INT16 activations with zero point zero and INT64 fully connected biases (and
`model_a16w8_qualcomm.tflite` provides the corresponding UINT16 export for
Qualcomm NPU compilation). **Mixed A8W8** keeps asymmetric INT8 fully connected
inputs/outputs and INT32 biases, with INT16 attention, residual and LayerNorm
boundaries. Both retain the same INT8 weight codes and scales; activation scales
are per-tensor.

The transformer graphs keep activations quantized across attention, residual,
reshape, transpose, softmax and GELU operations. LayerNorm uses composite
operators with quantized interfaces and floating-point decomposition subgraphs
for fallback execution. The patch embedding and public input/output remain
floating-point, so these models do not use integer arithmetic throughout.

A16W8 retains input-scale coarsening for fully connected layers that needed it
under a 32-bit accumulator profile, but omits explicit accumulator input
clipping. Validate accumulator behavior and accuracy on another backend.

TorchAO provides the linear quantization stage; recreating the complete TFLite
graph requires export support for the activation encodings above. The exact
calibration image list is not distributed, so this recipe does not guarantee
identical scales or files. See the
[TorchAO static quantization guide](https://docs.pytorch.org/ao/stable/eager_tutorials/static_quantization.html)
for the observer/calibration workflow.

## Accuracy evaluation

The results below use the same 5,000-image subset of ImageNet-1k validation:
five images per class, selected with seed `20261031`. The selection excludes
the 500 calibration images and 100 earlier diagnostic images. All runs use
the same input tensors, produced with the included Hugging Face PIL
preprocessing configuration.

The original PyTorch FP32 checkpoint scored **79.68% top-1** and **95.40%
top-5**, with TF32 and autocast disabled. The TFLite results below were
measured with AOT-compiled models fully delegated to each device's NPU; all
outputs were finite. FP32 denotes the source model, with FP16 execution
permitted by the NPU compiler.

| Model file | `SM-S937U1` top-1 (%) | `SM-S937U1` top-5 (%) | `LJ9` top-1 (%) | `LJ9` top-5 (%) |
| --- | ---: | ---: | ---: | ---: |
| `model_fp32.tflite` | 79.92 | 95.40 | 79.68 | 95.38 |
| `model_a16w8.tflite` | 79.72 | 95.32 | 79.66 | 95.32 |
| `model_a8w8.tflite` | 75.14 | 92.50 | 75.10 | 92.76 |

A16W8 stays close to the PyTorch baseline on this subset. Mixed A8W8 reduces
top-1 accuracy by 4.54 percentage points on `SM-S937U1` and 4.58 points on
`LJ9` relative to that baseline.

This subset was reused while comparing quantization recipes. It is not an
independent final test set or a full 50,000-image ImageNet validation run.
Small score differences should not be interpreted as general improvements
over the original checkpoint. Raw outputs, input images and diagnostic
reports are not included in this package.

The A16W8 scores were collected before unused tensor records were removed.
That cleanup preserved all live computation, tensor encodings and weights,
and left the Qualcomm compiled file identical. A fresh 100-image MTK check
reproduced the earlier logits exactly; the full 5,000-image MTK run was not
repeated after this metadata cleanup.

## NPU performance

These are representative measurements from a retail device to give a rough
sense of on-device runtime behavior, not a direct comparison between hardware
platforms. All latency numbers were collected with LiteRT's compiled-model
`benchmark_model` launched through `adb shell` on the connected device; they
are not app-level measurements from an integrated Android application.

Warm batch-one inference latency, rounded to the nearest tenth of a
millisecond. Each value is the median of three process means, with five warmup
invocations and 30 measured invocations per process. All configurations are
fully delegated in a single partition with no residual CPU operations (AOT
compiled for NPU, and pre-warmed OpenCL program cache for GPU).

| Model file | Backend / precision | `SM-S937U` (ms) |
| --- | --- | ---: |
| `model_fp32.tflite` | LiteRT GPU (MLDrift OpenCL, FP32) | 23.7 |
| `model_fp32.tflite` | LiteRT GPU (MLDrift OpenCL, FP16) | 12.8 |
| `model_fp32.tflite` | Qualcomm NPU (HTP, FP16) | 3.4 |
| `model_a16w8_qualcomm.tflite` | Qualcomm NPU (HTP, A16W8) | 2.5 |

On `SM-S937U`, Qualcomm NPU execution is **3.8× faster** than MLDrift OpenCL
GPU at matching FP16 precision (`3.4 ms` vs `12.8 ms`), and **5.0× faster**
with `A16W8` quantization (`2.5 ms` vs `12.8 ms`; **9.3× faster** than FP32
OpenCL GPU at `23.7 ms`).

FP32 identifies the source model, which Qualcomm compiles to FP16-based NPU
execution. The floating-source NPU timing therefore does not imply FP32
arithmetic throughout the NPU program. `model_a16w8_qualcomm.tflite` is the
dedicated UINT16 compilation input corresponding to `model_a16w8.tflite`. Mixed
A8W8 is omitted because a Qualcomm-specific UINT16 A8W8 compilation input is
not published in this package.

Measurements use LiteRT's compiled-model benchmark with persistent buffers
and deterministic random input (`clFinish()` synchronized per iteration on
OpenCL GPU). They exclude model initialization, shader/graph compilation, image
preprocessing, file I/O and explicit host input/output buffer transfers.

The `SM-S937U` ran Android 16 with Qualcomm QNN/QAIRT 2.52 in burst performance
mode (`O3` optimization) for NPU and LiteRT's MLDrift OpenCL backend on Adreno
for GPU. Power settings were not normalized, and energy consumption was not
measured. These results apply to the listed device/compiler/runtime
combinations and do not characterize JIT initialization latency.

**Hardware benchmark disclosure:** Results were measured by us on retail
devices. These results are not affiliated with, sponsored by, endorsed by, or
verified by Samsung, TECNO, Qualcomm, MediaTek, Google, MLCommons, or Hugging
Face. Results depend on device SKU, OS build, thermal state, battery mode,
backend, model quantization, runtime version, and benchmark settings.

## Input and output

- Input: FP32 image tensor, NCHW `[1, 3, 224, 224]`.
- Output: FP32 classifier logits, `[1, 1000]`, ordered by `labels.json`.
- Preprocessing: convert to RGB, resize to 224 × 224 with the included
  Transformers PIL processor, rescale by 1/255, then normalize each channel
  with mean and standard deviation 0.5. The example selects `backend="pil"`.

The models have static batch and spatial dimensions. All 25 LayerNorm
operations use TFLite `STABLEHLO_COMPOSITE` boundaries named `odml.group_norm`,
with FP32 decomposition subgraphs.

## Run a local image

Download the listed files into one directory and run the following from that
directory. The CPU example and dependencies were checked with Python 3.13 on
Linux:

```bash
uv venv --python 3.13
source .venv/bin/activate
uv pip install -r requirements.txt

python run_image.py --image image.jpg
python run_image.py --image image.jpg --model model_a8w8.tflite
python run_image.py --image image.jpg --model model_a16w8.tflite
```

The helper loads the included processor configuration and TFLite model, then
prints the top classes with their raw logits. A16W8 CPU execution may be slow;
the example is an interface check for that variant.

For NPU deployment, compile the selected `.tflite` for the target SoC using the
matching LiteRT compiler plugin, then load the compiled model with the matching
device dispatch runtime and select the NPU accelerator. Verify full delegation
on that device. The CPU example above does not select the NPU.

## Source and attribution

This artifact derives from the pinned Google checkpoint, whose model card
declares the Apache-2.0 license. Refer to that card for training details and
intended use. The architecture was introduced in
[An Image Is Worth 16x16 Words](https://arxiv.org/abs/2010.11929).
The accompanying `config.json` describes the original Transformers model;
inference here uses the `.tflite` file.