litert-community/inception_v3

Inception v3

Inception v3 pretrained on ImageNet-1k, introduced in Rethinking the Inception Architecture for Computer Vision by Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jonathon Shlens, and Zbigniew Wojna.

Model description

These LiteRT/TFLite models were converted from TorchVision's Inception_V3_Weights.IMAGENET1K_V1 checkpoint. TorchVision reports 77.294% top-1 and 93.450% top-5 ImageNet-1k accuracy for the original model, with 27,161,264 parameters.

The generic exports fold TorchVision's per-channel transform_input computation into the first convolution's weights and bias. Each starts directly with CONV_2D, and all 94 convolution/ReLU pairs are fused. Public preprocessing is unchanged. FP32 and dynamic INT8 contain 129 operators; static INT8 contains 131.

Available model files

File	Precision and interface	Recommended use
inception_v3.tflite	FP32 weights, activations and I/O; 95,314,608 bytes.	Full-precision CPU/GPU inference.
inception_v3_dynamic_wi8_afp32.tflite	Channelwise INT8 weights with FP32 activations, biases and I/O; 24,138,880 bytes (23.02 MiB).	Compact CPU/GPU inference.
inception_v3_int8_channelwise.tflite	Static INT8 weights, activations and I/O with INT32 biases; 24,364,920 bytes (23.24 MiB).	NPU deployment with a compatible vendor compiler/runtime.
inception_v3_Google_Tensor_G5_apply_plugin.tflite	Existing Google Tensor G5 AOT artifact.	Existing target-specific deployment.

Quantization

Both INT8 variants use AI Edge Quantizer 0.7.0 and symmetric channelwise INT8 weights for all 95 convolution/classifier operators.

• Dynamic INT8: recipe.dynamic_wi8_afp32(), using min/max weight ranges. The model retains FP32 activations, biases and public I/O. Hybrid CPU kernels may dynamically quantize activations internally; GPU execution uses floating-point activations with dequantized weights. CPU/GPU outputs can differ.
• Static INT8: recipe.static_wi8_ai8(), calibrated on 500 ImageNet-1k validation images, manifest rows 0–499. Intermediate ranges use an exponential moving average and final logits use global observed ranges. Bias scale and INT32 headroom safeguards are applied.

Preprocessing and interface

Use the checkpoint's preprocessing for every variant:

1. Convert to RGB.
2. Bilinear resize the shorter side to 342 pixels, preserving aspect ratio, with antialiasing.
3. Center crop to 299×299.
4. Divide pixel values by 255, subtract [0.485, 0.456, 0.406], and divide by [0.229, 0.224, 0.225].
5. Arrange the input as NHWC [1, 299, 299, 3].

The output is [1, 1000] ImageNet logits. FP32 and dynamic INT8 use float32 inputs/outputs. Static INT8 additionally quantizes the normalized input and requires dequantizing its output using the file's metadata. Do not apply TorchVision's internal transform_input operation again.

Validation

The following results use the same disjoint 100-image ImageNet-1k validation subset, manifest rows 1000–1099, with exact TorchVision preprocessing.

Variant	Backend	Top-1 correct / 100	Top-5 correct / 100
FP32	Builtin / XNNPACK-enabled CPU	70	91
FP32	GPU	70	91
Dynamic INT8	Builtin / XNNPACK-enabled CPU	68	90
Dynamic INT8	GPU	69	91
Static INT8	Builtin / XNNPACK-enabled CPU	68	89

These are subset results, not full ImageNet validation accuracy. The model-index metrics above are historical full-precision figures and were not remeasured over the full validation set for this release.

FP32 parity was checked against the preceding FP32 export on these 100 images plus zero and seeded-random inputs. All top-1 predictions matched; maximum absolute logit difference was 4.27e-5 across builtin and XNNPACK execution.

Compatibility

Variant	CPU	GPU	NPU
FP32	Supported	Fully delegated in the configuration below.	Use static INT8 for NPU deployment.
Dynamic INT8	Supported	Fully delegated in the configuration below.	Use static INT8 for NPU deployment.
Static INT8	Supported	Use FP32 or dynamic INT8 for GPU deployment.	Qualcomm / MediaTek validation below.
Google Tensor G5 AOT	Target-specific artifact	Target-specific artifact	Existing artifact, outside this refresh.

GPU evaluation used a Linux LiteRT 2.4 runtime with the IR frontend enabled, FP32 computation, and an NVIDIA RTX A5000 through WebGPU/Vulkan. Both FP32 and dynamic INT8 fully delegated. FP32 GPU logits matched builtin CPU on all 102 real/synthetic inputs (atol=1e-4, rtol=1e-3, max absolute error 2.15e-5). Dynamic INT8 GPU logits matched an independent FP32 reference with dequantized weights on all 102 inputs (max absolute error 1.55e-5).

Static INT8 XNNPACK-enabled CPU execution includes builtin fallback for the ten quantized AVERAGE_POOL_2D operators; it is not fully delegated to XNNPACK.

NPU validation

The published static INT8 file passed full AOT compilation with LiteRT's compiler plugins:

Vendor	Target	SDK	AOT result
Qualcomm	SM8750	QAIRT 2.50.0.260828	One dispatch partition; no remaining TFLite compute operators.
MediaTek	MT6897	Neuron 8.2.30	One dispatch partition; no remaining TFLite compute operators.
MediaTek	MT6993	Neuron 9.2.1	One dispatch partition; no remaining TFLite compute operators.

The MT6897 AOT model also ran through the NPU dispatch path on a TECNO LJ9 for ten held-out images (rows 1000–1009). All ten top-1 predictions matched the CPU static INT8 reference, with finite, nonconstant scores. This is a compatibility smoke check; the generic static INT8 file still needs compilation or JIT delegation with a compatible vendor plugin/runtime for the target device. Target-specific outputs from these checks are not included in this repository.

SHA-256

inception_v3.tflite
5e2e738b47666856f1e94bbf95e6defafba223c60fce73f7ea1385659617ab73

inception_v3_dynamic_wi8_afp32.tflite
643f0cd7920d22e7b44a937d06281e7aea1ae03e1cfa0d5a3843f0231ef0fb3e

inception_v3_int8_channelwise.tflite
129cf96f06a2aa83e4ca2aee344cc1b66f569607c46477aa49b6b738cb14d2ff

Intended uses & limitations

The models inherit the licenses and terms of their TorchVision checkpoints and the dataset used for training. Determine whether those terms permit your use.

How to use

Install the CPU example's dependencies:

pip install numpy Pillow huggingface_hub ai-edge-litert

For GPU execution, use a compatible LiteRT runtime with the IR frontend enabled and a supported GPU backend. The GPU validation above used LiteRT 2.4.

Save the following as classify.py. It defaults to dynamic INT8 on CPU. Static INT8 quantization parameters are read from the selected file.

#!/usr/bin/env python3
import argparse
import json

import numpy as np
from PIL import Image
from huggingface_hub import hf_hub_download
from ai_edge_litert.interpreter import Interpreter

VARIANTS = {
    "fp32": "inception_v3.tflite",
    "dynamic": "inception_v3_dynamic_wi8_afp32.tflite",
    "int8": "inception_v3_int8_channelwise.tflite",
}


def preprocess(img):
    img = img.convert("RGB")
    w, h = img.size
    if w < h:
        size = (342, int(342 * h / w))
    else:
        size = (int(342 * w / h), 342)
    img = img.resize(size, Image.Resampling.BILINEAR)
    left = int(round((img.width - 299) / 2.0))
    top = int(round((img.height - 299) / 2.0))
    img = img.crop((left, top, left + 299, top + 299))
    x = np.asarray(img, dtype=np.float32) / np.float32(255)
    x = (x - np.array([0.485, 0.456, 0.406], np.float32)) / np.array(
        [0.229, 0.224, 0.225], np.float32
    )
    return np.ascontiguousarray(x[None])


def classify(path, x, device="cpu"):
    if device == "gpu":
        from ai_edge_litert.compiled_model import CompiledModel
        from ai_edge_litert.hardware_accelerator import HardwareAccelerator
        from ai_edge_litert.options import Options, GpuOptions
        gpu = GpuOptions(
            enforce_f32=True,
            allow_src_quantized_fc_conv_ops=True,
            hint_fully_delegated_to_single_delegate=True,
        )
        if hasattr(gpu, "use_ir_model"):
            gpu.use_ir_model = True
        options = Options(hardware_accelerators=HardwareAccelerator.GPU,
                          gpu_options=gpu)
        with CompiledModel.from_file(str(path), options=options) as model:
            if not model.is_fully_accelerated():
                raise RuntimeError("The model requires full GPU delegation.")
            inputs = model.create_input_buffers(0)
            outputs = model.create_output_buffers(0)
            inputs[0].write(x)
            model.run_by_index(0, inputs, outputs)
            return outputs[0].read(1000, np.float32).reshape(1, 1000)

    interpreter = Interpreter(model_path=str(path), num_threads=4)
    interpreter.allocate_tensors()
    inp, = interpreter.get_input_details()
    out, = interpreter.get_output_details()
    if np.issubdtype(inp["dtype"], np.integer):
        scale, zero_point = inp["quantization"]
        limits = np.iinfo(inp["dtype"])
        x = np.clip(np.rint(x / scale + zero_point), limits.min, limits.max).astype(inp["dtype"])
    interpreter.set_tensor(inp["index"], np.ascontiguousarray(x))
    interpreter.invoke()
    scores = interpreter.get_tensor(out["index"])
    if np.issubdtype(scores.dtype, np.integer):
        scale, zero_point = out["quantization"]
        scores = (scores.astype(np.float32) - zero_point) * scale
    return scores


def main():
    parser = argparse.ArgumentParser()
    parser.add_argument("--image", required=True)
    parser.add_argument("--variant", choices=VARIANTS, default="dynamic")
    parser.add_argument("--device", choices=("cpu", "gpu"), default="cpu")
    args = parser.parse_args()
    if args.device == "gpu" and args.variant == "int8":
        parser.error("Use --variant dynamic or fp32 for GPU inference.")
    path = hf_hub_download("litert-community/inception_v3", VARIANTS[args.variant])
    with Image.open(args.image) as image:
        x = preprocess(image)
    scores = classify(path, x, args.device)
    labels_path = hf_hub_download(
        "huggingface/label-files", "imagenet-1k-id2label.json", repo_type="dataset"
    )
    with open(labels_path) as stream:
        labels = json.load(stream)
    prediction = int(scores.argmax())
    print(f"Top-1 class index: {prediction}")
    print(f"Top-1 label: {labels[str(prediction)]}")


if __name__ == "__main__":
    main()

Run the desired variant:

python classify.py --image cat.jpg
python classify.py --image cat.jpg --variant fp32
python classify.py --image cat.jpg --variant int8
python classify.py --image cat.jpg --variant dynamic --device gpu

The current static INT8 input scale/zero point are 0.017822088673710823 and -14; output scale/zero point are 0.12506833672523499 and -81. Always read these values from the file's metadata. Input quantization uses NumPy's nearest rounding with ties to even.

BibTeX entry and citation info

@inproceedings{szegedy2016rethinking,
  title={Rethinking the inception architecture for computer vision},
  author={Szegedy, Christian and Vanhoucke, Vincent and Ioffe, Sergey and Shlens, Jon and Wojna, Zbigniew},
  booktitle={Proceedings of the IEEE conference on computer vision and pattern recognition},
  pages={2818--2826},
  year={2016}
}