| --- |
| license: mit |
| pipeline_tag: image-feature-extraction |
| library_name: executorch |
| --- |
| |
| # clip-vit-base-patch32 |
|
|
| This repository hosts the **clip-vit-base-patch32** models exported for the |
| [React Native ExecuTorch](https://www.npmjs.com/package/react-native-executorch) |
| library as ExecuTorch `.pte` programs, ready to run on device. |
|
|
| Upstream model: [clip-vit-base-patch32](https://huggingface.co/openai/clip-vit-base-patch32) |
|
|
| ## Variants |
|
|
| | Path | Backend | Precision | |
| | --- | --- | --- | |
| | `coreml/clip_vit_base_patch32_image_coreml_fp16.pte` | coreml | fp16 | |
| | `coreml/clip_vit_base_patch32_text_coreml_fp16.pte` | coreml | fp16 | |
| | `coreml/clip_vit_base_patch32_text_coreml_int8.pte` | coreml | int8 | |
| | `mlx/clip_vit_base_patch32_image_mlx_int8.pte` | mlx | int8 | |
| | `vulkan/clip_vit_base_patch32_image_vulkan_fp16.pte` | vulkan | fp16 | |
| | `vulkan/clip_vit_base_patch32_text_vulkan_fp16.pte` | vulkan | fp16 | |
| | `xnnpack/clip_vit_base_patch32_image_xnnpack_fp32.pte` | xnnpack | fp32 | |
| | `xnnpack/clip_vit_base_patch32_image_xnnpack_int8.pte` | xnnpack | int8 | |
| | `xnnpack/clip_vit_base_patch32_text_xnnpack_fp32.pte` | xnnpack | fp32 | |
| | `xnnpack/clip_vit_base_patch32_text_xnnpack_int8.pte` | xnnpack | int8 | |
|
|
| ## XNNPACK int8 |
|
|
| The `xnnpack/*_int8.pte` towers are 8da8w (int8 dynamic activations, int8 per-channel |
| weights) on the Linear layers; the text tower also has an int8 token embedding table. |
| ImageNetV2 zero-shot top-1 (1000 images, one per class, "a photo of a {class}."): |
|
|
| | image | text | top-1 | |
| | --- | --- | --- | |
| | fp32 | fp32 | 51.4 | |
| | int8 | fp32 | 51.5 | |
| | fp32 | int8 | 51.2 | |
| | int8 | int8 | 51.3 | |
|
|
| ## Repository structure |
|
|
| ``` |
| config.json 43 B |
| coreml/clip_vit_base_patch32_image_coreml_fp16.pte 168 MB |
| coreml/clip_vit_base_patch32_text_coreml_fp16.pte 122 MB |
| coreml/clip_vit_base_patch32_text_coreml_int8.pte 61.0 MB |
| coreml/config.json 1.7 kB |
| mlx/clip_vit_base_patch32_image_mlx_int8.pte 93.7 MB |
| mlx/config.json 840 B |
| tokenizer.json 2.1 MB |
| vulkan/clip_vit_base_patch32_image_vulkan_fp16.pte 168 MB |
| vulkan/clip_vit_base_patch32_text_vulkan_fp16.pte 121 MB |
| vulkan/config.json 1.7 kB |
| xnnpack/clip_vit_base_patch32_image_xnnpack_fp32.pte 335 MB |
| xnnpack/clip_vit_base_patch32_image_xnnpack_int8.pte 91.5 MB |
| xnnpack/clip_vit_base_patch32_text_xnnpack_fp32.pte 242 MB |
| xnnpack/clip_vit_base_patch32_text_xnnpack_int8.pte 61.4 MB |
| xnnpack/config.json 2.9 kB |
| ``` |
|
|
| ## Compatibility |
|
|
| These files are published for the **ExecuTorch v1.4.1** runtime. ExecuTorch |
| gives no forward compatibility guarantee, so an older runtime may fail to load |
| them. |
|
|
| To use them in React Native ExecuTorch, pass the model constant shipped in the |
| library's model registry to the corresponding task pipeline. See the |
| [documentation](https://docs.swmansion.com/react-native-executorch/docs/fundamentals/downloading-models). |
|
|
| To load these files in your own ExecuTorch runtime, read the |
| [compatibility note](https://github.com/pytorch/executorch/blob/main/runtime/COMPATIBILITY.md) |
| first. |
|
|
