---
license: mit
base_model: openai/clip-vit-base-patch32
pipeline_tag: feature-extraction
library_name: coreai
tags:
  - core-ai
  - ios
  - clip
  - embeddings
---

# CLIP ViT-B/32 encoders for Core AI

[`openai/clip-vit-base-patch32`](https://huggingface.co/openai/clip-vit-base-patch32) exported to Apple Core AI as one `.aimodel` with separate text and image encoders, so text and images embed on their own into the same space.

Apple's [`models/clip`](https://github.com/apple/coreai-models/tree/main/models/clip) recipe traces one function that needs an image and a fixed batch of three prompts on every call, so this export uses [`export.py`](./export.py), based on it (coreai-torch 0.4.3, torch 2.13):

```sh
uv run --script export.py --output-dir exports --dtype float16
```

| Folder                    | Contents                                                                                                                                                                                                                                                   | Size   | Platforms  |
| ------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------ | ---------- |
| `clip-vit-base-patch32/`  | `clip-vit-base-patch32.aimodel` with `encode_text(input_ids, attention_mask) -> text_embeds` (`[batch, 77]` int32) and `encode_image(pixel_values) -> image_embeds` (`[batch, 3, 224, 224]` float16, CLIP mean and std), batch 1–64, 512-dim L2-normalized embeddings. `tokenizer/`: the CLIP tokenizer | 290 MB | iOS, macOS |

## Use

With [`@react-native-ai/core-ai`](https://github.com/callstackincubator/ai/tree/main/packages/core-ai), download `clip-vit-base-patch32/` into app storage, then embed text with the AI SDK and images with the native API:

```ts
import { coreAI, CoreAI } from '@react-native-ai/core-ai'
import { cosineSimilarity, embedMany } from 'ai'

const model = coreAI.embeddingModel({
  id: 'clip-vit-base-patch32',
  source: { type: 'file', uri: pathToFolder },
})
const { embeddings } = await embedMany({ model, values: ['a photo of a cat', 'a photo of a car'] })
const { embeddings: [image] } = await model.embedImages([imageUri])

cosineSimilarity(embeddings[0], image) // higher for the matching caption
```

## License

CLIP is MIT licensed. This repository only converts it to Core AI format.