metadata
license: mit
base_model: openai/clip-vit-base-patch32
pipeline_tag: feature-extraction
library_name: coreai
tags:
- core-ai
- ios
- clip
- embeddings
CLIP ViT-B/32 encoders for Core AI
openai/clip-vit-base-patch32 exported to Apple Core AI as one .aimodel with separate text and image encoders, so text and images embed on their own into the same space.
Apple's models/clip recipe traces one function that needs an image and a fixed batch of three prompts on every call, so this export uses export.py, based on it (coreai-torch 0.4.3, torch 2.13):
uv run --script export.py --output-dir exports --dtype float16
| Folder | Contents | Size | Platforms |
|---|---|---|---|
clip-vit-base-patch32/ |
clip-vit-base-patch32.aimodel with encode_text(input_ids, attention_mask) -> text_embeds ([batch, 77] int32) and encode_image(pixel_values) -> image_embeds ([batch, 3, 224, 224] float16, CLIP mean and std), batch 1–64, 512-dim L2-normalized embeddings. tokenizer/: the CLIP tokenizer |
290 MB | iOS, macOS |
Use
With @react-native-ai/core-ai, download clip-vit-base-patch32/ into app storage, then embed text with the AI SDK and images with the native API:
import { coreAI, CoreAI } from '@react-native-ai/core-ai'
import { cosineSimilarity, embedMany } from 'ai'
const model = coreAI.embeddingModel({
id: 'clip-vit-base-patch32',
source: { type: 'file', uri: pathToFolder },
})
const { embeddings } = await embedMany({ model, values: ['a photo of a cat', 'a photo of a car'] })
const { embeddings: [image] } = await model.embedImages([imageUri])
cosineSimilarity(embeddings[0], image) // higher for the matching caption
License
CLIP is MIT licensed. This repository only converts it to Core AI format.