| --- |
| license: apache-2.0 |
| base_model: Qwen/Qwen3-VL-2B-Instruct |
| pipeline_tag: image-text-to-text |
| tags: |
| - core-ai |
| - coreai-models |
| - ios |
| - vision-language-model |
| --- |
| |
| # Qwen3-VL 2B Instruct for Core AI |
|
|
| [Qwen/Qwen3-VL-2B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-2B-Instruct) exported with the [apple/coreai-models](https://github.com/apple/coreai-models) 1.0.0 VLM recipe, for `CoreAIVisionLanguageModel` (`CoreAILanguageModels`) and [`@react-native-ai/core-ai`](https://github.com/callstackincubator/ai/tree/main/packages/core-ai). |
|
|
| ## Folder |
|
|
| `qwen3_vl_2b/` is the bundle (`metadata.json` `kind: "vlm"`), 2.7 GB: |
|
|
| | File | Role | Size | |
| | --------------------- | -------------------------------------------------------- | ------ | |
| | `qwen3_vl_2b.aimodel` | Text decoder, 4-bit weights, stateful KV cache (4096) | 923 MB | |
| | `vision.aimodel` | Vision encoder, 448x448 input, 196 image tokens, float16 | 1.2 GB | |
| | `embed.aimodel` | Token embeddings | 594 MB | |
| | `tokenizer/` | Hugging Face tokenizer | 11 MB | |
|
|
| ## Platforms |
|
|
| - **iOS 27 (iPhone 17 Pro)**: works with the [increased memory limit](https://developer.apple.com/documentation/bundleresources/entitlements/com.apple.developer.kernel.increased-memory-limit) entitlement, which raises the app limit from 3,376 MB to 6,144 MB. Without it the app aborts while MPSGraph loads the weights. Answering three questions about a 320x240 photo, including loading the model, took 39 s. |
| - **macOS 27**: the recipe's target format. |
|
|
| The VLM recipe has no iOS preset, so this is the macOS export format (one `main` function per model, run on the GPU). |
|
|
| ## Export |
|
|
| ```bash |
| git clone https://github.com/apple/coreai-models && cd coreai-models && git checkout 1.0.0 |
| uv run coreai.vlm.export qwen3-vl --compression 4bit |
| ``` |
|
|
| In 1.0.0, `--compression 4bit` fails with `build_reference_inputs returned keys ['input_ids', ...], which are not a contiguous in-order prefix of forward's parameters ['inputs_embeds', ...]`, because `Qwen3VLForCausalLMEmbeddings` inherits token-id reference inputs. This export overrides `export_input_names`, `build_reference_inputs` and `build_dynamic_shapes` in `python/src/coreai_models/models/macos/qwen3_vl.py` to trace `inputs_embeds` (random `[1, query_len, hidden_size]` tensors) in place of `input_ids`. The model code is otherwise unchanged. |
|
|
| ## Use |
|
|
| ```ts |
| import { coreAI } from '@react-native-ai/core-ai' |
| import { generateText } from 'ai' |
| |
| const model = coreAI.languageModel({ |
| id: 'qwen3-vl-2b', |
| source: { type: 'file', uri: bundleFolder }, |
| }) |
| |
| const { text } = await generateText({ |
| model, |
| messages: [ |
| { |
| role: 'user', |
| content: [ |
| { type: 'file', mediaType: 'image/jpeg', data: new URL(`file://${imagePath}`) }, |
| { type: 'text', text: 'What animals are in this picture?' }, |
| ], |
| }, |
| ], |
| }) |
| ``` |
|
|
| Apple's VLM engine reads the first image of the prompt, decodes greedily, and needs an image in every prompt. |
|
|
| ## License |
|
|
| Apache-2.0, as the [base model](https://huggingface.co/Qwen/Qwen3-VL-2B-Instruct). |
|
|
