Hugging Face's logo

tayarndt
/
LFM3-VL-2B-CoreAI

Image-Text-to-Text
coreai
apple
apple-silicon
macos
on-device
float16
vision-language
lfm3
liquid
conversational
Model card Files
xet
 Community

Model Card for LFM3-VL-2B-CoreAI

LFM3-VL-2B-CoreAI is LiquidAI/LFM3-VL-2B converted to Apple's Core AI format (.aimodel) in float16. It answers questions about a picture on the Mac's GPU, from Swift, with no server. A Swift package to run it is included.

This is a converted and modified copy of LFM3-VL-2B by Liquid AI, Inc., shared under the LFM Open License v1.0 (see LICENSE).

Model Details

Model Description

The weights are the original LFM3-VL-2B weights, cast to float16. Nothing was retrained. The model was converted with Apple's coreai-torch, using only Core AI's built-in operations: no custom Metal kernels and no other inference engine.

It is split into two Core AI files:

  • lfm3-vision-float16.aimodel (0.38 GB): the vision encoder and projector. It takes a 512 by 512 picture and returns 256 image vectors.

  • lfm3-text-float16.aimodel (3.46 GB): the language model, with three functions that share one set of weights.

    • embed turns token ids into vectors.
    • prefill reads the whole prompt.
    • main makes one token per call.

  • Developed by: Liquid AI, Inc. (original model). Converted to Core AI by tayarndt.

  • Shared by: tayarndt

  • Model type: vision-language model (image and text in, text out)

  • Format: Apple Core AI .aimodel, float16 weights

  • Language(s): same as the original: English, Arabic, Chinese, French, German, Italian, Japanese, Korean, Portuguese, Spanish, Vietnamese, Thai, Indonesian, Hindi, Russian, Polish

  • License: LFM Open License v1.0 (lfm1.0)

  • Converted from model: LiquidAI/LFM3-VL-2B, revision 23387beda43c60310348e83e25f78f33c3140d3a

Model Sources

Uses

Direct Use

Asking questions about one picture in a Mac app or command line tool, fully on device. For example: describing a photo, reading text in a picture, or answering a question about what a picture shows.

Downstream Use

Developers can add the included Swift package to a macOS app and build their own features on top of it, such as picture descriptions for accessibility, photo search, or reading screenshots.

Out-of-Scope Use

  • iPhone and iPad. These are not tested, and the files are above Apple's 2 GB guidance for iOS.
  • Video, several pictures in one prompt, or more than 4,096 tokens in total. These are not supported by this conversion.
  • Uses the original model is not meant for. Liquid AI does not recommend LFM3-VL-2B for coding, multi-turn conversations, or complex reasoning.
  • High-stakes decisions. Do not rely on its answers without a person checking them, for example in medical, legal, safety, or identity decisions.
  • Any use the LFM Open License does not allow.

Bias, Risks, and Limitations

This conversion carries all the biases, risks, and limitations of the original LFM3-VL-2B. Like any vision-language model, it can describe things that are not in the picture, misread text, or state guesses as facts. See the original model card.

Limits that come from this conversion:

  • Picture size. The picture is stretched to 512 by 512. The original model keeps the aspect ratio and accepts other sizes, so pictures that are not square may give worse answers. Answers on real photos were only checked by looking at a few of them.
  • Decoding. The Swift package uses greedy decoding: it always picks the most likely token. Liquid AI recommends sampling with temperature=0.1, top_k=50, repetition_penalty=1.1. Sampling and repetition penalty are not implemented yet, so long answers can repeat more than the original would.
  • Thinking. Thinking (the <think> block) is off by default to make answers faster. The original model thinks by default, and Liquid AI's benchmark scores use thinking mode.
  • Vision model with CPU-only settings. On macOS 27.2, loading the vision model with CPU-only settings crashes inside Apple's CPU engine (BNNS). The default setting and the GPU both work.
  • Float16 rounding. Float16 changes the numbers slightly from the original. In testing, every checked token still matched the original model (see Evaluation).

Recommendations

  • Have a person review answers before acting on them.
  • Turn thinking on (thinking: true, with maxTokens around 2048) when quality matters more than speed.
  • Check stopReason so you can tell a finished answer from one that was cut off.
  • If you announce streamed text with VoiceOver, group the pieces into sentences first. Each piece is often only part of a word.

How to Get Started with the Model

Requirements: a Mac with Apple silicon, macOS 27.2 or later, and Xcode 27.2 or later.

Download the model and try it from the command line:

hf download tayarndt/LFM3-VL-2B-CoreAI --local-dir LFM3-VL-2B-CoreAI
cd LFM3-VL-2B-CoreAI/swift/LFM3VL
swift build -c release
.build/release/lfm3-describe ../.. photo.jpg "What is in this picture?"

The first argument is the folder that holds the .aimodel files. Add --think to let the model reason first.

If the build says swift-transformers 1.4.0 could not be resolved, your Swift package cache is out of date. Run swift build -c release --disable-dependency-cache instead.

To use it in an app, add swift/LFM3VL as a local Swift package, then:

import LFM3VL

let model = try await LFM3VisionLanguageModel(folder: modelFolderURL)
let result = try await model.answer(image: cgImage, question: "What is in this picture?") { piece in
    print(piece, terminator: "")   // the text streams in as it is made
}
print(result.reply)
if result.stopReason == .maxTokens { print("The answer was cut off.") }

What the Swift API does

answer(image:question:thinking:maxTokens:onText:) returns an LFM3Answer:

  • text: everything the model wrote
  • reply: the answer without the reasoning
  • reasoning: the text inside <think> ... </think>, when thinking is on
  • tokenIDs: the tokens the model wrote
  • stopReason: .endOfAnswer or .maxTokens

Behavior:

  • The picture must be upright. The package does not read photo rotation tags. The lfm3-describe example shows how to read a file upright with ImageIO. Transparent areas become white.
  • maxTokens defaults to 512. Raise it when thinking is on.
  • onText pieces, joined together, equal text.
  • Overlapping calls. LFM3VisionLanguageModel is an actor. A second call to answer waits until the first one finishes.
  • Cancelling the calling task stops the answer, and it throws CancellationError.

Training Details

Training Data

No training was done. For the original model's training, see the LFM3-VL-2B model card.

Training Procedure

None. The conversion steps:

  1. Load the original weights and cast them to float16 in PyTorch.
  2. Rewrite the text decoder so its memory (k_cache, v_cache, conv_states) is a read-only input. Each call returns only the new part (new_k, new_v, new_conv), and the Swift package copies it into place.
  3. Give the one-token step (main) fixed shapes. It always looks at all 4,096 memory slots, and a mask hides the empty ones.
  4. Move the picture preprocessing into the vision graph (scale to -1 to 1, then cut into 16 by 16 patches in the original order).
  5. Export all three text functions through one TorchConverter, so they share one copy of the weights.

Why steps 2 and 3: on macOS 27.2, the GPU gave wrong tokens after the first one in two cases. One was when the model updated its own memory as Core AI state. The other was when the one-token step had flexible shapes. This design gives the same tokens as the original model.

Preprocessing

The app stretches the picture to 512 by 512 sRGB and scales each pixel to 0 to 1. The vision graph does the rest. The prompt uses the original chat template, with 256 <|image|> tokens where the picture goes.

Speeds, Sizes, Times

These were measured on one Mac: an Apple M5 Max with 128 GB of memory, running macOS 27.2 with Core AI's default compute choice. Other Macs will differ.

  • Files: 3.84 GB in total
  • Answer speed: 44 to 47 tokens per second
  • First token: 0.9 to 1.8 seconds after the call
  • Load: 13.7 seconds the first time. Later loads took 1.6 to 5.4 seconds, because Core AI caches the compiled model.
  • Peak memory footprint: about 1.1 GB, measured with /usr/bin/time -l after the first load. Memory on a first load was not measured.

Evaluation

Testing Data, Factors & Metrics

Testing Data

  • A made-up 512 by 512 test picture with text and colored shapes.
  • A rotated copy of it with a rotation tag.
  • A PNG with a transparent background.
  • A few built-in macOS photos.

Factors

  • Compute choice: GPU, CPU, and Core AI's default
  • Thinking on and off
  • Answers in English, French, Japanese, and with emoji

Metrics

  • PSNR between this model's outputs and the original float32 PyTorch model. Higher means closer. Apple's bar for float16 is 50 dB.
  • Token match: whether greedy decoding picks the same tokens as the original model.

No benchmark scores were measured for this conversion.

Results

Check Result
Vision output vs original, GPU and default 73.4 dB
First-token scores vs original, GPU and default 77.3 dB
First-token scores vs original, CPU 56.7 dB
First 40 tokens vs original, GPU, CPU and default 40 of 40 match
Swift package, test picture, thinking on all 120 reference tokens match
Streamed pieces vs returned text (French, Japanese, emoji) equal
Photo with a rotation tag, read with the example's ImageIO code read upright
PNG with transparency background read as white
Two overlapping calls on one model (lfm3-selftest) both equal a single call
Cancelling an answer (lfm3-selftest) stops it, and the next answer is still correct

Summary

On every check, the converted model picked the same tokens as the original. For benchmark quality, see the original model card. Its scores should carry over, apart from the limits listed above.

Environmental Impact

No training was done. The conversion and tests ran on one Mac for a few hours, and their energy use was not measured.

Technical Specifications

Model Architecture and Objective

These are the same as the original LFM3-VL-2B:

  • A vision encoder and projector
  • A 24-layer hybrid text decoder: 13 short convolution layers, 6 full attention layers, and 5 sliding-window attention layers with a window of 256
  • Hidden size 2,048
  • A vocabulary of 128,000 tokens, with the token table tied to the output layer

Compute Infrastructure

Hardware

  • To run: a Mac with Apple silicon. It was tested only on an Apple M5 Max with 128 GB of memory.
  • To convert: the same Mac.

Software

  • To run: macOS 27.2, the Core AI framework, Swift 6.4, and huggingface/swift-transformers 1.4.0 for the tokenizer
  • To convert: coreai-torch 0.4.3, coreai-core 1.0.0b3, torch 2.13.0, transformers 5.22.2, Python 3.12.16, and Xcode 27.2

Citation

Please cite the original model:

@article{liquidAI2026LFM3,
  author  = {Liquid AI},
  title   = {Introducing LFM3: Every Modality for Specialized Intelligence},
  journal = {Liquid AI Blog},
  year    = {2026},
  note    = {www.liquid.ai/blog/lfm3},
}

License

LFM3-VL-2B is made by Liquid AI, Inc. and released under the LFM Open License v1.0, included here as LICENSE.

The two .aimodel files are modified: converted from PyTorch to Core AI, cast to float16, the text decoder split into embed, prefill and main, and the vision model changed to take a plain picture. The tokenizer files and LICENSE are unchanged.

This is a summary only, not legal advice: commercial use is allowed only for companies with under 10 million US dollars in annual revenue, and qualified non-profit organizations have separate terms for non-commercial and research use. Read the full LICENSE.

Model Card Authors

tayarndt

Model Card Contact

Use the Community tab of this repository. For questions about the original model, contact Liquid AI.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
Image-Text-to-Text
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for tayarndt/LFM3-VL-2B-CoreAI

Base model

LiquidAI/LFM3-1.7B-Base
Finetuned
LiquidAI/LFM3-VL-2B
Quantized
(42)
this model