OpenMed/North-Micro-Vision-Instruct-8bit-mlx

North Micro Vision Instruct — OpenMed MLX family

On-device, native-resolution vision-language inference for Apple Silicon, prepared for OpenMed's privacy-first clinical document workflows and the future Cohere Compass runtime in OpenMedKit.

These repositories contain MLX conversions of CohereLabs/North-Micro-Vision-Instruct, a compact 2.4B-parameter vision-language model released by Cohere under Apache 2.0. The same byte-identical README is used across the five precision variants; the repository name and config.json identify the precision.

Choose a precision

Repository	Weight payload	Intended trade-off
OpenMed/North-Micro-Vision-Instruct-4bit-mlx	2.02 GiB	Smallest affine variant; validate quality on your exact document set
OpenMed/North-Micro-Vision-Instruct-5bit-mlx	2.25 GiB	Compact middle ground
OpenMed/North-Micro-Vision-Instruct-6bit-mlx	2.48 GiB	Recommended first on-device quality/size trial
OpenMed/North-Micro-Vision-Instruct-8bit-mlx	2.93 GiB	Higher-fidelity quantized variant
OpenMed/North-Micro-Vision-Instruct-bf16-mlx	4.63 GiB	Full converted precision and quality reference

The 4/5/6/8-bit repositories use 64-element affine weight groups. Their vision tower remains in source precision; eligible language-model layers are quantized. This matters for document and OCR work because visual feature extraction is not compressed just to reduce the decoder footprint.

Why this model is interesting for OpenMedKit

North Micro Vision accepts interleaved text and images, preserves native image aspect ratios, and was trained for OCR, document understanding, charts, grounding, captioning, multilingual prompts, and multi-image conversations. That combination maps naturally to an OpenMedKit document pipeline:

1. Capture or import a page locally on iPhone, iPad, or Mac.
2. Keep image normalization, OCR/VLM inference, PII handling, and structured extraction on the user's device.
3. Validate generated fields against the source page before they enter a clinical record or workflow.
4. Redact or pseudonymize with OpenMedKit policies before any explicitly authorized export.
5. Never auto-trigger diagnosis, treatment, disclosure, or another consequential clinical decision from model output.

OpenMedKit's design defaults remain important even when the model is local: no telemetry by default, no cloud fallback for PHI, no raw clinical text in logs or analytics, synthetic fixtures in committed tests, and human review for consequential use.

Runtime status: read this before using Swift

The model weights use the standard MLX-VLM artifact layout and include the source tokenizer, chat template, native-resolution image processor metadata, and Cohere Compass configuration. They are validated today with the pinned Python MLX-VLM Compass runtime on Apple Silicon.

The released OpenMedKit Swift package does not yet contain a native Cohere Compass VLM implementation. Do not assume that downloading one of these repositories through OpenMedModelStore is sufficient for iOS inference. These artifacts are deliberately packaged without conversion-time Python dependencies so a future OpenMedKit Compass loader can consume the same weight payloads, but native Swift/iOS support must still land and pass physical-device memory, thermal, image, and text parity gates.

Until that runtime ships, use MLX-VLM on an Apple Silicon Mac for the model itself. OpenMedKit can still own capture, privacy policy, redaction, structured validation, and the surrounding app workflow. Do not send raw PHI from an iOS device to a Mac or service unless the user and deployment policy explicitly authorize that transfer.

Install the validated MLX runtime

Cohere Compass support is pinned to the exact MLX-VLM port used for conversion and validation:

python -m pip install -U \
  "git+https://github.com/Blaizzy/mlx-vlm.git@dd79a5d8caf3edafd6fa9e6326d7ce4977ddcbfc"

The validated environment uses MLX 0.32.0, MLX-LM 0.31.3, MLX-VLM 0.6.10 from that commit, and Transformers 5.15.0. Once Cohere Compass support is in a released MLX-VLM package, a normal pip install -U mlx-vlm can replace the commit pin after you rerun your own parity tests.

Image + text example

from mlx_vlm import apply_chat_template, generate, load

model_id = "OpenMed/North-Micro-Vision-Instruct-6bit-mlx"
model, processor = load(model_id)

prompt = apply_chat_template(
    processor,
    model.config,
    "Read this synthetic document and list the visible medication and dose.",
    num_images=1,
)
result = generate(
    model,
    processor,
    prompt,
    image="synthetic-clinical-note.png",
    max_tokens=128,
    temperature=0.0,
)
print(result.text)

Use synthetic or properly authorized content while prototyping. For a real clinical app, treat both the image and generated text as sensitive until the OpenMedKit privacy policy has been applied.

Text-only example

from mlx_vlm import apply_chat_template, generate, load

model, processor = load(
    "OpenMed/North-Micro-Vision-Instruct-6bit-mlx"
)
prompt = apply_chat_template(
    processor,
    model.config,
    "Explain in one sentence why local processing can improve document privacy.",
    num_images=0,
)
result = generate(
    model,
    processor,
    prompt,
    max_tokens=80,
    temperature=0.0,
)
print(result.text)

System prompts are not recommended by the source model authors. Keep instructions concise and place the task in the user message.

OpenMedKit integration gates

Before these weights are advertised as a native OpenMedKit/iOS runtime, the Swift implementation must demonstrate all of the following on a physical iPhone/iPad as well as Apple Silicon macOS:

• strict weight loading for bf16 and every affine bit width;
• tokenizer and chat-template parity with the validated Python path;
• native-resolution image preprocessing and visual-token count parity;
• coherent deterministic text-only generation;
• correct OCR/document answers on synthetic fixtures;
• bounded memory, launch latency, sustained thermals, and cancellation;
• no network fallback, telemetry, or plaintext PHI logging;
• provenance, model revision, and privacy-policy visibility in the app UI.

That is the bar for “OpenMedKit-ready”; an MLX file extension alone is not.

Validation performed before publication

Every OpenMed variant is loaded independently with strict tensor matching and must pass the same deterministic suite before any repository is made public:

• coherent text-only privacy explanation;
• exact fact extraction from a short synthetic text note;
• exact extraction of name, synthetic record ID, medication, dose, frequency, and allergy from a generated non-PHI clinical document image;
• exact category/value extraction from a generated chart image;
• model type, processor assets, quantization metadata, and weight-size checks;
• bf16 SHA-256 payload parity with an independently published Cohere-linked conversion made from the same pinned source and runtime port;
• quantized payload-size parity plus an explicit nonzero token-embedding gate (whole-file hashes are recorded but are not treated as a correctness proxy).

The fixtures are synthetic and are not clinical-quality evidence. Passing these checks establishes conversion/runtime coherence, not safety or accuracy for a particular medical workflow. Evaluate the exact document types, languages, device targets, and failure modes your app will encounter.

Source model scope and limitations

The source checkpoint supports native-resolution and multiple images, multilingual conversations, visual question answering, grounding, OCR, and chart/document understanding. Its language backbone supports a long context, but Cohere reports multimodal training and validation up to 8K tokens; do not silently claim longer multimodal reliability.

The source model is not a reasoning model, has limited math and code ability, does not support tool calling or agentic workflows, and is intended as a compact foundation for prototyping and specialization. Native-resolution images can materially increase memory use and latency.

This model and OpenMedKit are not medical devices. Outputs can be incomplete, incorrect, or fabricated. A qualified human must verify consequential use.

Reproducibility and credits

• Source: CohereLabs/North-Micro-Vision-Instruct
• Pinned source revision: 373bda96ac70bf89f99f7048f420cf00dc07c149
• MLX-VLM Compass port: dd79a5d8caf3edafd6fa9e6326d7ce4977ddcbfc
• OpenMed / OpenMedKit: github.com/maziyarpanahi/openmed

Thank you to Cohere for releasing North Micro Vision and to Prince Canuma and the MLX-VLM contributors for the Apple Silicon runtime and Compass port.

The converted weights retain the source model's Apache 2.0 license. OpenMed's SDK source is separately licensed under Apache 2.0.