Image-Text-to-Text
MLX
Safetensors
English
cohere_compass
mlx-vlm
vision-language
multimodal
cohere
north
quantized
conversational
8-bit precision
Instructions to use mlx-community/North-Micro-Vision-Instruct-8bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use mlx-community/North-Micro-Vision-Instruct-8bit with MLX:
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("mlx-community/North-Micro-Vision-Instruct-8bit") config = load_config("mlx-community/North-Micro-Vision-Instruct-8bit") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
Add 8-bit affine model card
Browse files
README.md
CHANGED
|
@@ -15,11 +15,62 @@ language:
|
|
| 15 |
- zh
|
| 16 |
- ar
|
| 17 |
tags:
|
|
|
|
| 18 |
- vision
|
| 19 |
- multimodal
|
| 20 |
- conversational
|
| 21 |
- multilingual
|
| 22 |
- native-resolution
|
| 23 |
-
- mlx
|
| 24 |
base_model: CohereLabs/North-Micro-Vision-Instruct
|
|
|
|
| 25 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 15 |
- zh
|
| 16 |
- ar
|
| 17 |
tags:
|
| 18 |
+
- mlx
|
| 19 |
- vision
|
| 20 |
- multimodal
|
| 21 |
- conversational
|
| 22 |
- multilingual
|
| 23 |
- native-resolution
|
|
|
|
| 24 |
base_model: CohereLabs/North-Micro-Vision-Instruct
|
| 25 |
+
base_model_relation: quantized
|
| 26 |
---
|
| 27 |
+
|
| 28 |
+
# North Micro Vision Instruct — 8-bit affine (MLX)
|
| 29 |
+
|
| 30 |
+
This repository contains an **MLX conversion** of [CohereLabs/North-Micro-Vision-Instruct](https://huggingface.co/CohereLabs/North-Micro-Vision-Instruct), prepared for inference with [MLX-VLM](https://github.com/Blaizzy/mlx-vlm).
|
| 31 |
+
|
| 32 |
+
North Micro Vision Instruct is a compact 2.4B-parameter vision-language model from Cohere. It supports native-resolution and multi-image inputs, multilingual conversations, visual question answering, captioning, grounding, OCR, and chart/document understanding.
|
| 33 |
+
|
| 34 |
+
## Conversion details
|
| 35 |
+
|
| 36 |
+
- **Quantization:** 8-bit affine
|
| 37 |
+
- **Bits:** 8
|
| 38 |
+
- **Group size:** 64
|
| 39 |
+
- **Vision tower:** Preserved in the source precision
|
| 40 |
+
|
| 41 |
+
The model weights were converted from the official checkpoint. Refer to the [original model card](https://huggingface.co/CohereLabs/North-Micro-Vision-Instruct) for architecture details, evaluation results, intended uses, and limitations.
|
| 42 |
+
|
| 43 |
+
## Installation
|
| 44 |
+
|
| 45 |
+
Cohere Compass support is currently available on the MLX-VLM port branch. Until it is included in a released package, install it with:
|
| 46 |
+
|
| 47 |
+
```bash
|
| 48 |
+
pip install -U "git+https://github.com/Blaizzy/mlx-vlm.git@pc/mlx-vlm-port"
|
| 49 |
+
```
|
| 50 |
+
|
| 51 |
+
After Compass support is released, the standard package can be used:
|
| 52 |
+
|
| 53 |
+
```bash
|
| 54 |
+
pip install -U mlx-vlm
|
| 55 |
+
```
|
| 56 |
+
|
| 57 |
+
## Usage
|
| 58 |
+
|
| 59 |
+
```bash
|
| 60 |
+
mlx_vlm.generate \
|
| 61 |
+
--model mlx-community/North-Micro-Vision-Instruct-8bit \
|
| 62 |
+
--image <path-or-URL-to-image> \
|
| 63 |
+
--prompt "What do you see?" \
|
| 64 |
+
--max-tokens 128 \
|
| 65 |
+
--temperature 0
|
| 66 |
+
```
|
| 67 |
+
|
| 68 |
+
The original model was trained with multimodal contexts up to 8K tokens. Although its language backbone supports a longer context, longer multimodal prompts were not validated by the model authors.
|
| 69 |
+
|
| 70 |
+
## Other MLX variants
|
| 71 |
+
|
| 72 |
+
BF16 and quantized conversions are available in the [North-Vision collection](https://huggingface.co/collections/mlx-community/north-vision-6a7c9be6ccc1cd992a83aecb).
|
| 73 |
+
|
| 74 |
+
## License
|
| 75 |
+
|
| 76 |
+
This conversion is released under the source model's [Apache 2.0 license](https://www.apache.org/licenses/LICENSE-2.0). Use is also subject to the limitations and responsible-use guidance in the [original model card](https://huggingface.co/CohereLabs/North-Micro-Vision-Instruct).
|