prince-canuma commited on
Commit
44bfc8f
·
verified ·
1 Parent(s): 4980ae4

Add 8-bit affine model card

Browse files
Files changed (1) hide show
  1. README.md +52 -1
README.md CHANGED
@@ -15,11 +15,62 @@ language:
15
  - zh
16
  - ar
17
  tags:
 
18
  - vision
19
  - multimodal
20
  - conversational
21
  - multilingual
22
  - native-resolution
23
- - mlx
24
  base_model: CohereLabs/North-Micro-Vision-Instruct
 
25
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
15
  - zh
16
  - ar
17
  tags:
18
+ - mlx
19
  - vision
20
  - multimodal
21
  - conversational
22
  - multilingual
23
  - native-resolution
 
24
  base_model: CohereLabs/North-Micro-Vision-Instruct
25
+ base_model_relation: quantized
26
  ---
27
+
28
+ # North Micro Vision Instruct — 8-bit affine (MLX)
29
+
30
+ This repository contains an **MLX conversion** of [CohereLabs/North-Micro-Vision-Instruct](https://huggingface.co/CohereLabs/North-Micro-Vision-Instruct), prepared for inference with [MLX-VLM](https://github.com/Blaizzy/mlx-vlm).
31
+
32
+ North Micro Vision Instruct is a compact 2.4B-parameter vision-language model from Cohere. It supports native-resolution and multi-image inputs, multilingual conversations, visual question answering, captioning, grounding, OCR, and chart/document understanding.
33
+
34
+ ## Conversion details
35
+
36
+ - **Quantization:** 8-bit affine
37
+ - **Bits:** 8
38
+ - **Group size:** 64
39
+ - **Vision tower:** Preserved in the source precision
40
+
41
+ The model weights were converted from the official checkpoint. Refer to the [original model card](https://huggingface.co/CohereLabs/North-Micro-Vision-Instruct) for architecture details, evaluation results, intended uses, and limitations.
42
+
43
+ ## Installation
44
+
45
+ Cohere Compass support is currently available on the MLX-VLM port branch. Until it is included in a released package, install it with:
46
+
47
+ ```bash
48
+ pip install -U "git+https://github.com/Blaizzy/mlx-vlm.git@pc/mlx-vlm-port"
49
+ ```
50
+
51
+ After Compass support is released, the standard package can be used:
52
+
53
+ ```bash
54
+ pip install -U mlx-vlm
55
+ ```
56
+
57
+ ## Usage
58
+
59
+ ```bash
60
+ mlx_vlm.generate \
61
+ --model mlx-community/North-Micro-Vision-Instruct-8bit \
62
+ --image <path-or-URL-to-image> \
63
+ --prompt "What do you see?" \
64
+ --max-tokens 128 \
65
+ --temperature 0
66
+ ```
67
+
68
+ The original model was trained with multimodal contexts up to 8K tokens. Although its language backbone supports a longer context, longer multimodal prompts were not validated by the model authors.
69
+
70
+ ## Other MLX variants
71
+
72
+ BF16 and quantized conversions are available in the [North-Vision collection](https://huggingface.co/collections/mlx-community/north-vision-6a7c9be6ccc1cd992a83aecb).
73
+
74
+ ## License
75
+
76
+ This conversion is released under the source model's [Apache 2.0 license](https://www.apache.org/licenses/LICENSE-2.0). Use is also subject to the limitations and responsible-use guidance in the [original model card](https://huggingface.co/CohereLabs/North-Micro-Vision-Instruct).