File size: 4,898 Bytes
f52bdee 500c95c f52bdee 04640f7 500c95c 04640f7 fddebe1 d32803d fddebe1 7095627 737dd59 6e02140 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 | ---
license: apple-amlr
license_link: https://huggingface.co/apple/LensVLM-9B/blob/main/LICENSE
base_model_relation: quantized
language: en
library_name: mlx
pipeline_tag: image-text-to-text
tags:
- mlx
- text-generation-inference
- apple-silicon
- 4-bit
- 8-bit
- mlx-vlm
base_model:
- apple/LensVLM-9B
---
# **LensVLM-9B-MLX**
> **LensVLM-9B** is a 9-billion-parameter vision-language model from Apple, built on Qwen3.5-9B, introduced in the paper "LensVLM: Selective Context Expansion for Compressed Visual Representation of Text." Its core mechanism scans a compressed image representation of text — at configurable compression ratios of 5x, 10x, or 15x — and then selectively expands only the pages relevant to a given question back to their uncompressed form via learned tools, allowing the model to process very long documents without holding the entire uncompressed text in context. It's run via the accompanying `ml-lensvlm` codebase with a simple demo script accepting a text file and a question, and is released under the Apple Machine Learning Research Model License (with the accompanying source code separately licensed under the Apple Sample Code License).
```plaintext
+-- prithivMLmods/LensVLM-9B-MLX (main)
+-- / (Root: bf16)
+-- 4bit/ (Quantized: 4-bit)
+-- 8bit/ (Quantized: 8-bit)
```
## Use with mlx
Install the required library:
```bash
pip install -U mlx-vlm
```
> **Model Note:** `LensVLM-9B` is a 9-billion parameter vision-language model developed by Apple, specialized in compressed visual text processing and document-level multi-modal understanding.
### BF16 Variant (Base Weights)
The unquantized BF16 weights reside directly in the root repository directory:
#### CLI (Terminal)
```bash
python -m mlx_vlm generate \
--model prithivMLmods/LensVLM-9B-MLX \
--max-tokens 512 \
--temperature 0.0 \
--prompt "Read and analyze the contents of this document in detail." \
--image <path_to_image>
```
#### Python API
```python
from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config
model_path = "prithivMLmods/LensVLM-9B-MLX"
model, processor = load(model_path)
config = load_config(model_path)
image = ["<path_to_image>"]
prompt = "Read and analyze the contents of this document in detail."
formatted_prompt = apply_chat_template(processor, config, prompt, num_images=len(image))
output = generate(
model,
processor,
formatted_prompt,
image=image,
max_tokens=512,
temperature=0.0
)
print(output.text)
```
### 8-bit Variant
Target the `8bit` subfolder:
#### CLI (Terminal)
```bash
python -m mlx_vlm generate \
--model prithivMLmods/LensVLM-9B-MLX/8bit \
--max-tokens 512 \
--temperature 0.0 \
--prompt "Read and analyze the contents of this document in detail." \
--image <path_to_image>
```
#### Python API
```python
from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config
model_path = "prithivMLmods/LensVLM-9B-MLX"
model, processor = load(model_path, subfolder="8bit")
config = load_config(model_path, subfolder="8bit")
image = ["<path_to_image>"]
prompt = "Read and analyze the contents of this document in detail."
formatted_prompt = apply_chat_template(processor, config, prompt, num_images=len(image))
output = generate(
model,
processor,
formatted_prompt,
image=image,
max_tokens=512,
temperature=0.0
)
print(output.text)
```
### 4-bit Variant
Target the `4bit` subfolder:
#### CLI (Terminal)
```bash
python -m mlx_vlm generate \
--model prithivMLmods/LensVLM-9B-MLX/4bit \
--max-tokens 512 \
--temperature 0.0 \
--prompt "Read and analyze the contents of this document in detail." \
--image <path_to_image>
```
#### Python API
```python
from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config
model_path = "prithivMLmods/LensVLM-9B-MLX"
model, processor = load(model_path, subfolder="4bit")
config = load_config(model_path, subfolder="4bit")
image = ["<path_to_image>"]
prompt = "Read and analyze the contents of this document in detail."
formatted_prompt = apply_chat_template(processor, config, prompt, num_images=len(image))
output = generate(
model,
processor,
formatted_prompt,
image=image,
max_tokens=512,
temperature=0.0
)
print(output.text)
```
## License and Attribution
* **Model:** [apple/LensVLM-9B](https://huggingface.co/apple/LensVLM-9B)
* **Base Model:** [Qwen/Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B)
* **License:** This MLX compression follows the [Apple AMLR License](https://huggingface.co/apple/LensVLM-9B/blob/main/LICENSE) of the original LensVLM-9B model.
* **MLX-VLM:** https://github.com/Blaizzy/mlx-vlm
* **MLX:** https://github.com/ml-explore/mlx |