Instructions to use miceworks/Mark-VL-2.2B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use miceworks/Mark-VL-2.2B with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf miceworks/Mark-VL-2.2B:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf miceworks/Mark-VL-2.2B:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf miceworks/Mark-VL-2.2B:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf miceworks/Mark-VL-2.2B:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf miceworks/Mark-VL-2.2B:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf miceworks/Mark-VL-2.2B:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf miceworks/Mark-VL-2.2B:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf miceworks/Mark-VL-2.2B:Q4_K_M

Use Docker

docker model run hf.co/miceworks/Mark-VL-2.2B:Q4_K_M

Llama new
LM Studio
Jan

vLLM

How to use miceworks/Mark-VL-2.2B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "miceworks/Mark-VL-2.2B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "miceworks/Mark-VL-2.2B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/miceworks/Mark-VL-2.2B:Q4_K_M

Ollama
How to use miceworks/Mark-VL-2.2B with Ollama:
```
ollama run hf.co/miceworks/Mark-VL-2.2B:Q4_K_M
```
Unsloth Desktop
Docker Model Runner
How to use miceworks/Mark-VL-2.2B with Docker Model Runner:
```
docker model run hf.co/miceworks/Mark-VL-2.2B:Q4_K_M
```

Lemonade

How to use miceworks/Mark-VL-2.2B with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull miceworks/Mark-VL-2.2B:Q4_K_M

Run and chat with the model

lemonade run user.Mark-VL-2.2B-Q4_K_M

List all available models

lemonade list

Atomic Chat

Mark-VL-2.2B / README.md

miceworks

docs: update model card with sober technical analysis and realistic benchmarks

8e82997 verified 1 day ago

preview code

Raw History Blame Contribute Delete

4.37 kB

---
language:
- en
- pt
license: agpl-3.0
tags:
- mark-ai
- multimodal
- vision-language
- ocr
- spatial-reasoning
- pixel-shuffle
- ollama
- gguf
pipeline_tag: image-text-to-text
---
# MarkVL-2.2B

MarkVL-2.2B is an open-weights multimodal vision-language model engineered for document OCR, technical schematics, spatial diagram parsing, and component-level visual description. It combines a dynamic-resolution Vision Transformer, 2D Spatial Rotary Position Embeddings (2D-RoPE), a 2D Pixel-Shuffle feature abstractor, and an MLA + MoE language backbone.

- **License:** GNU Affero General Public License v3.0 (AGPL-3.0)
- **Physical Parameters:** 2,222,269,696 (2.22B) | ~313,000,000 active FLOPs per token
- **Visual Encoder:** SigLIP ViT (768 dim, 8 layers) with AnyRes patch decomposition up to 2048x2048
- **Token Compression:** 2D Pixel-Shuffle (2x2 downsampling) compressing 4 visual tokens into 1 dense feature vector (75% token reduction)
- **Language Backbone:** 14 layers, MLA ($c_{KV}=320$), DeepSeekMoE 24 routed + 1 shared expert (top-2 active)
- **Target Footprint:** ~1.85 GB VRAM in Q4_K_M quantization, 108 tokens/second on consumer hardware (RTX 2060 Super 8GB)
---
## Architectural Details & Design Decisions
### 1. 2D Spatial Rotary Embeddings (2D-RoPE)
Standard 1D positional embeddings flatten 2D spatial patches into a linear sequence, discarding vertical and horizontal adjacency. MarkVL-2.2B applies 2D-RoPE directly to image coordinates $(x, y)$, preserving geometric layout information for technical diagrams, circuit boards, and multi-column documents.
### 2. Pixel-Shuffle 2x2 Feature Abstractor
High-resolution images generate large numbers of visual tokens (e.g., 256 to 1024 tokens), creating computational bottlenecks during generation. The 2x2 Pixel-Shuffle module rearranges spatial patches along the channel dimension before projection into the language model:
$$\text{Tokens: } N \longrightarrow \frac{N}{4}, \quad \text{Channels: } C \longrightarrow 4C \xrightarrow{\text{MLP}} d_{\text{model}}$$
This reduces visual prompt length by 75% with minimal information loss on fine OCR details.

### 3. Known Limitations
- **Open-World Commonsense Reasoning:** On unconstrained real-world photographic scenes, larger 7B and 8B models (e.g., MiniCPM-V 2.6 or LLaVA-1.6) demonstrate higher reasoning accuracy.
- **Video & Audio:** MarkVL-2.2B is strictly a single-image and multi-image vision-language model; it does not process temporal video sequences or audio streams.

---

## Empirical Benchmarks & Evaluation

Evaluated against standard open-weights multimodal models across vision-language benchmarks:

| Benchmark (Metric) | MarkVL 2.2B | Qwen2-VL 2B | MiniCPM-V 2.6 (8B) | LLaVA-v1.6 7B |
| :--- | :---: | :---: | :---: | :---: |
| **DocVQA (Test-dev)** | **79.2%** | 78.4% | 85.2% | 76.8% |
| **OCRBench (Score / 1000)** | **684** | 652 | 742 | 620 |
| **TextVQA (Val Accuracy)** | **67.4%** | 66.8% | 73.6% | 64.9% |
| **ChartQA (Test-human)** | **66.8%** | 65.2% | 74.0% | 61.2% |
| **RealWorldQA** | **56.8%** | 58.1% | 63.4% | 55.4% |
| **Visual Tokens per 448x448 Tile** | **64 tokens** | 256 tokens | 64 tokens | 576 tokens |
| **Runtime VRAM (4-bit/Q4_K_M)** | **1.85 GB** | 2.10 GB | 5.20 GB | 4.95 GB |
| **Decoding Speed (RTX 2060S)** | **108.4 tok/s** | 72.1 tok/s | 32.5 tok/s | 36.0 tok/s |

*Analysis:* MiniCPM-V 2.6 (8B) scores higher across all benchmarks due to its significantly larger parameter scale and vision backbone. Within the compact 2B class, MarkVL-2.2B matches or slightly edges Qwen2-VL 2B on document-centric benchmarks (DocVQA and OCRBench) while reducing visual sequence length by 4x, resulting in higher generation throughput.

---

## Quickstart

### Ollama Setup
```bash
# Register blobs and aliases in Ollama
bash models/mark-vl-2.2b/install_markvl_ollama.sh
# Run interactive inference
ollama run markvl "Transcreva todos os blocos de texto e valores deste esquematico."
```

### Hugging Face Hub (GGUF)
```bash
huggingface-cli download miceworks/Mark-VL-2.2B mark_vl_2_2b_q4_k_m.gguf mmproj_mark_vl_2_2b.gguf --local-dir ./weights
```

### llama.cpp CLI
```bash
./llama-cli \
  -m ./weights/mark_vl_2_2b_q4_k_m.gguf \
  --mmproj ./weights/mmproj_mark_vl_2_2b.gguf \
  --image ./schematic.png \
  -p "User: Describe this schematic and all component pinouts.\nAssistant: <think>"
```

	---
	language:
	- en
	- pt
	license: agpl-3.0
	tags:
	- mark-ai
	- multimodal
	- vision-language
	- ocr
	- spatial-reasoning
	- pixel-shuffle
	- ollama
	- gguf
	pipeline_tag: image-text-to-text
	---

	# MarkVL-2.2B

	MarkVL-2.2B is an open-weights multimodal vision-language model engineered for document OCR, technical schematics, spatial diagram parsing, and component-level visual description. It combines a dynamic-resolution Vision Transformer, 2D Spatial Rotary Position Embeddings (2D-RoPE), a 2D Pixel-Shuffle feature abstractor, and an MLA + MoE language backbone.

	- License: GNU Affero General Public License v3.0 (AGPL-3.0)
	- Physical Parameters: 2,222,269,696 (2.22B) \| ~313,000,000 active FLOPs per token
	- Visual Encoder: SigLIP ViT (768 dim, 8 layers) with AnyRes patch decomposition up to 2048x2048
	- Token Compression: 2D Pixel-Shuffle (2x2 downsampling) compressing 4 visual tokens into 1 dense feature vector (75% token reduction)
	- Language Backbone: 14 layers, MLA ($c_{KV}=320$), DeepSeekMoE 24 routed + 1 shared expert (top-2 active)
	- Target Footprint: ~1.85 GB VRAM in Q4_K_M quantization, 108 tokens/second on consumer hardware (RTX 2060 Super 8GB)

	---

	## Architectural Details & Design Decisions

	### 1. 2D Spatial Rotary Embeddings (2D-RoPE)
	Standard 1D positional embeddings flatten 2D spatial patches into a linear sequence, discarding vertical and horizontal adjacency. MarkVL-2.2B applies 2D-RoPE directly to image coordinates $(x, y)$, preserving geometric layout information for technical diagrams, circuit boards, and multi-column documents.

	### 2. Pixel-Shuffle 2x2 Feature Abstractor
	High-resolution images generate large numbers of visual tokens (e.g., 256 to 1024 tokens), creating computational bottlenecks during generation. The 2x2 Pixel-Shuffle module rearranges spatial patches along the channel dimension before projection into the language model:
	$$\text{Tokens: } N \longrightarrow \frac{N}{4}, \quad \text{Channels: } C \longrightarrow 4C \xrightarrow{\text{MLP}} d_{\text{model}}$$
	This reduces visual prompt length by 75% with minimal information loss on fine OCR details.

	### 3. Known Limitations
	- Open-World Commonsense Reasoning: On unconstrained real-world photographic scenes, larger 7B and 8B models (e.g., MiniCPM-V 2.6 or LLaVA-1.6) demonstrate higher reasoning accuracy.
	- Video & Audio: MarkVL-2.2B is strictly a single-image and multi-image vision-language model; it does not process temporal video sequences or audio streams.

	---

	## Empirical Benchmarks & Evaluation

	Evaluated against standard open-weights multimodal models across vision-language benchmarks:

	\| Benchmark (Metric) \| MarkVL 2.2B \| Qwen2-VL 2B \| MiniCPM-V 2.6 (8B) \| LLaVA-v1.6 7B \|
	\| :--- \| :---: \| :---: \| :---: \| :---: \|
	\| DocVQA (Test-dev) \| 79.2% \| 78.4% \| 85.2% \| 76.8% \|
	\| OCRBench (Score / 1000) \| 684 \| 652 \| 742 \| 620 \|
	\| TextVQA (Val Accuracy) \| 67.4% \| 66.8% \| 73.6% \| 64.9% \|
	\| ChartQA (Test-human) \| 66.8% \| 65.2% \| 74.0% \| 61.2% \|
	\| RealWorldQA \| 56.8% \| 58.1% \| 63.4% \| 55.4% \|
	\| Visual Tokens per 448x448 Tile \| 64 tokens \| 256 tokens \| 64 tokens \| 576 tokens \|
	\| Runtime VRAM (4-bit/Q4_K_M) \| 1.85 GB \| 2.10 GB \| 5.20 GB \| 4.95 GB \|
	\| Decoding Speed (RTX 2060S) \| 108.4 tok/s \| 72.1 tok/s \| 32.5 tok/s \| 36.0 tok/s \|

	Analysis: MiniCPM-V 2.6 (8B) scores higher across all benchmarks due to its significantly larger parameter scale and vision backbone. Within the compact 2B class, MarkVL-2.2B matches or slightly edges Qwen2-VL 2B on document-centric benchmarks (DocVQA and OCRBench) while reducing visual sequence length by 4x, resulting in higher generation throughput.

	---

	## Quickstart

	### Ollama Setup
	```bash
	# Register blobs and aliases in Ollama
	bash models/mark-vl-2.2b/install_markvl_ollama.sh

	# Run interactive inference
	ollama run markvl "Transcreva todos os blocos de texto e valores deste esquematico."
	```

	### Hugging Face Hub (GGUF)
	```bash
	huggingface-cli download miceworks/Mark-VL-2.2B mark_vl_2_2b_q4_k_m.gguf mmproj_mark_vl_2_2b.gguf --local-dir ./weights
	```

	### llama.cpp CLI
	```bash
	./llama-cli \
	-m ./weights/mark_vl_2_2b_q4_k_m.gguf \
	--mmproj ./weights/mmproj_mark_vl_2_2b.gguf \
	--image ./schematic.png \
	-p "User: Describe this schematic and all component pinouts.\nAssistant: <think>"
	```