cstr/deepseek-ocr2-crispembed-GGUF

DeepSeek-OCR2 — CrispEmbed GGUF

GGUF conversion of deepseek-ai/DeepSeek-OCR-2 for use with CrispEmbed.

Architecture

SAM-ViT-B (12L, 768d) → Qwen2 encoder (24L, 896d, bidirectional) → Linear projector (896→1280) → DeepSeek-V2 MoE decoder (12L, 1280d, 64 experts top-6 + 2 shared, layer 0 dense) → lm_head

Models

File	Quant	Size	Description
deepseek-ocr2-f16.gguf	F16	6.4 GB	Full precision
deepseek-ocr2-q8_0.gguf	Q8_0	~3.4 GB	Best quality/size balance
deepseek-ocr2-q4_k.gguf	Q4_K	~2.0 GB	Smallest, good quality

Performance features

• Per-row embedding dequant (saves ~655 MB peak RSS vs full table expansion)
• MoE decoder on Metal via ggml_mul_mat_id
• SAM patch-embed + neck on Metal via ggml_conv_2d
• Qwen2 encoder on Metal graph

Converted with models/convert-deepseek-ocr2-to-gguf.py from CrispEmbed.

Provenance and EU AI Act Art. 53 note

• Upstream model: deepseek-ai/DeepSeek-OCR-2 — published by deepseek-ai.
• Upstream licence: apache-2.0. This repository redistributes under the same terms; it grants no rights the upstream licence does not.
• What was done here: format conversion and/or quantisation only (GGUF). No training, no fine-tuning, no merging, no distillation, no change to architecture, vocabulary or capability. Only the numeric representation of the upstream weights differs.
• Training data: documented — where it is documented at all — by the upstream provider; see the upstream model card. No training data was used, added or selected by this repository. No training-content summary was found on the upstream model card at the time of writing; that documentation gap is upstream's and is not filled here.
• Provider status: under Regulation (EU) 2024/1689 the upstream authors remain the provider of this model. Converting the serialisation format does not make this repository the provider of a new general-purpose AI model, and no such claim is made. Questions about training content, copyright policy or model capability belong upstream.