Visual Document Retrieval
Transformers
Safetensors
ColPali
ops_colqwen3
feature-extraction
multimodal_embedding
embedding
multilingual-embedding
colqwen3
custom_code
Instructions to use OpenSearch-AI/Ops-Colqwen3-4B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use OpenSearch-AI/Ops-Colqwen3-4B with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("OpenSearch-AI/Ops-Colqwen3-4B", trust_remote_code=True, device_map="auto") - ColPali
How to use OpenSearch-AI/Ops-Colqwen3-4B with ColPali:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Add Sentence Transformers usage section to the model card
Browse files
README.md
CHANGED
|
@@ -6,6 +6,8 @@ pipeline_tag: visual-document-retrieval
|
|
| 6 |
library_name: transformers
|
| 7 |
tags:
|
| 8 |
- transformers
|
|
|
|
|
|
|
| 9 |
- multimodal_embedding
|
| 10 |
- embedding
|
| 11 |
- colpali
|
|
@@ -30,6 +32,47 @@ The model is trained using a multi-stage strategy that combines large-scale text
|
|
| 30 |
|
| 31 |
## Usage
|
| 32 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 33 |
**Requirements**
|
| 34 |
```
|
| 35 |
pillow
|
|
|
|
| 6 |
library_name: transformers
|
| 7 |
tags:
|
| 8 |
- transformers
|
| 9 |
+
- sentence-transformers
|
| 10 |
+
- multi-vector
|
| 11 |
- multimodal_embedding
|
| 12 |
- embedding
|
| 13 |
- colpali
|
|
|
|
| 32 |
|
| 33 |
## Usage
|
| 34 |
|
| 35 |
+
### Sentence Transformers
|
| 36 |
+
|
| 37 |
+
This model can be used with [Sentence Transformers](https://www.sbert.net/) as a multi-vector (ColBERT-style late interaction) retriever via the `MultiVectorEncoder`:
|
| 38 |
+
|
| 39 |
+
```bash
|
| 40 |
+
pip install "sentence-transformers[image]>=6.0.0"
|
| 41 |
+
```
|
| 42 |
+
|
| 43 |
+
```python
|
| 44 |
+
from sentence_transformers import MultiVectorEncoder
|
| 45 |
+
|
| 46 |
+
model = MultiVectorEncoder(
|
| 47 |
+
"OpenSearch-AI/Ops-Colqwen3-4B",
|
| 48 |
+
trust_remote_code=True,
|
| 49 |
+
model_kwargs={"dtype": "bfloat16"},
|
| 50 |
+
)
|
| 51 |
+
|
| 52 |
+
queries = [
|
| 53 |
+
"What is the variable represented on the y-axis of the graph?",
|
| 54 |
+
"Total outlay is maximum in which year?",
|
| 55 |
+
]
|
| 56 |
+
images = [
|
| 57 |
+
"https://huggingface.co/datasets/sentence-transformers/example-documents/resolve/main/doc1.jpg",
|
| 58 |
+
"https://huggingface.co/datasets/sentence-transformers/example-documents/resolve/main/doc2.jpg",
|
| 59 |
+
]
|
| 60 |
+
|
| 61 |
+
query_embeddings = model.encode_query(queries)
|
| 62 |
+
image_embeddings = model.encode_document(images)
|
| 63 |
+
print(query_embeddings[0].shape, image_embeddings[0].shape)
|
| 64 |
+
# torch.Size([25, 2560]) torch.Size([1254, 2560])
|
| 65 |
+
|
| 66 |
+
# Diagonal should have higher scores
|
| 67 |
+
scores = model.similarity(query_embeddings, image_embeddings)
|
| 68 |
+
print(scores)
|
| 69 |
+
# tensor([[17.4668, 12.9785],
|
| 70 |
+
# [ 7.0088, 15.9492]], device='cuda:0')
|
| 71 |
+
```
|
| 72 |
+
|
| 73 |
+
### Transformers
|
| 74 |
+
|
| 75 |
+
|
| 76 |
**Requirements**
|
| 77 |
```
|
| 78 |
pillow
|