Instructions to use NeuML/colbert-atto-9k-16d with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use NeuML/colbert-atto-9k-16d with sentence-transformers:
from pylate import models queries = [ "Which planet is known as the Red Planet?", "What is the largest planet in our solar system?", ] documents = [ ["Mars is the Red Planet.", "Venus is Earth's twin."], ["Jupiter is the largest planet.", "Saturn has rings."], ] model = models.ColBERT(model_name_or_path="NeuML/colbert-atto-9k-16d") queries_emb = model.encode(queries, is_query=True) docs_emb = model.encode(documents, is_query=False) - Transformers
How to use NeuML/colbert-atto-9k-16d with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("NeuML/colbert-atto-9k-16d", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
ColBERT Atto
Atto is a family of compact contextual models designed for parameter-efficient language representation learning.
This is a model fined-tuned using PyLate
Usage (txtai)
This model can be used to build embeddings databases with txtai for semantic search and/or as a knowledge source for retrieval augmented generation (RAG).
Note: txtai 9.0+ is required for late interaction model support
import txtai
model = "neuml/colbert-atto-9k-16d"
embeddings = txtai.Embeddings(
path=model,
content=True
)
embeddings.index(documents())
# Run a query
embeddings.search("query to run")
Late interaction models excel as reranker pipelines.
from txtai.pipeline import Reranker, Similarity
similarity = Similarity(path=model, lateencode=True)
ranker = Reranker(embeddings, similarity)
ranker("query to run")
Usage with Sentence Transformers
As of Sentence Transformers v6.0.0, this model loads directly as a multi-vector (ColBERT-style late interaction) retriever via the MultiVectorEncoder:
from sentence_transformers import MultiVectorEncoder
model = MultiVectorEncoder(model)
query = "What is the capital of France?"
documents = [
"Paris is the capital and largest city of France.",
"Berlin is the capital of Germany.",
]
query_embeddings = model.encode_query(query)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings[0].shape)
# MaxSim late-interaction scoring (higher is more relevant)
scores = model.similarity(query_embeddings, document_embeddings)
print(scores)
Evaluation
The following table shows a subset of BEIR scored with the txtai benchmarks script. Scores reported are ndcg@10.
| Model | Parameters | Dimensions | Vocab | ArguAna | NFCorpus | SciFact | Average |
|---|---|---|---|---|---|---|---|
| colbert-atto-840-8d | 840 | 8 | 250 | 0.2143 | 0.2585 | 0.4688 | 0.3139 |
| colbert-atto-9k-16d | 9K | 16 | 1K | 0.2459 | 0.2902 | 0.5604 | 0.3655 |
| colbert-atto-90k-32d | 90K | 32 | 4K | 0.3435 | 0.3263 | 0.6663 | 0.4454 |
| colbert-atto-950k-128d | 950K | 128 | 4K | 0.4149 | 0.3352 | 0.6946 | 0.4816 |
| colbert-atto-4m-384d | 4M | 384 | 30K | 0.4157 | 0.3349 | 0.6949 | 0.4818 |
| colbert-atto-14m-384d | 14M | 384 | 30K | 0.4248 | 0.3483 | 0.7057 | 0.4929 |
| colbert-atto-25m-384d | 25M | 384 | 30K | 0.4395 | 0.3461 | 0.7130 | 0.4995 |
| LateOn | 110M | 768 | 50K | 0.4979 | 0.3802 | 0.7609 | 0.5463 |
Training Parameters
| Parameter | Value |
|---|---|
| per_device_train_batch_size | 32 |
| num_train_epochs | 1 |
| learning_rate | 0.01 |
| warmup_steps | 0.05 |
| fp16 | True |
| eval_strategy | steps |
| dataloader_num_workers | 20 |
Full Model Architecture
ColBERT(
(0): Transformer({'max_seq_length': 399, 'do_lower_case': False, 'architecture': 'AttoModel'})
)
More Information
See the Introducing Atto article for benchmark results, architecture details, and training methodology.
- Downloads last month
- -
Model tree for NeuML/colbert-atto-9k-16d
Base model
NeuML/atto-9k-16d