Instructions to use opensearch-project/opensearch-neural-sparse-encoding-v1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use opensearch-project/opensearch-neural-sparse-encoding-v1 with sentence-transformers:
from sentence_transformers import SparseEncoder model = SparseEncoder("opensearch-project/opensearch-neural-sparse-encoding-v1") queries = ["Which planet is known as the Red Planet?"] documents = [ "Venus is often called Earth's twin because of its similar size and proximity.", "Mars, known for its reddish appearance, is often referred to as the Red Planet.", "Jupiter, the largest planet in our solar system, has a prominent red spot.", ] query_embeddings = model.encode_query(queries) document_embeddings = model.encode_document(documents) similarities = model.similarity(query_embeddings, document_embeddings) print(similarities) - Transformers
How to use opensearch-project/opensearch-neural-sparse-encoding-v1 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("feature-extraction", model="opensearch-project/opensearch-neural-sparse-encoding-v1")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForMaskedLM tokenizer = AutoTokenizer.from_pretrained("opensearch-project/opensearch-neural-sparse-encoding-v1") model = AutoModelForMaskedLM.from_pretrained("opensearch-project/opensearch-neural-sparse-encoding-v1", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Adding `safetensors` variant of this model
This is an automated PR created with https://huggingface.co/spaces/safetensors/convert
This new file is equivalent to pytorch_model.bin but safe in the sense that
no arbitrary code can be put into it.
These files also happen to load much faster than their pytorch counterpart:
https://colab.research.google.com/github/huggingface/notebooks/blob/main/safetensors_doc/en/speed.ipynb
The widgets on your model page will run using this model even if this is not merged
making sure the file actually works.
If you find any issues: please report here: https://huggingface.co/spaces/safetensors/convert/discussions
Feel free to ignore this PR.