Feature Extraction
Transformers
ONNX
Safetensors
multilingual
bidirectional_pplx_qwen3
sentence-similarity
conteb
contextual-embeddings
custom_code
text-embeddings-inference
Instructions to use perplexity-ai/pplx-embed-context-v1-4b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use perplexity-ai/pplx-embed-context-v1-4b with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("feature-extraction", model="perplexity-ai/pplx-embed-context-v1-4b", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("perplexity-ai/pplx-embed-context-v1-4b", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -97,6 +97,49 @@ embeddings = model_ctx.encode(doc_chunks)
|
|
| 97 |
|
| 98 |
</details>
|
| 99 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 100 |
## Technical Details
|
| 101 |
|
| 102 |
For comprehensive technical details and evaluation results, see our paper on arXiv.
|
|
|
|
| 97 |
|
| 98 |
</details>
|
| 99 |
|
| 100 |
+
<details>
|
| 101 |
+
<summary> Using ONNX models </summary>
|
| 102 |
+
|
| 103 |
+
```python
|
| 104 |
+
|
| 105 |
+
import onnxruntime as ort
|
| 106 |
+
from transformers import AutoTokenizer
|
| 107 |
+
import numpy as np
|
| 108 |
+
|
| 109 |
+
tokenizer = AutoTokenizer.from_pretrained("perplexity-ai/pplx-embed-v1-0.6b", trust_remote_code=True)
|
| 110 |
+
session = ort.InferenceSession("onnx/model.onnx")
|
| 111 |
+
|
| 112 |
+
|
| 113 |
+
texts = [
|
| 114 |
+
"Scientists explore the universe driven by curiosity.",
|
| 115 |
+
"Children learn through curious exploration.",
|
| 116 |
+
"Historical discoveries began with curious questions.",
|
| 117 |
+
"Animals use curiosity to adapt and survive.",
|
| 118 |
+
"Philosophy examines the nature of curiosity.",
|
| 119 |
+
]
|
| 120 |
+
|
| 121 |
+
tokenized = tokenizer(
|
| 122 |
+
texts,
|
| 123 |
+
padding=True,
|
| 124 |
+
truncation=True,
|
| 125 |
+
return_tensors="np"
|
| 126 |
+
)
|
| 127 |
+
|
| 128 |
+
onnx_inputs = {
|
| 129 |
+
"input_ids": tokenized["input_ids"].astype(np.int64),
|
| 130 |
+
"attention_mask": tokenized["attention_mask"].astype(np.int64),
|
| 131 |
+
}
|
| 132 |
+
|
| 133 |
+
# Run inference
|
| 134 |
+
onnx_embeddings = session.run([out.name for out in session.get_outputs()], onnx_inputs)
|
| 135 |
+
|
| 136 |
+
# ONNX produces both int8 and binary precision embeddings:
|
| 137 |
+
int8_embeddings = onnx_embeddings[2]
|
| 138 |
+
binary_embeddings = onnx_embeddings[3]
|
| 139 |
+
```
|
| 140 |
+
|
| 141 |
+
</details>
|
| 142 |
+
|
| 143 |
## Technical Details
|
| 144 |
|
| 145 |
For comprehensive technical details and evaluation results, see our paper on arXiv.
|