louis-pplx commited on
Commit
04f176b
·
verified ·
1 Parent(s): 373a895

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +43 -0
README.md CHANGED
@@ -97,6 +97,49 @@ embeddings = model_ctx.encode(doc_chunks)
97
 
98
  </details>
99
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
100
  ## Technical Details
101
 
102
  For comprehensive technical details and evaluation results, see our paper on arXiv.
 
97
 
98
  </details>
99
 
100
+ <details>
101
+ <summary> Using ONNX models </summary>
102
+
103
+ ```python
104
+
105
+ import onnxruntime as ort
106
+ from transformers import AutoTokenizer
107
+ import numpy as np
108
+
109
+ tokenizer = AutoTokenizer.from_pretrained("perplexity-ai/pplx-embed-v1-0.6b", trust_remote_code=True)
110
+ session = ort.InferenceSession("onnx/model.onnx")
111
+
112
+
113
+ texts = [
114
+ "Scientists explore the universe driven by curiosity.",
115
+ "Children learn through curious exploration.",
116
+ "Historical discoveries began with curious questions.",
117
+ "Animals use curiosity to adapt and survive.",
118
+ "Philosophy examines the nature of curiosity.",
119
+ ]
120
+
121
+ tokenized = tokenizer(
122
+ texts,
123
+ padding=True,
124
+ truncation=True,
125
+ return_tensors="np"
126
+ )
127
+
128
+ onnx_inputs = {
129
+ "input_ids": tokenized["input_ids"].astype(np.int64),
130
+ "attention_mask": tokenized["attention_mask"].astype(np.int64),
131
+ }
132
+
133
+ # Run inference
134
+ onnx_embeddings = session.run([out.name for out in session.get_outputs()], onnx_inputs)
135
+
136
+ # ONNX produces both int8 and binary precision embeddings:
137
+ int8_embeddings = onnx_embeddings[2]
138
+ binary_embeddings = onnx_embeddings[3]
139
+ ```
140
+
141
+ </details>
142
+
143
  ## Technical Details
144
 
145
  For comprehensive technical details and evaluation results, see our paper on arXiv.