Sentence Similarity
sentence-transformers
ONNX
Safetensors
Turkish
English
xlm-roberta
feature-extraction
cross-lingual-retrieval
scientific-literature
turkish
text-embeddings-inference
Instructions to use musapeker/tr-en-sci-e5-base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use musapeker/tr-en-sci-e5-base with sentence-transformers:
from sentence_transformers import SentenceTransformer model = SentenceTransformer("musapeker/tr-en-sci-e5-base") sentences = [ "The weather is lovely today.", "It's so sunny outside!", "He drove to the stadium." ] embeddings = model.encode(sentences) similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] - Notebooks
- Google Colab
- Kaggle
Add demo link and transformers.js usage
Browse files
README.md
CHANGED
|
@@ -24,6 +24,9 @@ datasets:
|
|
| 24 |
[intfloat/multilingual-e5-base](https://huggingface.co/intfloat/multilingual-e5-base) on ~380k Turkish–English
|
| 25 |
abstract pairs written by the authors of articles on [DergiPark](https://dergipark.org.tr).
|
| 26 |
|
|
|
|
|
|
|
|
|
|
| 27 |
> **Türkçe özet:** Türkçe sorguyla İngilizce bilimsel makale bulmak için eğitilmiş bir embedding modeli.
|
| 28 |
> DergiPark'taki yazarların kendi yazdığı Türkçe–İngilizce başlık, özet ve anahtar kelime çiftleriyle eğitildi.
|
| 29 |
> Eğitimde hiç görülmemiş dergilerden oluşan test kümesinde, kendisinin yaklaşık iki katı büyüklükteki
|
|
@@ -74,6 +77,18 @@ print(model.similarity(q_emb, d_emb)) # first document scores highest
|
|
| 74 |
|
| 75 |
For best results, index documents as `passage: <English title>. <English abstract>`.
|
| 76 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 77 |
## Training
|
| 78 |
|
| 79 |
| | |
|
|
|
|
| 24 |
[intfloat/multilingual-e5-base](https://huggingface.co/intfloat/multilingual-e5-base) on ~380k Turkish–English
|
| 25 |
abstract pairs written by the authors of articles on [DergiPark](https://dergipark.org.tr).
|
| 26 |
|
| 27 |
+
**Demo:** [huggingface.co/spaces/musapeker/tr-en-sci-search](https://huggingface.co/spaces/musapeker/tr-en-sci-search)
|
| 28 |
+
(runs entirely in the browser).
|
| 29 |
+
|
| 30 |
> **Türkçe özet:** Türkçe sorguyla İngilizce bilimsel makale bulmak için eğitilmiş bir embedding modeli.
|
| 31 |
> DergiPark'taki yazarların kendi yazdığı Türkçe–İngilizce başlık, özet ve anahtar kelime çiftleriyle eğitildi.
|
| 32 |
> Eğitimde hiç görülmemiş dergilerden oluşan test kümesinde, kendisinin yaklaşık iki katı büyüklükteki
|
|
|
|
| 77 |
|
| 78 |
For best results, index documents as `passage: <English title>. <English abstract>`.
|
| 79 |
|
| 80 |
+
### In the browser (transformers.js)
|
| 81 |
+
|
| 82 |
+
An int8 ONNX export (`onnx/model_quantized.onnx`, 279 MB) is included. On 1,000 test queries against 1,000 documents,
|
| 83 |
+
R@1 is 0.990 for the PyTorch model and 0.988 when int8 query embeddings are matched against PyTorch document embeddings.
|
| 84 |
+
|
| 85 |
+
```js
|
| 86 |
+
import { pipeline } from "@huggingface/transformers";
|
| 87 |
+
|
| 88 |
+
const extractor = await pipeline("feature-extraction", "musapeker/tr-en-sci-e5-base", { dtype: "q8" });
|
| 89 |
+
const out = await extractor(["query: tahrir defterleri ve Osmanlı nüfus tarihi"], { pooling: "mean", normalize: true });
|
| 90 |
+
```
|
| 91 |
+
|
| 92 |
## Training
|
| 93 |
|
| 94 |
| | |
|