Feature Extraction
Transformers
PyTorch
roberta
code-understanding
unixcoder
text-embeddings-inference
Instructions to use Lazyhope/RepoSim with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Lazyhope/RepoSim with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("feature-extraction", model="Lazyhope/RepoSim")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModel tokenizer = AutoTokenizer.from_pretrained("Lazyhope/RepoSim") model = AutoModel.from_pretrained("Lazyhope/RepoSim", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Add source id to self.device
Browse files- pipeline.py +1 -1
pipeline.py
CHANGED
|
@@ -144,7 +144,7 @@ class RepoEmbeddingPipeline(Pipeline):
|
|
| 144 |
+ [tokenizer.sep_token]
|
| 145 |
)
|
| 146 |
tokens_id = tokenizer.convert_tokens_to_ids(tokens)
|
| 147 |
-
source_ids = torch.tensor([tokens_id])
|
| 148 |
|
| 149 |
token_embeddings = self.model(source_ids)[0]
|
| 150 |
sentence_embeddings = token_embeddings.mean(dim=1)
|
|
|
|
| 144 |
+ [tokenizer.sep_token]
|
| 145 |
)
|
| 146 |
tokens_id = tokenizer.convert_tokens_to_ids(tokens)
|
| 147 |
+
source_ids = torch.tensor([tokens_id]).to(self.device)
|
| 148 |
|
| 149 |
token_embeddings = self.model(source_ids)[0]
|
| 150 |
sentence_embeddings = token_embeddings.mean(dim=1)
|