Sentence Similarity
sentence-transformers
Safetensors
Danish
Swedish
Norwegian
llama
feature-extraction
text-embedding
embeddings
information-retrieval
beir
text-classification
text-clustering
llm2vec
custom_code
text-embeddings-inference
Instructions to use jealk/TTC-L2V-supervised-2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use jealk/TTC-L2V-supervised-2 with sentence-transformers:
from sentence_transformers import SentenceTransformer model = SentenceTransformer("jealk/TTC-L2V-supervised-2", trust_remote_code=True) sentences = [ "The weather is lovely today.", "It's so sunny outside!", "He drove to the stadium." ] embeddings = model.encode(sentences) similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] - Notebooks
- Google Colab
- Kaggle
Add quantization / model size section to README
Browse files
README.md
CHANGED
|
@@ -89,6 +89,27 @@ instruction shown above. Documents are encoded without an instruction.
|
|
| 89 |
Pooling is mean pooling over the content tokens; the instruction and the
|
| 90 |
beginning-of-text token are excluded — exactly as in LLM2Vec.
|
| 91 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 92 |
### Notes
|
| 93 |
|
| 94 |
- `trust_remote_code=True` is required: it loads the bidirectional Llama
|
|
|
|
| 89 |
Pooling is mean pooling over the content tokens; the instruction and the
|
| 90 |
beginning-of-text token are excluded — exactly as in LLM2Vec.
|
| 91 |
|
| 92 |
+
### Model size and quantization
|
| 93 |
+
|
| 94 |
+
The model has ~8B parameters and loads in roughly 14 GB (bfloat16). To run on
|
| 95 |
+
smaller GPUs it can be loaded quantized via `bitsandbytes`:
|
| 96 |
+
|
| 97 |
+
```python
|
| 98 |
+
from sentence_transformers import SentenceTransformer
|
| 99 |
+
from transformers import BitsAndBytesConfig
|
| 100 |
+
|
| 101 |
+
model = SentenceTransformer(
|
| 102 |
+
"jealk/TTC-L2V-supervised-2", trust_remote_code=True,
|
| 103 |
+
model_kwargs={"quantization_config": BitsAndBytesConfig(load_in_8bit=True)},
|
| 104 |
+
)
|
| 105 |
+
```
|
| 106 |
+
|
| 107 |
+
Approximate memory footprint: **bfloat16 ≈ 14 GB**, **8-bit ≈ 8 GB**,
|
| 108 |
+
**4-bit ≈ 5 GB** (`load_in_4bit=True`).
|
| 109 |
+
|
| 110 |
+
> The quantized variants have not been evaluated on benchmarks — embedding
|
| 111 |
+
> quality may differ from the full-precision model.
|
| 112 |
+
|
| 113 |
### Notes
|
| 114 |
|
| 115 |
- `trust_remote_code=True` is required: it loads the bidirectional Llama
|