---
language:
- en
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- dense
- generated_from_trainer
- dataset_size:110608
- loss:MultipleNegativesRankingLoss
base_model: sentence-transformers/all-MiniLM-L6-v2
widget:
- source_sentence: proficiency in Yiddish required for this position
sentences:
- write Telugu
- Latvian
- Yiddish
- source_sentence: knowledge of international cinema is a necessity
sentences:
- SPARK
- Italian
- film studies
- source_sentence: experience with casting anode pins is a plus.
sentences:
- 'plan evaluation '
- synthetic natural environment
- cast cell covers
- source_sentence: Native speakers of Telugu are especially encouraged to apply.
sentences:
- Romanian
- Maltego
- Telugu
- source_sentence: looking for experienced animal hunters to join our team
sentences:
- animal hunting
- use databases
- electrical engineering
datasets:
- TechWolf/Synthetic-ESCO-skill-sentences
pipeline_tag: sentence-similarity
library_name: sentence-transformers
---
# SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2
This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) on the [synthetic-esco-skill-sentences](https://huggingface.co/datasets/TechWolf/Synthetic-ESCO-skill-sentences) dataset. It maps inputs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, classification, clustering, and more.
## Model Details
### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2)
- **Maximum Sequence Length:** 256 tokens
- **Output Dimensionality:** 384 dimensions
- **Similarity Function:** Cosine Similarity
- **Supported Modality:** Text
- **Training Dataset:**
- [synthetic-esco-skill-sentences](https://huggingface.co/datasets/TechWolf/Synthetic-ESCO-skill-sentences)
- **Language:** en
### Model Sources
- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/huggingface/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
### Full Model Architecture
```
SentenceTransformer(
(0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'})
(1): Pooling({'embedding_dimension': 384, 'pooling_mode': 'mean', 'include_prompt': True})
(2): Normalize({'module_input_name': 'sentence_embedding', 'module_output_name': 'sentence_embedding'})
)
```
## Usage
### Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
```bash
pip install -U sentence-transformers
```
Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer
# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
'looking for experienced animal hunters to join our team',
'animal hunting',
'electrical engineering',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]
# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000, 0.8278, -0.1120],
# [ 0.8278, 1.0000, -0.1666],
# [-0.1120, -0.1666, 1.0000]])
```
## Training Details
### Training Dataset
#### synthetic-esco-skill-sentences
* Dataset: [synthetic-esco-skill-sentences](https://huggingface.co/datasets/TechWolf/Synthetic-ESCO-skill-sentences) at [09fc487](https://huggingface.co/datasets/TechWolf/Synthetic-ESCO-skill-sentences/tree/09fc487d1394c1fdfd224cb621a89b14af925716)
* Size: 110,608 training samples
* Columns: anchor and positive
* Approximate statistics based on the first 100 samples:
| | anchor | positive |
|:---------|:-----------------------------------------------------------------------------------|:---------------------------------------------------------------------------------|
| type | string | string |
| modality | text | text |
| details |
Applicants should be skilled in examining legislative drafts for legal compliance and accuracy. | examine legislative drafts |
| Candidate must have knowledge in photographic film processing, including monitoring chemical baths. | monitor film development baths |
| we are looking for an experienced meat grinder to join our team | grind meat |
* Loss: [MultipleNegativesRankingLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
```json
{
"scale": 20.0,
"similarity_fct": "cos_sim",
"gather_across_devices": false,
"directions": [
"query_to_doc"
],
"partition_mode": "joint",
"hardness_mode": null,
"hardness_strength": 0.0
}
```
### Evaluation Dataset
#### synthetic-esco-skill-sentences
* Dataset: [synthetic-esco-skill-sentences](https://huggingface.co/datasets/TechWolf/Synthetic-ESCO-skill-sentences) at [09fc487](https://huggingface.co/datasets/TechWolf/Synthetic-ESCO-skill-sentences/tree/09fc487d1394c1fdfd224cb621a89b14af925716)
* Size: 27,652 evaluation samples
* Columns: anchor and positive
* Approximate statistics based on the first 100 samples:
| | anchor | positive |
|:---------|:-----------------------------------------------------------------------------------|:---------------------------------------------------------------------------------|
| type | string | string |
| modality | text | text |
| details | the position requires attention to detail and strong organizational skills in order to maintain accurate records of security incidents. | document security incidents in the store |
| We need someone with experience in export logistics to lead our transportation team | coordinate export transportation activities |
| The ideal candidate will have experience in developing food policy that meets the social objectives of our company. | develop food policy |
* Loss: [MultipleNegativesRankingLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
```json
{
"scale": 20.0,
"similarity_fct": "cos_sim",
"gather_across_devices": false,
"directions": [
"query_to_doc"
],
"partition_mode": "joint",
"hardness_mode": null,
"hardness_strength": 0.0
}
```
### Training Hyperparameters
#### Non-Default Hyperparameters
- `per_device_train_batch_size`: 128
- `num_train_epochs`: 50
- `learning_rate`: 0.0001
- `weight_decay`: 0.01
- `per_device_eval_batch_size`: 128
- `dataloader_pin_memory`: False
#### All Hyperparameters