Sentence Similarity
sentence-transformers
Safetensors
bert
feature-extraction
Generated from Trainer
dataset_size:991580
loss:MultipleNegativesRankingLoss
text-embeddings-inference
Instructions to use blackmlengineer/occucoder-en-venti-v0 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use blackmlengineer/occucoder-en-venti-v0 with sentence-transformers:
from sentence_transformers import SentenceTransformer model = SentenceTransformer("blackmlengineer/occucoder-en-venti-v0") sentences = [ "Draw Off Worker", "Computer Systems Analysts", "Meat, Poultry, and Fish Cutters and Trimmers", "Food Preparation Workers" ] embeddings = model.encode(sentences) similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [4, 4] - Notebooks
- Google Colab
- Kaggle
|
Download README.md from blackmlengineer/occucoder-en-venti-v0: direct link, hf CLI and curl.
- Browser
- Download file 14.4 kB
-
https://huggingface.co/blackmlengineer/occucoder-en-venti-v0/resolve/main/README.md
- Command line
-
hf download hf://blackmlengineer/occucoder-en-venti-v0/README.md
-
curl -L -o README.md https://huggingface.co/blackmlengineer/occucoder-en-venti-v0/resolve/main/README.md
14.4 kB
metadata
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:991580
- loss:MultipleNegativesRankingLoss
widget:
- source_sentence: Draw Off Worker
sentences:
- Computer Systems Analysts
- Meat, Poultry, and Fish Cutters and Trimmers
- Food Preparation Workers
- source_sentence: College Tutor
sentences:
- Endoscopy Technicians
- Mathematical Science Teachers, Postsecondary
- Tutors
- source_sentence: Carbon Plant Grinder
sentences:
- >-
Crushing, Grinding, and Polishing Machine Setters, Operators, and
Tenders
- Coating, Painting, and Spraying Machine Setters, Operators, and Tenders
- Adhesive Bonding Machine Operators and Tenders
- source_sentence: Roofing Supervisor
sentences:
- Solar Energy Installation Managers
- First-Line Supervisors of Construction Trades and Extraction Workers
- Agricultural Inspectors
- source_sentence: Meat Wrapper
sentences:
- Laundry and Dry-Cleaning Workers
- Printing Press Operators
- Packers and Packagers, Hand
pipeline_tag: sentence-similarity
library_name: sentence-transformers
SentenceTransformer
This is a sentence-transformers model trained on the train dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for retrieval.
Model Details
Model Description
- Model Type: Sentence Transformer
- Maximum Sequence Length: 32 tokens
- Output Dimensionality: 768 dimensions
- Similarity Function: Cosine Similarity
- Supported Modality: Text
- Training Dataset:
- train
Model Sources
- Documentation: Sentence Transformers Documentation
- Repository: Sentence Transformers on GitHub
- Hugging Face: Sentence Transformers on Hugging Face
Full Model Architecture
SentenceTransformer(
(0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'})
(1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'mean', 'include_prompt': True})
)
Usage
Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
pip install -U sentence-transformers
Then you can load this model and run inference.
from sentence_transformers import SentenceTransformer
# Download from the 🤗 Hub
model = SentenceTransformer("blackmlengineer/occucoder-en-venti-v0")
# Run inference
sentences = [
'Meat Wrapper',
'Packers and Packagers, Hand',
'Laundry and Dry-Cleaning Workers',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]
# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.6317, 0.1758],
# [0.6317, 1.0000, 0.1272],
# [0.1758, 0.1272, 1.0000]])
Training Details
Training Dataset
train
- Dataset: train
- Size: 991,580 training samples
- Columns:
anchor,positive, andnegative - Approximate statistics based on the first 100 samples:
anchor positive negative type string string string modality text text text details - min: 3 tokens
- mean: 4.61 tokens
- max: 8 tokens
- min: 3 tokens
- mean: 7.7 tokens
- max: 19 tokens
- min: 3 tokens
- mean: 7.95 tokens
- max: 22 tokens
- Samples:
anchor positive negative Personnel PsychologistIndustrial-Organizational PsychologistsEducational, Guidance, and Career Counselors and AdvisorsProperty Claims AdjusterClaims Adjusters, Examiners, and InvestigatorsAdministrative Law Judges, Adjudicators, and Hearing OfficersAssistant ProfessorArt, Drama, and Music Teachers, PostsecondaryArea, Ethnic, and Cultural Studies Teachers, Postsecondary - Loss:
MultipleNegativesRankingLosswith these parameters:{ "scale": 20.0, "similarity_fct": "cos_sim", "gather_across_devices": false, "directions": [ "query_to_doc" ], "partition_mode": "joint", "hardness_mode": null, "hardness_strength": 0.0 }
Training Hyperparameters
Non-Default Hyperparameters
per_device_train_batch_size: 128num_train_epochs: 5
All Hyperparameters
Click to expand
per_device_train_batch_size: 128num_train_epochs: 5max_steps: -1learning_rate: 5e-05lr_scheduler_type: linearlr_scheduler_kwargs: Nonewarmup_steps: 0optim: adamw_torch_fusedoptim_args: Noneweight_decay: 0.0adam_beta1: 0.9adam_beta2: 0.999adam_epsilon: 1e-08optim_target_modules: Nonegradient_accumulation_steps: 1average_tokens_across_devices: Truemax_grad_norm: 1.0label_smoothing_factor: 0.0bf16: Falsefp16: Falsebf16_full_eval: Falsefp16_full_eval: Falsetf32: Nonegradient_checkpointing: Falsegradient_checkpointing_kwargs: Nonetorch_compile: Falsetorch_compile_backend: Nonetorch_compile_mode: Noneuse_liger_kernel: Falseliger_kernel_config: Noneuse_cache: Falseneftune_noise_alpha: Nonetorch_empty_cache_steps: Noneauto_find_batch_size: Falselog_on_each_node: Truelogging_nan_inf_filter: Trueinclude_num_input_tokens_seen: nolog_level: passivelog_level_replica: warningdisable_tqdm: Falseproject: huggingfacetrackio_space_id: Nonetrackio_bucket_id: Nonetrackio_static_space_id: Noneper_device_eval_batch_size: 8prediction_loss_only: Trueeval_on_start: Falseeval_do_concat_batches: Trueeval_use_gather_object: Falseeval_accumulation_steps: Noneinclude_for_metrics: []batch_eval_metrics: Falsesave_only_model: Falsesave_on_each_node: Falseenable_jit_checkpoint: Falsepush_to_hub: Falsehub_private_repo: Nonehub_model_id: Nonehub_strategy: every_savehub_always_push: Falsehub_revision: Noneload_best_model_at_end: Falseignore_data_skip: Falserestore_callback_states_from_checkpoint: Falsefull_determinism: Falseseed: 42data_seed: Noneuse_cpu: Falseaccelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}parallelism_config: Nonedataloader_drop_last: Falsedataloader_num_workers: 0dataloader_pin_memory: Truedataloader_persistent_workers: Falsedataloader_prefetch_factor: Noneremove_unused_columns: Truelabel_names: Nonetrain_sampling_strategy: randomlength_column_name: lengthddp_find_unused_parameters: Noneddp_bucket_cap_mb: Noneddp_broadcast_buffers: Falseddp_static_graph: Noneddp_backend: Noneddp_timeout: 1800fsdp: Nonefsdp_config: Nonedeepspeed: Nonedebug: []skip_memory_metrics: Truedo_predict: Falseresume_from_checkpoint: Nonewarmup_ratio: Nonelocal_rank: -1prompts: Nonebatch_sampler: batch_samplermulti_dataset_batch_sampler: proportionalrouter_mapping: {}learning_rate_mapping: {}
Training Logs
| Epoch | Step | Training Loss |
|---|---|---|
| 0.0645 | 500 | 2.5686 |
| 0.1291 | 1000 | 1.9404 |
| 0.1936 | 1500 | 1.6797 |
| 0.2582 | 2000 | 1.5128 |
| 0.3227 | 2500 | 1.4104 |
| 0.3872 | 3000 | 1.3169 |
| 0.4518 | 3500 | 1.2508 |
| 0.5163 | 4000 | 1.1896 |
| 0.5809 | 4500 | 1.1410 |
| 0.6454 | 5000 | 1.1091 |
| 0.7100 | 5500 | 1.0743 |
| 0.7745 | 6000 | 1.0442 |
| 0.8390 | 6500 | 1.0229 |
| 0.9036 | 7000 | 0.9909 |
| 0.9681 | 7500 | 0.9812 |
| 1.0327 | 8000 | 0.9573 |
| 1.0972 | 8500 | 0.9400 |
| 1.1617 | 9000 | 0.9321 |
| 1.2263 | 9500 | 0.9085 |
| 1.2908 | 10000 | 0.8988 |
| 1.3554 | 10500 | 0.8988 |
| 1.4199 | 11000 | 0.8897 |
| 1.4844 | 11500 | 0.8875 |
| 1.5490 | 12000 | 0.8695 |
| 1.6135 | 12500 | 0.8551 |
| 1.6781 | 13000 | 0.8574 |
| 1.7426 | 13500 | 0.8489 |
| 1.8072 | 14000 | 0.8399 |
| 1.8717 | 14500 | 0.8389 |
| 1.9362 | 15000 | 0.8304 |
| 2.0008 | 15500 | 0.8286 |
| 2.0653 | 16000 | 0.8100 |
| 2.1299 | 16500 | 0.8146 |
| 2.1944 | 17000 | 0.8120 |
| 2.2589 | 17500 | 0.8063 |
| 2.3235 | 18000 | 0.8044 |
| 2.3880 | 18500 | 0.8059 |
| 2.4526 | 19000 | 0.7951 |
| 2.5171 | 19500 | 0.7924 |
| 2.5816 | 20000 | 0.7857 |
| 2.6462 | 20500 | 0.7888 |
| 2.7107 | 21000 | 0.7850 |
| 2.7753 | 21500 | 0.7801 |
| 2.8398 | 22000 | 0.7813 |
| 2.9044 | 22500 | 0.7772 |
| 2.9689 | 23000 | 0.7742 |
| 3.0334 | 23500 | 0.7706 |
| 3.0980 | 24000 | 0.7693 |
| 3.1625 | 24500 | 0.7641 |
| 3.2271 | 25000 | 0.7543 |
| 3.2916 | 25500 | 0.7555 |
| 3.3561 | 26000 | 0.7615 |
| 3.4207 | 26500 | 0.7583 |
| 3.4852 | 27000 | 0.7625 |
| 3.5498 | 27500 | 0.7599 |
| 3.6143 | 28000 | 0.7529 |
| 3.6788 | 28500 | 0.7554 |
| 3.7434 | 29000 | 0.7561 |
| 3.8079 | 29500 | 0.7523 |
| 3.8725 | 30000 | 0.7518 |
| 3.9370 | 30500 | 0.7481 |
| 4.0015 | 31000 | 0.7459 |
| 4.0661 | 31500 | 0.7404 |
| 4.1306 | 32000 | 0.7297 |
| 4.1952 | 32500 | 0.7408 |
| 4.2597 | 33000 | 0.7376 |
| 4.3243 | 33500 | 0.7394 |
| 4.3888 | 34000 | 0.7422 |
| 4.4533 | 34500 | 0.7311 |
| 4.5179 | 35000 | 0.7283 |
| 4.5824 | 35500 | 0.7356 |
| 4.6470 | 36000 | 0.7309 |
| 4.7115 | 36500 | 0.7377 |
| 4.7760 | 37000 | 0.7353 |
| 4.8406 | 37500 | 0.7286 |
| 4.9051 | 38000 | 0.7360 |
| 4.9697 | 38500 | 0.7317 |
Training Time
- Training: 4.1 hours
Framework Versions
- Python: 3.14.5
- Sentence Transformers: 5.6.0
- Transformers: 5.12.1
- PyTorch: 2.12.0
- Accelerate: 1.14.0
- Datasets: 5.0.0
- Tokenizers: 0.22.2
Citation
BibTeX
Sentence Transformers
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
MultipleNegativesRankingLoss
@misc{oord2019representationlearningcontrastivepredictive,
title={Representation Learning with Contrastive Predictive Coding},
author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
year={2019},
eprint={1807.03748},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/1807.03748},
}