KielEmbed-Vision

A multimodal text and image embedding model fine-tuned from sentence-transformers/clip-ViT-B-32.

This is a sentence-transformers model fine-tuned from sentence-transformers/clip-ViT-B-32. It maps text and images into a shared 512-dimensional dense vector space optimized for multimodal semantic similarity, zero-shot classification, and image-text retrieval tasks.

Model Details

Model Description

• Model Type: Sentence Transformer / Multimodal Embedding Backbone
• Base Model: sentence-transformers/clip-ViT-B-32
• Maximum Sequence Length: 77 tokens
• Output Dimensionality: 512 dimensions
• Similarity Function: Cosine Similarity
• Supported Modalities: Text, Image

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face Hub: kiel/KielEmbed-Vision

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'get_text_features', 'method_output_name': 'pooler_output'}, 'image': {'method': 'get_image_features', 'method_output_name': 'pooler_output'}}, 'module_output_name': 'sentence_embedding', 'architecture': 'CLIPModel'})
)
UsageDirect Usage (Sentence Transformers)First, install the Sentence Transformers library:Bashpip install -U sentence-transformers
Then load your model and run inference:Pythonfrom sentence_transformers import SentenceTransformer

# Load your custom fine-tuned CLIP model from the Hugging Face Hub
model = SentenceTransformer("kiel/KielEmbed-Vision")

# Run inference
sentences = [
    'Tornadoes , up to a foot of rain and hail as big as cantaloupes pounded southern Nebraska and northern Kansas , killing one man and destroying at least four homes .',
    'Up to a foot of rain and at least seven tornadoes pounded southern Nebraska and northern Kansas , killing a man and destroying at least four homes .',
    'The move follows a recent proposal by Mr Vajpayee , whoended an 18-month chill in relations by ordering normalisation of diplomatic links and restoration of air services with Pakistan .',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 512]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
Training DetailsTraining DatasetSemantic & Text-Image Alignment CorpusSize: 6,000 training samplesColumns: text1, text2, and labelApproximate Token Statistics (First 100 samples):Text 1: Min: 12 tokens | Mean: 28.37 tokens | Max: 44 tokensText 2: Min: 13 tokens | Mean: 28.16 tokens | Max: 45 tokensLabel Distribution: Class 0 (~34.62%), Class 1 (~65.38%)Loss Function: CosineSimilarityLoss with parameters:JSON{
    "loss_fct": "torch.nn.modules.loss.MSELoss",
    "cos_score_transformation": "torch.nn.modules.linear.Identity"
}
Training HyperparametersPer Device Train Batch Size: 8Gradient Accumulation Steps: 4 (Effective batch size = 32)Learning Rate: 2e-05Number of Epochs: 1Warmup Steps: 0.1Mixed Precision: FP16 EnabledOptimizer: adamw_torch_fusedTraining LogsEpochStepTraining Loss0.275035.46910.5310045.51420.8015045.2153Total Training Time: 2.5 minutesFramework VersionsPython: 3.13.15Sentence Transformers: 5.7.0Transformers: 5.16.1PyTorch: 2.11.0+cu128Accelerate: 1.14.0Datasets: 4.8.5Tokenizers: 0.23.1Additional ResourcesMultimodal Embedding & Reranker Models with Sentence Transformers: Use text, image, audio, and video models through the same API.Training and Finetuning Multimodal Embedding & Reranker Models with Sentence Transformers: End-to-end guide for training multimodal embedding models.CitationBibTeXCode snippet@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "[https://arxiv.org/abs/1908.10084](https://arxiv.org/abs/1908.10084)",
}