Instructions to use iamtatsuki05/Sentence-Llama-Bi-EN-1B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use iamtatsuki05/Sentence-Llama-Bi-EN-1B with sentence-transformers:
from sentence_transformers import SentenceTransformer model = SentenceTransformer("iamtatsuki05/Sentence-Llama-Bi-EN-1B", trust_remote_code=True) sentences = [ "That is a happy person", "That is a happy dog", "That is a very happy person", "Today is a sunny day" ] embeddings = model.encode(sentences) similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [4, 4] - Transformers
How to use iamtatsuki05/Sentence-Llama-Bi-EN-1B with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModel tokenizer = AutoTokenizer.from_pretrained("iamtatsuki05/Sentence-Llama-Bi-EN-1B", trust_remote_code=True) model = AutoModel.from_pretrained("iamtatsuki05/Sentence-Llama-Bi-EN-1B", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
Sentence-Llama-Bi-EN-1B
English / Japanese
Overview
Sentence-Llama-Bi-EN-1B fine-tunes iamtatsuki05/Sentence-Llama-Bi-EN-1B-PT with supervised contrastive learning on the supervised subsets of sentence-transformers/embedding-training-data (AllNLI, MS MARCO, Quora duplicate questions, NQ, SQuAD, and TriviaQA), yielding 1,792-dimensional English sentence embeddings.
Usage
Requirements
sentence-transformers>=4.1.0
transformers>=4.51.0
accelerate>=1.6.0
sentencepiece>=0.2.0
flash-attn>=2.7.3
Sample Code
import torch
from sentence_transformers import SentenceTransformer
model_name = "iamtatsuki05/Sentence-Llama-Bi-EN-1B"
model_kwargs = {
"torch_dtype": torch.bfloat16,
"attn_implementation": "flash_attention_2",
}
model = SentenceTransformer(model_name, model_kwargs=model_kwargs, trust_remote_code=True)
queries = ["What kind of character is Hachiware?"]
docs = [
"Hachiware is a cat-like character from Chiikawa with a bright, sociable, and positive personality. Hachiware enjoys everyday life with Chiikawa and friends and eagerly joins adventures such as subjugation missions.",
"Usagi is a carefree, free-spirited character whose bold actions and sharp intuition pull Chiikawa and friends along, while still caring deeply about friends.",
]
q_emb = model.encode(queries, normalize_embeddings=True)
d_emb = model.encode(docs, normalize_embeddings=True)
scores = model.similarity(q_emb, d_emb)
print(scores)
Model Details
- Base model: iamtatsuki05/Sentence-Llama-Bi-EN-1B-PT
- Architecture: Llama
- Maximum sequence length: 8,192 tokens
- Embedding dimension: 1792 (mean pooling)
- Tokenizer: SentencePiece / vocabulary size 102,400
- Positional encoding: RoPE
- Supported languages: English
- Similarity metric: cosine
Model Series
These encoders start from weakly supervised checkpoints and undergo supervised fine-tuning on the supervised subsets of sentence-transformers/embedding-training-data.
| ID | Architecture | #Param. | #Param. w/o Emb. |
MTEB-Avg | MTEB-Retrieval | MTEB-STS | MTEB-Classification | MTEB-Reranking | MTEB-Clustering |
|---|---|---|---|---|---|---|---|---|---|
| iamtatsuki05/Sentence-ModernBERT-EN-1B | ModernBERT | 1304M | 1120M | 54.76 | 42.27 | 76.72 | 68.47 | 43.87 | 42.46 |
| iamtatsuki05/Sentence-Llama-Bi-EN-1B (this model) |
Llama | 1224M | 1040M | 52.94 | 39.04 | 75.45 | 66.46 | 42.47 | 41.28 |
| iamtatsuki05/Sentence-Sarashina-Bi-EN-1B | Llama | 1224M | 1040M | 57.56 | 44.90 | 79.78 | 73.48 | 45.98 | 43.64 |
Scores are measured on the 37-task English subset of MTEB (eng, v2) that covers the same five task families as JMTEB. Each task-family score averages all datasets, splits, and subsets in the family, and MTEB-Avg is the macro average over the five families.
Licence
This model is distributed under the MIT License.
How to Cite
@article{MIREI
title={同一条件下における Encoder/Decoder アーキテクチャによる文埋め込みの性能分析},
author={岡田 龍樹 and 杉本 徹},
journal={言語処理学会第 32 回年次大会 (NLP2026)},
year={2026}
}
- Downloads last month
- 21
Model tree for iamtatsuki05/Sentence-Llama-Bi-EN-1B
Base model
iamtatsuki05/Llama-EN-1B-PT-stage1