Universal Sentence Encoder Multilingual Dialogue (v.1.11-11-2026)

A fine-tuned version of Google's Universal Sentence Encoder Multilingual QA for conversational response retrieval: given a chat message, find the best-fitting response from a pool of candidate responses, in and across 16 languages.

Architecture: dual encoder (question_encoder for the message, response_encoder for the response)
Embedding size: 512, trained with [Matryoshka Representation Learning], so embeddings can be truncated to 256, 128 or 64 dimensions
Languages: Arabic, Chinese (Simplified and Traditional), Dutch, English, French, German, Italian, Japanese, Korean, Polish, Portuguese, Russian, Spanish, Thai, Turkish
Formats: TensorFlow SavedModel, ONNX (fp32 / int8), TFLite / LiteRT (fp32 / fp16 / int8)

Files

Path	Format	Variants
`saved_model/`	TensorFlow SavedModel (signatures `question_encoder`, `response_encoder`; tokenization included)	fp32
`onnx/`	ONNX, one graph per encoder sharing one weights file	`none` (fp32, 346 MB), `int8` (87 MB)
`tflite/`	TFLite / LiteRT, one file with both encoders as signatures	`none` (fp32, 347 MB), `float16` (174 MB), `int8` (88 MB)

The ONNX and TFLite models take token IDs, not text. Tokenize with the SentencePiece model in spm.model (add BOS and EOS, int32 IDs, one text per call). tokenizer.json describes the exact inputs. The ONNX *.onnx files load their weights from encoders_<variant>.onnx_data, so keep both in the same folder.

Usage

All encoders return L2-normalized 512-dim vectors, so the dot product is the cosine similarity. The response_encoder additionally takes a context (the text surrounding the response); if there is none, pass the response itself.

TensorFlow SavedModel

# pip install tensorflow tensorflow-text huggingface_hub
import tensorflow as tf
import tensorflow_text  # noqa: F401  registers the SentencePiece ops
from huggingface_hub import snapshot_download

path = snapshot_download("ereinhardt/universal-sentence-encoder-multilingual-dialogue", allow_patterns=["saved_model/*"])
model = tf.saved_model.load(f"{path}/saved_model")

messages = ["How was your weekend?"]
responses = ["Great, I went hiking!", "The train leaves at 5 pm.", "Ich mag Pizza."]

q = model.signatures["question_encoder"](tf.constant(messages))["outputs"]
r = model.signatures["response_encoder"](input=tf.constant(responses), context=tf.constant(responses))["outputs"]
print(tf.matmul(q, r, transpose_b=True))

ONNX Runtime

# pip install onnxruntime sentencepiece numpy huggingface_hub
import numpy as np
import onnxruntime as ort
import sentencepiece as spm
from huggingface_hub import snapshot_download

path = snapshot_download("ereinhardt/universal-sentence-encoder-multilingual-dialogue", allow_patterns=["onnx/*int8*", "onnx/spm.model"])
sp = spm.SentencePieceProcessor(model_file=f"{path}/onnx/spm.model")
q_enc = ort.InferenceSession(f"{path}/onnx/question_encoder_int8.onnx")
r_enc = ort.InferenceSession(f"{path}/onnx/response_encoder_int8.onnx")

def ids(text):
    return np.array(sp.encode(text, add_bos=True, add_eos=True), dtype=np.int32)

message = "How was your weekend?"
responses = ["Great, I went hiking!", "The train leaves at 5 pm.", "Ich mag Pizza."]

q = q_enc.run(None, {"input_ids": ids(message)})[0][0]
r = np.stack([r_enc.run(None, {"input_ids": ids(t), "context_ids": ids(t)})[0][0] for t in responses])
print(r @ q)

TFLite / LiteRT

# pip install ai-edge-litert sentencepiece numpy huggingface_hub
import numpy as np
import sentencepiece as spm
from ai_edge_litert.interpreter import Interpreter
from huggingface_hub import snapshot_download

path = snapshot_download("ereinhardt/universal-sentence-encoder-multilingual-dialogue", allow_patterns=["tflite/encoders_int8.tflite", "tflite/spm.model"])
sp = spm.SentencePieceProcessor(model_file=f"{path}/tflite/spm.model")
interpreter = Interpreter(f"{path}/tflite/encoders_int8.tflite")
q_enc = interpreter.get_signature_runner("question_encoder")
r_enc = interpreter.get_signature_runner("response_encoder")

def ids(text):
    return np.array(sp.encode(text, add_bos=True, add_eos=True), dtype=np.int32)

message = "How was your weekend?"
responses = ["Great, I went hiking!", "The train leaves at 5 pm.", "Ich mag Pizza."]

q = q_enc(input_ids=ids(message))["outputs"][0]
r = np.stack([r_enc(input_ids=ids(t), context_ids=ids(t))["outputs"][0] for t in responses])
print(r @ q)

Smaller embeddings (Matryoshka)

Keep the first d dimensions and re-normalize. The model was trained for d in 512, 256, 128 and 64.

def truncate(e, d=128):
    e = e[..., :d]
    return e / np.linalg.norm(e, axis=-1, keepdims=True)

Training

Training starts from the original USE Multilingual QA weights and fine-tunes both encoders on (message → response) pairs from multi-turn dialogues.

Objective: symmetric contrastive loss (InfoNCE, message → response and response → message) with in-batch negatives. Identical messages or responses in a batch are masked out rather than treated as negatives.
Matryoshka: the loss is averaged over the embedding sizes 512, 256, 128 and 64.
Large batches: 4096 pairs per step via GradCache, processed in chunks of 512.
Batching: each batch comes from a single source file (one language and one corpus), so negatives come from the same language and domain. Batches are shuffled.
Model selection: the checkpoint with the best mean validation MRR over all Matryoshka sizes is kept.

Hyperparameter	Value
Batch size (in-batch negatives)	4096
Learning rate	1e-4, linear warmup (1000 steps), then cosine decay
Optimizer	Adam, gradient clipping (norm 1.0)
Similarity scale (temperature⁻¹)	20
Epochs	1
Max text length	160 characters (longer texts were truncated)
Matryoshka dimensions	512, 256, 128, 64
Seed	42

Training data

About 79 million (message → response) pairs, published in this repository under multilingual-dialogue-pairs-train-data. The English source dialogues (DailyDialog, ProsocialDialog, SAMSum, SODA, Topical-Chat and a small set of WhatsApp-style chats) were machine-translated into all 16 languages with google/madlad400-10b-mt. A cross-lingual subset pairs a message in one language with its response in another. Pairs that occur in the validation set were removed from the training data.

Note:

Apart from the English originals, the training data is machine-translated. Translation errors and unnatural phrasing carry over into the model.
The model is trained for short, casual chat turns, not for long documents or factual question answering.
Texts longer than 160 characters were truncated during training, so quality on long messages may be lower.
The model retrieves responses from a pool you provide. It does not generate text and does not check whether a response is appropriate or factually correct.

Downloads last month: 26