rolf-mozilla/french-embedding-e5-distilled

French embedding model (e5-small student, distilled) — q8 ONNX

A compact French sentence-embedding model: intfloat/multilingual-e5-small with its vocabulary pruned to French (16k tokens) and distilled from a fine-tuned French HPLT-BERT teacher. 28.3 MB as int8 ONNX, 256-d output.

Built for on-device use (transformers.js / onnxruntime). The 256-d projection head is folded into the graph, so the standard mean pooling + L2 normalize feature-extraction recipe gives the embedding directly.

Results (French, q8 ONNX, same evaluation pipeline for every row)

model	size	memory retrieval nDCG@5	SyntecRetrieval nDCG@5	MASSIVE-fr probe acc
this model	28.3 MB	0.740	0.739	0.804
ibm-granite/granite-embedding-97m-multilingual-r2 (q8)	98.2 MB	0.724	0.720	0.730
Xenova/multilingual-e5-small (q8)	118.3 MB	0.560	0.729	0.749
Xenova/all-MiniLM-L6-v2 (q8)	23.0 MB	0.563	0.537	0.724

Memory retrieval = a French per-query retrieval set (500 queries, 11–21 candidate memories each, 2–4 gold). SyntecRetrieval uses a 100-query sample. Differences under ~0.01 are within run-to-run noise of dynamic int8 quantization.

Usage

transformers.js

import { pipeline } from "@huggingface/transformers";
const extractor = await pipeline("feature-extraction", "rolf-mozilla/french-embedding-e5-distilled", { dtype: "q8" });
const out = await extractor(["Rappelle-moi d'acheter du pain."], { pooling: "mean", normalize: true });

Python (onnxruntime)

import numpy as np, onnxruntime as ort
from huggingface_hub import hf_hub_download
from transformers import AutoTokenizer

repo = "rolf-mozilla/french-embedding-e5-distilled"
tok = AutoTokenizer.from_pretrained(repo)
sess = ort.InferenceSession(hf_hub_download(repo, "onnx/model_quantized.onnx"))

def embed(texts):
    e = tok(texts, padding=True, truncation=True, max_length=256, return_tensors="np")
    h = sess.run(None, {"input_ids": e["input_ids"], "attention_mask": e["attention_mask"]})[0]
    m = e["attention_mask"][..., None]
    v = (h * m).sum(1) / m.sum(1)
    return v / np.linalg.norm(v, axis=1, keepdims=True)

No query: / passage: prefixes are needed.

How it was made

1. Teacher: HPLT/hplt_bert_base_fr fine-tuned (supervised SimCSE) on commercially usable French pairs: Wikipedia title→lead and sentence→paragraph, web question→answer from HPLT 2.0, OpenAssistant oasst1 / Aya prompt→reply.
2. Target space: teacher mean-pooled vectors, whitened (fit on the distillation corpus) and truncated to 256 dims.
3. Student: intfloat/multilingual-e5-small, vocabulary pruned to the 16k XLM-R pieces used by French text, distilled on 1.2M French texts (HPLT 2.0, French Wikipedia, oasst1, Aya): cosine + in-batch similarity-distribution (KL) + contrastive pair loss, 3 epochs, then a short pass with k-means near-neighbour batches (which fixed fine-grained ranking within a narrow domain).
4. Export: ONNX (opset 17), dynamic int8 quantization (onnx/model_quantized.onnx); onnx/model.onnx is the fp32 graph.

Data and licensing notes

• Distillation/fine-tuning data: HPLT 2.0 (CC0 packaging; web content remains its owners'), Wikipedia FR (CC BY-SA 3.0 / GFDL), OpenAssistant oasst1 and Aya (Apache 2.0). MS MARCO and XNLI were deliberately not used.
• Lineage caveat: the base model intfloat/multilingual-e5-small is MIT-licensed, but Microsoft's training data for it includes MS MARCO (non-commercial terms). Review this before commercial use.
• No evaluation data (the memory-retrieval set, SyntecRetrieval, MASSIVE) was used in training.

Limitations

• French only (tokenizer pruned to French); English and cross-lingual use are untested.
• Trained on inputs up to 256 tokens.
• Similarity scores are not calibrated to other models' scales — retune any similarity thresholds.