Initial upload: French e5-student embedding / intent, q8 ONNX

2d27425 verified 1 day ago

4.18 kB

---
language:
- fr
license: mit
library_name: transformers.js
pipeline_tag: feature-extraction
tags:
- sentence-embeddings
- onnx
- french
- distillation
base_model: intfloat/multilingual-e5-small
---
# French embedding model (e5-small student, distilled) — q8 ONNX

A compact **French** sentence-embedding model: `intfloat/multilingual-e5-small` with its
vocabulary pruned to French (16k tokens) and distilled from a fine-tuned French
HPLT-BERT teacher. **28.3 MB** as int8 ONNX, **256-d** output.

Built for on-device use (transformers.js / onnxruntime). The 256-d projection head is
folded into the graph, so the standard *mean pooling + L2 normalize* feature-extraction
recipe gives the embedding directly.

## Results (French, q8 ONNX, same evaluation pipeline for every row)

| model | size | memory retrieval nDCG@5 | SyntecRetrieval nDCG@5 | MASSIVE-fr probe acc |
|---|---|---|---|---|
| **this model** | **28.3 MB** | **0.740** | **0.739** | **0.804** |
| ibm-granite/granite-embedding-97m-multilingual-r2 (q8) | 98.2 MB | 0.724 | 0.720 | 0.730 |
| Xenova/multilingual-e5-small (q8) | 118.3 MB | 0.560 | 0.729 | 0.749 |
| Xenova/all-MiniLM-L6-v2 (q8) | 23.0 MB | 0.563 | 0.537 | 0.724 |

*Memory retrieval* = a French per-query retrieval set (500 queries, 11–21 candidate
memories each, 2–4 gold). SyntecRetrieval uses a 100-query sample. Differences under
~0.01 are within run-to-run noise of dynamic int8 quantization.

## Usage

**transformers.js**
```js
import { pipeline } from "@huggingface/transformers";
const extractor = await pipeline("feature-extraction", "rolf-mozilla/french-embedding-e5-distilled", { dtype: "q8" });
const out = await extractor(["Rappelle-moi d'acheter du pain."], { pooling: "mean", normalize: true });
```

**Python (onnxruntime)**
```python
import numpy as np, onnxruntime as ort
from huggingface_hub import hf_hub_download
from transformers import AutoTokenizer
repo = "rolf-mozilla/french-embedding-e5-distilled"
tok = AutoTokenizer.from_pretrained(repo)
sess = ort.InferenceSession(hf_hub_download(repo, "onnx/model_quantized.onnx"))
def embed(texts):
    e = tok(texts, padding=True, truncation=True, max_length=256, return_tensors="np")
    h = sess.run(None, {"input_ids": e["input_ids"], "attention_mask": e["attention_mask"]})[0]
    m = e["attention_mask"][..., None]
    v = (h * m).sum(1) / m.sum(1)
    return v / np.linalg.norm(v, axis=1, keepdims=True)
```
No `query:` / `passage:` prefixes are needed.

## How it was made

1. **Teacher:** `HPLT/hplt_bert_base_fr` fine-tuned (supervised SimCSE) on commercially
   usable French pairs: Wikipedia title→lead and sentence→paragraph, web question→answer
   from HPLT 2.0, OpenAssistant oasst1 / Aya prompt→reply.
2. **Target space:** teacher mean-pooled vectors, whitened (fit on the distillation
   corpus) and truncated to 256 dims.
3. **Student:** `intfloat/multilingual-e5-small`, vocabulary pruned to the 16k
   XLM-R pieces used by French text, distilled on 1.2M French texts (HPLT 2.0, French
   Wikipedia, oasst1, Aya): cosine + in-batch similarity-distribution (KL) + contrastive
   pair loss, 3 epochs, then a short pass with k-means near-neighbour batches (which
   fixed fine-grained ranking within a narrow domain).
4. **Export:** ONNX (opset 17), dynamic int8 quantization (`onnx/model_quantized.onnx`);
   `onnx/model.onnx` is the fp32 graph.

## Data and licensing notes

- Distillation/fine-tuning data: HPLT 2.0 (CC0 packaging; web content remains its
  owners'), Wikipedia FR (CC BY-SA 3.0 / GFDL), OpenAssistant oasst1 and Aya (Apache 2.0).
  MS MARCO and XNLI were deliberately **not** used.
- **Lineage caveat:** the base model `intfloat/multilingual-e5-small` is MIT-licensed,
  but Microsoft's training data for it includes MS MARCO (non-commercial terms).
  Review this before commercial use.
- No evaluation data (the memory-retrieval set, SyntecRetrieval, MASSIVE) was used in training.

## Limitations

- French only (tokenizer pruned to French); English and cross-lingual use are untested.
- Trained on inputs up to 256 tokens.
- Similarity scores are not calibrated to other models' scales — retune any
  similarity thresholds.

	---
	language:
	- fr
	license: mit
	library_name: transformers.js
	pipeline_tag: feature-extraction
	tags:
	- sentence-embeddings
	- onnx
	- french
	- distillation
	base_model: intfloat/multilingual-e5-small
	---

	# French embedding model (e5-small student, distilled) — q8 ONNX

	A compact French sentence-embedding model: `intfloat/multilingual-e5-small` with its
	vocabulary pruned to French (16k tokens) and distilled from a fine-tuned French
	HPLT-BERT teacher. 28.3 MB as int8 ONNX, 256-d output.

	Built for on-device use (transformers.js / onnxruntime). The 256-d projection head is
	folded into the graph, so the standard mean pooling + L2 normalize feature-extraction
	recipe gives the embedding directly.

	## Results (French, q8 ONNX, same evaluation pipeline for every row)

	\| model \| size \| memory retrieval nDCG@5 \| SyntecRetrieval nDCG@5 \| MASSIVE-fr probe acc \|
	\|---\|---\|---\|---\|---\|
	\| this model \| 28.3 MB \| 0.740 \| 0.739 \| 0.804 \|
	\| ibm-granite/granite-embedding-97m-multilingual-r2 (q8) \| 98.2 MB \| 0.724 \| 0.720 \| 0.730 \|
	\| Xenova/multilingual-e5-small (q8) \| 118.3 MB \| 0.560 \| 0.729 \| 0.749 \|
	\| Xenova/all-MiniLM-L6-v2 (q8) \| 23.0 MB \| 0.563 \| 0.537 \| 0.724 \|

	Memory retrieval = a French per-query retrieval set (500 queries, 11–21 candidate
	memories each, 2–4 gold). SyntecRetrieval uses a 100-query sample. Differences under
	~0.01 are within run-to-run noise of dynamic int8 quantization.

	## Usage

	transformers.js
	```js
	import { pipeline } from "@huggingface/transformers";
	const extractor = await pipeline("feature-extraction", "rolf-mozilla/french-embedding-e5-distilled", { dtype: "q8" });
	const out = await extractor(["Rappelle-moi d'acheter du pain."], { pooling: "mean", normalize: true });
	```

	Python (onnxruntime)
	```python
	import numpy as np, onnxruntime as ort
	from huggingface_hub import hf_hub_download
	from transformers import AutoTokenizer

	repo = "rolf-mozilla/french-embedding-e5-distilled"
	tok = AutoTokenizer.from_pretrained(repo)
	sess = ort.InferenceSession(hf_hub_download(repo, "onnx/model_quantized.onnx"))

	def embed(texts):
	e = tok(texts, padding=True, truncation=True, max_length=256, return_tensors="np")
	h = sess.run(None, {"input_ids": e["input_ids"], "attention_mask": e["attention_mask"]})[0]
	m = e["attention_mask"][..., None]
	v = (h * m).sum(1) / m.sum(1)
	return v / np.linalg.norm(v, axis=1, keepdims=True)
	```
	No `query:` / `passage:` prefixes are needed.

	## How it was made

	1. Teacher: `HPLT/hplt_bert_base_fr` fine-tuned (supervised SimCSE) on commercially
	usable French pairs: Wikipedia title→lead and sentence→paragraph, web question→answer
	from HPLT 2.0, OpenAssistant oasst1 / Aya prompt→reply.
	2. Target space: teacher mean-pooled vectors, whitened (fit on the distillation
	corpus) and truncated to 256 dims.
	3. Student: `intfloat/multilingual-e5-small`, vocabulary pruned to the 16k
	XLM-R pieces used by French text, distilled on 1.2M French texts (HPLT 2.0, French
	Wikipedia, oasst1, Aya): cosine + in-batch similarity-distribution (KL) + contrastive
	pair loss, 3 epochs, then a short pass with k-means near-neighbour batches (which
	fixed fine-grained ranking within a narrow domain).
	4. Export: ONNX (opset 17), dynamic int8 quantization (`onnx/model_quantized.onnx`);
	`onnx/model.onnx` is the fp32 graph.

	## Data and licensing notes

	- Distillation/fine-tuning data: HPLT 2.0 (CC0 packaging; web content remains its
	owners'), Wikipedia FR (CC BY-SA 3.0 / GFDL), OpenAssistant oasst1 and Aya (Apache 2.0).
	MS MARCO and XNLI were deliberately not used.
	- Lineage caveat: the base model `intfloat/multilingual-e5-small` is MIT-licensed,
	but Microsoft's training data for it includes MS MARCO (non-commercial terms).
	Review this before commercial use.
	- No evaluation data (the memory-retrieval set, SyntecRetrieval, MASSIVE) was used in training.

	## Limitations

	- French only (tokenizer pruned to French); English and cross-lingual use are untested.
	- Trained on inputs up to 256 tokens.
	- Similarity scores are not calibrated to other models' scales — retune any
	similarity thresholds.