Ozankj/modernbert-tr-embed-ONNX

ModernBERT-TR Embed ONNX (Unofficial)

Unofficial private ONNX mirror of ytu-ce-cosmos/modernbert-tr-embed. The main value of this mirror is the custom 8-bit int8 derivative: ~152 MB instead of ~298 MB, while remaining near FP16 on the tested inputs.

This repository is not affiliated with, endorsed by, or maintained by YTU-CE / Cosmos.

• Original model: https://huggingface.co/ytu-ce-cosmos/modernbert-tr-embed
• Base model: https://huggingface.co/ytu-ce-cosmos/modernbert-tr-base
• Original training data and attribution: see the original model card
• Original authors / copyright: YTU-CE Cosmos / model authors listed there
• License: Apache License 2.0
• ONNX conversion and int8 quantization: Ozankj
• Original model page and its Benchmarks remain the authoritative source for model quality claims

This repository contains the tokenizer, sentence-transformer config, pooling/normalize modules, an independently regenerated FP16 ONNX export, and an independently quantized int8 ONNX derivative. It does not contain the PyTorch safetensors.

Accuracy

The regenerated FP16 export was checked against the local FP32 reference and is included mainly as a compatibility/comparison baseline. The important artifact is onnx/model_quantized.onnx.

Quantized accuracy

Against the custom FP16 ONNX, using ONNX Runtime CPU with the included tokenizer. Both models emit last_hidden_state; evaluation casts outputs to float32, applies attention-mask-weighted mean pooling, L2-normalizes, and compares sentence embeddings.

Smoke tests

Test	Custom int8	Upstream official quantized
10 representative Turkish examples	avg 0.999892 / min 0.999857	avg 0.9704 / min 0.9610
512-token repeated example	cosine 0.9998	—
3 literal bracket/special-token-like strings	0.9998–0.9999	—

Wiki40B Turkish validation

120 independently sampled Turkish wiki40b validation passages retained after cleaning (len >= 100), truncated to 512 tokens, and evaluated as pooled normalized sentence embeddings.

Test	Custom int8 (Ozankj)	Upstream official quantized
Average cosine vs FP16	0.999795	0.959056
Minimum cosine	0.999552	0.915724
1st percentile cosine	0.999625	0.919562
5th percentile cosine	0.999664	0.937027
Examples >= 0.999	100%	0%
Max absolute pairwise-similarity drift	0.00559	0.0919
Top-1 nearest-neighbor match vs FP16	100%	71.7%
Top-5 overlap vs FP16	98.8%	79.7%

Absolute pairwise-similarity drift for custom int8 had mean -0.000527 and standard deviation 0.001271.

These are limited CPU smoke/robustness tests, not a full Turkish MTEB evaluation. Runtime parity on web, WASM, DirectML, or other execution providers is not guaranteed. Use FP16 only when maximum compatibility matters, or as a direct reference; the custom 8-bit artifact is the compact version this mirror is meant to distribute.

Files

File	Format	Size	Notes
onnx/model_fp16.onnx	FP16 ONNX	~298 MB	Compatibility/reference baseline; regenerated from upstream safetensors
onnx/model_quantized.onnx	custom int8 / MatMulNBits	~152 MB	Primary custom 8-bit artifact; see QUANTIZATION.md

Outputs: last_hidden_state (batch, seq_len, 768). Sentence pooling and L2 normalization remain in 1_Pooling/ and 2_Normalize/; use the original SentenceTransformers pipeline as shown below.

Usage

from sentence_transformers import SentenceTransformer

model = SentenceTransformer(
    "Ozankj/modernbert-tr-embed-ONNX",
    backend="onnx",
    model_kwargs={"file_name": "onnx/model_fp16.onnx"},
)

# Custom int8 variant; requires MatMulNBits support
quantized_model = SentenceTransformer(
    "Ozankj/modernbert-tr-embed-ONNX",
    backend="onnx",
    model_kwargs={"file_name": "onnx/model_quantized.onnx"},
)
lines = ["Türkiye'nin başkenti Ankara'dır.", "Bugün hava çok güzel."]
embeddings = model.encode(lines)

License and attribution

This derivative is redistributed under Apache-2.0. Keep upstream attribution intact. See NOTICE and QUANTIZATION.md.

No implied endorsement by YTU-CE / Cosmos.