| ---
|
| language:
|
| - ba
|
| - ru
|
| license: apache-2.0
|
| pretty_name: Bashkir-Russian Pair Scorer
|
| library_name: onnxruntime
|
| pipeline_tag: text-classification
|
| tags:
|
| - bashkir
|
| - russian
|
| - alignment
|
| - pair-scoring
|
| - onnx
|
| - onnxruntime
|
| - low-resource
|
| - open-source
|
| ---
|
|
|
| # Bashkir-Russian Pair Scorer
|
|
|
| > Compact ONNX models that score whether a Bashkir-Russian sentence pair is
|
| > parallel, for fast corpus filtering.
|
|
|
| ## Overview
|
|
|
| Compact ONNX models for estimating whether a Bashkir-Russian sentence pair is
|
| parallel and suitable for corpus filtering. The release is designed for fast,
|
| repeatable scoring of large bilingual corpora and contains three model sizes:
|
| `nano`, `mini` and `medium`. This is an alignment-quality scorer, not a
|
| translation model: it returns a compatibility score for a pair of texts and can
|
| be used to build a filtering cascade.
|
|
|
| | At a glance | |
|
| | --- | --- |
|
| | Task | Parallel-pair quality estimation |
|
| | Default artifact | `models/mini/model.onnx` (recommended) |
|
| | Source | Reviewed parallel Bashkir-Russian data |
|
| | Version / license | v1 / Apache-2.0 |
|
|
|
| ## Contents
|
|
|
| ### Files and Configurations
|
|
|
| | Path | Purpose | Size |
|
| | --- | --- | ---: |
|
| | `models/nano/model.onnx` | Fastest first-pass scorer | 9.1 MB |
|
| | **`models/mini/model.onnx`** (recommended) | Balanced default for most batch jobs | 15.0 MB |
|
| | `models/medium/model.onnx` | Compact higher-quality verifier | 23.2 MB |
|
| | `spm_bpe_16k.model` | Shared SentencePiece tokenizer | — |
|
| | `config.json` | Runtime contract and model metadata | — |
|
| | `benchmark_summary.json` | Machine-readable evaluation results | — |
|
| | `META.json` | Release passport and artifact hashes | — |
|
| | `LICENSE` | Full license text | — |
|
| | `SHA256SUMS` | Release checksums | — |
|
|
|
| All configurations use the same SentencePiece vocabulary and the `fast128`
|
| profile: FP16 weights, dynamic batch dimension and a maximum of 128 tokens per
|
| side. This is a deliberate production design for stable speed and predictable
|
| memory use; longer inputs are truncated before scoring.
|
|
|
| ### Model Architecture
|
|
|
| | Property | Description |
|
| | --- | --- |
|
| | Task | Parallel-pair compatibility scoring |
|
| | Directions | `ba_ru`, `ru_ba` |
|
| | Profile | `fast128` (FP16, dynamic batch, ≤128 tokens per side) |
|
| | Output | `score_logit` (Sigmoid to 0.0–1.0) |
|
| | Tokenizer | `spm_bpe_16k.model` (SentencePiece BPE) |
|
| | Runtime | ONNX Runtime on CPU or GPU |
|
| | Starting thresholds | `accepted` ≥ 0.80, `review` ≥ 0.50 (heuristics; calibrate per corpus) |
|
|
|
| ### Examples
|
|
|
| Scores from the `medium` model on CPU (Sigmoid of `score_logit`):
|
|
|
| | Bashkir | Russian | Score |
|
| | --- | --- | ---: |
|
| | Бөгөн һауа бик йылы. | Сегодня очень тепло. | 0.981 |
|
| | Башҡортостан Республикаһы — Рәсәй Федерацияһы субъекты. | Республика Башкортостан — субъект Российской Федерации. | 0.984 |
|
| | Мин башҡорт телен яратам. | Квантовая механика описывает поведение микрочастиц и полей. | 0.000 |
|
|
|
| ## Method
|
|
|
| ```text
|
| reviewed parallel data + hard negatives + teacher signals
|
| ├── 1. construct cross-encoder input pairs (ba + ru)
|
| ├── 2. train compact cross-encoder with margin-calibrated distillation
|
| ├── 3. optimize inference graphs with fast128 sequence profiling
|
| └── 4. export ONNX family: nano, mini & medium (FP16 & INT8)
|
| ```
|
|
|
| The models were trained for Bashkir-Russian parallel-corpus quality estimation
|
| using reviewed parallel data, hard negative pairs and teacher/reference scoring
|
| signals. The release contains model weights and tokenizer assets only; the source
|
| corpus text is not included.
|
|
|
| ### Benchmark
|
|
|
| Comparison on a calibrated bilingual evaluation set (10,000 positive pairs and
|
| 20,000 deterministic global/local hard negatives, excluded from training). Full
|
| numbers are in [`benchmark_summary.json`](benchmark_summary.json).
|
|
|
| | Model | ROC-AUC | Average Precision | RTX 4060 throughput |
|
| | --- | ---: | ---: | ---: |
|
| | LaBSE | 0.9944 | 0.9895 | 44.8 pairs/s |
|
| | LaBSE + LASER | 0.9939 | 0.9871 | 44.8 pairs/s |
|
| | DevLake BERT | 0.9896 | 0.9706 | 112.5 pairs/s |
|
| | **Medium** | 0.9633 | 0.9155 | 2,555.9 pairs/s |
|
| | **Mini** | 0.9590 | 0.9040 | 4,673.5 pairs/s |
|
| | **Nano** | 0.9498 | 0.8785 | 7,472.2 pairs/s |
|
|
|
| The compact scorers trade a small amount of discrimination for a large
|
| throughput gain. A practical cascade is `nano`/`mini` for the first pass,
|
| `medium` for uncertain pairs and LaBSE/LASER only for the final review layer.
|
|
|
| ## Quality and Use
|
|
|
| Scores are not calibrated human probabilities. The `accepted`, `review` and
|
| `quarantine` thresholds must be calibrated for a target corpus and a reviewed
|
| sample. The `fast128` profile truncates long inputs; for long literary or document
|
| segments use a review layer with a longer-context scorer.
|
|
|
| ### Limitations
|
|
|
| - Alignment scores are model estimates, not human judgments.
|
| - The benchmark is intended for comparison and does not guarantee production precision on every domain.
|
| - `fast128` truncates inputs beyond 128 tokens per side.
|
| - Starting thresholds are heuristics and require per-corpus calibration.
|
|
|
| ## Related Resources
|
|
|
| - [Bashkir-Russian Wikipedia Parallel Corpus](https://huggingface.co/datasets/failed09/bashkir-wikipedia-parallel) —
|
| uses the `medium` model to produce its `pair_score` column, a concrete
|
| application of this scorer.
|
|
|
| ## Usage
|
|
|
| ```bash
|
| pip install onnxruntime sentencepiece numpy
|
| ```
|
|
|
| ```python
|
| import numpy as np
|
| import onnxruntime as ort
|
| import sentencepiece as spm
|
|
|
| session = ort.InferenceSession(
|
| "models/mini/model.onnx",
|
| providers=["CUDAExecutionProvider", "CPUExecutionProvider"],
|
| )
|
| tokenizer = spm.SentencePieceProcessor(model_file="spm_bpe_16k.model")
|
|
|
|
|
| def encode(ba, ru, max_len=128):
|
| limit = (max_len - 3) // 2
|
| ru_ids = tokenizer.encode(ru, out_type=int)[:limit]
|
| ba_ids = tokenizer.encode(ba, out_type=int)[:limit]
|
| ids = [1, *ru_ids, 2, *ba_ids, 2]
|
| ids += [tokenizer.pad_id()] * (max_len - len(ids))
|
| ids = np.asarray([ids], dtype=np.int64)
|
| return {"input_ids": ids, "attention_mask": ids != tokenizer.pad_id()}
|
|
|
|
|
| logit = session.run(["score_logit"], encode("Сәләм донъя", "Привет мир"))[0][0]
|
| score = float(1 / (1 + np.exp(-logit)))
|
| print(score)
|
| ```
|
|
|
| For corpus-scale jobs, use the direct batch runner to avoid HTTP/JSON overhead and
|
| stream Parquet, CSV/TSV or JSONL inputs.
|
|
|
| ## License
|
|
|
| Distributed under the [Apache-2.0 license](https://huggingface.co/failed09/bashkir-pair-scorer/blob/main/LICENSE).
|
| This release contains derived alignment-scoring software and model assets, not
|
| source corpus text, scans or original page layouts.
|
|
|
| ## Citation
|
|
|
| ```bibtex
|
| @software{failed09_bashkir_pair_scorer_2026,
|
| title = {Bashkir-Russian Pair Scorer},
|
| author = {failed09},
|
| year = {2026},
|
| publisher = {Hugging Face},
|
| url = {https://huggingface.co/failed09/bashkir-pair-scorer},
|
| note = {Open-source compact alignment models for the Bashkir language}
|
| }
|
| ```
|
|
|
| ## Open Bashkir Data and Sources 🐝
|
|
|
| This release is part of an open-source effort to support the development,
|
| preservation and practical use of the Bashkir language. Other related models,
|
| datasets and tools are available on the author's Hugging Face profile.
|
|
|
| The author does not claim ownership or authorship of the source texts or other
|
| materials used to derive this release; rights and licensing remain with the
|
| original authors, publishers and dataset providers. Source texts are not
|
| redistributed in this repository, so users should follow the licenses and
|
| attribution requirements of the relevant upstream resources.
|
|
|
