Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,49 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: bsd-3-clause
|
| 3 |
+
tags:
|
| 4 |
+
- hebrew
|
| 5 |
+
- diacritization
|
| 6 |
+
- nikud
|
| 7 |
+
- plane-factorized
|
| 8 |
+
- onnx
|
| 9 |
+
- int8
|
| 10 |
+
library_name: onnx
|
| 11 |
+
---
|
| 12 |
+
|
| 13 |
+
# heb-diac-plane-2.0
|
| 14 |
+
|
| 15 |
+
Hebrew nikud diacritization, plane-factorized (run-022): ByT5-**base**
|
| 16 |
+
encoder + per-character nikud-class head, K=3 mask-predict decode,
|
| 17 |
+
int8 (dynamic QInt8) ONNX, 416 MB. Trained on hebrew-v4 (nakdimon +
|
| 18 |
+
sefaria + distilled + expanded) on HF Jobs.
|
| 19 |
+
|
| 20 |
+
## Metrics (runtime protocol)
|
| 21 |
+
|
| 22 |
+
| benchmark | value |
|
| 23 |
+
|---|---|
|
| 24 |
+
| nakdimon test DER (greedy, int8, py runtime) | **8.18** (1.0: 12.48) |
|
| 25 |
+
| phonikud heb-g2p-benchmark CER / no-stress (via nikud_ipa rules) | 0.2393 / 0.1389 |
|
| 26 |
+
|
| 27 |
+
## Features
|
| 28 |
+
|
| 29 |
+
- `translate(text, preserve_diacritics=True)` — user-supplied nikud is
|
| 30 |
+
pinned inside the decode and round-trips byte-exactly (leading marks
|
| 31 |
+
included).
|
| 32 |
+
- Language-generic plane contract (`kind: plane`); same runtime as the
|
| 33 |
+
Arabic plane family.
|
| 34 |
+
- nikud→IPA composition: `interscript.ml.nikud_ipa.plane_to_ipa`.
|
| 35 |
+
|
| 36 |
+
## Provenance
|
| 37 |
+
|
| 38 |
+
- Training: TODO.sota/04 (run-022, HF Jobs a100-large); corpus
|
| 39 |
+
`Interscript/hebrew-v4` (private); recipe mirrors run-021 at
|
| 40 |
+
byt5-base/4ep/K=3.
|
| 41 |
+
- Artifact sha256: `9c8f0432294ebae6abc847ddb79572c1ab979f4851f03d44a9bb6495ba767a9e`.
|
| 42 |
+
- License: BSD-3-Clause (code + weights).
|
| 43 |
+
|
| 44 |
+
|
| 45 |
+
## Mirror provenance
|
| 46 |
+
|
| 47 |
+
Canonical channel: GitHub Releases (interscript/interscript-models),
|
| 48 |
+
sha256 `9c8f0432294ebae6abc847ddb79572c1ab979f4851f03d44a9bb6495ba767a9e`.
|
| 49 |
+
This HF repo is a 1:1 mirror of the index entry `heb-diac-plane-2.0`.
|