Automatic Speech Recognition
PyTorch
Safetensors
GGUF
MLX
Tamil
English
speech
asr
streaming
rnnt
ctc
tamil
tanglish
code-switching
indian-english
verisafe
Instructions to use junafinity/VeriSafe-Tanglish-1-STT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use junafinity/VeriSafe-Tanglish-1-STT with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] hf download junafinity/VeriSafe-Tanglish-1-STT --local-dir VeriSafe-Tanglish-1-STT
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
File size: 1,759 Bytes
ec01e2d | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 | """normalize() from indic_stt/text.py, copied verbatim so the MLX runner needs no PyTorch: NFC, lower case, no punctuation,
single spaces. The same function scored every reported number. Keep it equal to indic_stt/text.py (export_mlx.py verify checks)."""
from __future__ import annotations
import re
import unicodedata
# Danda, double danda, ASCII and common Unicode punctuation. Hyphens become spaces; apostrophes
# inside words (don't, it's) survive via _INNER_APOS, all others become spaces.
_PUNCT = re.compile(r"[।॥!\"#$%&'()*+,\-./:;<=>?@\[\\\]^_`{|}~‘’“”–—… ]")
_SPACE = re.compile(r"\s+")
_INNER_APOS = re.compile(r"(?<=\w)['’](?=\w)")
_KEEP = "" # private-use placeholder for a protected apostrophe
_NUKTA = "़"
# Precomposed nukta letters -> base letter (क़ ख़ ग़ ज़ ड़ ढ़ फ़ य़)
_NUKTA_FOLD = str.maketrans({"क़": "क", "ख़": "ख", "ग़": "ग", "ज़": "ज",
"ड़": "ड", "ढ़": "ढ", "फ़": "फ", "य़": "य"})
def normalize(text: str, fold_nukta: bool = False) -> str:
"""Training target / scoring normalization: NFC, no punctuation, single spaces.
fold_nukta=True also maps nukta letters to their base letter (क़ -> क). Use it for a
"lenient" WER that ignores the nukta spelling variation common in Hindi transcripts.
"""
text = unicodedata.normalize("NFC", text).lower() # lower() only changes cased scripts (English)
text = _INNER_APOS.sub(_KEEP, text)
text = _PUNCT.sub(" ", text).replace(_KEEP, "'")
if fold_nukta:
text = unicodedata.normalize("NFD", text).replace(_NUKTA, "")
text = unicodedata.normalize("NFC", text).translate(_NUKTA_FOLD)
return _SPACE.sub(" ", text).strip()
|