README.md

metadata

license: other
language:
  - hi
  - mr
  - ne
  - mai
  - sa
  - brx
  - kok
  - doi
  - gu
  - pa
  - or
  - ta
  - te
  - kn
  - ml
  - mni
  - bn
  - as
  - ur
  - sd
  - ks
  - sat
tags:
  - transliteration
  - code-mixing
  - text-to-speech
  - indic

indictrans: English words in Indian-language text, written in the request's script

Runtime assets of the code-mix transliteration layer of the Finesse TTS (utils/codemix_xlit): English words inside Indic text are rewritten in the language's script, spelled the way an Indian-English speaker says them, so an Indic voice reads them correctly; text typed entirely in Latin is read word by word, the language's own words back into its script. 22 languages (Dogri and Santali through the Hindi model).

Version 1.3.0+c0ad24b04aaa. Built by tools/codemix_xlit/build_assets.py; manifest.json lists every file with its size and SHA-256.

Contents

• model/: the IndicXlit en->indic v1.0 transformer converted to CTranslate2
• lexicon/<code>.reviewed.json, lexicon/<code>.generated.json: reviewed and generated spellings
• rescore/: pruned IndicXlit word-frequency dictionaries
• cmudict.json, english_*: English pronunciations, vocabulary and reviewed corrections
• indic_words.json: Indian words read natively; ranker.json: candidate ranker
• roman/<code>.json: how people type each language's words in Latin, and how often

Sources and licences

• IndicXlit model and word-frequency dictionaries: AI4Bharat, MIT licence (https://github.com/AI4Bharat/IndicXlit)
• CMU Pronouncing Dictionary: Carnegie Mellon University, BSD-style licence
• English word lists derived from wordfreq (https://github.com/rspeer/wordfreq): CC BY-SA 4.0
• Dakshina dataset (Roark et al. 2020, https://github.com/google-research-datasets/dakshina): CC BY-SA 4.0; the romanized tables of bn, gu, hi, kn, ml, mr, pa, sd, ta, te, ur use counts derived from its romanized sentences, and those tables are shared under CC BY-SA 4.0
• Lexicons, corrections, ranker and the other romanized tables: derived from the above and reviewed