Automatic Speech Recognition
PyTorch
Safetensors
GGUF
MLX
Tamil
English
speech
asr
streaming
rnnt
ctc
tamil
tanglish
code-switching
indian-english
verisafe
Instructions to use junafinity/VeriSafe-Tanglish-1-STT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use junafinity/VeriSafe-Tanglish-1-STT with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] hf download junafinity/VeriSafe-Tanglish-1-STT --local-dir VeriSafe-Tanglish-1-STT
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
File size: 5,994 Bytes
ec01e2d | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 | # Code
Shared by the format folders. Python 3.12.
| Path | What |
|---|---|
| `transcribe.py` | The PyTorch runner for `../safetensors/` and `../gguf/` (`--weights FILE`; default `../safetensors/stt2_model.safetensors`). A file with `fork_cm.*` tensors needs `"codemix_fork": true` in its settings and the reverse (strict check on load). `--mode codemix` (default) or `--mode tamil`, `--head rnnt` (default) or `ctc`, `--streaming` with `--finals two-pass` (default) or `--finals f`, `--ctc-assist` (default in code-mix mode: offline, and with `--streaming` for the final text, partials greedy) / `--no-ctc-assist` |
| `ctc_assist.py` | `--ctc-assist`: J(0.3, 0) CTC-assisted rescoring. Its decoding functions are verbatim copies of the project's `scripts/rnnt_beam.py` and `scripts/decode_ctc_assist.py` (sha256 of both in its docstring) |
| `indic_stt/model.py` | The model. Same as the project's `src/indic_stt/model.py` except at the three places marked `PACKAGE CHANGE` (plus two imports): the audio tower is built from a local `config.json` instead of downloading EmbeddingGemma 2 (two places), (STT-1.2) `encode()` is split into `mix()` (tower and layer mix) and `read()` (the reader), the same operations in the same order, so that a streaming caller can run the reader twice on one tower output, and (STT-2) a second post-tower stack, the code-mix reader fork `fork_cm` (layer mix, input norm and projection, position conv, 6 reader layers, output norm: 81 tensors), which the code-mix heads read when the file holds it; `mix`, `read` and `encode` then take `mode="codemix"` or `mode="tamil"` |
| `indic_stt/text.py`, `indic_stt/tower_ctx.py` | Tokenizer wrapper and text normalization; tower context helper. Unchanged from the project |
| `indic_stt/gguf_io.py` | GGUF writer and reader for this release |
| `export/` | How the formats were built and checked: `export_release.py` (safetensors, GGUF, MLX), `extract_tower_rest.py`, `make_manifest.py`, and the MLX sources copied into `../mlx/` |
| `cm_main_training_config.yaml` | The training config of STT-1 (run `cm-main`), whose tower, Tamil stack and Tamil heads STT-2 keeps, for the record. The fork run's own config is not shipped; `../safetensors/stt_config.json` records the fork's provenance. Its local paths are not read |
STT-2 has one tower, two post-tower stacks and two head sets, each an RNN-T head and a CTC head: the Tamil stack and heads (`transducer`, `ctc_head`; `--mode tamil`, Tamil script throughout; unchanged from STT-1) and the code-mix reader fork with the code-mix heads (`fork_cm.*`, `transducer_cm`, `ctc_head_cm`; `--mode codemix`, English words in Latin script). In the Python API, `Release.mix`, `read`, `encode`, `greedy` and `ctc_assist` need `mode=` on a fork file. In each mode the CTC head also gives the start cue of RNN-T greedy decoding: a start penalty of 1 on blank while nothing has been emitted, from the first frame where that CTC head leaves blank.
The tokenizer, `../tokenizer/spm.model`, is a SentencePiece model with 2,058 pieces; model id = piece id + 1, and id 0 is the blank.
`transcribe.py` has one switch for the default: `CTC_ASSIST_DEFAULT` (shipped `True`, for the modes in `CTC_ASSIST_DEFAULT_MODES` = code-mix only: code-mix RNN-T rows use the J(0.3, 0) rescoring unless `--no-ctc-assist`; `--mode tamil` is greedy unless `--ctc-assist`; `--head ctc` rows are greedy either way). STT-1.1 adds `STREAMING_FINALS_DEFAULT` (shipped `True`): the default also applies with `--streaming`, where J re-ranks each utterance's final text once, at its end, on the whole utterance's streaming reader output; the partials (the live text) stay RNN-T greedy with the start cue, which is `--streaming --no-ctc-assist` and STT-1's streaming row. STT-1.2 adds `--finals` and its default `FINALS_DEFAULT` (shipped `"two-pass"`): with `--streaming`, at the end of each utterance the reader runs once more, with full attention, over the kept `mix()` output, and the final text is the offline decode of the same flags (J(0.3, 0) in code-mix; greedy with `--no-ctc-assist`, `--mode tamil` or `--head ctc`). So since STT-1.2 the partials' text, STT-1's streaming row, is `--streaming --finals f --no-ctc-assist` (Tamil: `--mode tamil --streaming --finals f`); `--finals f` is STT-1.1's finals, J on the streaming reader output. Utterances over `TWO_PASS_MAX_SECS` (30 s) get `f`, with a note on stderr. `FINALS_DEFAULT = "f"` restores STT-1.1; with `STREAMING_FINALS_DEFAULT = False` as well, STT-1's greedy `--streaming`. `Release` refuses a model with tower right context (`offline_right` > 0), since one `mix()` feeds both reader passes. A live caller runs `mix()` and `read(..., streaming=True)` + `greedy()` for the partials, starts the tower and position-conv state fresh at each utterance, and at its end runs `read(..., streaming=False)` over that utterance's frames only. The MLX runner, `../mlx/run_mlx.py`, has the same `CTC_ASSIST_DEFAULT` with the same value, but no streaming finals and no `--finals`: its `--streaming` stays greedy.
Rebuild the formats from the fork candidate (`f_B0.5.pt`, written by the project's `scripts/build_fork.py` from STT-1's `ema.pt` and the fork run's EMA; neither checkpoint is published; needs torch, safetensors, gguf, sentencepiece, numpy). The safetensors step needs STT-1's safetensors file (`STT1_FILE` below; it is not in this repository) and checks the Tamil half against it byte for byte:
```
python -X utf8 export/extract_tower_rest.py EG2/model.safetensors ema.pt tower_rest.safetensors
python -X utf8 export/export_release.py safetensors f_B0.5.pt tower_rest.safetensors RELEASE_DIR STT1_FILE
python -X utf8 export/export_release.py gguf RELEASE_DIR f16 (also q8_0)
python -X utf8 export/export_release.py mlx RELEASE_DIR (the float32 MLX file, not published; the input of mlx8)
python -X utf8 export/export_release.py mlx8 RELEASE_DIR (the 8-bit file; needs MLX)
python export/make_manifest.py verify RELEASE_DIR
```
|