| --- |
| license: other |
| license_name: mixed-see-below |
| tags: |
| - automatic-speech-recognition |
| - translation |
| - whisper.cpp |
| - ctranslate2 |
| - vrstt |
| --- |
| |
| # vrSTT models |
|
|
| Model weights for [vrSTT](https://github.com/kittynXR/vrSTT) — local speech-to-text |
| for VR and streaming. Everything vrSTT runs is here so the app can fetch the one |
| model a user chose, verify it, and never touch the network again. |
|
|
| **This repository redistributes third-party models. It trains nothing and |
| modifies nothing except file format.** Licences differ per directory and are |
| stated below; the upstream terms travel with the weights. |
|
|
| ## Layout |
|
|
| ``` |
| manifest.json # what the fetcher reads first |
| whisper/ggml-<name>.bin # whisper.cpp ggml format, one file each |
| translation/<name>/ # CTranslate2 format, a directory each |
| ``` |
|
|
| `manifest.json` carries a `sha256` and a byte count for every file. The fetcher |
| verifies both before handing a path to whisper.cpp or CTranslate2 — a truncated |
| ggml file otherwise fails at load with an error about magic bytes, which sends |
| you looking at the wrong thing. |
|
|
| ## What is here |
|
|
| **Speech recognition** — [whisper.cpp](https://github.com/ggerganov/whisper.cpp) |
| ggml conversions, byte-identical to |
| [ggerganov/whisper.cpp](https://huggingface.co/ggerganov/whisper.cpp). Mirrored |
| rather than linked so the bytes vrSTT downloads are pinned and hashed. |
|
|
| `tiny`, `tiny.en`, `base`, `base.en`, `small`, `small.en`, `medium`, |
| `medium.en`, `large-v1`, `large-v2`, `large-v3`, `large-v3-turbo` |
|
|
| vrSTT defaults to **`small.en` for commits and `base.en` for live previews** — |
| the split is deliberate, previews are allowed to be wrong and are replaced within |
| a second, so they should not spend the larger model. |
|
|
| **Translation** — [NLLB-200](https://ai.meta.com/research/no-language-left-behind/) |
| converted to CTranslate2 with `ct2-transformers-converter --quantization |
| float16`, from `facebook/nllb-200-*`. |
| |
| `nllb-200-distilled-600M` (default), `nllb-200-distilled-1.3B`, `nllb-200-1.3B` |
| |
| float16 rather than int8 because CTranslate2 can cast **down** at load time but |
| not up — one artifact serves both, and converting to int8 here would cap quality |
| for everyone. 600M is the default because vrSTT's stated minimum-VRAM target is |
| 8 GB, and a 1.3B model beside the ASR stages and a game does not fit |
| comfortably there. |
| |
| ## Licences — read this before redistributing further |
| |
| | Directory | Licence | Commercial use | |
| |---|---|---| |
| | `whisper/` | **MIT** (OpenAI Whisper; ggml conversions by Georgi Gerganov) | permitted | |
| | `translation/` | **CC-BY-NC-4.0** (Meta AI, NLLB-200) | **NOT permitted** | |
| |
| **NLLB-200 is non-commercial.** Redistribution is allowed with attribution and |
| the licence must travel with it — which is what this section is — but neither |
| Meta's weights nor anything derived from them may be used commercially. If vrSTT |
| ever becomes a paid product, the translation models have to be replaced rather |
| than relicensed. |
| |
| NLLB-200 also carries Meta's own caveats, which apply unchanged here: it is |
| research work, not certified for production translation, it was trained on |
| Wikimedia-domain text and does not target long-form or domain-specific input, |
| and translation quality varies widely by language pair. Mistranslations can |
| carry real consequences and vrSTT displays them live on a stream — the |
| `include_original` setting exists so the source text stays visible beside the |
| translation for exactly this reason, and it is on by default. |
| |
| ## Citation |
| |
| Whisper — Radford et al., *Robust Speech Recognition via Large-Scale Weak |
| Supervision*, OpenAI, 2022. |
| |
| NLLB-200 — NLLB Team et al., *No Language Left Behind: Scaling Human-Centered |
| Machine Translation*, Meta AI, 2022. |
| |
