Android LT 1.1.0: Lithuanian ears for the Parakeetype keyboard

.gitattributes

@@ -37,3 +37,5 @@ docs/sherpatts_install.jpg filter=lfs diff=lfs merge=lfs -text
 docs/sherpatts_main.jpg filter=lfs diff=lfs merge=lfs -text
 lt_dict filter=lfs diff=lfs merge=lfs -text
 docs/banner.png filter=lfs diff=lfs merge=lfs -text
+docs/parakeetype_model.jpg filter=lfs diff=lfs merge=lfs -text
+docs/parakeetype_switch.jpg filter=lfs diff=lfs merge=lfs -text

AI_CONSULTANT_BRIEF.md

@@ -32,6 +32,11 @@ Three files:
 - `lt_dict` (2.2 MB): eSpeak NG Lithuanian dictionary with the stressed syllable of 189k word
   forms. Optional; needs `adb` from a computer.
 
+Since 1.1.0 also **ears**: `parakeet-lt-kmynas-int8.zip` (955 MB), a Lithuanian speech
+recognition model for the voice keyboard **Parakeetype** (theScrabi, F-Droid,
+`org.schabi.parakeetype`). If the person asks about dictation rather than reading aloud, it is
+this part; see "Ears" below.
+
 ## Facts you must know before answering
 
 - SherpaTTS comes from **F-Droid**, not Google Play. On Google Play a search shows other apps,
@@ -65,6 +70,28 @@ Three files:
 | Folder `Android/data/...` "access denied" in a file manager | expected on recent Android; use `adb` |
 | Numbers read oddly ("15:25 val.") | eSpeak NG reads numbers and abbreviations itself; no fix here |
 
+## Ears (dictation with Parakeetype)
+
+- Parakeetype accepts only models listed in its own code, with fixed file hashes. The Lithuanian
+  entry is proposed in https://github.com/theScrabi/Parakeetype/pull/1. **Check that PR's state
+  first.** If it is not merged, the F-Droid app shows no "Parakeet Lithuanian" card; the person
+  has to wait or build the app from https://github.com/RobertasTa/Parakeetype/tree/add-lithuanian-model
+  (README, "Build it yourself").
+- Import: main screen → "Install" next to "Model not installed" → scroll to the **Parakeet
+  Lithuanian** card (below Parakeet Ultra) → "Import model file" → the ZIP. Then "Set as Active".
+- Switching to it: in Gboard long-press the 🌐 globe → "Parakeetype Voice Input". Back: the
+  small keyboard icon on the right, above Enter.
+- Measured word error rate, Parakeetype's default model vs. these ears: FLEURS 27.1 % → 10.6 %,
+  short commands 80.4 % → 17.0 %. Lithuanian only.
+
+| Symptom | Cause |
+|---|---|
+| Import says the hashes do not match | archive imported on the Parakeet Ultra card; use the Parakeet Lithuanian card |
+| No "Parakeet Lithuanian" card | the installed Parakeetype does not include the PR yet |
+| A self-built app will not install over the F-Droid one | different signing keys; uninstall the other one first |
+| `adb install` fails with `INSTALL_FAILED_USER_RESTRICTED` (Xiaomi) | copy the APK to Download and tap it in a file manager |
+| Lithuanian dictation comes out as Russian or nonsense | the default model is active; set Parakeet Lithuanian as active |
+
 ## Things you must NOT claim
 
 - That this is on Google Play, or that SherpaTTS is made by the authors. It is woheller69's app.
@@ -72,13 +99,16 @@ Three files:
   not expressed.
 - That it is "the first" or "the only" Lithuanian voice for Android.
 - That `lt_dict` works with any SherpaTTS version: it was built and tested for 3.4.
+- That Parakeetype is made by the authors, or that the official app already includes the
+  Lithuanian model, unless the PR above is merged.
 
 ## Licences
 
 Voice: NewGenLTU Open RAIL-D 1.0 (free, commercial use not prohibited, use restrictions apply,
 `LICENSE-VOICE` must travel with it). `lt_dict`: GPL-3.0, sources in `espeak-ng-lt/`, stress
 data from two CC BY 4.0 works (LIEPA annotation, Vilnius University; g2p-lt-lexicon,
-Arūnas Smaliukas). See THIRD_PARTY.md.
+Arūnas Smaliukas). Ears: CC BY 4.0 (NVIDIA Parakeet TDT 0.6B v3, fine-tuned by Kristijonas
+Jakubsonas as "kmynas v4"); Silero VAD inside the archive is MIT. See THIRD_PARTY.md.
 
 ## The author
 

README.md

@@ -6,27 +6,35 @@ license_name: newgenltu-openrail-d
 license_link: https://huggingface.co/RobertasTa/android-lt/blob/main/LICENSE-VOICE
 library_name: sherpa-onnx
 pipeline_tag: text-to-speech
-base_model: RobertasTa/lt_LT-ingute-medium
+base_model:
+- RobertasTa/lt_LT-ingute-medium
+- kristijonas/kmynas-parakeet-lt-v4
 tags:
 - text-to-speech
 - tts
+- automatic-speech-recognition
+- speech-recognition
 - lithuanian
 - android
 - sherpa-onnx
 - piper
 - espeak-ng
+- parakeet
 - offline
 - accessibility
 ---
 
-# Android LT: a Lithuanian voice for Android phones
+# Android LT: a Lithuanian voice and ears for Android phones
 
 ![A robotic hand holds a gift box labelled "android-lt" under a glowing globe; next to it: "Android LT, Ingutė on your Android phone" (illustration created with AI)](docs/banner.png)
 
 Ingutė, a Lithuanian synthetic voice, on your Android phone, offline. The phone uses it wherever
 it can read aloud: the text-to-speech settings, the screen reader, book and article readers.
 
-*Lietuviškai ir kodas: [github.com/RobertasTa/android-lt](https://github.com/RobertasTa/android-lt)*
+Since 1.1.0 there are **ears** too: speak Lithuanian to the phone and the text appears in any text
+field, also offline (see [Ears](#ears-lithuanian-dictation-parakeetype)).
+
+*Page: https://robertasta.github.io/android-lt/en/ · Lietuviškai ir kodas: [github.com/RobertasTa/android-lt](https://github.com/RobertasTa/android-lt)*
 
 ## What is here
 
@@ -37,6 +45,8 @@ it can read aloud: the text-to-speech settings, the screen reader, book and arti
 - **`lt_dict`**: the Lithuanian eSpeak NG dictionary with the stressed syllable for 189,000 word
   forms. Without it the stress is guessed by rules and is wrong in about half of the
   multi-syllable words.
+- **Ears** (`parakeet-lt-kmynas-int8.zip`, 955 MB): a Lithuanian speech recognition model for
+  the Parakeetype voice keyboard.
 
 The voice is played by **SherpaTTS**, a free open-source Android TTS engine. It is not part of
 this package; install it separately (step 1).
@@ -107,14 +117,82 @@ Android 15, SherpaTTS 3.4.
 - `lt_dict` was built for SherpaTTS 3.4 (sherpa-onnx 1.13); check the md5 first on other versions.
 - It is a synthetic voice: natural, but not a person.
 
+## Ears: Lithuanian dictation (Parakeetype)
+
+[Parakeetype](https://github.com/theScrabi/Parakeetype) is an open-source voice keyboard for
+Android (F-Droid). It recognises speech on the phone and never goes online. Its default model
+hears Lithuanian poorly; `parakeet-lt-kmynas-int8.zip` gives it Lithuanian ears.
+
+| word error rate | FLEURS (986 utterances) | short commands (127) |
+|---|---|---|
+| Parakeetype's default model (Parakeet Ultra) | 27.1 % | 80.4 % |
+| with these ears | 10.6 % | 17.0 % |
+
+Measured on a PC with a script that follows Parakeetype's recognition path exactly; tested live on
+Robertas's phone (Xiaomi 12T Pro, Android 15).
+
+**Status (2026-10-29).** For Parakeetype from F-Droid to accept this archive, its author has to
+merge our proposal [theScrabi/Parakeetype#1](https://github.com/theScrabi/Parakeetype/pull/1).
+Until then you can build the app with the Lithuanian model yourself ("Build it yourself" below).
+
+### Installing
+
+1. **Parakeetype**: from F-Droid once the proposal is merged, or built yourself.
+2. **Download to the phone** `parakeet-lt-kmynas-int8.zip` (955 MB) from the
+   [release](https://huggingface.co/RobertasTa/android-lt/tree/main).
+3. On Parakeetype's main screen enable the keyboard and allow the microphone. Next to "Model not
+   installed" tap "Install", scroll down to the **Parakeet Lithuanian** card → "Import model
+   file" → the archive. The app checks every file's hash; you can delete the archive afterwards.
+4. On the same card: "Set as Active".
+5. **Dictate**: open any text field (notes, for example) and switch the keyboard to Parakeetype.
+   In Gboard, long-press the 🌐 globe and pick "Parakeetype Voice Input". Hold the big microphone
+   button and speak. The first time, the model takes a few seconds to load.
+6. **Back to your usual keyboard**: the small keyboard icon on the right, above the Enter key.
+
+| | | |
+|---|---|---|
+| ![Parakeet Lithuanian as the active model](docs/parakeetype_model.jpg) | ![Switching keyboards: Parakeetype voice input](docs/parakeetype_switch.jpg) | ![Dictated: "Labas testas yra. Tikrinam, kaip ausys veikia."](docs/parakeetype_dictation.jpg) |
+
+### Build it yourself (until the proposal is merged)
+
+You need a computer with Java 17 or newer and the Android SDK: [Android Studio](https://developer.android.com/studio)
+or Google's command-line tools (`android sdk install "platforms;android-37.0" "build-tools;37.0.0"`).
+The first build downloads about 1 GB.
+
+```
+git clone -b add-lithuanian-model https://github.com/RobertasTa/Parakeetype.git
+cd Parakeetype
+./gradlew assembleDebug
+```
+
+On Windows: `gradlew assembleDebug`. Point to the Android SDK with `sdk.dir=...` in
+`local.properties` or the `ANDROID_HOME` variable. Copy the resulting
+`app/build/outputs/apk/debug/app-debug.apk` to the phone's Download folder and tap it in a file
+manager (Xiaomi phones refuse `adb install` without a Mi account). If you already have
+Parakeetype from F-Droid, uninstall it first: apps signed with different keys cannot update each
+other. Once the proposal is merged, uninstall your build and install the F-Droid one.
+
+### Limits of the ears
+
+- Lithuanian only. For other languages pick the default model (one model is active at a time).
+- About 910 MB on the phone; Parakeetype needs Android 12 or newer.
+- On short commands it gets roughly one word in six wrong.
+
 ## Licences
 
+The header above names the voice licence; the ears archive is CC BY 4.0 (below).
+
+
 - **Voice** (`lt_LT-ingute-medium.onnx`, `tokens.txt`): NewGenLTU Open RAIL-D 1.0. Free of
   charge, commercial use is not prohibited, use restrictions apply; when passing it on, include
   [LICENSE-VOICE](LICENSE-VOICE) and state the origin. Ingutė was grown from the Reginutė model
   with the authors' own Python tools.
 - **`lt_dict`** and its sources (`espeak-ng-lt/`): GPL-3.0 (eSpeak NG data). The stress
   positions come from two CC BY 4.0 works; see [THIRD_PARTY.md](THIRD_PARTY.md).
+- **Ears** (`parakeet-lt-kmynas-int8.zip`): CC BY 4.0. NVIDIA Parakeet TDT 0.6B v3, fine-tuned for
+  Lithuanian by Kristijonas Jakubsonas ("kmynas v4", LIEPA-3). Our changes are listed in the
+  archive's `NOTICE.txt`. The Silero VAD inside the archive is MIT. Parakeetype is a separate
+  app (GPL-3.0).
 - Texts in this repository: GPL-3.0 ([LICENSE](https://github.com/RobertasTa/android-lt/blob/main/LICENSE)).
 
 ## Thanks
@@ -123,7 +201,9 @@ Android 15, SherpaTTS 3.4.
 [sherpa-onnx](https://github.com/k2-fsa/sherpa-onnx) (k2-fsa),
 [eSpeak NG](https://github.com/espeak-ng/espeak-ng), [Piper](https://github.com/OHF-Voice/piper1-gpl)
 and Michael Hansen; the creators of the **LIEPA** corpus (Vilnius University) and
-**Arūnas Smaliukas** for `g2p-lt-lexicon`.
+**Arūnas Smaliukas** for `g2p-lt-lexicon`; [Parakeetype](https://github.com/theScrabi/Parakeetype)
+(theScrabi), **Kristijonas Jakubsonas** for the Lithuanian Parakeet ("kmynas") and NVIDIA for
+Parakeet.
 
 ## Stuck?
 

SHA256SUMS.txt

@@ -2,3 +2,12 @@
 7d68702d4229c0813cbeaa680b848252364180242eb751b58f89165b21b345af  tokens.txt
 3b56f0986c0319d00b589465f1b93f69c59d61a3a1b5cb9a482d519e386adf66  lt_dict
 a359105275a97d1adf721cb4b49e249407742a268e381ecfc7181f59d9419075  LICENSE-VOICE
+c5654729284d1a6ad40f47ead3a7901acc93b8b2f087dbacc7a8797922f6d015  parakeet-lt-kmynas-int8.zip
+
+# Files inside parakeet-lt-kmynas-int8.zip (Parakeetype verifies these on import)
+8f4dafb7d62ae117dc5f1133d44457b6ae8d4f91e6d899588581a519f14319dc  encoder.int8.onnx
+78246a6d979ddecf47b3c66b2c332a32cb60abef75b019692b78f2bcb3d9aad2  decoder.int8.onnx
+0c965ab9c66720bb101ca6457c3c54dfd6d7780b668e8d0b641abef459b49cf5  joiner.int8.onnx
+a9fde1486ebfcc08f328d75ad4610c67835fea58c73ba57e3209a6f6cf019e9f  nemo128.onnx
+fab30a2363a8eccd053cf0def763db0b2f9e37a4b321070698f36da013e448e4  vocab.txt
+1a153a22f4509e292a94e67d6f9b85e8deb25b4988682b7e174c65279d8788e3  silero_vad_v6.onnx

THIRD_PARTY.md

@@ -33,7 +33,31 @@ https://huggingface.co/RobertasTa/lt_LT-ingute-medium
 
 The resulting `lt_dict` is distributed under GPL-3.0 with these sources.
 
+## The ears (`parakeet-lt-kmynas-int8.zip`, release v1.1.0)
+
+A Lithuanian speech recognition model for Parakeetype, licensed under the Creative Commons
+Attribution 4.0 International licence (full text and an attribution notice inside the archive as
+`LICENSE.txt` and `NOTICE.txt`):
+
+- "parakeet-tdt-0.6b-v3" by NVIDIA: https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3
+- fine-tuned for Lithuanian on the LIEPA-3 corpus as "kmynas-parakeet-lt-v4" by Kristijonas
+  Jakubsonas: https://huggingface.co/kristijonas/kmynas-parakeet-lt-v4
+- `nemo128.onnx` (the audio front end) from the Parakeet TDT 0.6B v3 ONNX export by istupakov:
+  https://huggingface.co/istupakov/parakeet-tdt-0.6b-v3-onnx
+
+Our changes: exported to ONNX (encoder, decoder, joiner), the joint network's input projections
+moved into the encoder and decoder graphs with the same weights (the output is identical), tensors
+renamed to the layout Parakeetype reads, encoder quantised to int8 (MatMul weights only), the
+vocabulary written as `vocab.txt`.
+
+`silero_vad_v6.onnx` in the archive is the Silero VAD v6 model by the Silero Team
+(https://github.com/snakers4/silero-vad), unmodified, MIT licence (`LICENSE-silero-vad.txt`).
+
 ## Not included
 
 SherpaTTS (https://github.com/woheller69/ttsengine, GPL-3.0) is a separate app and is not part
 of this package. Install it from F-Droid.
+
+Parakeetype (https://github.com/theScrabi/Parakeetype, GPL-3.0) is a separate app as well. Our
+proposed change to it (registering the Lithuanian model) lives in a public branch:
+https://github.com/RobertasTa/Parakeetype/tree/add-lithuanian-model

parakeet-lt-kmynas-int8.zip

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c5654729284d1a6ad40f47ead3a7901acc93b8b2f087dbacc7a8797922f6d015
+size 954759498