| --- |
| license: other |
| license_name: apache-2.0-and-gemma |
| license_link: LICENSE.md |
| base_model: |
| - google/gemma-4-e4b-it-qat-q4_0-unquantized |
| - google/gemma-4-E2B-it-qat-q4_0-unquantized |
| - google/gemma-3-1b-it-qat-q4_0-unquantized |
| tags: |
| - dictation |
| - speech-to-text |
| - text-normalization |
| - grammatical-error-correction |
| - gguf |
| - lora |
| language: |
| - en |
| --- |
| |
| # Utter cleanup models |
|
|
| The cleanup models of [Utter](https://github.com/ductai199x/utter), a local voice dictation app. |
| They take what the speech recognizer (NVIDIA Parakeet) heard and return the text the speaker |
| meant to type: |
| - **Punctuation and capitals** nobody said. |
| - **Spoken symbols and numbers** written as people type them ("four x faster" -> "4× faster", |
| "about two forty" -> "~240"). |
| - **Fillers and false starts** removed. |
| - **Corrections applied:** "at 2, actually 3"; "scratch that" (the last sentence); "let's start |
| over" (everything). |
| - **Far-back edits:** "actually, change the eggs to three boxes". |
| - **Spoken lists as bullets,** numbered for steps. |
| - **Grammar fixed,** in the speaker's own words. |
| - **The person's own words spelled right** when they are in their dictionary. |
| - **Dictated text is typed, never answered:** a question or an instruction to an AI comes out |
| as text. |
|
|
| Utter offers Quality and Light on a computer with a graphics card, and Light and Extra light on |
| one without; the person chooses in the app. |
|
|
| ## Files |
|
|
| | File | Model | What | |
| |---|---|---| |
| | `utter-quality-v2-q4_0.gguf` | **Quality**, v2 | Gemma 4 E4B fine-tuned, Q4_0, for llama.cpp (b11535 or later); about 3.3 GB of GPU memory when the per-layer embeddings stay in system RAM | |
| | `adapters/utter-quality-v2/` | | Its LoRA adapter (rank 16) on the base checkpoint, for merging or further training | |
| | `utter-light-v2-q4_0.gguf` | **Light**, v2 | Gemma 4 E2B fine-tuned, Q4_0; about 1.8 GB of GPU memory, for machines with little of it (Utter then runs speech recognition on the CPU) | |
| | `adapters/utter-light-v2/` | | Its LoRA adapter (rank 16) | |
| | `utter-mini-v1-q4_0.gguf` | **Extra light**, v1 | Gemma 3 1B fully fine-tuned, Q4_0 (720 MB): for computers without a graphics card, and Utter's phone app. Gemma 3 terms (see License) | |
| |
| v1 (`utter-quality-v1-q4_0.gguf`, `utter-light-v1-q4_0.gguf` and their adapters) stays here: |
| it was trained before the data was published, on pairs that overlap today's test split. |
| |
| ## How it must be prompted |
| |
| The rules are in the weights, so the system prompt is short, and it must be exactly this: |
| |
| ``` |
| Turn this dictation into the text the speaker meant to type: punctuated, formatted and corrected the way a careful writer would type it, in the speaker's own words. |
| ``` |
| |
| The user message is the transcript after `Dictated: `, optionally preceded by context lines: |
| |
| ``` |
| App: email |
| Before: "text before the cursor" |
| Lists: off |
| Dictionary: Utter, Nguyen |
| Dictated: <what the recognizer heard> |
| ``` |
| |
| `Dictionary:` carries only the person's terms that sound like something in this dictation, not |
| the whole list. The reply is the cleaned text only. Thinking must be off |
| (`chat_template_kwargs: {"enable_thinking": false}`). |
|
|
| ## Training |
|
|
| - **Method:** LoRA (rank 16, alpha 32) on Google's unquantized QAT checkpoint of Gemma 4 (E4B |
| for Quality, E2B for Light), then merged and quantized to Q4_0, the format the QAT was |
| trained for. |
| - **Both tiers, one recipe:** one pass over the train split of |
| [ductai199x/dictation-to-text](https://huggingface.co/datasets/ductai199x/dictation-to-text) |
| (79,246 pairs), lr 2e-4, 1 epoch. A share of the pairs carry `Dictionary:` lines: some hold |
| the right spelling of a misheard word, some near-miss words that must not be forced in. |
| - **Extra light:** every weight of Google's unquantized QAT checkpoint of Gemma 3 1B trained |
| (not LoRA: a rank-16 adapter left the small model its habit of running on in lists), on the |
| same train split, lr 2e-5, 2 epochs, then quantized to Q4_0. |
| - **Data:** pairs of (recognizer output, intended text): |
| - **Written side:** real passages from Common Pile (Stack Exchange, GitHub, arXiv abstracts, |
| Ubuntu IRC) and Wikipedia, plus lists and LLM-written everyday messages. |
| - **Spoken side:** written by Gemma 4 26B-A4B in eight kinds (plain, spoken punctuation, |
| disfluent, self-correction, retraction command, sentence restatement, grammar mistakes, |
| far-back edit), checked by code, spoken by Kokoro TTS and recognized by Parakeet. |
| - **Answer keys:** fixed for typos and grammar, with each fix verified as grammar by ERRANT. |
| - **Held out:** the dataset splits by passage; no test or evaluation passage was trained on. |
| |
| ## Evaluation |
|
|
| **Ready to send:** the share of dictations a careful writer would send exactly as typed. Gemma 4 |
| 26B-A4B judges each one against the intended text, with a rubric calibrated |
| on human review. |
|
|
| **Test sets:** |
| - **Realistic:** 950 held-out dictations, everyday messages, emails, notes and AI prompts plus |
| short real forum and chat posts. Each is spoken one way: plainly, with fillers, a |
| self-correction, a command, a later edit, spoken punctuation or broken grammar. |
| - **Directed:** 292 dictations of questions, requests to AI assistants, instructions to coding |
| agents, and text telling the dictation app itself to do something. |
|
|
| | Ready to send | Quality v2 | Light v2 | |
| |---|---|---| |
| | Realistic, Kokoro voices | 88.1% | 85.4% | |
| | Realistic, with a 300-term dictionary | 87.8% | 85.7% | |
| | Realistic, unseen Chatterbox voices | 80.1% | 75.5% | |
| | Directed, with a dictionary | 87.7% | 83.6% | |
| | Cleanup time, median (RTX 3090 Ti) | about 250 ms | about 180 ms | |
|
|
| **Extra light**, judged the same way on a larger held-out set (25,469 dictations), beside |
| Light on the same set: |
|
|
| | Ready to send | Light v2 | Extra light v1 | |
| |---|---|---| |
| | Kokoro voices, real passages | 85.7% | 79.7% | |
| | Unseen Chatterbox voices | 74.6% | 67.0% | |
| | Typed text test | 64.3% | 51.9% | |
| | Cleanup time, median (Pixel 10 Pro XL CPU) | 2.69 s | 1.49 s | |
|
|
| Utter retries a reply that runs on (one more pass that forbids a leading list marker); with |
| that retry, 1 of the 25,469 still ran on (Light: none). |
|
|
| **Known gaps:** |
| - A leading filler with a comma ("Uh, so...") can survive. |
| - Pronoun case ("me and him went") is not always fixed. |
| - Speech in the evaluation is synthetic (two TTS engines), not recorded dictation. |
|
|
| ## License |
|
|
| - **Quality and Light** (Gemma 4): Apache 2.0, as their base models. |
| - **Extra light** (`utter-mini-*`, Gemma 3): Gemma is provided under and subject to the Gemma |
| Terms of Use found at [ai.google.dev/gemma/terms](https://ai.google.dev/gemma/terms), including |
| the [Gemma Prohibited Use Policy](https://ai.google.dev/gemma/prohibited_use_policy). This |
| model is a modification of Gemma 3 1B; anyone who uses or redistributes it must follow |
| those terms and pass them on. |
|
|
| The training text came from openly licensed sources (public domain, CC0, CC BY-SA, MIT, |
| Apache-2.0). |
|
|
