Infatoshi/phonon-refiner-1.7b

Phonon refiner 1.7B

The correction model in Phonon, a local voice-typing app for Apple Silicon Macs. Phonon turns speech into a raw transcript with Parakeet TDT 0.6B v2. This model rewrites that transcript into the text the speaker meant to type: it fixes misrecognized words and technical terms, removes filler and false starts, keeps only the final version when the speaker corrects themselves, and formats the text for the app it is going to (writing modes, below). It is fine-tuned from LiquidAI/LFM3-1.7B on transcripts of public audio and on synthetic transcripts. No personal dictation went into it.

This is the refiner in Phonon 1.0.1. What changed since 1.0:

• It ends a question with "?" when the transcript ended it with a period or no mark at all, which the 1.0 refiner usually did not.
• It keeps short answer words and confirmation tags ("Yeah, ...", "..., right?") that the 1.0 refiner removed as filler.
• It writes line breaks itself: dictated steps become a numbered list, one step per line, in every mode but casual, and in formal mode a dictated greeting and sign-off go on their own lines.
• On 2,522 of the Phonon author's own dictations, a local judge preferred its text over the 1.0 refiner's 575 times and the 1.0 refiner's 248 times (1,200 outputs were identical), and on keeping the speaker's meaning 156 to 98. Details below.

The 1.0 refiner is commit 2b0ab74 of this repository and the 0.2.3 refiner is commit 74ea4b6.

This repository holds what the app downloads on first launch:

File	What it is	Bytes	md5
phonon-refiner-1.7b-Q4_K_S-e4.gguf	the refiner, GGUF Q4_K_S, for the native engine	988,453,696	444a71a5b60f73a158e27770843b4f9c
phonon-refiner-1.7b-dflash2-drafter.gguf	DFlash2 speculative drafter (unchanged from 0.2.3)	214,803,168	1d7cf3084db8567106d540ef5832e5fe
model.safetensors	the refiner, MLX 4/6/8-bit, the fallback	1,133,072,403	59df3375745fcb258082eda0c41cbe19
model.safetensors.index.json	MLX weight index (unchanged)	41,831	62fd9b86a96e6a0a1fef69ff050f4ff9
config.json	model config (MLX quantization included, unchanged)	28,490	5ad9fe4a82c4dfd5c6a7edab1590a63a
tokenizer.json	tokenizer	17,908,634	f16e3a324b7c8650672fde1b46eba60a
tokenizer_config.json	tokenizer config	1,896	5dd81ef5773d9baddb35765f387bc7ab
chat_template.jinja	chat template (the base model's)	12,129	c04953adf0eb362de060fa5a9a82316e
generation_config.json	generation defaults (no repetition penalty)	328	c2be237d5e4a810893ba46925a50c99a
phonon_refiner.json	marks a Phonon refiner: prompt format, writing-mode lines, recipe, eval numbers	3,389	badf2c9c68a3fa3ceed32ba61728ee7a
phonon_engine.json	which GGUF files the engine loads	136	3d91848deb89562f6e8abfcb63081176
LICENSE, NOTICE	LFM Open License v1.0 and the change notice

Hugging Face shows each file's SHA-256 on its file page. The GGUF target and the drafter run on Phonon's native Metal engine (phonon_engine.json). The MLX weights (model.safetensors, config.json and the tokenizer files) are the same refiner, which the app loads when the engine cannot run on a Mac.

Writing modes

Phonon picks a mode from the app the text is going to (for a browser, from the site) and sends it as one line of the system prompt:

Mode	Where Phonon uses it	Line
minimal	documents, notes, and any app not listed below	Writing mode: minimal (notes and documents).
casual	personal chat apps and sites: Discord, Messages, WhatsApp, Telegram, X	Writing mode: casual (a chat message).
formal	email apps and webmail (Mail, Outlook, Gmail, Superhuman) and work chat (Slack, Teams, Google Chat)	Writing mode: formal (an email or document).
prompt	AI assistants (ChatGPT, Claude, Gemini, Codex), terminals and code editors	Writing mode: prompt (an instruction to an AI assistant).

The rules in every mode:

• Fix misrecognized words and terms only with evidence: the vocabulary line or unambiguous context.
• Remove filler: hesitations, stutters, repeats, meaningless "like", "you know" and "I mean", checks to the listener in the middle of speech, and sentence-opening linking words.
• Keep what only looks like filler: a confirmation tag that turns a statement into a question ("the meeting is at ten, right?"), "now what?"-style questions, and a "Yes", "No" or "Yeah" that answers someone.
• Keep only the speaker's final version of a false start or a self-correction, without the correction words.
• Write digits and symbols the way an engineer types them.
• End a spoken question with "?", whatever end mark the transcript has.
• Never invent a word and never drop meaning.

Each mode then adds its own:

• minimal: changes nothing else; explicitly dictated ordered steps become a numbered list.
• casual: keeps the speaker's informal words, writes sentences in lowercase except names, acronyms and "I", and leaves off the final period. A question keeps its "?". One paragraph, no lists.
• formal: writes complete sentences and fixes grammar slips without rephrasing. Dictated ordered steps become a numbered list, and a dictated greeting and sign-off go on their own lines.
• prompt: keeps a revised or restated request once, in its final version. Separate steps become a numbered list. It never answers or extends the request.

Line breaks come from the model: numbered lists put one item per line, and the formal greeting and sign-off get their own lines. Phonon 1.0.1 also runs a fixed layout rule after the model for the same two cases (lists in every mode but casual; greeting and sign-off only for email apps), so a list left on one line is still broken up.

Without a mode line the model behaves like the 0.2.3 refiner, because over a quarter of its training rows are that model's rows without a line.

Training data

Two stages, both on public or synthetic text only.

Stage 1, supervised (132,228 rows).

• Writing-mode rows (93,748): targets in all four modes written by DeepSeek-V4.1-Flash-1017 (MIT license) from a written specification of the intended text (the rules above), over DeepSeek's hosted API and over OpenRouter with the provider pinned to DeepSeek. Every target went through an edit gate that undoes word substitutions the transcript cannot support. The inputs are Parakeet TDT 0.6B v2 transcripts, or text shaped like them, of:

  • chunks of public technical audio (talks, tutorials, courses) and of public programming livestreams, and synthetic dictations written by DeepSeek, such as chat messages, emails and notes (43,716);
  • spoken requests to AI coding assistants, cut from public programming livestreams (20,729);
  • synthetic turns of a developer talking to a coding agent (20,918), written by Qwen3.8-27B around commits from permissively licensed repositories (CommitPackFT), then passed through a model of Parakeet's errors;
  • synthetic layout dictations written by DeepSeek (8,385): ordered steps, emails with a spoken greeting and sign-off, and narration that says "first" or "then" without being a list. Their topics are disjoint from the layout probes below.

  5,661 of these rows are copies with the input's punctuation and casing removed, so the model also sees transcripts with no end marks. Since 1.0 the targets were revised in three ways: 5,797 targets were relabeled so that answer words and confirmation tags stay, 146 dictated-step rows in minimal and formal mode were relabeled as lists, and the line breaks the labeler wrote were restored into the targets (1.0's targets had been joined with single spaces; words never changed in that step).

• Rows of the 0.2.3 refiner (38,480, no mode line): its teacher-labeled chunks of public YouTube technical talks, tutorials and courses, plus term-repair rows and hard negatives. A term-repair row has a corrupted technical term in the input and a vocabulary line listing the true term; a hard negative has an unrelated vocabulary and must not change.

Stage 2, reinforcement learning for question marks (7,369 prompts). Short and full-length pieces of the stage-1 public rows: 2,114 questions whose "?" was replaced with "." or removed, and 3,999 non-questions; 210 public sentences that hold a question word but are not questions ("I don't know why", "let me know when"); and 523 pairs written by DeepSeek-V4.1-Flash-1017 of a non-question (exclamation, indirect question, request) and its direct-question twin. Rewards are scripted, with no judge model: a question must end with "?" and a non-question must gain none, and the words must stay close to what the stage-1 model itself writes for that input (its own greedy output, with the "?" set on questions). That keeps this stage to the end marks: earlier runs that scored the words against the stage-1 labels instead pulled the model back toward the labels' habits and lost quality.

• Never used: the Phonon author's dictations, any user data, and the evaluation sets below. Videos used for evaluation were held out by video id. The question-mark probe and Jim Faster's test (below) were removed from the stage-2 data, along with any generated pair whose words overlapped one of their items by half or more.
• Leak check: 3,036 evaluation texts were checked against both stages: the author's dictations, the bench sets, the lecture captions and the self-correction probe. No training input or target equals one of them, or contains half or more of one's 8-word phrases. 239 row sides share at least one 8-word phrase with one of them; those phrases are generic, for example "what we're going to do is we're going".
• No audio is redistributed with these weights.

Recipe:

• Stage 1: three runs of a rank-16 LoRA on all linear layers of LFM3-1.7B (learning rate 2e-4, 2 epochs, batch 16, up to 1,024 tokens, batches grouped by length), identical except for the seed. Their LoRA weight changes were averaged and merged into the weights.
• Stage 2: GRPO (TRL) with a fresh rank-16 LoRA on all linear layers of the stage-1 model, which is also the KL reference (beta 0.04): 8 samples per prompt at temperature 0.9, learning rate 2e-5, one epoch (920 steps), merged into the weights.
• Then quantized two ways, exactly as in 1.0:
  • Engine target: GGUF Q4_K_S with a 4-bit token embedding and output head, and an importance matrix from the public training rows.
  • Fallback: MLX 4-bit (group 64) with 6-bit attention and 8-bit embeddings, about 5.2 bits per weight.

Prompt format

Use the format the model was trained on, or quality drops:

• System: Rewrite the raw dictation transcript into the exact text the speaker intended. Fix recognition errors and technical terms, keep the speaker's wording and casing style, do not add or drop content.
  • Then one writing-mode line from the table above.
  • Then, optionally, Vocabulary the speaker may use: term1, term2, ..., the user's dictionary terms that may appear. Phonon sends 30 to 80, most confident first.
• User: the bare transcript.
• Decoding: greedy, thinking off (enable_thinking: false in the chat template), and no repetition penalty. The GGUF carries the base model's sampling defaults (temp 0.1, top_k 50), so pass --temp 0 to llama.cpp.
• Cap the output at 1.5 x input tokens + 32. Phonon also falls back to the raw transcript when the output has more than twice the input's words or, past 8 words, fewer than 0.4 of them.

Evaluation

All numbers below come from the shipped Q4 GGUF unless a section says otherwise.

Real dictations against the 1.0 refiner (one speaker)

2,522 items from the Phonon author's own dictations: the 608 dictations of the Wispr Flow comparison below, 1,055 more from his dictation history in the mode Phonon picked for them, and 859 of these rendered in another mode (minimal, casual and formal). A local blind judge (Gemma 4 31B, both orders, a verdict only when both agree) read each pair against the written specification above, with a second speech recognizer's transcript as evidence. 1,200 outputs were identical to the 1.0 refiner's and count as ties.

Judge	Won	Tied	Lost	Won on meaning	Tied	Lost
spec as above	575	1,695	248	156	2,264	98
same, a dictation-opening "Yes" or "So" never counts against cleanup	637	1,655	229	178	2,230	113

Ties include the identical outputs. 4 and 1 pairs got no usable verdict and are left out. "Meaning" is the judge's separate verdict on whether the text keeps what the speaker meant. Between two training runs of the same recipe that differ only in seed, the overall margin (won minus lost) spreads by about 35, so read small margins as noise. These dictations stay private; only the totals are published.

Real dictations against Wispr Flow (one speaker)

On 608 of the Phonon author's dictations, the same local judge, reading the earlier specification (before answer words and tags were kept), compared this model's text with Wispr Flow's, each in the mode Phonon would pick for the app it was dictated into (552 in prompt mode).

	Won	Tied	Lost	Win rate
This model vs Wispr Flow	251	212	145	0.59
1.0 refiner vs Wispr Flow	235	209	164	0.56

The win rate counts a tie as half.

Question marks

A hand-written probe of 40 questions and 40 non-questions (indirect questions, exclamations, statements that start with a question word), each in two forms: as Parakeet writes it (cased, a question ending in ".") and raw (lowercase, no punctuation). The table counts questions that came out with "?", per mode and form, out of 40. Both models ran on llama.cpp on the CPU with the same flags.

Mode, form	This model	1.0 refiner
minimal, Parakeet	40	19
minimal, raw	40	36
casual, Parakeet	40	0
casual, raw	40	0
formal, Parakeet	39	38
formal, raw	38	39
prompt, Parakeet	40	34
prompt, raw	40	37
non-questions given a "?" (of 320)	0	0

The 1.0 refiner's casual mode dropped the "?" together with the final period. The probe was written for this check, so it is small and narrow; it measures the end mark, not the rest of the text.

Lists and email layout

Public probes written by DeepSeek and shaped like Parakeet output: 95 dictations of 3 to 5 ordered steps and 34 dictated emails with a spoken greeting and sign-off, each also in raw form, run with no layout rule after the model. The 1.0 refiner was trained on targets with no line breaks and writes none.

Probe, Parakeet / raw form	minimal	casual	formal	prompt
steps written as a numbered list, one per line (of 95)	83 / 90	0 / 0	85 / 91	95 / 95
email greeting on its own line (of 34)	0 / 0	0 / 0	34 / 34	1 / 3
email sign-off on its own line (of 34)	0 / 0	0 / 0	33 / 33	6 / 25

Only formal mode is asked to lay out an email; in prompt mode the model still put the sign-off on its own line in some outputs, mostly for raw input.

Breaks where none were asked for: on 1,183 held-out public livestream transcripts the model broke lines in 0 of 301 minimal, 0 of 297 casual, 23 of 285 prompt and 8 of 300 formal outputs. Every break was read: greetings, a dictated list, and paragraph breaks at a new request.

An outside tester's 57 cases

Jim Faster wrote a 57-case test of the public refiner and sent it to us (thank you). It runs the refiner in minimal mode with the prompt above and scores each output against what the case expects. This model passes 55 of 57; the 1.0 refiner passed 41 in his run and 40 when we reran it with the same scorer as this model. Questions with no "?" in the input now get one in 12 of 12 cases (1.0: 3), and inputs with no end mark at all get one in 20 of 22 (1.0: 4). The two misses are one wrong capital left by a pause and one small word that carries meaning.

Training seeds

The supervised stage is the average of three runs that differ only in seed, so no single seed is shipped. The question-mark stage ran once on that average (seed 1). Against the average alone it changed little else: on the 2,522 dictations above, 1,740 outputs are identical to the average's, and the two judges put this model +4 and +30 ahead, inside the seed spread of about 35. The question-mark table and Jim Faster's test ran on llama.cpp on the CPU; the judged comparisons ran on a llama.cpp server on a GPU.

The fallback weights

The MLX fallback, run through Phonon's correction sidecar on an M4 Max, gave the same text as the Q4 GGUF on llama.cpp on 64 of 64 probe requests (16 questions in four modes). That is a smoke test, not a full comparison; the 1.0 fallback matched its GGUF on 296 of 384 requests.

Measured on the 1.0 refiner, not re-measured for this version

These numbers are from the 1.0 refiner (commit 2b0ab74). This version shares its public sources and adds data on top, but these tests were not rerun, so treat them as context, not as this model's results.

• Three lecturers against their own captions (120 segments of 30 s; Jeremy Howard, fast.ai 2022, CC BY; Ana Bell, MIT 6.100L; Charles Leiserson, MIT 6.172; f16 weights): words differing from the captions 8.51% [7.48, 9.57], against 7.03% for 0.2.3 and 10.89% for Wispr Flow with the same dictionary. A blind pairwise judge (Claude Opus 5.5, both orders) preferred the 1.0 refiner over Wispr Flow on 90 of 119 segments, 6 even, 23 for Wispr Flow (0.78).
• Self-corrections: 96 synthetic dictations, each revising one detail ("tuesday, actually no, thursday"); the 1.0 refiner resolved 0.56 / 0.52 / 0.58 / 0.48 (minimal / casual / formal / prompt), 0.2.3 none.
• Word error rate on one speaker's dictations against references derived from Wispr Flow: 0.1164 and 0.1058 on two sets (580 and 606 rows), a tie with 0.2.3 within the 95% intervals.

Caveats, read these before quoting the numbers:

• The real dictations are from one speaker, mostly technical. They show how the model does on that workload, not on English speech in general. They stay private.
• Judges are models. The judges read the same written spec the labeler was given, so they reward following that spec, and the spec changed between 1.0 and this version (answer words and tags now stay).
• The question-mark probe and the stage-2 rewards were built around the same failure, and the layout probes and the synthetic layout rows around another, so those probes show the fixes worked, not how often real dictations need them.

Limitations

• "now what?" questions: the specification keeps them whole, but most training labels still cut them, so the model often does too.
• Greeting lines in work chat: formal mode puts a dictated greeting ("Hi Sam,") and sign-off on their own lines. Phonon also uses formal mode for Slack, Teams and Google Chat, where that layout can look out of place.
• Self-corrections: the 1.0 refiner resolved about half of them; not re-measured for this version.
• It deletes more than 0.2.3 on lecture-style speech (measured on 1.0). Some of that is repeated or abandoned phrasing, some is content.
• English only. Dictation-length inputs, up to about a minute of speech.
• It is a cleanup model, not an assistant. In prompt mode it rewrites the request and never answers it. Check its output before relying on it for medical, legal or other high-stakes text.

Intended use

Post-processing of speech-recognition output for dictation, run locally. Phonon downloads this repository pinned to a commit and runs it without network access afterwards.

Running it

• In Phonon: nothing to do. The app downloads this repository on first launch and picks the mode from the target app.
• llama.cpp: Phonon runs the GGUF through llama.cpp 39febd2 with Phonon's patch. Upstream builds with LFM3 support (since PR #31252, 2026-10-27) should load it, but that is untested. Use the chat template in the file, --temp 0, and the prompt format above.
• MLX: mlx-lm 0.31.3 has no LFM3 model. Add the model definition from mlx-lm PR 2016; Phonon ships it as sidecar/models/lfm3.py.
• The drafter (phonon-refiner-1.7b-dflash2-drafter.gguf) is the 0.2.3 release's DFlash2 block drafter: 3 layers, FFN 1024, a 24,576-token draft vocabulary, 7 drafted tokens per step, trained on the 0.2.3 refiner's outputs on public transcripts. With greedy verification the output is the target's own; the drafter changes speed, not text. It was trained on an older model's outputs, so it accepts fewer drafted tokens with this one; on Phonon 0.2.4's engine (M4 Max, 384 requests) the refine step took 73 ms median for the 1.0 refiner against 69 ms for 0.2.3. Not re-measured for this version, whose weights have the same size and layout.
  • Its GGUF layout is read by Phonon's engine (pe, kind dflash2n) and by llama.cpp with Phonon's patch (--spec-type draft-dflash), not by stock llama.cpp.

License

The weights are a derivative of LFM3-1.7B and are distributed under the LFM Open License v1.0, as the base model is. In particular:

• Commercial use is licensed only to organizations with under 10 million US dollars in annual revenue (section 5). A larger organization's commercial use is not licensed by it.
• Redistributions must include the license and keep the notice that these files were modified (see NOTICE).
• "Liquid AI" and "LFM" are Liquid AI's names, used here only to describe the model's origin.

Phonon itself is GPL-3.0. Parakeet TDT 0.6B v2 is NVIDIA's, under CC-BY-4.0.