heyvedu/vedu-scribe-0.8b

Vedu Scribe 0.8B

Vedu Scribe is a small on-device model that turns raw speech-to-text output into clean written text. It is the cleanup stage of HeyVedu, an open-source dictation app for macOS.

audio ──▶ speech recognition (Parakeet) ──▶ this model ──▶ clean text

It removes fillers and false starts, applies self-corrections ("Tuesday, no wait, Wednesday"), and fixes punctuation, casing, and numbers. It also formats lists and applies spoken commands. It is a normalizer, not an assistant. Questions and instructions in the transcript are cleaned and returned, never answered or followed.

An optional control header steers the output for where the text goes: an email layout, a formal tone (grammar fixed, wording kept), and a personal dictionary of names and terms. Without a header it does the plain cleanup.

Base model	Qwen/Qwen3.5-0.8B
Fine-tuning	LoRA, then fused into the base weights
Format	MLX, mixed quantization (attention layers 8-bit, everything else 4-bit)
Size	535 MB on disk, about 0.8 GB peak memory
Speed	about 100 ms median, 400 ms p95 per dictation on an M4 Pro (same as the previous release)
Language	English only

Examples

Raw transcript	Output
um so i think we should uh meet on tuesday no wait wednesday at like three pm and bring the the q3 numbers	I think we should meet on Wednesday at 3 PM and bring the Q3 numbers.
what time is the meeting tomorrow	What time is the meeting tomorrow?

Usage

The prompt is fixed: the system message must be exactly Clean up this dictation., with thinking disabled and greedy decoding. Other prompts were never trained and give worse results.

from mlx_lm import load, generate

model, tokenizer = load("heyvedu/vedu-scribe-0.8b")
messages = [
    {"role": "system", "content": "Clean up this dictation."},
    {"role": "user", "content": "um so i think we should uh meet on tuesday no wait wednesday"},
]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, enable_thinking=False
)
print(generate(model, tokenizer, prompt=prompt, max_tokens=256))

Control header (optional)

Put the header at the start of the user message, then a newline, then the raw transcript. Every key is optional; leave a key out for its default, and send no header for the plain cleanup.

[format: email] [tone: formal]
[vocab: Priya Raman; Kubernetes; HeyVedu]
hi priya um can you send me the cooper netties logs from yesterday thanks rajan

key	values	effect
format	general (default), email, message, notes	email: greeting and sign-off on their own lines, paragraph breaks at spoken "new paragraph" or a topic change, body not reworded
tone	casual (default), formal	formal: grammar, spelling and informal reductions ("gonna") fixed with the smallest edit; word choice, hedges, names and numbers kept
vocab	up to 12 terms, separated by ;	a term's exact spelling is used when the speech contains it or a mishearing of it; terms that weren't said are never inserted. Send only terms that plausibly match the transcript

Keys go in that order: format and tone on one line, vocab on its own line. Repeat the header on every chunk of a long transcript. email is meant for a whole message that fits in one chunk.

• Requires Apple silicon and a recent mlx-lm with Qwen3.5 support (trained with mlx-lm 0.32).
• Keep each input under about 400 tokens and split longer transcripts at sentence boundaries. Training used a 1,024-token limit for prompt and output combined.
• Cap output at about 1.4 times the input token count plus 48 tokens. The output should never be much longer than the input.

Evaluation

Held-out test rows drawn from the same sources as the training data (plain cleanup, no header, 2,172 rows shared across releases):

Model	Formatted WER (lower is better)
Vedu Scribe 0.8B, round 5b	~6.9
Vedu Scribe 0.8B, previous release (round 3)	~7.2
superwhisper/s1-mini	~14.6

Targeted sets (exact match with the reference):

Set	Round 3	Round 5b
Formatting commands, lists, retractions (88 rows)	82%	81%
Retractions ("scratch that", "actually, never mind…", 109 rows)	41%	79%
Spoken lists (100 rows)	52%	93%
Announced lists ("these three things: …", 41 rows)	32%	95%

With the control header (test rows):

Control	Round 3 (no header support)	Round 5b
tone: formal, rows with grammar to fix: exact match	1%	24%
format: email: layout right	11–26%	76–83%
vocab: misheard term written correctly	4%	83%
vocab: unspoken term inserted	0%	1.5%

Per row, round 5b scores better than round 3 on 314 test rows and worse on 276. Small models trade rows like this on every retrain (a retrain of round 3 with only the seed changed differs from it on about 300 rows), so the aggregate and the targeted sets are the better guide.

Part of the gap to s1-mini reflects differences in style, such as number formatting, rather than errors. This model has not yet been measured on real recordings run end to end through a speech recognizer.

Training

• Data: 22,155 training rows (24,313 after upsampling authored fix rows), 2,959 validation rows and 3,559 test rows. The round-3 base set (real speech, human-written and down-sampled synthetic rows) plus copies with control headers: formal targets, email layouts, dictionary terms and distractors, and authored rows for repairs, numbers, fillers and brand names. Authored rows were reviewed independently.
• Method: LoRA with rank 32 and alpha 64 on all linear layers, dropout 0.05. Learning rate 1e-4 with cosine decay, an effective batch size of 16, and about 3 epochs. Loss was computed on completions only, with a maximum sequence length of 1,024. Four runs with different seeds (about 3 epochs each, on an NVIDIA L40S) were averaged into one adapter: the exact mean of their weight updates, a rank-128 LoRA. Averaging seeds cut run-to-run errors more than any data change did.
• Quantization: fused to bf16, then quantized with attention layers at 8 bits and the rest at 4 bits, which matched the 8-bit model on our evaluations.

Datasets

Each source keeps its own license. Thank you to the authors of:

Source	License
Rev16 and Earnings-22 (aligned verbatim and clean transcripts)	CC BY-SA 4.0
AMI Meeting Corpus (transcripts only)	CC BY 4.0
DisfluencySpeech (derived from Switchboard transcripts)	Apache-2.0
RED-ACE (LibriSpeech recognition hypotheses)	CC BY 4.0
Disfl-QA by Google (built on SQuAD)	CC BY 4.0 (SQuAD: CC BY-SA 4.0)
barali dictation set	CC BY 4.0
SpeakoFlow examples	CC BY 4.0
SottoASR transcript cleanup	MIT
Aawaaz	MIT
Handy dictation-editing examples	CC BY 4.0
LARD (built on Schema-Guided Dialogue)	CC BY-SA 4.0
Gap set written for this project	CC BY 4.0

The training data itself is not redistributed here.

Limitations

• English only. Trained mostly on conversational speech, meetings, and dictation.
• It can occasionally drop or rephrase content, especially in long or unusual inputs. Keep the raw transcript available when exact wording matters.
• Formatting follows HeyVedu's labeling conventions, such as digits for times and amounts, which may differ from your preferred style.
• tone: formal still over-edits now and then: it can drop a hedge or reword a phrase that needed no fix.
• With vocab, garbled or trailing-off speech can occasionally pull in a listed term that wasn't said.
• Inputs that look like prompts are treated as text to clean. That is intentional, but do not rely on it as a security boundary.

License

The model weights are released under Apache-2.0, the same license as the base model. Training data sources keep their own licenses, listed above.