Automatic Speech Recognition
Laya
Safetensors
speech-to-text
disfluency-removal
punctuation
token-classification
whistle
cactus
mmbert
Instructions to use Lucasgitguy-co/Fleet-Clean-0.36B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Laya
How to use Lucasgitguy-co/Fleet-Clean-0.36B with Laya:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
|
Download README.md from Lucasgitguy-co/Fleet-Clean-0.36B: direct link, hf CLI and curl.
- Browser
- Download file 7.53 kB
-
https://huggingface.co/Lucasgitguy-co/Fleet-Clean-0.36B/resolve/main/README.md
- Command line
-
hf download hf://Lucasgitguy-co/Fleet-Clean-0.36B/README.md
-
curl -L -o README.md https://huggingface.co/Lucasgitguy-co/Fleet-Clean-0.36B/resolve/main/README.md
7.53 kB
| license: apache-2.0 | |
| language: [en, de, fr, es, it, nl, pl, ko] | |
| pipeline_tag: automatic-speech-recognition | |
| base_model: | |
| - Lucasgitguy-co/Fleet-2 | |
| - convaiinnovations/laya-multilingual | |
| datasets: | |
| - Lucasgitguy-co/fleet-clean-tags | |
| - amaai-lab/DisfluencySpeech | |
| - facebook/voxpopuli | |
| - Bingsu/zeroth-korean | |
| - google/fleurs | |
| tags: [speech-to-text, disfluency-removal, punctuation, token-classification, whistle, cactus, mmbert, laya] | |
| # Fleet-Clean-0.36B | |
| Speech in, clean text out. It is two open models with 363M parameters in total: | |
| 1. **[Fleet-2](https://huggingface.co/Lucasgitguy-co/Fleet-2)** (56M, Cactus Whistle) writes down what was said. | |
| 2. **A word tagger** (307M) cleans that transcript. It is the encoder of [convaiinnovations/laya-multilingual](https://huggingface.co/convaiinnovations/laya-multilingual) (mmBERT) with a new head that decides, for every word: | |
| - keep or delete it (fillers, discourse markers, repeats, false starts) | |
| - which mark follows it (none , . ? !) | |
| - whether its first letter changes case | |
| The tagger only removes Fleet-2's words and fixes punctuation and capitals. **It never writes a word of its own**, so it can't invent content or answer what was dictated. | |
| **GGUF version:** [Fleet-Clean-0.36B-GGUF](https://huggingface.co/Lucasgitguy-co/Fleet-Clean-0.36B-GGUF) · **Training data:** [fleet-clean-tags](https://huggingface.co/datasets/Lucasgitguy-co/fleet-clean-tags) · **Collection:** [Speech recognition](https://huggingface.co/collections/Lucasgitguy-co/speech-recognition-6aef2dcbe0ea208136b0ddc9) | |
| ## Usage | |
| ```bash | |
| pip install cactus-compute torch "transformers>=5" safetensors | |
| hf download Lucasgitguy-co/Fleet-Clean-0.36B --local-dir fleet-clean && cd fleet-clean | |
| python fleet_clean.py clip.wav # clean transcript | |
| python fleet_clean.py --json clip.wav # raw, clean and the per-word tags | |
| python fleet_clean.py --text "so I I think, you know, we should go" | |
| python fleet_clean.py --punct full clip.wav # also re-punctuate words away from deletions | |
| ``` | |
| Audio is 16 kHz WAV (`ffmpeg -i in.m4a -ar 16000 -ac 1 clip.wav`). Fleet-2 detects the language, or you can pass `--language`. The delete threshold is 0.5. It was calibrated on validation data so that fluent speech keeps its words, and `--threshold` changes it. | |
| ## Examples | |
| These are from the DisfluencySpeech test set (English): | |
| - **Fleet-2:** Yeah. Well, robots are darling. That would be fun if you could get them trained. Otherwise, they're pretty smelly. | |
| **Fleet-Clean:** Yeah, robots are darling. That would be fun if you could get them trained. Otherwise, they're pretty smelly. | |
| - **Fleet-2:** Well, McNew, New, and Lara, I guess it's leaving one of them is a leaving. | |
| **Fleet-Clean:** McNew, New, and Lara, I guess one of them is a leaving. | |
| - **Fleet-2:** Well, I don't want to. I don't want this to be a unilateral conversation, but yes, we do. | |
| **Fleet-Clean:** I don't want to. I don't want this to be a unilateral conversation, but yes, we do. | |
| ## Evaluation | |
| **Real disfluent English speech.** Fleet-2 transcribes the audio, and the output is compared with DisfluencySpeech's own cleaned transcript: | |
| | DisfluencySpeech test (250 clips) | Fleet-2 alone | Fleet-Clean | | |
| |---|---:|---:| | |
| | WER against the cleaned transcript (`transcript_c`), punctuation ignored | 18.8% | **14.6%** | | |
| | Punctuation F1 against `transcript_c` | **0.6700** | 0.6695 | | |
| **Fluent speech (FLEURS test).** These are the same 100 clips per language as Fleet-2's card. A cleanup model should leave fluent speech alone: | |
| | Language | Words deleted from fluent speech | WER before | WER after | | |
| |---|---:|---:|---:| | |
| | English | 0.47% | 14.5% | 14.3% | | |
| | German | 0.05% | 22.0% | 22.0% | | |
| | French | 0.04% | 24.9% | 25.0% | | |
| | Spanish | 0.05% | 11.9% | 11.9% | | |
| | Italian | 0.06% | 16.7% | 16.7% | | |
| | Dutch | 0.19% | 30.1% | 30.0% | | |
| | Polish | 0.29% | 41.6% | 41.6% | | |
| | Korean | 0.27% | 42.8% (CER) | 42.9% (CER) | | |
| <details> | |
| <summary>Synthetic disfluencies (held-out tag test rows)</summary> | |
| Fillers, discourse markers, repeats, restarts and self-corrections were inserted into Fleet-2 transcripts of held-out VoxPopuli / Zeroth / FLEURS clips. The table shows whether the tagger finds exactly the inserted words. "Weak" means F1 below 0.80. | |
| | Language | Delete F1 | Precision | Recall | | |
| |---|---:|---:|---:| | |
| | English | 0.981 | 0.967 | 0.996 | | |
| | German | 0.984 | 0.968 | 1.000 | | |
| | French | 0.970 | 0.946 | 0.995 | | |
| | Spanish | 0.971 | 0.978 | 0.964 | | |
| | Italian | 0.969 | 0.959 | 0.979 | | |
| | Dutch | 0.952 | 0.920 | 0.986 | | |
| | Polish | 0.990 | 0.980 | 1.000 | | |
| | Korean | 0.979 | 0.975 | 0.984 | | |
| </details> | |
| Limits: | |
| - **Real disfluent speech was only tested in English.** The other seven languages were trained and tested on synthetic disfluencies, so their numbers show how well the tagger removes those inserted patterns, not how it handles real conversation. | |
| - **Fleet-2 already leaves out most "uh/um".** It learned from Whisper transcripts. What is left to clean is mostly "you know", "I mean", "like", repeats and restarts. | |
| - **Recognition errors stay.** The tagger can only delete or re-punctuate words, so it can't fix a misheard word. | |
| - Clips are up to 30 s per call. The tagger reads up to 512 tokens and leaves anything past that unchanged. | |
| ## How it was built | |
| <details> | |
| <summary>Data, training and gates</summary> | |
| - **Transcripts:** Fleet-2 (`whistle.cact`, native Cactus engine, automatic language detection) transcribed: | |
| - DisfluencySpeech train/validation/test | |
| - VoxPopuli validation (7 languages, a split Fleet-2 never trained on) | |
| - Zeroth-Korean train minus Fleet-2's own training rows | |
| - FLEURS Korean train minus Fleet-2's rows | |
| - **Labels:** | |
| - **English deletions:** each Fleet-2 word is aligned to DisfluencySpeech's verbatim transcript, and that transcript's words are mapped to its cleaned version. | |
| - **Punctuation and case:** taken from the human reference wherever Fleet-2's word matches it. Zeroth has no punctuation, so its rows only teach deletion. | |
| - **Synthetic rows:** disfluencies are only ever inserted, so the clean text is always recoverable by deletion. | |
| - **Training:** | |
| - Laya-multilingual's encoder, with its 256k-token embedding table frozen, plus a new 768→10 head. | |
| - PyTorch on an Apple M4 (MPS), fp32. | |
| - Best validation checkpoint kept, then the delete threshold calibrated on validation only. | |
| - **Release gates:** | |
| - G1: cleaning lowers WER on real disfluent English, and punctuation F1 doesn't drop. WER dropped from 18.76% to 14.61%, but punctuation F1 went from 0.6700 to 0.6695 (−0.0005), so this part of the gate did not pass. It was published anyway by the owner's decision, and the number is reported as measured. | |
| - G2: at most 1.0% of words deleted from fluent FLEURS speech in every language. Passed in all 8. | |
| - **Punctuation mode:** by default, Fleet-2's own punctuation is kept, and the tagger's punctuation and case are used only next to removed words. This was chosen on DisfluencySpeech *validation*, where re-punctuating everything scored lower against the human clean transcripts. `--punct full` re-punctuates every word; on fluent validation speech in all 8 languages, full mode was slightly more accurate than Fleet-2's own marks. | |
| </details> | |
| ## License and attribution | |
| Apache-2.0. See `NOTICE` for every component: | |
| - Fleet-2 / Cactus Whistle (Apache-2.0) | |
| - Laya-multilingual (Apache-2.0), which builds on mmBERT (MIT) | |
| - DisfluencySpeech (Apache-2.0) | |
| - VoxPopuli (CC0) | |
| - Zeroth-Korean and FLEURS (CC BY 4.0) | |
| No audio is included. | |