|
Download README.md from Lucasgitguy-co/Fleet-2: direct link, hf CLI and curl.
- Browser
- Download file 6.28 kB
-
https://huggingface.co/Lucasgitguy-co/Fleet-2/resolve/main/README.md
- Command line
-
hf download hf://Lucasgitguy-co/Fleet-2/README.md
-
curl -L -o README.md https://huggingface.co/Lucasgitguy-co/Fleet-2/resolve/main/README.md
6.28 kB
| license: apache-2.0 | |
| base_model: Lucasgitguy-co/Fleet-1 | |
| language: [en, de, fr, es, it, nl, pl, ko] | |
| pipeline_tag: automatic-speech-recognition | |
| datasets: [facebook/voxpopuli, Bingsu/zeroth-korean, google/fleurs] | |
| tags: [whistle, cactus, lora, speech-to-text, distillation, pseudo-labels] | |
| # Fleet-2 | |
| A 30 MB on-device speech-to-text model (Cactus Whistle architecture) for English, German, French, Spanish, Italian, Dutch, Polish and Korean, trained on transcripts written by [openai/whisper-large-v3-turbo](https://huggingface.co/openai/whisper-large-v3-turbo). | |
| **Collection:** [Speech recognition](https://huggingface.co/collections/Lucasgitguy-co/speech-recognition-6aef2dcbe0ea208136b0ddc9) | |
| **Clean text:** [Fleet-Clean-0.36B](https://huggingface.co/Lucasgitguy-co/Fleet-Clean-0.36B) chains Fleet-2 with a word tagger that removes fillers, repeats and false starts and fixes punctuation ([GGUF](https://huggingface.co/Lucasgitguy-co/Fleet-Clean-0.36B-GGUF)). | |
| ## What changed from Fleet-1 | |
| - **Teacher:** Whisper-large-v3-turbo transcribed new audio, and Fleet-2 learned from those transcripts. This is sequence-level distillation, as in Distil-Whisper. Whisper and Whistle use different tokenizers, so no logits were matched. | |
| - **New audio:** VoxPopuli (European Parliament speech) for the seven European languages and Zeroth-Korean for Korean. Fleet-1 used FLEURS only. | |
| - **Start point:** chosen by validation score between Sky-Kim's Korean Whistle and Fleet-1. This run started from `Lucasgitguy-co/Fleet-1`. | |
| ## Evaluation | |
| Each language has 100 short FLEURS test clips. These are the same clips, filtered the same way, as Fleet-1's table. All models ran in the same native Cactus engine with automatic language detection. Lower is better. Whistle has no Korean. | |
| | Language | Metric | Whistle | Sky-Kim | Fleet-1 | Fleet-2 | | |
| |---|---|---:|---:|---:|---:| | |
| | en | WER | 13.86% | 14.87% | 14.39% | **14.50%** | | |
| | de | WER | 22.45% | 22.71% | 22.18% | **22.02%** | | |
| | fr | WER | 22.15% | 25.73% | 25.18% | **24.93%** | | |
| | es | WER | 11.83% | 11.83% | 11.93% | **11.93%** | | |
| | it | WER | 17.44% | 16.60% | 16.32% | **16.66%** | | |
| | nl | WER | 32.95% | 29.39% | 29.53% | **30.06%** | | |
| | pl | WER | 45.38% | 43.68% | 43.13% | **41.62%** | | |
| | ko | CER | n/a | 33.68% | 52.35% | **42.80%** | | |
| Fleet-2 beats Fleet-1 in 4 of 8 languages (de, fr, pl, ko). Korean CER is no worse than Sky-Kim: no. **Improvement over Fleet-1 was NOT demonstrated. Check each language; some may regress.** | |
| <details> | |
| <summary>Out-of-domain clips (VoxPopuli / Zeroth-Korean test splits)</summary> | |
| | Language | Source | Clips | Whistle | Sky-Kim | Fleet-1 | Fleet-2 | | |
| |---|---|---:|---:|---:|---:|---:| | |
| | en | VoxPopuli | 50 | 10.56% | 12.69% | 10.91% | **11.98%** | | |
| | de | VoxPopuli | 50 | 17.32% | 19.66% | 19.53% | **18.36%** | | |
| | fr | VoxPopuli | 50 | 13.06% | 12.24% | 12.04% | **13.57%** | | |
| | es | VoxPopuli | 50 | 9.99% | 11.89% | 11.70% | **11.04%** | | |
| | it | VoxPopuli | 50 | 34.45% | 40.21% | 38.22% | **33.19%** | | |
| | nl | VoxPopuli | 50 | 23.06% | 23.29% | 23.17% | **22.95%** | | |
| | pl | VoxPopuli | 50 | 22.02% | 23.53% | 22.98% | **23.53%** | | |
| | ko | Zeroth-Korean | 50 | n/a | 25.19% | 15.65% | **13.24%** | | |
| Fleet-2 trained on other clips from these corpora, so this table shows in-corpus generalisation, not a fully independent test. | |
| </details> | |
| These results come from short clips of 12 s or less. They are not full benchmark scores and not significance tests. Accuracy on long, noisy or conversational audio was not measured. Upstream models may have seen FLEURS training data before. | |
| ## Training | |
| - **Data:** 4326 clips kept out of 6400 that the teacher labeled: 3014 new VoxPopuli/Zeroth clips and 1312 of Fleet-1's FLEURS training clips. All clips are 12 s or less. | |
| - **Teacher decoding:** language forced, temperature 0, no conditioning on previous text. Runtime: the MLX conversion `mlx-community/whisper-large-v3-turbo`. | |
| - **Filter:** a clip was kept only if all of these held: | |
| - compression ratio ≤ 2.4 | |
| - average log-probability ≥ -1.0 | |
| - no-speech probability ≤ 0.6 | |
| - teacher text within 10% WER of the human transcript (CER for Korean) | |
| The training target is the teacher's text. | |
| - **Excluded from training:** validation and test clips, and any transcript that matches an evaluation transcript. | |
| - **Optimisation:** the start adapter (LoRA plus Korean vocabulary rows) continued with AdamW at 2e-05, warmup/cosine schedule, batch 1, 2 epochs (8652 steps), clip 1.0, W4/A8 quantisation-aware training on CPU (Apple M4). | |
| - **Checkpoint selection:** step 2163, chosen by macro validation error on Fleet-1's FLEURS validation clips (0.2212, against 0.2283 for the start model). Test clips were not used for selection. | |
| - **Release:** published whatever the scores. The comparison above is reported as measured, including any regressions. | |
| `training_manifest.json` records every clip, teacher transcript and pinned revision. `evaluation.json` has every score. `fleet2.py`, `teacher.py` and `run.sh` reproduce the run. | |
| ## Usage | |
| ```python | |
| import cactus | |
| from huggingface_hub import hf_hub_download | |
| path = hf_hub_download("Lucasgitguy-co/Fleet-2", "whistle.cact") | |
| print(cactus.transcribe("clip.wav", weights=path)["text"]) | |
| ``` | |
| Use 16 kHz mono WAV and `cactus-compute` at revision `ee2ffc67fe0f9537d4928e158440c7d228a63100`. This is a Cactus model, not a Transformers checkpoint. | |
| ## License and attribution | |
| - **Model:** Apache License 2.0. The original Whistle LICENSE is kept verbatim. Architecture and pretrained weights: Cactus Compute's Whistle (Mroz et al., 2026). Korean vocabulary and adapter lineage: [Sky-Kim/whistle-finetuned-ko](https://huggingface.co/Sky-Kim/whistle-finetuned-ko) and [Lucasgitguy-co/Fleet-1](https://huggingface.co/Lucasgitguy-co/Fleet-1). See `UPSTREAM_MODEL_CARD.md`. | |
| - **Teacher:** [openai/whisper-large-v3-turbo](https://huggingface.co/openai/whisper-large-v3-turbo), MIT License. Only its transcripts were used; none of its weights are included. | |
| - **Data:** | |
| - [VoxPopuli](https://huggingface.co/datasets/facebook/voxpopuli): CC0 | |
| - [Zeroth-Korean](https://www.openslr.org/40/): CC BY 4.0 | |
| - [FLEURS](https://huggingface.co/datasets/google/fleurs) (Conneau et al., 2022): CC BY 4.0 | |
| No audio is included in this repository. See `NOTICE`. | |