Instructions to use Lucasgitguy-co/Fleet-Clean-0.36B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use Lucasgitguy-co/Fleet-Clean-0.36B-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible embeddings server: llama serve -hf Lucasgitguy-co/Fleet-Clean-0.36B-GGUF:F16 --embeddings
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible embeddings server: llama serve -hf Lucasgitguy-co/Fleet-Clean-0.36B-GGUF:F16 --embeddings
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible embeddings server: ./build/bin/llama serve -hf Lucasgitguy-co/Fleet-Clean-0.36B-GGUF:F16 --embeddings
- Llama new
- LM Studio
- Jan
- Ollama
How to use Lucasgitguy-co/Fleet-Clean-0.36B-GGUF with Ollama:
ollama run hf.co/Lucasgitguy-co/Fleet-Clean-0.36B-GGUF:F16
- Unsloth Desktop
- Docker Model Runner
How to use Lucasgitguy-co/Fleet-Clean-0.36B-GGUF with Docker Model Runner:
docker model run hf.co/Lucasgitguy-co/Fleet-Clean-0.36B-GGUF:F16
- Lemonade
How to use Lucasgitguy-co/Fleet-Clean-0.36B-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull Lucasgitguy-co/Fleet-Clean-0.36B-GGUF:F16
Run and chat with the model
lemonade run user.Fleet-Clean-0.36B-GGUF-F16
List all available models
lemonade list
- Atomic Chat
Fleet-Clean-0.36B-GGUF
GGUF files for Fleet-Clean-0.36B: speech in, clean text out, with no Cactus or PyTorch needed.
| File | What | Runs in |
|---|---|---|
whistle-fleet2-F16.gguf (recommended), -F32.gguf |
Fleet-2 speech-to-text (Cactus Whistle architecture) | whistle/whistle-cli (C++, in this repo) |
fleet-clean-tagger-Q8_0.gguf / -F16.gguf |
the cleanup tagger's encoder (modern-bert) |
llama.cpp llama-server --embeddings --pooling none |
fleet-clean-tagger-head.gguf |
the 768โ10 head and its labels, read by fleet_clean_gguf.py |
numpy |
Usage
hf download Lucasgitguy-co/Fleet-Clean-0.36B-GGUF --local-dir fleet-clean-gguf && cd fleet-clean-gguf
sh whistle/build.sh # whistle-cli: Accelerate on macOS, OpenBLAS or plain C++ elsewhere
pip install numpy gguf # plus llama.cpp's llama-server on PATH (a build with modern-bert support)
python fleet_clean_gguf.py clip.wav # 16 kHz WAV -> clean text
python fleet_clean_gguf.py --text "so, um, I I think we should, you know, go"
whistle/whistle-cli -m whistle-fleet2-F16.gguf -f clip.wav --json # speech only
Important: GGUF speech is less accurate than whistle.cact
No existing GGUF runtime supports Whistle, so whistle-cli is a new C++ port of Cactus's open JAX model. Cactus's own .cact files run in a closed native engine, and on the same weights that engine scores better:
| Language | Clips | whistle.cact (native engine) |
whistle-fleet2-F32/F16.gguf (open model) |
|---|---|---|---|
| English | 100 | 14.5% | 17.6% |
| German | 100 | 22.0% | 25.1% |
| French | 30 | 27.4% | 30.7% |
| Spanish | 30 | 11.3% | 12.5% |
| Italian | 30 | 18.6% | 19.3% |
| Dutch | 30 | 32.5% | 36.6% |
| Polish | 30 | 43.2% | 48.6% |
| Korean | 30 | 34.4% CER | 45.3% CER |
The weights in these GGUF files are exactly Fleet-2's: they were dequantised from whistle.cact, and a re-export check confirmed the tensor mapping. The gap comes from the engines. Beam search, padding, silence trimming and 8-bit activations were all tested and don't explain it. For the best accuracy, use whistle.cact with Fleet-Clean-0.36B.
Parity checks
parity/ has every number.
| File | Transcripts identical to the open JAX model | Shipped |
|---|---|---|
whistle-fleet2-F32.gguf |
380/380 | yes |
whistle-fleet2-F16.gguf |
378/380 | yes |
whistle-fleet2-Q8_0.gguf |
344/380 | no (fails the gate) |
| Tagger file | Word tags identical to PyTorch | Token-id mismatches | Shipped |
|---|---|---|---|
fleet-clean-tagger-F16.gguf |
99.99% | 0 | yes |
fleet-clean-tagger-Q8_0.gguf |
99.82% | 0 | yes |
Notes
- The Whistle Q8_0 file failed its gate (at least 95% of transcripts identical to the open model, and WER no more than 0.5 points worse), so it isn't included.
- The tagger GGUF uses the original jhu-clsp/mmBERT-base tokenizer files. They have the same vocabulary and merges as Laya's, which transformers 5 re-saved in a form llama.cpp rejects. Token ids were checked against the HF tokenizer on every evaluation text.
- In F16, a few tokens with outlier activations ("I", ".") reach about 0.98 cosine against PyTorch on every backend, while the mean is โฅ 0.9999. The word tags are what matter, and they are compared above.
whistle-clireads GGUF itself (F32 / F16 / Q8_0), uses float32 throughout and decodes greedily, the same way as Cactus's reference decoder. It handles 16-bit, 24-bit and 32-bit PCM or float WAV, up to 30 s.
License
Apache-2.0. whistle/whistle.cpp is a port of Apache-2.0 Cactus code. See NOTICE.
- Downloads last month
- 154
Model tree for Lucasgitguy-co/Fleet-Clean-0.36B-GGUF
Base model
Cactus-Compute/whistle