Fleet-Clean-0.36B-GGUF

GGUF files for Fleet-Clean-0.36B: speech in, clean text out, with no Cactus or PyTorch needed.

File What Runs in
whistle-fleet2-F16.gguf (recommended), -F32.gguf Fleet-2 speech-to-text (Cactus Whistle architecture) whistle/whistle-cli (C++, in this repo)
fleet-clean-tagger-Q8_0.gguf / -F16.gguf the cleanup tagger's encoder (modern-bert) llama.cpp llama-server --embeddings --pooling none
fleet-clean-tagger-head.gguf the 768โ†’10 head and its labels, read by fleet_clean_gguf.py numpy

Usage

hf download Lucasgitguy-co/Fleet-Clean-0.36B-GGUF --local-dir fleet-clean-gguf && cd fleet-clean-gguf
sh whistle/build.sh                      # whistle-cli: Accelerate on macOS, OpenBLAS or plain C++ elsewhere
pip install numpy gguf                   # plus llama.cpp's llama-server on PATH (a build with modern-bert support)
python fleet_clean_gguf.py clip.wav      # 16 kHz WAV -> clean text
python fleet_clean_gguf.py --text "so, um, I I think we should, you know, go"
whistle/whistle-cli -m whistle-fleet2-F16.gguf -f clip.wav --json        # speech only

Important: GGUF speech is less accurate than whistle.cact

No existing GGUF runtime supports Whistle, so whistle-cli is a new C++ port of Cactus's open JAX model. Cactus's own .cact files run in a closed native engine, and on the same weights that engine scores better:

Language Clips whistle.cact (native engine) whistle-fleet2-F32/F16.gguf (open model)
English 100 14.5% 17.6%
German 100 22.0% 25.1%
French 30 27.4% 30.7%
Spanish 30 11.3% 12.5%
Italian 30 18.6% 19.3%
Dutch 30 32.5% 36.6%
Polish 30 43.2% 48.6%
Korean 30 34.4% CER 45.3% CER

The weights in these GGUF files are exactly Fleet-2's: they were dequantised from whistle.cact, and a re-export check confirmed the tensor mapping. The gap comes from the engines. Beam search, padding, silence trimming and 8-bit activations were all tested and don't explain it. For the best accuracy, use whistle.cact with Fleet-Clean-0.36B.

Parity checks

parity/ has every number.

File Transcripts identical to the open JAX model Shipped
whistle-fleet2-F32.gguf 380/380 yes
whistle-fleet2-F16.gguf 378/380 yes
whistle-fleet2-Q8_0.gguf 344/380 no (fails the gate)
Tagger file Word tags identical to PyTorch Token-id mismatches Shipped
fleet-clean-tagger-F16.gguf 99.99% 0 yes
fleet-clean-tagger-Q8_0.gguf 99.82% 0 yes
Notes
  • The Whistle Q8_0 file failed its gate (at least 95% of transcripts identical to the open model, and WER no more than 0.5 points worse), so it isn't included.
  • The tagger GGUF uses the original jhu-clsp/mmBERT-base tokenizer files. They have the same vocabulary and merges as Laya's, which transformers 5 re-saved in a form llama.cpp rejects. Token ids were checked against the HF tokenizer on every evaluation text.
  • In F16, a few tokens with outlier activations ("I", ".") reach about 0.98 cosine against PyTorch on every backend, while the mean is โ‰ฅ 0.9999. The word tags are what matter, and they are compared above.
  • whistle-cli reads GGUF itself (F32 / F16 / Q8_0), uses float32 throughout and decodes greedily, the same way as Cactus's reference decoder. It handles 16-bit, 24-bit and 32-bit PCM or float WAV, up to 30 s.

License

Apache-2.0. whistle/whistle.cpp is a port of Apache-2.0 Cactus code. See NOTICE.

Downloads last month
154
GGUF
Model size
0.3B params
Architecture
modern-bert
Hardware compatibility
Log In to add your hardware

8-bit

16-bit

32-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for Lucasgitguy-co/Fleet-Clean-0.36B-GGUF

Collection including Lucasgitguy-co/Fleet-Clean-0.36B-GGUF