Instructions to use semfence/pingo-ptbr with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use semfence/pingo-ptbr with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] hf download semfence/pingo-ptbr --local-dir pingo-ptbr
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
Download docs/EXPERIMENTS.md from semfence/pingo-ptbr: direct link, hf CLI and curl.
- Browser
- Download file 6.86 kB
-
https://huggingface.co/semfence/pingo-ptbr/resolve/main/docs/EXPERIMENTS.md
- Command line
-
hf download hf://semfence/pingo-ptbr/docs/EXPERIMENTS.md
-
curl -L -o EXPERIMENTS.md https://huggingface.co/semfence/pingo-ptbr/resolve/main/docs/EXPERIMENTS.md
Experiments
Everything we measured on the way to 0.87 MB, in order, including what did not work. WER numbers use the MiniMax pt set (100 sentences per
voice, Qwen3-ASR); they move by about ±0.2 points between runs, so small differences are noise.
1. Quantization without retraining
Starting point: pivo-ptbr in float32 (acoustic 2.34 MB + vocoder 3.27 MB = 5.61 MB). Symmetric per-output-channel quantization, no calibration. Size counts the weights (int4 packed two per byte, plus one scale per channel).
| Bits (acoustic / vocoder) | Weights | WER F_s4 | WER M_sm4 |
|---|---|---|---|
| float32 | 5.61 MB | 0.95% | 0.95% |
| fp16 / fp16 | 2.81 MB | 1.09% | 0.81% |
| int8 / int8 | 1.44 MB | 0.76% | 0.86% |
| int8 / fp16 | 2.24 MB | 0.76% | 1.05% |
| fp16 / int8 | 2.01 MB | 0.95% | 0.76% |
| int8 / int4 | 1.04 MB | 1.14% | 0.95% |
| int4 / int8 | 1.15 MB | 1.09% | 1.24% |
| int4 / int4 | 0.75 MB | 1.28% | 1.28% |
- WER barely moves even at 4 bits (naive post-training quantization): the ASR is tolerant. WER does not measure timbre.
- Listening (author, informal): 8 bits everywhere is fine; 4 bits everywhere is audibly noisier. Everything stays intelligible with a "phone call" sound, which comes from the 16 kHz FARGAN vocoder and is already there in float32.
- Feature error (float32 vs quantized, 20 sentences): fp16 0.0002, int8 0.0048 (mean |feature| about 1). With int8 the predicted durations change by a frame in most sentences (rounding), but the features stay close.
2. A noise proxy for the vocoder
Sample-by-sample SNR is meaningless for FARGAN (autoregressive; 8 bits gives 3.5 dB). We use the log-mel spectral distance (LSD) between the float32 vocoder output and the quantized one, running both on the same features.
| Vocoder scheme | Size | LSD |
|---|---|---|
| int8 | 0.83 MB | 0.214 |
| int6 per channel | 0.62 MB | 0.383 |
| int5 per channel | 0.52 MB | 0.467 |
| int4 per channel | 0.42 MB | 0.729 |
| int4, groups of 16 inputs | 0.51 MB | 0.474 |
| mixed (per-layer, see below) | 0.66 MB | 0.28 |
3. Per-layer sensitivity of the vocoder
All layers int8, one layer at a time lowered; increase in LSD:
| Layer | Parameters | ΔLSD at 4 bits | ΔLSD at 6 bits |
|---|---|---|---|
| skip_dense | 88,064 | 0.280 | 0.063 |
| frame_conv | 62,976 | 0.184 | 0.040 |
| gru1.input | 130,560 | 0.170 | 0.007 |
| signal_out | 5,120 | 0.170 | 0.018 |
| cond.dense1 | 2,048 | 0.153 | 0.109 |
| cond.conv | 24,576 | 0.123 | 0.018 |
| gru1.hidden | 76,800 | 0.102 | 0.003 |
| cond.dense2 | 40,960 | 0.100 | 0.001 |
| gru2.input | 92,160 | 0.093 | 0.024 |
| gru2.hidden | 49,152 | 0.049 | -0.001 |
| cond.pitch_embedding | 2,688 | 0.042 | 0.004 |
| glu1 | 25,600 | 0.040 | 0.008 |
| gru3.input | 79,872 | 0.039 | -0.001 |
| glu2 | 16,384 | 0.034 | 0.003 |
| gru3.hidden | 49,152 | 0.033 | 0.002 |
| skip_glu | 16,384 | 0.031 | 0.005 |
| frame_glu | 36,864 | 0.021 | -0.002 |
| cond_gain.w | 80 | 0.020 | 0.002 |
| pitch_gain.w | 768 | 0.018 | -0.001 |
| glu3 | 16,384 | 0.016 | 0.004 |
Almost every layer takes 6 bits at no cost; four (cond.dense1, skip_dense, frame_conv, gru2.input) do not. A greedy allocation
(additive model from this table, then a real LSD check) gave the scheme in pingo/qpack.py:
0.66 MB at LSD 0.28, against 0.62 MB at LSD 0.38 for uniform 6 bits.
4. Per-layer sensitivity of the acoustic model
Feature error with float32's durations forced, and log-duration error:
| Scheme | Size | Δfeature | Δlog-duration |
|---|---|---|---|
| all 8 bits | 0.60 MB | 0.0048 | 0.0040 |
| all 6 bits | 0.46 MB | 0.0222 | 0.0097 |
| all 5 bits | 0.38 MB | 0.0480 | 0.0293 |
| all 4 bits | 0.31 MB | 0.0975 | 0.0681 |
| only embeddings at 4 (rest 8) | 0.59 MB | 0.0638 | 0.0394 |
| only decoder at 4 (rest 8) | 0.52 MB | 0.0529 | 0.0040 |
| only attention at 4 (rest 8) | 0.56 MB | 0.0161 | 0.0084 |
| only FFN at 4 (rest 8) | 0.52 MB | 0.0277 | 0.0176 |
| only convolutions at 4 (rest 8) | 0.53 MB | 0.0234 | 0.0151 |
| only duration predictor at 4 (rest 8) | 0.59 MB | 0.0048 | 0.0375 |
| chosen: encoder 5, decoder 6, durations 6, embeddings 8 | 0.41 MB | 0.0232 | 0.0127 |
The embeddings (12k weights, about 12 KB at 8 bits) and the decoder are the most sensitive, so they get more bits.
5. The mixed scheme on the wide model (pingo-ptbr-w96)
Acoustic 0.41 MB + vocoder 0.66 MB = 1.07 MB of weights. Simulated quantize-dequantize: WER 0.76% / 1.19%. Loaded from the packed file
(weights/pingo-ptbr-w96.bin, 1.086 MB including a 14 KB header): WER 0.81% / 1.00% (float32: 0.95% / 0.95%). Listening: the author judged it
the same quality as the 8-bit-everywhere version. Reconstruction error per tensor is at most 3.3% (the 5-bit layers).
A bug found on the way: a channel with all zeros (the padding row of the embedding) had an fp16 scale of 0 and produced NaN when packing; fixed with a floor of 1e-4 on the scale.
6. Narrower acoustic model (width 64) → pingo-ptbr
Width 64, FFN 128: 270,081 parameters (46% of 584,897). 40,000 steps on a rented RTX 3090 (about 37 min, ~US$ 0.14). Validation loss:
| Step | 2k | 6k | 10k | 20k | 30k | 40k |
|---|---|---|---|---|---|---|
| feat + 0.1 × dur | 0.2028 | 0.1841 | 0.1789 | 0.1736 | 0.1707 | 0.1699 |
pivo-ptbr's width-96 model reached 0.1682 (at 20k steps, then overfitting). The width-64 model had not overfit at 40k.
Packed with the same bit scheme: weights/pingo-ptbr.bin, 0.871 MB (acoustic 0.22 MB + vocoder 0.66 MB, header 14 KB). WER from the packed file:
F_s4 0.95%, M_sm4 1.00%.
Training notes (details in HOW-WE-TRAINED.md): the Mac ran out of memory with the full set (about 19 GB of swap in use on a 16 GB machine), so we trained
on a rented GPU; the first GPU run gave NaN because of a fully masked attention row in padding frames, fixed by keeping the mask diagonal open (after the fix the
numbers match the Mac's: step 300 feature loss 0.2800, duration loss 0.1176).
7. English words
The phonemizer reads foreign words with Portuguese rules. pingo/pronunciation.py rewrites a short list before phonemizing; we checked the resulting phonemes:
| Term | Rewritten as | Note |
|---|---|---|
| Hugging Face | Ráguin Feice | "Hâguim" lost the initial H (silent in Portuguese); R gives /x/ |
| GitHub | Guit rub | "Guitrráb" produced a stray /tʃi/ |
| startup | istártape | |
| deploy | diplói | |
| pitch | pítxi | "pítchi" produced an extra /ʃ/ |
| feedback | fídbéque |
8. What we did not do
- ONNX that computes in low precision: the shipped ONNX stores int8 weights but the runtime dequantizes to float32 on load.
- Quantization-aware training, fine-tuning or pruning of the vocoder, a listening test with several people.
- Compare directly with other small pt-BR voices on a shared listening panel.