| --- |
| license: agpl-3.0 |
| library_name: pytorch |
| language: |
| - th |
| datasets: |
| - google/fleurs |
| tags: |
| - forced-alignment |
| - phoneme-recognition |
| - phoneme-segmentation |
| - speech |
| - multilingual |
| --- |
| |
| # mbfa-thai (p4mbfa · thai_khmer) |
| |
| A context-free CNN phoneme encoder and forced aligner (p4mbfa, the successor of CUPE / p3cupe), |
| for the **thai_khmer** language group of [standard_g2p](https://github.com/tabahi/CharsiuG2P). Each 5 ms frame is classified |
| from at most 120 ms of audio (38.9 ms receptive field), so the model cannot learn any language's |
| phonotactics. Phone boundaries come from a segmental Viterbi decoder over the known phone |
| sequence, refined to sub-frame precision at the crossing of neighbouring posteriors. |
| |
| | head | labels | size | |
| |---|---|---| |
| | `ph` | local tokens of `thai_khmer` (incl. `<blank> SIL noise <unk>`) | 38 | |
| | `phg` | gold phoneme groups, shared by every language group | 15 | |
| | `tone` | tone values, shared by every language group; trained on this group's tone layer | 22 | |
|
|
| Languages (the FLEURS languages it was trained on): th. standard_g2p maps every member of the group |
| onto the same tokens, so other members may align too, untested. |
| |
| ## Training |
| |
| **Data:** FLEURS thai_khmer without Khmer (exclude_langs [km]): Thai only, all 7.8 h, training noise_level 0.02. |
| **Checkpoint:** experiment `mt01a`, epoch 8, selected on FLEURS val_loss (best of 20 epochs; final epoch 2.565). |
| Trunk and phg_head from latin ma02a (final epoch), ph_head and tone_head trained fresh. Khmer is in the thai_khmer label space but was not trained (standard_g2p folds 19.8% of its tokens to <unk>). |
|
|
| | metric | value | |
| |---|---| |
| | `val_loss` | 2.5226 | |
| | `val_frame_acc` | 0.4881 | |
| | `val_frame_acc_groups` | 0.5973 | |
| | `val_peak_ce_tone` | 0.863 | |
|
|
| `val_*` are on held-out FLEURS clips but are scored against the model's own alignments (there are no boundary labels outside English), so they measure self-consistency, not boundary accuracy. |
|
|
| Labels are standard_g2p dictionary pronunciations (gold inventory `9438371ed6dd`), |
| not phonetic transcriptions of what was said. |
| |
| ## Usage |
| |
| The code is in [https://github.com/tabahi/bfa_models](https://github.com/tabahi/bfa_models) (`p4mbfa/`); clone it and run from its root. |
| |
| ```python |
| from p4mbfa.inference import MbfaAligner |
| |
| aligner = MbfaAligner.from_pretrained("Tabahi/mbfa-thai") |
| wav = aligner.load_audio("clip.wav") # mono, 16000 Hz |
| phones = ["SIL", "h", "ɛ", "l", "o", "SIL"] # this group's tokens (config.json labels.tokens) |
| for seg in aligner.align(wav, phones): |
| print(seg["token"], seg["start_ms"], seg["end_ms"]) |
| ``` |
| |
| `aligner.encode(wav)` returns the per-frame log posteriors of all three heads. |
| Text -> tokens is standard_g2p's job (`goldG2P.phonemize_sentence` then |
| `lang_group_inventory.to_local`); `config.json` lists the token strings. `from_pretrained` |
| downloads only `config.json` and `model.safetensors`. |
|
|
| ## Fine-tuning |
|
|
| `ckpt/thai_fleurs8h_mt01a_e8_val_loss=2.523.ckpt` is the training checkpoint (PyTorch Lightning, weights only, a pickle: load |
| it only if you trust this repo). Download it, then point a p4mbfa yaml at it to continue |
| training, or to train a new language group on its trunk and shared `phg` head: |
|
|
| ```bash |
| hf download Tabahi/mbfa-thai "ckpt/thai_fleurs8h_mt01a_e8_val_loss=2.523.ckpt" --local-dir tmp/hf/mbfa-thai |
| ``` |
|
|
| ```yaml |
| ckpt_path: "tmp/hf/mbfa-thai/ckpt/thai_fleurs8h_mt01a_e8_val_loss=2.523.ckpt" |
| reset_fine_heads: true # new lang_group: rebuild ph_head / tone_head; false = same group |
| ``` |
|
|
| License: AGPL-3.0. |
|
|
