Upload mbfa-thai (thai_fleurs8h_mt01a_e8_val_loss=2.523.ckpt)

README.md

@@ -0,0 +1,85 @@
+---
+license: agpl-3.0
+library_name: pytorch
+language:
+- th
+datasets:
+- google/fleurs
+tags:
+- forced-alignment
+- phoneme-recognition
+- phoneme-segmentation
+- speech
+- multilingual
+---
+
+# mbfa-thai (p4mbfa · thai_khmer)
+
+A context-free CNN phoneme encoder and forced aligner (p4mbfa, the successor of CUPE / p3cupe),
+for the **thai_khmer** language group of [standard_g2p](https://github.com/tabahi/CharsiuG2P). Each 5 ms frame is classified
+from at most 120 ms of audio (38.9 ms receptive field), so the model cannot learn any language's
+phonotactics. Phone boundaries come from a segmental Viterbi decoder over the known phone
+sequence, refined to sub-frame precision at the crossing of neighbouring posteriors.
+
+| head | labels | size |
+|---|---|---|
+| `ph` | local tokens of `thai_khmer` (incl. `<blank> SIL noise <unk>`) | 38 |
+| `phg` | gold phoneme groups, shared by every language group | 15 |
+| `tone` | tone values, shared by every language group; trained on this group's tone layer | 22 |
+
+Languages (the FLEURS languages it was trained on): th. standard_g2p maps every member of the group
+onto the same tokens, so other members may align too, untested.
+
+## Training
+
+**Data:** FLEURS thai_khmer without Khmer (exclude_langs [km]): Thai only, all 7.8 h, training noise_level 0.02.  
+**Checkpoint:** experiment `mt01a`, epoch 8, selected on FLEURS val_loss (best of 20 epochs; final epoch 2.565).  
+Trunk and phg_head from latin ma02a (final epoch), ph_head and tone_head trained fresh. Khmer is in the thai_khmer label space but was not trained (standard_g2p folds 19.8% of its tokens to <unk>).
+
+| metric | value |
+|---|---|
+| `val_loss` | 2.5226 |
+| `val_frame_acc` | 0.4881 |
+| `val_frame_acc_groups` | 0.5973 |
+| `val_peak_ce_tone` | 0.863 |
+
+`val_*` are on held-out FLEURS clips but are scored against the model's own alignments (there are no boundary labels outside English), so they measure self-consistency, not boundary accuracy.
+
+Labels are standard_g2p dictionary pronunciations (gold inventory `9438371ed6dd`),
+not phonetic transcriptions of what was said.
+
+## Usage
+
+The code is in [https://github.com/tabahi/bfa_models](https://github.com/tabahi/bfa_models) (`p4mbfa/`); clone it and run from its root.
+
+```python
+from p4mbfa.inference import MbfaAligner
+
+aligner = MbfaAligner.from_pretrained("Tabahi/mbfa-thai")
+wav = aligner.load_audio("clip.wav")                     # mono, 16000 Hz
+phones = ["SIL", "h", "ɛ", "l", "o", "SIL"]   # this group's tokens (config.json labels.tokens)
+for seg in aligner.align(wav, phones):
+    print(seg["token"], seg["start_ms"], seg["end_ms"])
+```
+
+`aligner.encode(wav)` returns the per-frame log posteriors of all three heads.
+Text -> tokens is standard_g2p's job (`goldG2P.phonemize_sentence` then
+`lang_group_inventory.to_local`); `config.json` lists the token strings. `from_pretrained`
+downloads only `config.json` and `model.safetensors`.
+
+## Fine-tuning
+
+`ckpt/thai_fleurs8h_mt01a_e8_val_loss=2.523.ckpt` is the training checkpoint (PyTorch Lightning, weights only, a pickle: load
+it only if you trust this repo). Download it, then point a p4mbfa yaml at it to continue
+training, or to train a new language group on its trunk and shared `phg` head:
+
+```bash
+hf download Tabahi/mbfa-thai "ckpt/thai_fleurs8h_mt01a_e8_val_loss=2.523.ckpt" --local-dir tmp/hf/mbfa-thai
+```
+
+```yaml
+ckpt_path: "tmp/hf/mbfa-thai/ckpt/thai_fleurs8h_mt01a_e8_val_loss=2.523.ckpt"
+reset_fine_heads: true      # new lang_group: rebuild ph_head / tone_head; false = same group
+```
+
+License: AGPL-3.0.

ckpt/thai_fleurs8h_mt01a_e8_val_loss=2.523.ckpt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:01f0eff82e4d34b6db95833daebbd1bc5e12aa2f7b86487e1369fdbf799b7292
+size 58834891

config.json

@@ -0,0 +1,172 @@
+{
+  "format": "p4mbfa",
+  "format_version": 1,
+  "model_config": {
+    "input_wav_length": 1920,
+    "cnn_channels": 192,
+    "embed_dim": 512,
+    "context_kernels": [
+      5,
+      3
+    ],
+    "cnn_dropout": 0.2,
+    "head_dropout": 0.3,
+    "noise_level": 0.02,
+    "frame_blocks": 4,
+    "ffn_mult": 4,
+    "attn_radius": 0,
+    "attn_heads": 8,
+    "block_dropout": 0.3,
+    "n_phg": 15,
+    "lang_group": "thai_khmer",
+    "n_tokens": 38,
+    "n_tones": 22
+  },
+  "geometry": {
+    "sample_rate": 16000,
+    "window_size_ms": 120,
+    "window_stride_ms": 80
+  },
+  "decoder": {
+    "min_phone_frames": 6,
+    "soft_phone_frames": 10,
+    "soft_frame_cost": 1.0,
+    "sil_skippable": true,
+    "optional_sil": true,
+    "class_prior_norm": true,
+    "boundary_bonus_weight": 30.0,
+    "boundary_contrast_frames": 17,
+    "boundary_flux_source": "mel"
+  },
+  "labels": {
+    "lang_group": "thai_khmer",
+    "tokens": [
+      "<blank>",
+      "SIL",
+      "noise",
+      "<unk>",
+      "a",
+      "s",
+      "i",
+      "e",
+      "n",
+      "t",
+      "l",
+      "k",
+      "o",
+      "r",
+      "m",
+      "d",
+      "u",
+      "f",
+      "j",
+      "b",
+      "p",
+      "ɛ",
+      "w",
+      "ʔ",
+      "ɔ",
+      "ŋ",
+      "ɯ",
+      "h",
+      "tʰ",
+      "t͡ɕ",
+      "kʰ",
+      "ɤ",
+      "pʰ",
+      "k̚",
+      "t͡ɕʰ",
+      "t̚",
+      "p̚",
+      "a̯"
+    ],
+    "special_tokens": {
+      "<blank>": 0,
+      "SIL": 1,
+      "noise": 2,
+      "<unk>": 3
+    },
+    "phg_names": [
+      "special",
+      "vowel_close",
+      "vowel_mid",
+      "vowel_open",
+      "stop_voiceless",
+      "stop_voiced",
+      "affricate",
+      "fricative_vl",
+      "fricative_vd",
+      "nasal",
+      "lateral",
+      "rhotic",
+      "glide",
+      "glottal",
+      "other"
+    ],
+    "tone_vocab": [
+      "",
+      "35",
+      "21",
+      "33",
+      "45",
+      "51",
+      "32",
+      "214",
+      "55",
+      "24",
+      "3",
+      "312",
+      "5",
+      "13",
+      "2",
+      "212",
+      "42",
+      "44",
+      "53",
+      "22",
+      "4",
+      "114"
+    ],
+    "member_langs": [
+      "th",
+      "tts"
+    ],
+    "gold_fingerprint": "9438371ed6dd"
+  },
+  "training": {
+    "experiment": "mt01a",
+    "note": "first thai_khmer run: trunk + phg_head from latin ma02a, fresh ph_head (38) and tone_head",
+    "data_src": "fleurs",
+    "train_limit": null,
+    "val_limit": null,
+    "exclude_langs": [
+      "km"
+    ],
+    "duration_max": 20,
+    "ckpt_path": "tmp/training/p4mbfa/ma02a/m_ma02a_final_e30.pt",
+    "trunk_init_path": null,
+    "reset_fine_heads": true,
+    "ramp_frames": 1.0,
+    "soft_onset_mix": 1.0,
+    "noise_level": 0.02,
+    "lr": "1e-4",
+    "epochs": 20,
+    "epoch": 8,
+    "global_step": 504,
+    "source_checkpoint": "m_mt01a_epoch=8_step=504_val_loss=2.5226.ckpt",
+    "selection": {
+      "trained_langs": [
+        "th"
+      ],
+      "train_data": "FLEURS thai_khmer without Khmer (exclude_langs [km]): Thai only, all 7.8 h, training noise_level 0.02",
+      "selected_on": "FLEURS val_loss (best of 20 epochs; final epoch 2.565)",
+      "metrics": {
+        "val_loss": 2.5226,
+        "val_frame_acc": 0.4881,
+        "val_frame_acc_groups": 0.5973,
+        "val_peak_ce_tone": 0.863
+      },
+      "note": "Trunk and phg_head from latin ma02a (final epoch), ph_head and tone_head trained fresh. Khmer is in the thai_khmer label space but was not trained (standard_g2p folds 19.8% of its tokens to <unk>)."
+    }
+  }
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1bc1177caea282961073310368a4cad50ef07da70eb5b019f7d0d6b7ae75750e
+size 58810452