Gatozu35 commited on
Commit
8c8c7a3
·
verified ·
1 Parent(s): 9e29d9e

Remove local export path from metadata; add model card

Browse files
README.md ADDED
@@ -0,0 +1,76 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ library_name: pytorch
4
+ tags:
5
+ - forced-alignment
6
+ - word-alignment
7
+ - speech
8
+ - audio
9
+ - gguf
10
+ ---
11
+
12
+ # FuseAlign multilingual (CJK fine-tune, step 2,500)
13
+
14
+ Word-level forced aligner: given a recording and its transcript, it predicts each word's start and end on a 1 ms
15
+ grid plus a presence probability. A reproduction of FuseAlign ([Vaidya et al., 2026](https://arxiv.org/abs/2609.33650))
16
+ by Zyphra: the multilingual model (100,000 steps), fine-tuned 2,500 steps on CJK junction-corrected labels.
17
+
18
+ Code, usage, long-form alignment and the C++/ggml CLI:
19
+ [github.com/Zyphra/fusealign-multilingual](https://github.com/Zyphra/fusealign-multilingual).
20
+
21
+ ## Files
22
+
23
+ | File | Size | Use |
24
+ | --- | ---: | --- |
25
+ | `fusealign-multilingual-cjkft-step2500.safetensors` | 1.34 GB | PyTorch package (`torch.hub`, `fusealign_hub`): FuseAlign fp32, Dia text encoder bf16 |
26
+ | `fusealign-multilingual-cjkft-step2500-f32.gguf` | 1.35 GB | ggml port: FuseAlign F32, Dia BF16, Silero VAD F32 |
27
+ | `fusealign-multilingual-cjkft-step2500-q8_0.gguf` | 0.51 GB | ggml port: transformer matmuls Q8_0, other Dia tensors BF16, the rest F32 |
28
+
29
+ ## Usage
30
+
31
+ ```python
32
+ import soundfile as sf
33
+ import torch
34
+
35
+ aligner = torch.hub.load("Zyphra/fusealign-multilingual", "fusealign", trust_repo=True, device="cuda")
36
+ wav, sr = sf.read("speech.wav", dtype="float32")
37
+ words = aligner.align(torch.from_numpy(wav), sr, "the transcript of the recording")
38
+ # [WordAlignment(word='the', start=..., end=..., presence=..., placed=True), ...]
39
+ ```
40
+
41
+ ## Evaluation
42
+
43
+ % of word boundaries off by more than 50 ms (the paper's asymmetric error):
44
+
45
+ | Corpus | % >50 |
46
+ | --- | ---: |
47
+ | TIMIT | 16.55 |
48
+ | Buckeye | 24.58 |
49
+ | AMI-silver | 49.38 |
50
+ | TIMIT-long | 17.25 |
51
+ | BZNSYP Mandarin (gold) | 15.94 |
52
+ | FLEURS Mandarin | 17.95 |
53
+ | FLEURS Japanese | 14.02 |
54
+ | FLEURS Korean | 8.39 |
55
+ | FLEURS Spanish | 19.66 |
56
+ | FLEURS Portuguese | 9.30 |
57
+ | FLEURS French | 13.15 |
58
+ | FLEURS German | 13.29 |
59
+ | FLEURS Russian | 12.74 |
60
+
61
+ ## License
62
+
63
+ Apache-2.0. The checkpoint and GGUFs include the Dia text encoder weights from
64
+ [nari-labs/Dia-1.6B-0626](https://huggingface.co/nari-labs/Dia-1.6B-0626) (Apache-2.0); the GGUFs include
65
+ [Silero VAD](https://github.com/snakers4/silero-vad) weights (MIT).
66
+
67
+ ## Citation
68
+
69
+ ```bibtex
70
+ @article{vaidya2026fusealign,
71
+ title = {FuseAlign: Forced Alignment in the Wild},
72
+ author = {Vaidya, Mithilesh and Bailey, Stephen and Badam, Sumukh and Bendel, Matthew and He, Xingzhe},
73
+ journal = {arXiv preprint arXiv:2609.33650},
74
+ year = {2026}
75
+ }
76
+ ```
fusealign-multilingual-cjkft-step2500-f32.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:f01b2e27c2a3e404f3ea9442fd94a5494423501a09a7eead62c3d3d8741b25ea
3
- size 1345210592
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a54a49f309c96dd5ff92876986bc9875ac32227794161a79337422af9781ee9b
3
+ size 1345210432
fusealign-multilingual-cjkft-step2500-q8_0.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e3fa988089e78697af1be1050727aaf040e2ca8f9858e283de318df3e7945cbe
3
- size 514803936
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:183b4247afe5de7c0d163eea4a41057fa8dd0da27f6a898632b1aba53db77a3f
3
+ size 514803776
fusealign-multilingual-cjkft-step2500.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:bad29a3533eaa7e593df19673f772ed5a67d81bd849113fb6d47b124f433da18
3
- size 1343979116
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1c8c4243e1362453d7e8277da3b39480ea1c45507b87e5dfa7f496fade5d72c6
3
+ size 1343979004