File size: 2,919 Bytes
8c8c7a3 59f2668 8c8c7a3 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 | ---
license: apache-2.0
library_name: pytorch
tags:
- forced-alignment
- word-alignment
- speech
- audio
- gguf
---
# FuseAlign multilingual (CJK fine-tune, step 2,500)
Word-level forced aligner: given a recording and its transcript, it predicts each word's start and end on a 1 ms
grid plus a presence probability. A reproduction of FuseAlign ([Vaidya et al., 2026](https://arxiv.org/abs/2609.33650))
by Zyphra: the multilingual model (100,000 steps), fine-tuned 2,500 steps on CJK junction-corrected labels.
Code, usage, long-form alignment and the C++/ggml CLI:
[github.com/Zyphra/fusealign-multilingual](https://github.com/Zyphra/fusealign-multilingual).
## Files
| File | Size | Use |
| --- | ---: | --- |
| `fusealign-multilingual-cjkft-step2500.safetensors` | 1.34 GB | PyTorch package (`torch.hub`, `fusealign_hub`): FuseAlign fp32, Dia text encoder bf16 |
| `fusealign-multilingual-cjkft-step2500-f32.gguf` | 1.35 GB | ggml port: FuseAlign F32, Dia BF16, Silero VAD F32 |
| `fusealign-multilingual-cjkft-step2500-q8_0.gguf` | 0.51 GB | ggml port: transformer matmuls Q8_0, other Dia tensors BF16, the rest F32 |
## Usage
```python
import soundfile as sf
import torch
aligner = torch.hub.load("Zyphra/fusealign-multilingual", "fusealign", trust_repo=True, device="cuda")
wav, sr = sf.read("speech.wav", dtype="float32")
words = aligner.align(torch.from_numpy(wav), sr, "the transcript of the recording")
# [WordAlignment(word='the', start=..., end=..., presence=..., placed=True), ...]
```
## Evaluation
% of word boundaries off by more than 50 ms (the paper's asymmetric error), against
[Qwen3-ForcedAligner-0.6B](https://huggingface.co/Qwen/Qwen3-ForcedAligner-0.6B) (qwen-asr 0.0.6, bf16, whole clips)
on the same clips, transcripts and references:
| Corpus | FuseAlign | Qwen3-FA |
| --- | ---: | ---: |
| TIMIT | 16.55 | 20.17 |
| Buckeye | 24.58 | 24.60 |
| AMI-silver | 49.38 | 58.22 |
| TIMIT-long | 17.25 | 20.04 |
| BZNSYP Mandarin (gold) | 15.94 | 28.04 |
| FLEURS Mandarin | 17.95 | 11.06 |
| FLEURS Japanese | 14.02 | 8.38 |
| FLEURS Korean | 8.39 | 11.53 |
| FLEURS Spanish | 19.66 | 11.71 |
| FLEURS Portuguese | 9.30 | 10.24 |
| FLEURS French | 13.15 | 12.35 |
| FLEURS German | 13.29 | 13.91 |
| FLEURS Russian | 12.74 | 11.18 |
FLEURS references are Montreal Forced Aligner output, not hand labels; BZNSYP's are hand-labelled.
## License
Apache-2.0. The checkpoint and GGUFs include the Dia text encoder weights from
[nari-labs/Dia-1.6B-0626](https://huggingface.co/nari-labs/Dia-1.6B-0626) (Apache-2.0); the GGUFs include
[Silero VAD](https://github.com/snakers4/silero-vad) weights (MIT).
## Citation
```bibtex
@article{vaidya2026fusealign,
title = {FuseAlign: Forced Alignment in the Wild},
author = {Vaidya, Mithilesh and Bailey, Stephen and Badam, Sumukh and Bendel, Matthew and He, Xingzhe},
journal = {arXiv preprint arXiv:2609.33650},
year = {2026}
}
```
|