| --- |
| license: apache-2.0 |
| language: |
| - en |
| pipeline_tag: text-to-speech |
| tags: |
| - arxiv:2401.12345 |
| --- |
| |
| # 🧠 AphaVoice: Text-to-Speech Model for Aphasia Patient Simulation |
|
|
| **AphaVoice** is a specialized Text-to-Speech (TTS) model designed to simulate non-fluent aphasic speech for medical student training and clinical simulation. AphaVoice fine-tunes the non-autoregressive architecture of Matcha-TTS to accurately map the distinct temporal and phonetic dysfluencies characteristic of post-stroke aphasia. |
|
|
| ## 🚀 Key Features |
| * **Pathological Realism:** Outperforms off-the-shelf TTS models in accurately simulating aphasic Words-Per-Minute (WPM) and pathological pause ratios. |
| * **Acoustic Stability:** Achieves high Speaker Encoder Cosine Similarity (SECS) scores, ensuring the underlying voice profile remains perfectly consistent despite severe simulated dysfluency. |
| * **Multi-Speaker Latent Shift:** Supports dynamic latent shifting to generate both Male and Female baseline speaker profiles. |
| * **Clinical Dysfluency Engine:** Includes a custom linguistic pipeline to inject mathematically weighted phonetic false starts, repetitions, and conversational fillers prior to acoustic generation. |
|
|
| ## 💻 Quick Start (Python) |
|
|
| You can download and run the model locally using the `huggingface_hub` library. |
| Before you could use the model, as this is built on top of Matcha-TTS, you need to install all their dependencies to make it work. Please refer to this github: https://github.com/varrelkusuma/AphaVoice |
|
|
| **Prerequisites:** |
| ```bash |
| pip install huggingface_hub soundfile spacy pyyaml torch safetensors noisereduce |
| python -m spacy download en_core_web_sm |
| sudo apt-get install espeak-ng # Required for Linux/Colab phonemization |
| ``` |
|
|
| **Inference Code:** |
| ```python |
| import os |
| import sys |
| import soundfile as sf |
| import IPython.display as ipd |
| from huggingface_hub import snapshot_download |
| |
| # 1. Download the AphaVoice repository |
| repo_path = snapshot_download(repo_id="varrelkusuma/AphaVoice") |
| sys.path.insert(0, repo_path) |
| from inference import AphaVoicePipeline |
| |
| # 2. Initialize the model |
| pipeline = AphaVoicePipeline( |
| config_path=os.path.join(repo_path, "config.yaml"), |
| model_path=os.path.join(repo_path, "model.ckpt"), |
| vocoder_path=os.path.join(repo_path, "vocoder") |
| ) |
| |
| # 3. Generate Speech |
| text = "The patient needs a glass of water." |
| sample_rate, audio_data, final_script = pipeline.synthesise( |
| text=text, |
| speaker_id=0, # 0 for Female Model, 1 for Male Model |
| apply_aphasia=True, # Applies the clinical dysfluency script, change to "False" for normal speech generation |
| severity=0.4, # Probability of dysfluency, value from 0-1 |
| length_scale=1.0 # Pacing adjustment |
| ) |
| |
| # 4. Save output |
| print(f"Synthesized script: {final_script}") |
| display(ipd.Audio(audio_data, rate=22050)) |
| ``` |
|
|
| ## 📝 Citation |
| If you use AphaVoice in your research, please cite our paper: |
|
|
| ```bibtex |
| @misc{kusuma2026aphavoicetexttospeechmodelaphasia, |
| title={AphaVoice: Text-to-Speech Model for Aphasia Patient Simulation}, |
| author={Al Varrel Putra Kusuma and Fernando Bello and Joshua Brown}, |
| year={2026}, |
| eprint={2610.14479}, |
| archivePrefix={arXiv}, |
| primaryClass={cs.CL}, |
| url={https://arxiv.org/abs/2610.14479}, |
| } |
| ``` |
