Voice cloning, inflection tuning, llama.cpp
It would be great if it had these features.
- simple voice cloning.
- inflection/intonation cloning using a few wav files of the original speaker.
- simple gguf interfacing with llama.cpp
nice work though!
Thank you! I'll look into these - though simple voice cloning likely would be extremely difficult to add for a model so small.
maybe more help can be given to the model. like say a larger set of texts for the person being cloned to read, and an easier-to-use end2end approach for finetuning the model after. Or maybe, a larger text2speech model with voice cloning can be used to distill the voice into this.
Voice cloning requires a dynamic prosody encoder that decouples speaker’s voice from what is being said and must fully replace speaker encoder.
Even if something like that is implemented, perfect cloning is not guaranteed. Range of 2000-5000 speakers in the dataset is needed (minimum) to fully saturate the latent space!