| --- |
| license: other |
| license_name: nvidia-open-model-license |
| license_link: https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/ |
| base_model: nvidia/personaplex-7b-v1 |
| tags: |
| - full-duplex |
| - speech-to-speech |
| - emotion-generation |
| language: |
| - en |
| --- |
| |
| # MAGO PersonaPlex emotion finetune |
|
|
| Full-duplex speech-to-speech LoRA finetune of [PersonaPlex](https://huggingface.co/nvidia/personaplex-7b-v1), merged into one `model.safetensors`. Trained on duplex conversations whose speech was synthesized with [Boson AI Higgs Audio v3 TTS 4B](https://huggingface.co/bosonai/higgs-tts-3-4b) (Higgs TTS 3). The agent was trained to follow: |
|
|
| `Answer with a {emotion} tone.` |
|
|
| where `{emotion}` is `happy`, `sad`, `angry`, or `neutral`. |
|
|
| This repo is not affiliated with NVIDIA or Boson AI. See `NOTICE` and `LICENSE`. |
|
|
| ## Use |
|
|
| Same stack as PersonaPlex. `model.safetensors` replaces the base LM weights. Mimi, the text tokenizer, and voice prompts still come from `nvidia/personaplex-7b-v1` unless those files are also in this repo. |
|
|
| Accept the PersonaPlex license at <https://huggingface.co/nvidia/personaplex-7b-v1> before the base files will download. |
|
|
| Offline, one WAV in and one WAV out. Download the LM weights from [mago-ai/emotionplex-7b](https://huggingface.co/mago-ai/emotionplex-7b) first. Mimi, the text tokenizer, and voice prompts still come from `nvidia/personaplex-7b-v1`. |
|
|
| ```bash |
| hf download mago-ai/emotionplex-7b model.safetensors --local-dir weights |
| |
| python -m moshi.offline \ |
| --moshi-weight weights/model.safetensors \ |
| --hf-repo nvidia/personaplex-7b-v1 \ |
| --voice-prompt NATF2.pt \ |
| --text-prompt "Answer with a sad tone." \ |
| --input-wav user.wav \ |
| --output-wav agent.wav \ |
| --output-text out.json |
| ``` |
|
|
| For the web server, download the same weights, plus the PersonaPlex web UI (`dist.tgz`). `--hf-repo nvidia/personaplex-7b-v1` still supplies Mimi, the tokenizer, and voice prompts. |
|
|
| ```bash |
| hf download mago-ai/emotionplex-7b model.safetensors --local-dir weights |
| hf download nvidia/personaplex-7b-v1 dist.tgz --local-dir weights |
| tar -xzf weights/dist.tgz -C weights |
| |
| python -m moshi.server \ |
| --moshi-weight weights/model.safetensors \ |
| --hf-repo nvidia/personaplex-7b-v1 \ |
| --static weights/dist |
| ``` |
|
|
| `moshi` is the PersonaPlex server package (`personaplex/moshi`). Voice prompts such as `NATF2.pt` come from `voices.tgz` on `nvidia/personaplex-7b-v1`. |
|
|
| ## What this is |
|
|
| LoRA rank 512, scaling 2, merged in bfloat16 into PersonaPlex 7B. The text prompt is the system prompt PersonaPlex reads before the conversation. |
|
|
