Promote verified live Alpha comparison demo

0900a68 verified 23 days ago

8.2 kB

---
license: cc-by-4.0
language:
- en
- de
- fr
- nl
- es
pipeline_tag: text-to-speech
base_model: laion/Humaneness-Voice-Large-1.3-Alpha
base_model_relation: adapter
tags:
- audio
- text-to-speech
- lora
- reinforcement-learning
- custom-code
---
# Humane Voice Large Alpha 1.4

A **rank-128 reinforcement-learning adapter** for expressive multilingual
text-to-speech. This release adds the completed 1,452-update RL adapter to
[Humaneness Voice Large 1.3 Alpha](https://huggingface.co/laion/Humaneness-Voice-Large-1.3-Alpha).
It is not a standalone model and does not duplicate the base weights.

**[Open the live Alpha comparison demo](https://laion-humaneness-voice-large-alpha-comparisons.static.hf.space/)**
to hear the same 70 cases from the 1.2 SFT baseline, 1.3 SFT baseline and this
completed RL adapter side by side, with aggregate quality and emotion tables.
All 215 players, including five speaker references, are publicly accessible.

The [legacy comparison](https://laion-humaneness-voice-large-1-2-alpha-listening.static.hf.space/listening.html)
is retained as a secondary link. Its historical “1.3 + RL” / green FINAL RL
column refers to the adapter released here. The separately planned 20/40/40
continuation will be an additional comparison column after completion, not a
silent replacement of these released weights.

## What changed?

The frozen Large 1.3 model was trained with a new attention LoRA to favor
pleasant speech, source-emotion fidelity and natural-sounding delivery. Rewards
combine **33% Content Enjoyment, 33% matching the source audio's two strongest
emotions, and 34% genuineness**. Both reference-audio and prompt-only generations
were trained. This version does **not** include the later requested 20/40/40
continuation; that is a separate experiment and checkpoint.

The policy is hierarchical: a roughly 4B semantic transformer advances once
per 80-ms frame, and a local Talker predicts twelve codec indices within that
frame. Twelve codebooks at 12.5 frames/s means 150 codec indices per second,
not twelve indices per second. The external MOSS v2 codec reconstructs 48-kHz
mono audio. See the pinned base model for the full architecture and SFT history.

## Matched evaluation

All 70 fixed cases were retained: 35 scenes, five languages, with and without
the same speaker reference, using the same literal text, directing prompts,
seeds and frame budgets as the SFT baseline. No best-of-N selection, edited
waveforms or discarded poor outputs. Scores are computed from the actual MP3s.

| Model | Reference | N | Content Enjoyment ↑ | Genuineness ↑ | Mean per-clip WER ↓ |
|---|---|---:|---:|---:|---:|
| Large 1.3 SFT | No | 35 | 6.274 | 4.257 | 4.358% |
| **Alpha 1.4 adapter** | No | 35 | **6.341** | **4.323** | **3.689%** |
| Large 1.3 SFT | Yes | 35 | 6.243 | 4.143 | 2.118% |
| **Alpha 1.4 adapter** | Yes | 35 | **6.314** | **4.276** | **1.797%** |

These are small improvements in learned proxy scores, not human MOS or a
statistical-significance claim. EARS Medium estimates CE and genuineness;
Parakeet v3 supplies ASR for WER. WER is the arithmetic mean of per-clip error
rates, not a corpus token-weighted rate. See [aggregate statistics](stats/evaluation.json)
for sample standard deviations and [run statistics](stats/training.json) for
training cost and QA. [Group learning curves](stats/learning_curves.json) and
[optimizer-step LR/gradient traces](stats/optimizer_curve.json) are included.
No certified speaker-similarity or emotion-target ground
truth is claimed on these scripted listening cases.

## Install and generate

The tested stack used Python 3.13, PyTorch 2.9.1/CUDA 12.8 and Transformers
5.14.1 on GH200. A CUDA GPU is required for the tested path. On another server,
install the appropriate CUDA PyTorch build first, then the remaining requirements.
The model repository uses reviewed remote model code; `trust_remote_code=True`
is necessary for the custom MOSS architecture. Keep caches outside a quota-limited
home directory. The example downloads the **pinned base revision**, not its
possibly-changing main branch.

```bash
hf download laion/Humane-Voice-Large-Alpha-1.4 --local-dir humane-alpha14
cd humane-alpha14
pip install -r requirements.txt
python infer.py --language en --seconds 7 --seed 777 \
  --text 'I really thought you had forgotten. This is such a wonderful surprise.' \
  --prompt 'GENERAL: warm, spontaneous, quietly delighted.\nSCRIPT:\n(warm relief, 3.0 seconds) I really thought you had forgotten.\n(brighter gratitude, 4.0 seconds) This is such a wonderful surprise.' \
  --output example.wav
```

For speaker conditioning add `--reference-wav clean-reference.wav`.
Use a clean single-speaker recording **at least 5.04 seconds long**; the loader
preserves up to **14.96 seconds**, not the historical 2.96-second limit.
Spoken words are supplied separately via `--text`; GENERAL/SCRIPT contains the
performance directions and matching text, with round parentheses for inline
directions and optional duration controls. Reference matching and exact timing
are not guaranteed. Directions are usually English; spoken text can use any
listed language. A registered fictional alias from the base catalog can be
supplied via `--voice-name` instead of audio, but a good reference is preferred.

### Important adapter format

This is **not PEFT format**. Do not feed `adapter.safetensors` directly to
`PeftModel.from_pretrained`. Our loader validates every tensor name and shape,
then inserts FP32 `a`/`b` residuals into semantic q/k/v/o and local-Talker
c_attn/c_proj attention projections. Rank=alpha=128, residual scaling=1.0.
The frozen base uses BF16 for deployment; adapter multiplication remains FP32.
The published listening evaluation used this same unmerged deployment path.
Deployment uses temperature 1.0, top-p 0.95, top-k 50 and repetition penalty 1,
with a 625-frame ceiling in the fixed comparison. Training rollouts instead used
full-support temperature 0.7 sampling, as required by likelihood replay.
The adapter SHA-256 is
`c26031b9e1932ae0e76b29ec894b49bb6f719a9ff997cf733eae8bbf443f9fb2`.

```python
import infer
model, processor, base_api = infer.load(device='cuda:0')
base_api.load_codec(processor, 'OpenMOSS-Team/MOSS-Audio-Tokenizer-v2')
# base_api.build_user / generate / decode then work with the adapted model.
```

## Training and reproducibility

See [TRAINING.md](TRAINING.md) for the objective, exact hyperparameters,
prepared SQLite schema, source filtering, generic distributed launch and resume
requirements. The portable trainer and numerical objective implementations are
in [training/](training/). Run `python -m unittest discover -s tests -v` for
CPU adapter-shape/integrity and objective checks; see `VALIDATION.md` for the
allocated-GPU canary result.

The run used a curated mixture of real and synthetic speech across English,
German, French, Dutch and Spanish, with equal language/source/conditioning
representation. Provider-identifying metadata, local paths, raw data and input
identifiers are intentionally omitted. Consequently the supplied code reproduces
the **method with compatible prepared inputs**, not a byte-identical input map
or optimizer/RNG resume of the original private run. No optimizer pickle is
distributed. The complete original optimizer states remain preserved locally.

## License, limitations and responsible use

New adapter weights and release materials: **CC BY 4.0**; see [LICENSE](LICENSE)
and [NOTICE.md](NOTICE.md). The base, codec, dependencies and source recordings
retain their own upstream terms. This does not grant permission to impersonate
people or to use someone else's voice without necessary consent.

This alpha release is intended primarily for research and responsibly licensed
dataset augmentation. Assess content, identity, pronunciation, bias and audio
quality before any downstream deployment. Do not use it for deceptive
impersonation, fraud or misleading attributed recordings. LAION contributors
do not endorse generated statements or assume responsibility for downstream
use. These are usage recommendations and safety guidance, not additional
restrictions that replace the CC BY 4.0 license.

	---
	license: cc-by-4.0
	language:
	- en
	- de
	- fr
	- nl
	- es
	pipeline_tag: text-to-speech
	base_model: laion/Humaneness-Voice-Large-1.3-Alpha
	base_model_relation: adapter
	tags:
	- audio
	- text-to-speech
	- lora
	- reinforcement-learning
	- custom-code
	---

	# Humane Voice Large Alpha 1.4

	A rank-128 reinforcement-learning adapter for expressive multilingual
	text-to-speech. This release adds the completed 1,452-update RL adapter to
	[Humaneness Voice Large 1.3 Alpha](https://huggingface.co/laion/Humaneness-Voice-Large-1.3-Alpha).
	It is not a standalone model and does not duplicate the base weights.

	[Open the live Alpha comparison demo](https://laion-humaneness-voice-large-alpha-comparisons.static.hf.space/)
	to hear the same 70 cases from the 1.2 SFT baseline, 1.3 SFT baseline and this
	completed RL adapter side by side, with aggregate quality and emotion tables.
	All 215 players, including five speaker references, are publicly accessible.

	The [legacy comparison](https://laion-humaneness-voice-large-1-2-alpha-listening.static.hf.space/listening.html)
	is retained as a secondary link. Its historical “1.3 + RL” / green FINAL RL
	column refers to the adapter released here. The separately planned 20/40/40
	continuation will be an additional comparison column after completion, not a
	silent replacement of these released weights.

	## What changed?

	The frozen Large 1.3 model was trained with a new attention LoRA to favor
	pleasant speech, source-emotion fidelity and natural-sounding delivery. Rewards
	combine **33% Content Enjoyment, 33% matching the source audio's two strongest
	emotions, and 34% genuineness**. Both reference-audio and prompt-only generations
	were trained. This version does not include the later requested 20/40/40
	continuation; that is a separate experiment and checkpoint.

	The policy is hierarchical: a roughly 4B semantic transformer advances once
	per 80-ms frame, and a local Talker predicts twelve codec indices within that
	frame. Twelve codebooks at 12.5 frames/s means 150 codec indices per second,
	not twelve indices per second. The external MOSS v2 codec reconstructs 48-kHz
	mono audio. See the pinned base model for the full architecture and SFT history.

	## Matched evaluation

	All 70 fixed cases were retained: 35 scenes, five languages, with and without
	the same speaker reference, using the same literal text, directing prompts,
	seeds and frame budgets as the SFT baseline. No best-of-N selection, edited
	waveforms or discarded poor outputs. Scores are computed from the actual MP3s.

	\| Model \| Reference \| N \| Content Enjoyment ↑ \| Genuineness ↑ \| Mean per-clip WER ↓ \|
	\|---\|---\|---:\|---:\|---:\|---:\|
	\| Large 1.3 SFT \| No \| 35 \| 6.274 \| 4.257 \| 4.358% \|
	\| Alpha 1.4 adapter \| No \| 35 \| 6.341 \| 4.323 \| 3.689% \|
	\| Large 1.3 SFT \| Yes \| 35 \| 6.243 \| 4.143 \| 2.118% \|
	\| Alpha 1.4 adapter \| Yes \| 35 \| 6.314 \| 4.276 \| 1.797% \|

	These are small improvements in learned proxy scores, not human MOS or a
	statistical-significance claim. EARS Medium estimates CE and genuineness;
	Parakeet v3 supplies ASR for WER. WER is the arithmetic mean of per-clip error
	rates, not a corpus token-weighted rate. See [aggregate statistics](stats/evaluation.json)
	for sample standard deviations and [run statistics](stats/training.json) for
	training cost and QA. [Group learning curves](stats/learning_curves.json) and
	[optimizer-step LR/gradient traces](stats/optimizer_curve.json) are included.
	No certified speaker-similarity or emotion-target ground
	truth is claimed on these scripted listening cases.

	## Install and generate

	The tested stack used Python 3.13, PyTorch 2.9.1/CUDA 12.8 and Transformers
	5.14.1 on GH200. A CUDA GPU is required for the tested path. On another server,
	install the appropriate CUDA PyTorch build first, then the remaining requirements.
	The model repository uses reviewed remote model code; `trust_remote_code=True`
	is necessary for the custom MOSS architecture. Keep caches outside a quota-limited
	home directory. The example downloads the pinned base revision, not its
	possibly-changing main branch.

	```bash
	hf download laion/Humane-Voice-Large-Alpha-1.4 --local-dir humane-alpha14
	cd humane-alpha14
	pip install -r requirements.txt
	python infer.py --language en --seconds 7 --seed 777 \
	--text 'I really thought you had forgotten. This is such a wonderful surprise.' \
	--prompt 'GENERAL: warm, spontaneous, quietly delighted.\nSCRIPT:\n(warm relief, 3.0 seconds) I really thought you had forgotten.\n(brighter gratitude, 4.0 seconds) This is such a wonderful surprise.' \
	--output example.wav
	```

	For speaker conditioning add `--reference-wav clean-reference.wav`.
	Use a clean single-speaker recording at least 5.04 seconds long; the loader
	preserves up to 14.96 seconds, not the historical 2.96-second limit.
	Spoken words are supplied separately via `--text`; GENERAL/SCRIPT contains the
	performance directions and matching text, with round parentheses for inline
	directions and optional duration controls. Reference matching and exact timing
	are not guaranteed. Directions are usually English; spoken text can use any
	listed language. A registered fictional alias from the base catalog can be
	supplied via `--voice-name` instead of audio, but a good reference is preferred.

	### Important adapter format

	This is not PEFT format. Do not feed `adapter.safetensors` directly to
	`PeftModel.from_pretrained`. Our loader validates every tensor name and shape,
	then inserts FP32 `a`/`b` residuals into semantic q/k/v/o and local-Talker
	c_attn/c_proj attention projections. Rank=alpha=128, residual scaling=1.0.
	The frozen base uses BF16 for deployment; adapter multiplication remains FP32.
	The published listening evaluation used this same unmerged deployment path.
	Deployment uses temperature 1.0, top-p 0.95, top-k 50 and repetition penalty 1,
	with a 625-frame ceiling in the fixed comparison. Training rollouts instead used
	full-support temperature 0.7 sampling, as required by likelihood replay.
	The adapter SHA-256 is
	`c26031b9e1932ae0e76b29ec894b49bb6f719a9ff997cf733eae8bbf443f9fb2`.

	```python
	import infer
	model, processor, base_api = infer.load(device='cuda:0')
	base_api.load_codec(processor, 'OpenMOSS-Team/MOSS-Audio-Tokenizer-v2')
	# base_api.build_user / generate / decode then work with the adapted model.
	```

	## Training and reproducibility

	See [TRAINING.md](TRAINING.md) for the objective, exact hyperparameters,
	prepared SQLite schema, source filtering, generic distributed launch and resume
	requirements. The portable trainer and numerical objective implementations are
	in [training/](training/). Run `python -m unittest discover -s tests -v` for
	CPU adapter-shape/integrity and objective checks; see `VALIDATION.md` for the
	allocated-GPU canary result.

	The run used a curated mixture of real and synthetic speech across English,
	German, French, Dutch and Spanish, with equal language/source/conditioning
	representation. Provider-identifying metadata, local paths, raw data and input
	identifiers are intentionally omitted. Consequently the supplied code reproduces
	the method with compatible prepared inputs, not a byte-identical input map
	or optimizer/RNG resume of the original private run. No optimizer pickle is
	distributed. The complete original optimizer states remain preserved locally.

	## License, limitations and responsible use

	New adapter weights and release materials: CC BY 4.0; see [LICENSE](LICENSE)
	and [NOTICE.md](NOTICE.md). The base, codec, dependencies and source recordings
	retain their own upstream terms. This does not grant permission to impersonate
	people or to use someone else's voice without necessary consent.

	This alpha release is intended primarily for research and responsibly licensed
	dataset augmentation. Assess content, identity, pronunciation, bias and audio
	quality before any downstream deployment. Do not use it for deceptive
	impersonation, fraud or misleading attributed recordings. LAION contributors
	do not endorse generated statements or assume responsibility for downstream
	use. These are usage recommendations and safety guidance, not additional
	restrictions that replace the CC BY 4.0 license.