DriftAudio

DriftAudio applies marginal drifting to post-train one-step text-to-audio generators. It matches the generated audio distribution to real audio in a frozen PANNs feature space while retaining text-conditioned, one-step inference.

Paper · Code · Audio demo

Checkpoints

File	Initialization	Selected training step
driftaudio_from_meanaudio_s_full_step4400.pth	MeanAudio-S-Full	4,400
driftaudio_from_fdaudio_step1800.pth	FdAudio	1,800

These files contain generator weights only. Inference also requires the upstream text encoders, VAE, vocoder, and related assets. See the code repository for setup, asset downloads, and generation commands.

To download the checkpoints:

huggingface-cli download xingyuchen-8/DriftAudio \
  driftaudio_from_meanaudio_s_full_step4400.pth \
  driftaudio_from_fdaudio_step1800.pth \
  --local-dir assets

Evaluation

The following are the archived paper-run results on 877 available AudioCaps test clips, using one fixed caption per clip. They are not measurements from a new run of this model-card release.

Checkpoint	FAD ↓	FD ↓	KL ↓	IS ↑	CLAP ↑
From MeanAudio-S-Full	1.111	12.715	1.207	9.834	0.325
From FdAudio	0.987	12.609	1.244	10.169	0.335

FAD uses VGGish features; FD, KL, and IS use PANNs; CLAP is the LAION-CLAP score. The evaluation protocol and its limitations are described in the code repository.

Intended use and limitations

These checkpoints are released for research and non-commercial use. Generated audio can contain artifacts or fail to follow a prompt. Results outside the reported evaluation setting have not been established. Reproducing the reported numbers requires the original data and preprocessing inputs, which are not redistributed here.

License and acknowledgments

The checkpoint files in this repository are provided under Creative Commons Attribution–NonCommercial–ShareAlike 4.0 International. The DriftAudio code is separately licensed under MIT. Upstream models, components, and datasets retain their respective terms.

DriftAudio builds on MeanAudio and FdAudio. The FdAudio-initialized checkpoint derives from the released FdAudio model.

Citation

@article{chen2026driftaudio,
  title={DriftAudio: Marginal Drifting for Distributional Post-Training of One-Step Text-to-Audio Generators},
  author={Chen, Xingyu and Ma, Fei and Zhao, Sipei},
  journal={arXiv preprint arXiv:2609.27598},
  year={2026}
}