LILAC: An Idempotent Neural Speech Codec (0.75 kb/s)

Anonymous review copy — AAAI-27 submission.

LILAC is a neural speech codec for 24 kHz speech at 0.75 kb/s (9.375 Hz frame rate, 20 FSQ coordinates × 4 bits per frame) that is idempotent by construction: re-encoding its own decoded output reproduces the token stream exactly — encode(decode(codes)) == codes — for any number of decode/re-encode cycles. Quality, intelligibility, and downstream metrics are therefore invariant under repeated cycles, where comparison codecs degrade.

The codec is convolution-only: an invertible analysis chart (orthogonal 1×1 mixes + additive couplings), finite scalar quantization on a selected coordinate subset, a deterministic fill network for the discarded coordinates, and an invertible anti-imaging stem acting on the five waveform phases.

Files

lilac_swa10.pt — the shipping checkpoint (58.5 M generator parameters, stochastic weight averaging over the final ten 1k-spaced training checkpoints).

Usage

With the companion code repository (see the paper's code link):

uv run python infer.py --checkpoint lilac_swa10.pt \
    --input in.wav --output out.wav
# idempotence demo: 100 decode->re-encode cycles, codes bit-exact
uv run python infer.py --checkpoint lilac_swa10.pt \
    --input in.wav --output out100.wav --cycles 100

Input is resampled to 24 kHz mono; output is 24 kHz PCM16.

Evaluation (full test splits, matched-rate comparison in the paper)

Corpus	UTMOS ↑	dWER ↓	PESQ ↑	STOI ↑
LibriSpeech test-clean (out-of-domain)	4.14	0.101	2.60	0.935
LibriTTS-R test (out-of-domain)	4.24	0.086	2.60	0.944

Token agreement across 100 decode/re-encode cycles: 1.000 (bit-exact).

Training data

Trained on the 44.1 kHz subset of HiFiTTS-2 (LibriVox-derived, ~31,700 h, 4,629 speakers), resampled to 24 kHz. LibriSpeech and LibriTTS(-R) readers are excluded by ID filter; evaluation corpora are out-of-domain.

License

Downloads last month: -; Downloads are not tracked for this model. How to track