| --- |
| library_name: peace |
| license: cc-by-nc-4.0 |
| tags: |
| - audio |
| - audio-effects |
| - audio-retrieval |
| - code-retrieval |
| - faust |
| - jax |
| - flax |
| --- |
| |
| # PEACE: effect code and audio embeddings |
|
|
| PEACE embeds Faust audio effect code and effected ("wet") audio in one space, so |
| an audio query retrieves the effect chain that produced it, and a program |
| retrieves audio processed by it. It accompanies **PEACE: Joint Embeddings of DSP |
| Effects Code and Audio**, by David Braun and Adam Finkelstein (ISMIR 2026). |
|
|
| [Inference code](https://github.com/DBraun/PEACE) · |
| [Interactive map](https://dbraun.github.io/PEACE-map/) |
|
|
| Each subfolder holds a complete model: the AFx-Rep audio encoder (a mid/side |
| CNN14 trained from scratch), a code encoder, and both modalities' SLAP projector |
| and predictor heads. Retrieval uses cosine similarity of the predictions `q`. |
|
|
| | Subfolder | Code encoder | Audio→Code R@1 | R@10 | Code→Audio R@1 | R@10 | |
| |---|---|---:|---:|---:|---:| |
| | `boxgraph` | BoxGraph GNN over the Faust Box API graph | 48.5% | 77.7% | 50.8% | 77.5% | |
| | `t5` | T5 v1.1 small, fine-tuned, over Faust source text | 50.6% | 78.1% | 48.4% | 76.3% | |
|
|
| Scores are the paper's, on the 4,096-pair test set from 6-second excerpts, with |
| exactly one correct item per query. BoxGraph generalizes less well than T5 to |
| chains longer than the one to three effects seen in training, but its |
| embeddings do not depend on how the program is written. |
|
|
| ## Usage |
|
|
| ```bash |
| pip install "peace @ git+https://github.com/DBraun/PEACE" |
| ``` |
|
|
| ```python |
| import numpy as np |
| from audiotree import AudioTree |
| from peace import PEACE |
| |
| model = PEACE.from_pretrained("davidbraun/peace", subfolder="boxgraph") |
| programs = [ |
| model.library.program([("reverb_freeverb", [0.8, 0.5, 0.3, 0.4])]), |
| model.library.program( |
| [("distortion", [1, 3, 0.6, 0.5, 0.2, 0.7, 1.0]), ("delay", [0, 0.3, 0.5, 0.2, 0.5, 0.2, 0.5, 0.5, 0.4])] |
| ), |
| ] |
| gallery = np.asarray(model.encode_code(programs)) # [2, 768] |
| query = np.asarray(model.encode_audio(AudioTree.from_file("wet.wav", duration=6))) |
| ranking = np.argsort(-(query @ gallery.T), axis=-1) |
| ``` |
|
|
| `encode_code(programs, mask_parameters=True)` hides every parameter value, so |
| embeddings describe only which effects are used and in what order. The package |
| also ships the 21 effects as Faust source and renders programs with DawDreamer. |
|
|
| ## Training |
|
|
| - **Data**: 200K pairs of Faust code and 10-second, 48 kHz wet audio. Dry audio |
| comes from 23 public datasets across bass, drums, guitar, instruments, |
| multitrack stems, music, piano, singing, and speech. Each program chains one |
| to three of 21 Faust effects with randomly sampled parameters. |
| - **Audio**: mid/side log-mel spectrograms (2048-point FFT, hop 1024, 128 bands, |
| 20–20,000 Hz) of 6-second excerpts, after normalizing clips to −18 LUFS. |
| - **Objective**: SLAP (Guinot et al., 2025), a BYOL-style objective without |
| negative samples, for 50,000 steps at effective batch size 256. |
| - **Selection**: the checkpoint with the best validation mean reciprocal rank. |
|
|
| PEACE's audio encoder shares AFx-Rep's architecture but not its exact input |
| features: its training log-mel frames are zero-padded at the ends of each clip, |
| where AFx-Rep pads by reflection. The ISMIR proceedings computed PEACE's features |
| with reflection padding in the RIR and chain retrieval evaluations; the arXiv |
| version recomputes them on the training features, so those numbers differ |
| slightly between the two versions. |
|
|
| ## Limitations |
|
|
| - The model knows only the 21 effects in its library, applied in series. |
| Programs with other effects or with parallel or feedback routing are rejected. |
| - Effects with near-identical outputs, such as commuting linear effects, cannot |
| be told apart from audio alone, so some retrieval "errors" are equivalent. |
| - Retrieval has not been evaluated in a user study. |
|
|
| ## Licensing |
|
|
| The inference code is MIT-licensed. The weights are released under |
| **CC BY-NC 4.0**: non-commercial use with attribution. The `t5` weights are |
| fine-tuned from [google/t5-v1_1-small](https://huggingface.co/google/t5-v1_1-small), |
| Apache License 2.0, and its `tokenizer.json` is derived from that model's |
| SentencePiece vocabulary. The Faust effect sources in the package keep their own |
| terms, listed in its NOTICE file; `compressor.dsp` is GPL-3.0-or-later. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @inproceedings{braun2026peace, |
| title = {{PEACE}: Joint Embeddings of {DSP} Effects Code and Audio}, |
| author = {Braun, David and Finkelstein, Adam}, |
| booktitle = {Proc. of the 27th Int. Society for Music Information Retrieval Conference (ISMIR)}, |
| year = {2026}, |
| } |
| ``` |
|
|
