ORB detail model β research only
Code Β· Planner (Apache-2.0) Β· Project page
Non-commercial. These weights were trained on ESC-50 (Piczak, 2015), which is CC BY-NC 3.0, so this model is released under CC BY-NC-4.0. It is not part of the Apache-2.0 ORB engine and must not be used commercially until it is retrained on permissively licensed audio.
Detail v6: a 9.06M-parameter (9,064,387) complex-STFT U-Net that predicts a high-frequency residual on top of an ORB DSP render.
DSP render βββΊ this model βββΊ residual (above 2 kHz) βββΊ refined WAV
What it does, honestly
It is a subtle texture stage, not a restoration model. It passes ORB's
no-noise gate (scripts/eval_detail_artifacts.py): it adds no tonal lines, no
energy above 10 kHz and almost nothing in quiet frames. On the ESC-50
degradation harness it reaches only a +0.84% MR-STFT improvement, against a
10% restoration target, which it does not meet.
| no-noise gate (400 eval clips) | v6 | limit |
|---|---|---|
| tonal prominence | 2.20 dB | < 6.0 |
| periodicity, output / residual | 0.006 / 0.252 | β€ 0.2 / β€ 0.3 |
| eval-pairs prominence, max / median | 3.71 / 0.86 dB | < 6.0 |
| energy ratio > 10 kHz / > 12 kHz | 1.000Γ / 1.000Γ | β€ 1.25 |
| quiet-frame rise | +0.063 dB | β€ 0.5 |
| MR-STFT improvement | +0.84% | β₯ 0 |
| onset F1 (identity 0.749) | 0.749 | β₯ identity β 0.02 |
| gate | PASS |
The previous weights (v5) reached 1.4% MR-STFT but failed this gate: their
ConvTranspose2d upsampling left checkerboard tonal stripes around 10β13 kHz
plus a ~4 kHz line. v6 is more conservative and cleaner.
The DSP path is the shippable engine. This model is included for research and for the texture option.
Architecture
- Complex-STFT U-Net, base 64, 4 levels, GroupNorm, SiLU, FiLM conditioning, bilinear resize-conv upsampling.
- Input:
(2, 513, T)real/imag STFT; no noise channel. - Output: a smoothed mask residual (kernel 9Γ3) restricted to bins above 2 kHz, tapered off between 8 and 9.5 kHz, with an energy gate and a quiet-frame gate (30 dB, 10 dB ramp).
- Losses: band-energy 2.0, reconstruction 0.05, envelope 0.2, overshoot 1.0, idempotence 5.0; no adversarial loss.
- Trained on 256-frame crops; inference uses 256-frame windows with 50% overlap and a Hann crossfade.
Usage
pip install "orb-audio[train,hub] @ git+https://github.com/stdevMac/orb-audio"
from huggingface_hub import hf_hub_download
from orb.api import load_detail, render_spec, save_wav
path = hf_hub_download("marcosmaceo/orb-detail", "detail.safetensors")
hf_hub_download("marcosmaceo/orb-detail", "config.json") # read from the same folder
detail = load_detail(path)
spec = {
"duration": 6.0, "texture": "rain", "mood": "calm",
"events": [{"role": "impact", "t": 1.0, "dur": 0.9, "gain_db": -6.0, "pan": 0.0, "seed": 7}],
}
audio = render_spec(spec, detail=detail, strength=1.0)
save_wav(audio, "cue.wav")
strength scales the residual from 0 (DSP only) to 1 (full).
Training data
ESC-50 folds 1β4 (1599 pairs), degraded to a 40-band mel envelope plus a 2β6 kHz lowpass. Evaluation on fold 5 (400 clips). ESC-50 is not redistributed here. Batch 8; these weights are step 1250 of a 1500-step run on one NVIDIA H100 80GB (about 2 minutes).
Limitations
- Does not meet its own restoration gate (10% MR-STFT).
- Trained on a synthetic degradation, not on real recording pairs.
- Non-commercial only.
Citation
If you use this model, please also cite ESC-50:
@inproceedings{piczak2015esc,
title = {ESC: Dataset for Environmental Sound Classification},
author = {Piczak, Karol J.},
booktitle = {Proceedings of the 23rd ACM International Conference on Multimedia},
pages = {1015--1018},
year = {2015}
}
- Downloads last month
- -