--- license: apache-2.0 library_name: inworld-voice-isolation pipeline_tag: audio-to-audio tags: [audio, speech, speech-enhancement, speech-denoising, noise-suppression, voice-isolation, 48khz, full-band, streaming, realtime] ---
Open-license (Apache-2.0) 48 kHz voice isolation for noisy speech recordings and live audio.
--- ## TL;DR Inworld Voice Isolator v1 ships as two models that share one architecture: | | `voice-isolator-v1` | `voice-isolator-v1-realtime` | |---|---|---| | made for | recorded audio and anything not latency-sensitive, at the best quality | live audio, as a stream in your pipeline or production | | how it runs | offline: every frame sees its whole 8 s chunk | a causal stream with 20 ms of look-ahead | | parameters | 64M | 58M | **Use them for** cleaning recordings before playback or publication, curating speech datasets and voice pipelines where the speaker's identity must survive. With the realtime model, use them for live calls and streams. Subtract the isolated voice from the input to extract background noise or a backing-track residual for vocals/music separation. **They are not** a speaker selector or a restoration model. Restoration is planned for version 2 (see [Plans](https://github.com/inworld-ai/voice-isolator#plans)). ## Quick start Using a coding agent? Point it at the [setup section of AGENTS.md](https://github.com/inworld-ai/voice-isolator/blob/main/AGENTS.md#setting-up-the-voice-isolator-for-a-user): it installs the package, checks the machine with `voice-isolate-doctor` and cleans a first file. ```bash pip install inworld-voice-isolation # or from a clone: git clone https://github.com/inworld-ai/voice-isolator && cd voice-isolator && pip install -e . # a noisy example; a clone also has it at demo/examples/sample_14.wav curl -LO https://raw.githubusercontent.com/inworld-ai/voice-isolator/main/demo/examples/sample_14.wav # offline isolation (downloads the offline model on first use): # the voice goes to sample_14_clean.wav and the removed background # to sample_14_clean_background.wav, beside it voice-isolate sample_14.wav sample_14_clean.wav ``` Both are float32 WAVs at 48 kHz, with their original levels preserved; together they reconstruct the aligned mono input. Use `--output-sr 0` to keep the input rate, `--background noise.wav` to choose the background path for one file, or `--no-background` to save only the voice. ```python from inworld import voice_isolation # isolate a recording offline offline = voice_isolation.VoiceIsolator("inworld/voice-isolator") wav, sample_rate = voice_isolation.load_audio("sample_14.wav") voice = offline.isolate(wav, sample_rate) voice_isolation.save_audio("sample_14_clean.wav", voice, sample_rate, subtype="FLOAT") voice_isolation.save_audio("sample_14_clean_background.wav", wav - voice, sample_rate, subtype="FLOAT") # what was removed ``` ### Streaming The realtime model isolates the voice as the audio arrives, with 20 ms of look-ahead: ```python import torch from inworld import voice_isolation live = voice_isolation.VoiceIsolator("inworld/voice-isolator", mode="streaming", lookahead_frames=2) session = live.stream() # takes 48 kHz mono, in pieces of any size wav, sample_rate = voice_isolation.load_audio("sample_14.wav") # 48 kHz mono, standing in for a live source voice = [] for start in range(0, len(wav), 960): # 20 ms at a time, as a microphone delivers it voice.append(session.push(wav[start:start + 960])) # the samples that are final: play or send them voice.append(session.flush()) # the rest, once the source ends voice_isolation.save_audio("sample_14_streamed.wav", torch.cat(voice), sample_rate) ``` The streaming CLI writes only the isolated voice, after the whole file has gone through: ```bash voice-isolate sample_14.wav sample_14_streamed.wav --mode streaming --lookahead 2 ``` The weights download on first use, and their checksums are verified. The mode picks the checkpoint: `voice-isolator-v1` offline, `voice-isolator-v1-realtime` for streaming. CUDA is used when available, Apple's Metal backend (`mps`) on an Apple-silicon Mac with PyTorch 2.6 or later, and the CPU otherwise; `--device` picks one. The code, the [interactive demo](https://github.com/inworld-ai/voice-isolator/blob/main/demo/README.md) and the [benchmark](https://github.com/inworld-ai/voice-isolator/blob/main/bench/README.md) are on GitHub: [inworld-ai/voice-isolator](https://github.com/inworld-ai/voice-isolator). The demo loads both models at startup: **Offline** processes files and samples into voice and background audio; **Online** streams the microphone for up to 60 seconds with updating waveforms and spectrograms, pausing capture before playback. ## Hardware Tested on NVIDIA Ampere, Hopper and Blackwell GPUs (A100, H100, B200 and B300) and on CPU. Smaller GPUs are not tested yet (see [Plans](https://github.com/inworld-ai/voice-isolator#plans)). ## The VIBE benchmark
Built for people with ❤️ by Inworld.