README.md

metadata

title: SAM Audio WebGPU
emoji: 😻
colorFrom: blue
colorTo: indigo
sdk: gradio
app_file: app.py
python_version: '3.12'
short_description: Meta's SAM Audio model running locally
startup_duration_timeout: 30m

SAM Audio Studio — browser ONNX app

A Gradio-hosted interface for running the SAM Audio Small ONNX pipeline locally in the browser. The Space hosts the UI and model files; audio processing runs in the visitor’s browser.

Features

• Interactive waveform players for the original, target, and residual audio.
• CPU/WASM and GPU/WebGPU modes, with automatic device selection.
• ODE step control.
• Supports audio files up to 60 seconds (WAV, MP3, FLAC, OGG, M4A, subject to browser decoding support).
• Optional Automatic Span Prediction using PEAFrame. This option downloads peaframe.onnx, peaframe.onnx.data, peaframe_config.json, and the PEAFrame tokenizer on demand. Model assets are cached in browser Cache Storage when quota and browser policy allow.
• Optional Manual Anchors using one range per line, formatted as + start_seconds end_seconds or - start_seconds end_seconds. Positive anchors focus on a range; negative anchors mark a range to ignore. Manual anchors take priority if both extra options are enabled.
• Base ONNX models and external .onnx.data weights are cached in browser Cache Storage.

Important notes

• Automatic Span Prediction uses a very large PEAFrame external weight file (several GB). It is downloaded only when the option is enabled. The browser may not have enough storage, RAM, or WebGPU/WASM memory to load it; this depends on the browser, device, and GPU driver. Manual Anchors do not require these extra weights.
• In CPU/WASM mode, inference may be slow, especially for longer clips. The 60-second limit is an input limit, not a performance guarantee.
• Clearing the model cache removes the stored ONNX assets and requires downloading them again.
• This is an experimental port. Compare tensor shapes and output quality with the Python reference before relying on the results.

Launch

pip install gradio spaces
python app.py

The app uses demo.load(..., js=CLIENT_JS) to initialize client-side inference.

Reference

Model: https://github.com/facebookresearch/sam-audio

Model: https://huggingface.co/matbee/sam-audio-small-onnx