README.md

---
title: SAM Audio WebGPU
emoji: 😻
colorFrom: blue
colorTo: indigo
sdk: gradio
app_file: app.py
python_version: '3.12'
short_description: Meta's SAM Audio model running locally
startup_duration_timeout: 30m
---

# SAM Audio Studio — browser ONNX app

A Gradio-hosted interface for running the SAM Audio Small ONNX pipeline locally in the browser. The Space hosts the UI and model files; audio processing runs in the visitor’s browser.

## Features

- Interactive waveform players for the original, target, and residual audio.
- CPU/WASM and GPU/WebGPU modes, with automatic device selection.
- ODE step control.
- Supports audio files up to 60 seconds (WAV, MP3, FLAC, OGG, M4A, subject to browser decoding support).
- Optional **Automatic Span Prediction** using PEAFrame. This option downloads `peaframe.onnx`, `peaframe.onnx.data`, `peaframe_config.json`, and the PEAFrame tokenizer on demand. Model assets are cached in browser Cache Storage when quota and browser policy allow.
- Optional **Manual Anchors** using one range per line, formatted as `+ start_seconds end_seconds` or `- start_seconds end_seconds`. Positive anchors focus on a range; negative anchors mark a range to ignore. Manual anchors take priority if both extra options are enabled.
- Base ONNX models and external `.onnx.data` weights are cached in browser Cache Storage.

## Important notes

- Automatic Span Prediction uses a very large PEAFrame external weight file (several GB). It is downloaded only when the option is enabled. The browser may not have enough storage, RAM, or WebGPU/WASM memory to load it; this depends on the browser, device, and GPU driver. Manual Anchors do not require these extra weights.
- In CPU/WASM mode, inference may be slow, especially for longer clips. The 60-second limit is an input limit, not a performance guarantee.
- Clearing the model cache removes the stored ONNX assets and requires downloading them again.
- This is an experimental port. Compare tensor shapes and output quality with the Python reference before relying on the results.

## Launch

```bash
pip install gradio spaces
python app.py
```

The app uses `demo.load(..., js=CLIENT_JS)` to initialize client-side inference.

## Reference

Model: https://github.com/facebookresearch/sam-audio

Model: https://huggingface.co/matbee/sam-audio-small-onnx