Instructions to use soniqo/HTDemucs-Android with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- LiteRT
How to use soniqo/HTDemucs-Android with LiteRT:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Publish Android CPU and OpenCL model exports
Browse files- LICENSE +21 -0
- README.md +80 -0
- htdemucs.onnx +3 -0
- separator-fused-norm.tflite +3 -0
LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
MIT License
|
| 2 |
+
|
| 3 |
+
Copyright (c) Meta Platforms, Inc. and affiliates.
|
| 4 |
+
|
| 5 |
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
| 6 |
+
of this software and associated documentation files (the "Software"), to deal
|
| 7 |
+
in the Software without restriction, including without limitation the rights
|
| 8 |
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
| 9 |
+
copies of the Software, and to permit persons to whom the Software is
|
| 10 |
+
furnished to do so, subject to the following conditions:
|
| 11 |
+
|
| 12 |
+
The above copyright notice and this permission notice shall be included in all
|
| 13 |
+
copies or substantial portions of the Software.
|
| 14 |
+
|
| 15 |
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
| 16 |
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
| 17 |
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
| 18 |
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
| 19 |
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
| 20 |
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
| 21 |
+
SOFTWARE.
|
README.md
ADDED
|
@@ -0,0 +1,80 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: mit
|
| 3 |
+
language: []
|
| 4 |
+
base_model: smank/htdemucs-onnx
|
| 5 |
+
library_name: tflite
|
| 6 |
+
pipeline_tag: audio-to-audio
|
| 7 |
+
tags:
|
| 8 |
+
- music-source-separation
|
| 9 |
+
- onnx
|
| 10 |
+
- litert
|
| 11 |
+
- android
|
| 12 |
+
- opencl
|
| 13 |
+
---
|
| 14 |
+
|
| 15 |
+
# HTDemucs for Android
|
| 16 |
+
|
| 17 |
+
FP32 music source separation for 44.1 kHz stereo audio. The model estimates
|
| 18 |
+
drums, bass, other instruments and vocals. The runtime can sum the first
|
| 19 |
+
three stems to produce an instrumental track.
|
| 20 |
+
|
| 21 |
+
| File | Format | Size | Use |
|
| 22 |
+
| --- | --- | ---: | --- |
|
| 23 |
+
| `htdemucs.onnx` | ONNX FP32 | 304,321,552 bytes | ONNX Runtime CPU |
|
| 24 |
+
| `separator-fused-norm.tflite` | LiteRT FP32 | 244,796,736 bytes | LiteRT OpenCL GPU |
|
| 25 |
+
| `LICENSE` | MIT | | License and attribution |
|
| 26 |
+
|
| 27 |
+
Both use the same original HTDemucs weights and process 7.8-second windows.
|
| 28 |
+
The CPU file exactly matches the pinned upstream ONNX export. The GPU export
|
| 29 |
+
changes tensor layouts and how the graph is executed. It uses host-side
|
| 30 |
+
framing, inverse STFT and overlap-add.
|
| 31 |
+
|
| 32 |
+
| SHA-256 | File |
|
| 33 |
+
| --- | --- |
|
| 34 |
+
| `d2b401f322558cd57d67a752ed7be3fa55178a0626011eda8ac7bb74e17280c0` | `htdemucs.onnx` |
|
| 35 |
+
| `3b4aeed464fc0ce5cde49a3404d1cb47ff64a33bc51e88f6de98eaa0fa45d445` | `separator-fused-norm.tflite` |
|
| 36 |
+
|
| 37 |
+
## Android measurements
|
| 38 |
+
|
| 39 |
+
| Samsung Galaxy S23 Ultra, 60-second clip | Elapsed | Real-time factor |
|
| 40 |
+
| --- | ---: | ---: |
|
| 41 |
+
| LiteRT OpenCL with a warm kernel cache | 20.89 s | 0.348 |
|
| 42 |
+
| LiteRT OpenCL with an empty kernel cache | 27.53 s | 0.459 |
|
| 43 |
+
|
| 44 |
+
These runs used LiteRT 2.3.2 with the model already on the device. Timing
|
| 45 |
+
includes model verification and initialization, processing and WAV export.
|
| 46 |
+
Warm-run peak sampled process PSS was 1.93 GiB. The five PCM16 outputs
|
| 47 |
+
matched the qualified FP32 LiteRT reference across 26.46 million samples.
|
| 48 |
+
|
| 49 |
+
## Download
|
| 50 |
+
|
| 51 |
+
```sh
|
| 52 |
+
hf download soniqo/HTDemucs-Android htdemucs.onnx
|
| 53 |
+
hf download soniqo/HTDemucs-Android separator-fused-norm.tflite
|
| 54 |
+
```
|
| 55 |
+
|
| 56 |
+
Select one export for the target runtime. The CPU export accepts one
|
| 57 |
+
normalized `[1, 2, 343980]` stereo float tensor. The GPU signature also takes
|
| 58 |
+
`[672, 4096]` reflection-padded audio frames. Its outputs are a
|
| 59 |
+
`[1, 16, 2048, 336]` complex spectrogram representation and a
|
| 60 |
+
`[1, 4, 2, 343980]` waveform tensor. Converting the spectrogram to audio
|
| 61 |
+
requires the original window and inverse STFT. The runtime adds the waveform
|
| 62 |
+
output, reverses input scaling and overlap-adds consecutive windows.
|
| 63 |
+
|
| 64 |
+
```python
|
| 65 |
+
from huggingface_hub import hf_hub_download
|
| 66 |
+
from ai_edge_litert.interpreter import Interpreter
|
| 67 |
+
|
| 68 |
+
path = hf_hub_download("soniqo/HTDemucs-Android", "separator-fused-norm.tflite")
|
| 69 |
+
model = Interpreter(model_path=path)
|
| 70 |
+
for tensor in model.get_input_details() + model.get_output_details():
|
| 71 |
+
print(tensor["name"], tensor["shape"], tensor["dtype"])
|
| 72 |
+
```
|
| 73 |
+
|
| 74 |
+
## Source
|
| 75 |
+
|
| 76 |
+
The [original Demucs implementation](https://github.com/facebookresearch/demucs)
|
| 77 |
+
and [HTDemucs paper](https://arxiv.org/abs/2211.08553) describe the model.
|
| 78 |
+
The [pinned ONNX export](https://huggingface.co/smank/htdemucs-onnx/tree/fb1fd25aa6260c253d86bbb4aa26b73aebc79f10)
|
| 79 |
+
is the source of both artifacts. See [LiteRT on Android](https://developers.google.com/edge/litert/android)
|
| 80 |
+
for runtime integration and [Soniqo Music](https://soniqo.audio/music) for the app.
|
htdemucs.onnx
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d2b401f322558cd57d67a752ed7be3fa55178a0626011eda8ac7bb74e17280c0
|
| 3 |
+
size 304321552
|
separator-fused-norm.tflite
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3b4aeed464fc0ce5cde49a3404d1cb47ff64a33bc51e88f6de98eaa0fa45d445
|
| 3 |
+
size 244796736
|