aufklarer commited on
Commit
38d55b5
·
verified ·
1 Parent(s): 3e06ed4

Publish Android CPU and OpenCL model exports

Browse files
Files changed (4) hide show
  1. LICENSE +21 -0
  2. README.md +80 -0
  3. htdemucs.onnx +3 -0
  4. separator-fused-norm.tflite +3 -0
LICENSE ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ MIT License
2
+
3
+ Copyright (c) Meta Platforms, Inc. and affiliates.
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
README.md ADDED
@@ -0,0 +1,80 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ language: []
4
+ base_model: smank/htdemucs-onnx
5
+ library_name: tflite
6
+ pipeline_tag: audio-to-audio
7
+ tags:
8
+ - music-source-separation
9
+ - onnx
10
+ - litert
11
+ - android
12
+ - opencl
13
+ ---
14
+
15
+ # HTDemucs for Android
16
+
17
+ FP32 music source separation for 44.1 kHz stereo audio. The model estimates
18
+ drums, bass, other instruments and vocals. The runtime can sum the first
19
+ three stems to produce an instrumental track.
20
+
21
+ | File | Format | Size | Use |
22
+ | --- | --- | ---: | --- |
23
+ | `htdemucs.onnx` | ONNX FP32 | 304,321,552 bytes | ONNX Runtime CPU |
24
+ | `separator-fused-norm.tflite` | LiteRT FP32 | 244,796,736 bytes | LiteRT OpenCL GPU |
25
+ | `LICENSE` | MIT | | License and attribution |
26
+
27
+ Both use the same original HTDemucs weights and process 7.8-second windows.
28
+ The CPU file exactly matches the pinned upstream ONNX export. The GPU export
29
+ changes tensor layouts and how the graph is executed. It uses host-side
30
+ framing, inverse STFT and overlap-add.
31
+
32
+ | SHA-256 | File |
33
+ | --- | --- |
34
+ | `d2b401f322558cd57d67a752ed7be3fa55178a0626011eda8ac7bb74e17280c0` | `htdemucs.onnx` |
35
+ | `3b4aeed464fc0ce5cde49a3404d1cb47ff64a33bc51e88f6de98eaa0fa45d445` | `separator-fused-norm.tflite` |
36
+
37
+ ## Android measurements
38
+
39
+ | Samsung Galaxy S23 Ultra, 60-second clip | Elapsed | Real-time factor |
40
+ | --- | ---: | ---: |
41
+ | LiteRT OpenCL with a warm kernel cache | 20.89 s | 0.348 |
42
+ | LiteRT OpenCL with an empty kernel cache | 27.53 s | 0.459 |
43
+
44
+ These runs used LiteRT 2.3.2 with the model already on the device. Timing
45
+ includes model verification and initialization, processing and WAV export.
46
+ Warm-run peak sampled process PSS was 1.93 GiB. The five PCM16 outputs
47
+ matched the qualified FP32 LiteRT reference across 26.46 million samples.
48
+
49
+ ## Download
50
+
51
+ ```sh
52
+ hf download soniqo/HTDemucs-Android htdemucs.onnx
53
+ hf download soniqo/HTDemucs-Android separator-fused-norm.tflite
54
+ ```
55
+
56
+ Select one export for the target runtime. The CPU export accepts one
57
+ normalized `[1, 2, 343980]` stereo float tensor. The GPU signature also takes
58
+ `[672, 4096]` reflection-padded audio frames. Its outputs are a
59
+ `[1, 16, 2048, 336]` complex spectrogram representation and a
60
+ `[1, 4, 2, 343980]` waveform tensor. Converting the spectrogram to audio
61
+ requires the original window and inverse STFT. The runtime adds the waveform
62
+ output, reverses input scaling and overlap-adds consecutive windows.
63
+
64
+ ```python
65
+ from huggingface_hub import hf_hub_download
66
+ from ai_edge_litert.interpreter import Interpreter
67
+
68
+ path = hf_hub_download("soniqo/HTDemucs-Android", "separator-fused-norm.tflite")
69
+ model = Interpreter(model_path=path)
70
+ for tensor in model.get_input_details() + model.get_output_details():
71
+ print(tensor["name"], tensor["shape"], tensor["dtype"])
72
+ ```
73
+
74
+ ## Source
75
+
76
+ The [original Demucs implementation](https://github.com/facebookresearch/demucs)
77
+ and [HTDemucs paper](https://arxiv.org/abs/2211.08553) describe the model.
78
+ The [pinned ONNX export](https://huggingface.co/smank/htdemucs-onnx/tree/fb1fd25aa6260c253d86bbb4aa26b73aebc79f10)
79
+ is the source of both artifacts. See [LiteRT on Android](https://developers.google.com/edge/litert/android)
80
+ for runtime integration and [Soniqo Music](https://soniqo.audio/music) for the app.
htdemucs.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d2b401f322558cd57d67a752ed7be3fa55178a0626011eda8ac7bb74e17280c0
3
+ size 304321552
separator-fused-norm.tflite ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3b4aeed464fc0ce5cde49a3404d1cb47ff64a33bc51e88f6de98eaa0fa45d445
3
+ size 244796736