HY-2012 commited on
Commit
1dd61a9
·
verified ·
1 Parent(s): 6940904

AudioSR.AXERA AX650 models (UNet, VAE, split vocoder)

Browse files
.gitattributes CHANGED
@@ -33,3 +33,15 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ models/audiosr_unet.axmodel filter=lfs diff=lfs merge=lfs -text
37
+ models/audiosr_vae_decoder.axmodel filter=lfs diff=lfs merge=lfs -text
38
+ models/audiosr_vae_encoder_nhwc.axmodel filter=lfs diff=lfs merge=lfs -text
39
+ models/voc_pre_fp32.axmodel filter=lfs diff=lfs merge=lfs -text
40
+ models/voc_split_nhwc_post_fp32.axmodel filter=lfs diff=lfs merge=lfs -text
41
+ models/voc_split_nhwc_up2_mix.axmodel filter=lfs diff=lfs merge=lfs -text
42
+ models/voc_split_nhwc_up3_mix.axmodel filter=lfs diff=lfs merge=lfs -text
43
+ models/voc_split_nhwc_up4_mix.axmodel filter=lfs diff=lfs merge=lfs -text
44
+ models/voc_up0a_mix.axmodel filter=lfs diff=lfs merge=lfs -text
45
+ models/voc_up0b_mix.axmodel filter=lfs diff=lfs merge=lfs -text
46
+ models/voc_up1a_mix.axmodel filter=lfs diff=lfs merge=lfs -text
47
+ models/voc_up1b_mix.axmodel filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,127 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # AudioSR.AXERA
2
+
3
+ **AX650 / NPU3 语音超分:低采样率语音 → 48 kHz。完整链路 ONNX 拆分 + Pulsar2 混合量化,vocoder 板端相似度 0.9984,完整链路 vs PyTorch 0.9917。**
4
+
5
+ ![AX650](https://img.shields.io/badge/AX650-NPU3-blue) ![vocoder](https://img.shields.io/badge/vocoder%20corr-0.9984-brightgreen) ![full](https://img.shields.io/badge/full%20chain%20corr-0.9917-brightgreen) [![HuggingFace](https://img.shields.io/badge/%F0%9F%A4%97-AudioSR.AXERA-yellow)](https://huggingface.co/HY-2012/AudioSR.AXERA)
6
+
7
+ - HuggingFace(模型从这里下载):<https://huggingface.co/HY-2012/AudioSR.AXERA>
8
+ - GitHub:<https://github.com/ZY-2012/AudioSR_AXERA>
9
+ - 上游:<https://github.com/haoheliu/versatile_audio_super_resolution>
10
+ - 输入:8 / 16 / 24 / 48 kHz 单声道或立体声 WAV;输出:48 kHz
11
+ - 5.12 s 输入,AudioSR DDIM-10;vocoder 拆成 9 个小模型以适配 AX650 内存和精度
12
+
13
+ ## 目录
14
+
15
+ - 快速开始
16
+ - 结果
17
+ - 推理速度
18
+ - 模型
19
+ - 模型转换
20
+ - 数据来源
21
+ - License
22
+
23
+ ## 快速开始(AX650 板端)
24
+
25
+ 从 HuggingFace 下载整个仓库,或只下载 `models/`,放在包根目录旁:
26
+
27
+ ```bash
28
+ # 模型目录结构
29
+ # AudioSR_AXERA/
30
+ # └── models/
31
+ # ├── audiosr_unet.axmodel
32
+ # ├── audiosr_vae_encoder_nhwc.axmodel
33
+ # ├── audiosr_vae_decoder.axmodel
34
+ # └── voc_*.axmodel
35
+
36
+ pip install -r requirements.txt
37
+
38
+ python python/demo.py \
39
+ samples/listen/16k/speech_2_input_16k.wav \
40
+ -o output_48k.wav
41
+ ```
42
+
43
+ 输入 8 kHz / 16 kHz 均可;SDK 会重采样到 48 kHz,并按 AudioSR 的 5.12 s 窗口处理后送 NPU。
44
+
45
+ ## 工程结构
46
+
47
+ ```text
48
+ AudioSR_AXERA/
49
+ ├── model_convert/ # ONNX 导出 → 拆分 → 校准数据 → Pulsar2 U16 量化
50
+ ├── python/ # Python 板端推理 SDK(axengine)
51
+ ├── cpp/ # C++ 板端推理源码 + 交叉编译脚本
52
+ ├── models/ # 仅 README,量化模型从 HuggingFace 下载
53
+ ├── samples/ # 中英文示例输入
54
+ └── requirements.txt
55
+ ```
56
+
57
+ ## 结果
58
+
59
+ ### 精度
60
+
61
+ | 模块 / 链路 | 板端 vs ONNX/PyTorch |
62
+ |---|---:|
63
+ | VAE encoder(NHWC) | 0.99975 |
64
+ | UNet | 0.99989 |
65
+ | VAE decoder | 0.99999 |
66
+ | vocoder 9-model chain | **0.99838** |
67
+ | full AudioSR chain | **0.99169** |
68
+
69
+ ### 示例
70
+
71
+ | 输入 | 输出 |
72
+ |---|---|
73
+ | [English 16 kHz](samples/listen/16k/speech_2_input_16k.wav) | [48 kHz](samples/listen/output/speech_2_output_48k.wav) |
74
+ | [English 8 kHz](samples/listen/8k/OSR_us_000_0010_input_8k.wav) | 运行 SDK 生成 |
75
+ | [中文 8 kHz](samples/listen/8k/OSR_cn_000_0072_input_8k.wav) | 运行 SDK 生成 |
76
+
77
+ ## 推理速度
78
+
79
+ 5.12 s 输入、DDIM-10、Python SDK(AX650 NPU3):
80
+
81
+ ```text
82
+ 约 20–22 s / 5.12 s 音频,RTF ≈ 4.0–4.3
83
+ ```
84
+
85
+ 短音频主要受 DDIM 10 步 UNet 和 vocoder 拆分模型调度开销影响;这是当前 Python 参考实现的实测值。
86
+
87
+ ## 模型
88
+
89
+ 模型不放进 GitHub,统一从 HuggingFace 下载:
90
+
91
+ - `audiosr_unet.axmodel`:273 MB
92
+ - `audiosr_vae_encoder_nhwc.axmodel`:102 MB
93
+ - `audiosr_vae_decoder.axmodel`:143 MB
94
+ - `voc_pre_fp32.axmodel`:11 MB
95
+ - `voc_up0a_mix.axmodel`:55 MB
96
+ - `voc_up0b_mix.axmodel`:202 MB
97
+ - `voc_up1a_mix.axmodel`:12 MB
98
+ - `voc_up1b_mix.axmodel`:52 MB
99
+ - `voc_split_nhwc_up2_mix.axmodel`:17 MB
100
+ - `voc_split_nhwc_up3_mix.axmodel`:7.7 MB
101
+ - `voc_split_nhwc_up4_mix.axmodel`:5.8 MB
102
+ - `voc_split_nhwc_post_fp32.axmodel`:99 KB
103
+
104
+ 详见 [`models/README.md`](models/README.md)。
105
+
106
+ ## 模型转换
107
+
108
+ ```bash
109
+ # 1. 上游 AudioSR 代码 / checkpoint
110
+ # 2. 导出 VAE encoder / UNet / decoder / vocoder 子图
111
+ # 3. 校准数据
112
+ # 4. Pulsar2 8.0 量化
113
+ PULSAR2_IMAGE=docker-registry.aitsw.axera-tech.com/pulsar2:8.0 \
114
+ bash model_convert/compile.sh voc_pre_fp32 voc_up0a_mix ...
115
+ ```
116
+
117
+ 具体命令、BF16/U16/S8 配置和各阶段校准见 [`model_convert/README.md`](model_convert/README.md)。
118
+
119
+ ## 数据来源
120
+
121
+ - 上游 AudioSR:<https://github.com/haoheliu/versatile_audio_super_resolution>
122
+ - 示例英文 / 中文语音:Open Speech Repository、公开 16 kHz 人声样例
123
+ - 示例输出由本项目 AX650 Python SDK 生成
124
+
125
+ ## License
126
+
127
+ AudioSR 上游代码与模型 checkpoint 遵循其原始许可证;本仓库的转换脚本和 SDK 以 Apache-2.0 提供。模型权重请遵循上游许可证。
model_convert/audiosr_ddim_10.npz ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ef451a92d0b86fc9999e66fbaf99f5312a4676d54963e96a7b7387480559ee86
3
+ size 10074
models/README.md ADDED
@@ -0,0 +1,25 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Models
2
+
3
+ GitHub 仓库不包含 axmodel。请从 HuggingFace 下载:
4
+
5
+ <https://huggingface.co/HY-2012/AudioSR.AXERA/tree/main/models>
6
+
7
+ 需要的文件:
8
+
9
+ ```text
10
+ audiosr_unet.axmodel
11
+ audiosr_vae_encoder_nhwc.axmodel
12
+ audiosr_vae_decoder.axmodel
13
+ voc_pre_fp32.axmodel
14
+ voc_up0a_mix.axmodel
15
+ voc_up0b_mix.axmodel
16
+ voc_up1a_mix.axmodel
17
+ voc_up1b_mix.axmodel
18
+ voc_split_nhwc_up2_mix.axmodel
19
+ voc_split_nhwc_up3_mix.axmodel
20
+ voc_split_nhwc_up4_mix.axmodel
21
+ voc_split_nhwc_post_fp32.axmodel
22
+ ```
23
+
24
+ 所有模型均为 Pulsar2 8.0 / AX650 NPU3 编译。vocoder 采用拆分 + 混合精度:
25
+ `pre/post` 为 FP32,`up0/up1` 拆成两段,其余 up2/up3/up4 为混合精度。
models/audiosr_unet.axmodel ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bddeb3a64d6bc60ffa58e20e4d93640c19f3dfc2a28f8ce49f0648289fbf6fcb
3
+ size 285800456
models/audiosr_vae_decoder.axmodel ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4e0a8ea7a0415128860be4ba01347c5d693b3065b5e79195c4edaeced316fde5
3
+ size 149047475
models/audiosr_vae_encoder_nhwc.axmodel ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9d2cbb9dbbbf29f5e945adf3e96bd0395e6fd1a79c54634b52fd30fc542b71a5
3
+ size 106617540
models/voc_pre_fp32.axmodel ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:44efd5c09a3077eeb1c8a38da058ffb28150e791a94b89c63b5dd93bba7d899c
3
+ size 11040377
models/voc_split_nhwc_post_fp32.axmodel ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e29e568cb2d2cdbbeecd9eafdfe46bcc6e8fd46ed8511d69f32ae45a73bd0c07
3
+ size 101017
models/voc_split_nhwc_up2_mix.axmodel ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:884c8a57cad5a15dce9b87ffeecd7a6fa9b1325fe76dc3c78e2b695ab8e839ac
3
+ size 20096630
models/voc_split_nhwc_up3_mix.axmodel ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f7952e2ba46b30c6e9e3ae68f50f3fe1cb239147342b63e3f6b5331a2f38db45
3
+ size 8025046
models/voc_split_nhwc_up4_mix.axmodel ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:17c4576ea8252720c01188f1876e27a801eb987f101dffb0b68f77f409817ec2
3
+ size 6074768
models/voc_up0a_mix.axmodel ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:29dafebef2300121f4fea0afb9992cd373d09791296407895b64332b1ce11052
3
+ size 56930619
models/voc_up0b_mix.axmodel ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6861cc29b8247fc3cf3753e48352092bca47bc61e958f460623c5ff714b25f16
3
+ size 211010547
models/voc_up1a_mix.axmodel ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9e3762be4918dc25fcb99ba9807269bb8924b2bd8c7d18d645d98513858acb3d
3
+ size 11983860
models/voc_up1b_mix.axmodel ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:043c0efd16293aacd6f704a2bd06b733304c410f6f6de78b2fee367c47a0c3c8
3
+ size 53545172