Add files using upload-large-folder tool

.gitattributes

@@ -3,33 +3,25 @@
 *.bin filter=lfs diff=lfs merge=lfs -text
 *.bz2 filter=lfs diff=lfs merge=lfs -text
 *.ckpt filter=lfs diff=lfs merge=lfs -text
-*.ftz filter=lfs diff=lfs merge=lfs -text
 *.gz filter=lfs diff=lfs merge=lfs -text
 *.h5 filter=lfs diff=lfs merge=lfs -text
 *.joblib filter=lfs diff=lfs merge=lfs -text
-*.lfs.* filter=lfs diff=lfs merge=lfs -text
-*.mlmodel filter=lfs diff=lfs merge=lfs -text
 *.model filter=lfs diff=lfs merge=lfs -text
 *.msgpack filter=lfs diff=lfs merge=lfs -text
 *.npy filter=lfs diff=lfs merge=lfs -text
 *.npz filter=lfs diff=lfs merge=lfs -text
 *.onnx filter=lfs diff=lfs merge=lfs -text
-*.ot filter=lfs diff=lfs merge=lfs -text
 *.parquet filter=lfs diff=lfs merge=lfs -text
 *.pb filter=lfs diff=lfs merge=lfs -text
-*.pickle filter=lfs diff=lfs merge=lfs -text
-*.pkl filter=lfs diff=lfs merge=lfs -text
 *.pt filter=lfs diff=lfs merge=lfs -text
 *.pth filter=lfs diff=lfs merge=lfs -text
-*.rar filter=lfs diff=lfs merge=lfs -text
 *.safetensors filter=lfs diff=lfs merge=lfs -text
-saved_model/**/* filter=lfs diff=lfs merge=lfs -text
-*.tar.* filter=lfs diff=lfs merge=lfs -text
 *.tar filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
 *.tflite filter=lfs diff=lfs merge=lfs -text
-*.tgz filter=lfs diff=lfs merge=lfs -text
-*.wasm filter=lfs diff=lfs merge=lfs -text
-*.xz filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
-*tfevents* filter=lfs diff=lfs merge=lfs -text
+artifacts/ascend_310p/vits_flow_decoder_dynamic_fp16.om filter=lfs diff=lfs merge=lfs -text
+artifacts/ascend_310p/vits_flow_decoder_t639_fp16.om filter=lfs diff=lfs merge=lfs -text
+artifacts/ascend_310p/vits_flow_decoder_t681_fp16.om filter=lfs diff=lfs merge=lfs -text
+assets/snapshot_iter_150000.pdz filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,315 @@
+---
+license: apache-2.0
+pipeline_tag: text-to-speech
+language:
+- zh
+tags:
+- vits
+- text-to-speech
+- tts
+- csmsc
+- paddlespeech
+- chinese
+- robotics
+- IB-Robot
+- ascend
+- edge-deployment
+---
+
+# Model Card for VITS-CSMSC (IB-Robot, hybrid CPU + Ascend 310P)
+
+PaddleSpeech VITS-CSMSC — single-speaker Mandarin text-to-speech, 22.05 kHz —
+packaged for the [IB-Robot](https://atomgit.com/openeuler/IB_Robot) framework
+as a **hybrid deployment**: the text encoder and duration predictor run as ONNX
+on CPU, and the flow decoder (which contains the vocoder and dominates compute)
+runs as an `.om` on Ascend 310P1.
+
+This is the speech-output stage of the robot pipeline. Accepts arbitrary
+Chinese text; the voice is the fixed CSMSC speaker.
+
+## Deployments
+
+| deployment | backend | flow artifact | frames | notes |
+|---|---|---|---:|---|
+| `ascend_310p` | ONNX CPU + Ascend ACL | `vits_flow_decoder_dynamic_fp16.om` | variable | **recommended**, any utterance length |
+| `torch_cpu` | ONNX Runtime CPU | `artifacts/onnx/vits_flow_decoder.onnx` | variable | full CPU reference path |
+
+Two fixed-shape flow decoders also ship in the repo without their own
+deployment entry, matching the convention across this organisation that one
+hardware target carries one deployment:
+
+| file | frames | audio | notes |
+|---|---:|---:|---|
+| `vits_flow_decoder_t639_fp16.om` | 639 | 7.42 s | ~20 % lower flow latency than dynamic, 44 % of its size |
+| `vits_flow_decoder_t681_fp16.om` | 681 | 7.91 s | second fixed-shape build |
+
+Both are declared in `bundle.files` and load by pointing an ACL session at the
+file directly. They accept **exactly** their frame count, so an utterance has
+to be padded or trimmed to fit; the dynamic build takes any length, which is
+why it is the one wired to `ascend_310p`. Accuracy for all three is in
+[Validation](#validation) — they are within 1e-5 cosine of each other.
+
+The `ascend_310p` row is a **composite deployment** in the manifest: there is no
+single `runtime_profile`, because there is no single backend. `role_identities`
+and `role_runtime_profiles` name the backend per role — `onnx` on CPU for
+`text_encoder` and `duration`, `ascend`/`acl` for `flow_decoder`. `torch_cpu`
+runs all three roles on ONNX Runtime and so keeps one shared profile.
+
+The two host-side steps between duration and flow (`duration_expand_path`,
+`sample_z_p`) are listed in each deployment's `metadata.host_stages`; they are
+not model roles and have no artifact.
+
+Shared contract: `tensor_model/vits/synthesize`
+(`tts.phone_ids [1,-1] -> tts.audio [1,-1]`), preprocessing
+`paddlespeech-zh-frontend-jieba-pypinyin-phoneid-v1`, output
+`mono-float32-pcm-22050hz-v1`, logical revision
+`paddlespeech-vits-csmsc@onnx-export-v1`.
+
+`ascend_310p` uses the dynamic-shape flow decoder, so no padding or trimming
+is needed. For latency-sensitive use with a known utterance length, load one of
+the two fixed-shape builds above directly.
+
+## Pipeline
+
+```text
+Chinese text
+  → PaddleSpeech frontend / pypinyin      → phone ids
+  → text encoder     (ONNX, CPU)          → x, m_p, logs_p (192-d), x_mask
+  → duration predictor (ONNX, CPU)        → logw
+  → host: alignment expansion + z_p       (see below)
+  → flow decoder     (.om, Ascend 310P1)  → waveform
+  → 22.05 kHz mono wav
+```
+
+### Host stage (between duration and flow)
+
+This is not a thin glue layer — it is part of the model's semantics and must be
+reproduced exactly. See `assets/reference_impl/run_vits_hybrid_310p.py`:
+
+```python
+dur    = ceil(exp(logw) * float32(x_mask))  # per-phone frame counts
+path   = generate_path(dur, ...)           # [1, T_text, T_frames] alignment
+m      = (path.transpose @ m_p.transpose).transpose      # expand to frames
+logs   = (path.transpose @ logs_p.transpose).transpose
+y_mask = ones([1, 1, T_frames])
+z_p    = m + randn_like(m) * exp(logs) * 0.667           # noise_scale 0.667
+```
+
+The reference uses **seed 1234** for reproducibility; the sampling of `z_p` is
+genuinely stochastic, so runs with different seeds produce different (equally
+valid) audio, and bit-exact comparison against a reference requires pinning it.
+
+## Tensor contract
+
+```text
+text encoder   text        int64  [1, -1]      phone ids
+               text_lengths int64  [1]
+             → x           float32 [1, 192, -1]   outputs are positional and
+               m_p         float32 [1, 192, -1]   come back in this order
+               logs_p      float32 [1, 192, -1]
+               x_mask      bool    [1, 1, -1]
+
+duration       x           float32 [1, 192, -1]
+               x_mask      bool    [1, 1, -1]     bool, not float
+             → logw        float32 [1, 1, -1]
+
+flow decoder   z_p         float32 [1, 192, T]   T = 639 / 681 / -1
+               y_mask      float32 [1, 1, T]
+             → wav         float32 [1, T*256]    256 samples per frame
+```
+
+Sample rate 22050 Hz, hop 256 → one frame ≈ 11.61 ms.
+
+## Repository Structure
+
+- `inference_manifest.json` �� deployment routing (schema v3)
+- `assets/adapter.json` — adapter identity (`vits` / `synthesize`)
+- `assets/phone_id_map.txt` — 268-entry phone→id map used by the frontend
+- `assets/snapshot_iter_150000.pdz` — the upstream Paddle training checkpoint
+  (1.1 GB), byte-identical to the one in `vits_csmsc_ckpt_1.4.0.zip`
+- `assets/default.yaml` — that checkpoint's own training config
+- `assets/reference_impl/run_vits_hybrid_310p.py` — the hybrid runner, and the
+  normative source for the host stage above
+- `assets/reference_impl/synthesize_vits_310p.py` — text → wav CLI
+- `artifacts/onnx/*.onnx` — the three ONNX graphs (CPU stages + CPU reference)
+- `artifacts/ascend_310p/*.om` — three flow decoder builds
+
+## Source Model
+
+Upstream is **PaddleSpeech VITS-CSMSC**, released checkpoint
+**`vits_csmsc_ckpt_1.4.0`**. The manifest's logical revision is
+`paddlespeech-vits-csmsc@onnx-export-v1` — it names the ONNX export this bundle
+runs, not the upstream release tag, which is `1.4.0`.
+
+```bash
+wget https://paddlespeech.cdn.bcebos.com/Parakeet/released_models/vits/\
+vits_csmsc_ckpt_1.4.0.zip
+# 1,027,863,302 bytes
+# sha256 994340036777b24f3bf02967bf37f570d40e5514b54ff0001c5b4732921ae064
+```
+
+The archive holds exactly three files, and **all three are shipped here
+byte-for-byte** (sha256 verified against a fresh download and extraction):
+
+| shipped as | in archive | bytes | sha256 |
+|---|---|---:|---|
+| `assets/snapshot_iter_150000.pdz` | `snapshot_iter_150000.pdz` | 1,116,898,380 | `ef0faad8186cdd1aa278b61e0b2a3cb7dee9216d1b84f1bc760ec887bcd935c7` |
+| `assets/default.yaml` | `default.yaml` | 7,835 | `7eebdd044a663f2485b6abd54f6179e90da64c0c7c109778295f9a23be6dcbfc` |
+| `assets/phone_id_map.txt` | `phone_id_map.txt` | 2,128 | `ca00d60618a00a2dc1e42002faaa9024c9e5057e04b1e834325950111cee59c3` |
+
+`phone_id_map.txt` pins the phone inventory, and therefore the text frontend
+contract, to this exact release. `default.yaml` is the checkpoint's own training
+config and independently confirms the numbers this bundle's tensor contract
+depends on: `fs: 22050`, `n_shift: 256` (the 256 samples per frame above) and
+`hidden_channels: 192` (the 192-d `z_p`).
+
+Shipping the checkpoint follows the convention in the rest of this
+organisation, where the upstream weights live under `assets/`. Note that it is
+**not** the artifact any deployment loads — every deployment runs the ONNX
+graphs below. It is there so the bundle carries its own provenance.
+
+The three ONNX graphs were exported from that checkpoint in separate passes,
+which is why their opsets differ (12 / 13 / 11):
+
+| file | bytes | opset | sha256 |
+|---|---:|---:|---|
+| `vits_text_encoder_clean.onnx` | 55,329,745 | 12 | `15c5fb01538ce38c4748c9bc5892fc0af9bee7c65b228cd116651237a3f72536` |
+| `vits_duration.onnx` | 2,644,862 | 13 | `f75fd325801a62976ecea0f52ba03a384197f574fedf0486598e5c0fc840c89c` |
+| `vits_flow_decoder.onnx` | 86,596,889 | 11 | `3cb9dfe036b843abce71b867626f23d60745c0f0f35f29787ecec7fbfaf0f7fa` |
+| `vits_flow_decoder_t639_fp16.om` | 91,708,818 | — | `b7da515895536b7baf75314b908344164efb916ffc8cc2246c6dac0e4c4cf91b` |
+| `vits_flow_decoder_t681_fp16.om` | 91,706,422 | — | `df192baf48cefb732f52945cd411d3c88bbd310cac784dc51e11a8a7ed0d2b61` |
+| `vits_flow_decoder_dynamic_fp16.om` | 210,986,822 | — | `c8b26160a0bc7e4de987424fab228bcc8dfd3fb053bd910737443999412c40d8` |
+
+Shipping the checkpoint does **not** make the ONNX byte-reproducible from it:
+the export script is not part of this bundle, and PaddleSpeech static-model
+export behaviour changed with PaddlePaddle ≥ 2.5. What *is* established is the
+accuracy of the export — cosine `0.9999999999998159` against the Paddle
+reference, below — plus the exact checkpoint and config it came from.
+
+## Validation
+
+All measured on a real Ascend 310P1, against ONNX Runtime CPU fp32.
+
+**ONNX export vs Paddle reference** (text encoder + duration) — the export
+itself is lossless:
+
+```text
+cosine   0.9999999999998159
+max_abs  4.291534423828125e-06
+```
+
+**Flow decoder OM vs ONNX**, on the flow subgraph alone:
+
+| build | cosine | max_abs | latency mean | p95 | subgraph RTF |
+|---|---:|---:|---:|---:|---:|
+| t639 | 0.9999842557218173 | 0.01094835 | 96.17 ms | 96.38 ms | 0.01296 |
+| t681 | 0.9999692545131995 | 0.01450025 | 102.72 ms | — | 0.01299 |
+
+Those are the conversion-time numbers. They have since been **independently
+reproduced on the shipped artifacts** — same board, same `artifacts/onnx/
+vits_flow_decoder.onnx` reference, but with `z_p` built through the real
+encoder → duration → host-alignment path rather than synthetic tensors
+(`_tools/board/verify_vits_flow_om_vs_onnx.py`):
+
+| build | frames | cosine | max_abs | rel_l2 | latency mean | p95 | subgraph RTF |
+|---|---:|---:|---:|---:|---:|---:|---:|
+| t639 | 639 | 0.99998847 | 0.008860 | 0.00480 | 95.89 ms | 96.00 ms | 0.01292 |
+| t681 | 681 | 0.99998923 | 0.008666 | 0.00464 | 102.47 ms | 102.71 ms | 0.01296 |
+| dynamic | 399 | 0.99998542 | 0.006929 | 0.00540 | 77.32 ms | 77.90 ms | 0.01669 |
+
+Latency matches the conversion report to within 0.3 ms on every build, and
+accuracy comes out slightly *better* on realistic `z_p` than on the synthetic
+input used at conversion time. The dynamic build's higher RTF at 399 frames is
+the fixed per-call overhead spread over less audio, not a slower kernel — at
+639 frames it lands within ~40 ms of the fixed build (see below).
+
+**End-to-end hybrid pipeline**, 20 runs, 7.42 s of output audio. Measured with
+the fixed **t639** build, which is what the reference runner loads
+(`assets/reference_impl/run_vits_hybrid_310p.py:15`); the same pipeline on the
+`ascend_310p` dynamic build is the `baseline` row two tables down, 367.63 ms:
+
+```text
+cosine_vs_onnx   0.9999853236698978
+max_abs          0.008778512477874756
+latency  mean    327.38 ms   p50 323.81 ms   p95 355.89 ms
+RTF              0.0441
+```
+
+Per-stage breakdown (mean, ms):
+
+| stage | backend | ms | share |
+|---|---|---:|---:|
+| text encoder | ONNX CPU | 48.71 | 15% |
+| duration | ONNX CPU | 25.38 | 8% |
+| host alignment + z_p | numpy CPU | 156.64 | 48% |
+| flow decoder | Ascend OM | 96.53 | 29% |
+
+The host alignment stage is now the single largest cost — larger than the NPU
+work it feeds. It is pure numpy (`generate_path` plus two matmuls) and is the
+obvious next optimisation target; moving it onto the NPU or vectorising it
+would cut end-to-end latency by roughly a third.
+
+**Dynamic build, variable text length:**
+
+| sample | phones | frames | audio | latency mean | RTF |
+|---|---:|---:|---:|---:|---:|
+| baseline | 101 | 639 | 7.4188 s | 367.63 ms | 0.04955 |
+| new_text | 144 | 915 | 10.6231 s | 414.97 ms | 0.03906 |
+
+Latency grows sub-linearly with length, so RTF improves on longer utterances.
+The dynamic build costs ~40 ms over the fixed t639 build at the same length.
+
+## Usage
+
+```python
+from inference_manifest import load_inference_manifest
+
+validated = load_inference_manifest("models/vits-csmsc", "ascend_310p")
+```
+
+The hybrid path needs both an ONNX Runtime CPU session (encoder, duration) and
+an ACL session (flow), plus the host stage between them — see
+`assets/reference_impl/run_vits_hybrid_310p.py`, which is the normative
+implementation.
+
+## Known Limitations
+
+- **Not a pure-NPU TTS.** Text encoder and duration remain on CPU; only the
+  flow decoder is compiled to OM.
+- **Chinese only.** English and other languages are not a validation target.
+- **Fixed speaker.** Single-speaker CSMSC voice, no speaker conditioning.
+- **The two extra fixed-shape OMs require exact frame counts** (639 / 681).
+  The `ascend_310p` deployment uses the dynamic build and has no such limit;
+  the fixed builds are for callers who know their length and want the ~40 ms.
+- Output is stochastic (`noise_scale = 0.667`) unless the RNG seed is pinned.
+
+## Environment
+
+Validated on: Ascend 310P1 with CANN and `ais_bench`, Python 3.11.6,
+ONNX Runtime 1.23.2.
+
+## License
+
+Code and packaging: Apache-2.0. Model weights follow the upstream
+[PaddleSpeech](https://github.com/PaddlePaddle/PaddleSpeech) license
+(Apache-2.0); the CSMSC corpus is subject to its own terms.
+
+## Citation
+
+    @inproceedings{vits2021,
+      title     = {Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech},
+      author    = {Kim, Jaehyeon and Kong, Jungil and Son, Juhee},
+      booktitle = {arXiv preprint arXiv:2106.06103},
+      year      = {2021}
+    }
+    @article{paddlespeech2022,
+      title   = {PaddleSpeech: An Easy-to-Use All-in-One Speech Toolkit},
+      author  = {Zhang, Hui and Yuan, Tian and Chen, Junkun and Li, Xintong and Zheng, Renjie and Huang, Yuxin and Chen, Xiaojie and Gong, Enlei and Chen, Zeyu and Hu, Xiaoguang and Yu, Dianhai and Ma, Yanjun and Huang, Liang},
+      journal = {arXiv preprint arXiv:2205.12007},
+      year    = {2022}
+    }
+    @software{ib_robot,
+      title   = {IB-Robot: Intelligence Boom Robot},
+      url     = {https://atomgit.com/openeuler/IB_Robot},
+      license = {Apache-2.0}
+    }

artifacts/ascend_310p/vits_flow_decoder_dynamic_fp16.om

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c8b26160a0bc7e4de987424fab228bcc8dfd3fb053bd910737443999412c40d8
+size 210986822

artifacts/ascend_310p/vits_flow_decoder_t639_fp16.om

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b7da515895536b7baf75314b908344164efb916ffc8cc2246c6dac0e4c4cf91b
+size 91708818

artifacts/ascend_310p/vits_flow_decoder_t681_fp16.om

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:df192baf48cefb732f52945cd411d3c88bbd310cac784dc51e11a8a7ed0d2b61
+size 91706422

artifacts/onnx/vits_duration.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f75fd325801a62976ecea0f52ba03a384197f574fedf0486598e5c0fc840c89c
+size 2644862

artifacts/onnx/vits_flow_decoder.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3cb9dfe036b843abce71b867626f23d60745c0f0f35f29787ecec7fbfaf0f7fa
+size 86596889

artifacts/onnx/vits_text_encoder_clean.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:15c5fb01538ce38c4748c9bc5892fc0af9bee7c65b228cd116651237a3f72536
+size 55329745

assets/adapter.json

@@ -0,0 +1,16 @@
+{
+  "checkpoint": "onnx/vits_flow_decoder.onnx",
+  "checkpoint_sha256": "3cb9dfe036b843abce71b867626f23d60745c0f0f35f29787ecec7fbfaf0f7fa",
+  "hop_length": 256,
+  "interface": "tensor_model",
+  "logical_model_revision": "paddlespeech-vits-csmsc@onnx-export-v1",
+  "model_type": "vits",
+  "noise_scale": 0.667,
+  "noise_scale_dur": 0.0,
+  "operation": "synthesize",
+  "phone_id_map": "phone_id_map.txt",
+  "postprocessing": "mono-float32-pcm-22050hz-v1",
+  "preprocessing": "paddlespeech-zh-frontend-jieba-pypinyin-phoneid-v1",
+  "sample_rate": 22050,
+  "speaker": "csmsc-single-speaker"
+}

assets/default.yaml

@@ -0,0 +1,185 @@
+# This configuration tested on 4 GPUs (V100) with 32GB GPU
+# memory. It takes around 2 weeks to finish the training
+# but 100k iters model should generate reasonable results.
+###########################################################
+#                FEATURE EXTRACTION SETTING               #
+###########################################################
+
+fs: 22050         # sr
+n_fft: 1024        # FFT size (samples).
+n_shift: 256       # Hop size (samples). 12.5ms
+win_length: null   # Window length (samples). 50ms
+                   # If set to null, it will be the same as fft_size.
+window: "hann"     # Window function.
+
+
+##########################################################
+#                  TTS MODEL SETTING                     #
+##########################################################
+model:
+    # generator related
+    generator_type: vits_generator
+    generator_params:
+        hidden_channels: 192
+        spks: -1
+        global_channels: -1
+        segment_size: 32
+        text_encoder_attention_heads: 2
+        text_encoder_ffn_expand: 4
+        text_encoder_blocks: 6
+        text_encoder_positionwise_layer_type: "conv1d"
+        text_encoder_positionwise_conv_kernel_size: 3
+        text_encoder_positional_encoding_layer_type: "rel_pos"
+        text_encoder_self_attention_layer_type: "rel_selfattn"
+        text_encoder_activation_type: "swish"
+        text_encoder_normalize_before: True
+        text_encoder_dropout_rate: 0.1
+        text_encoder_positional_dropout_rate: 0.0
+        text_encoder_attention_dropout_rate: 0.1
+        use_macaron_style_in_text_encoder: True
+        use_conformer_conv_in_text_encoder: False
+        text_encoder_conformer_kernel_size: -1
+        decoder_kernel_size: 7
+        decoder_channels: 512
+        decoder_upsample_scales: [8, 8, 2, 2]
+        decoder_upsample_kernel_sizes: [16, 16, 4, 4]
+        decoder_resblock_kernel_sizes: [3, 7, 11]
+        decoder_resblock_dilations: [[1, 3, 5], [1, 3, 5], [1, 3, 5]]
+        use_weight_norm_in_decoder: True
+        posterior_encoder_kernel_size: 5
+        posterior_encoder_layers: 16
+        posterior_encoder_stacks: 1
+        posterior_encoder_base_dilation: 1
+        posterior_encoder_dropout_rate: 0.0
+        use_weight_norm_in_posterior_encoder: True
+        flow_flows: 4
+        flow_kernel_size: 5
+        flow_base_dilation: 1
+        flow_layers: 4
+        flow_dropout_rate: 0.0
+        use_weight_norm_in_flow: True
+        use_only_mean_in_flow: True
+        stochastic_duration_predictor_kernel_size: 3
+        stochastic_duration_predictor_dropout_rate: 0.5
+        stochastic_duration_predictor_flows: 4
+        stochastic_duration_predictor_dds_conv_layers: 3
+    # discriminator related
+    discriminator_type: hifigan_multi_scale_multi_period_discriminator
+    discriminator_params:
+        scales: 1
+        scale_downsample_pooling: "AvgPool1D"
+        scale_downsample_pooling_params:
+            kernel_size: 4
+            stride: 2
+            padding: 2
+        scale_discriminator_params:
+            in_channels: 1
+            out_channels: 1
+            kernel_sizes: [15, 41, 5, 3]
+            channels: 128
+            max_downsample_channels: 1024
+            max_groups: 16
+            bias: True
+            downsample_scales: [2, 2, 4, 4, 1]
+            nonlinear_activation: "leakyrelu"
+            nonlinear_activation_params:
+                negative_slope: 0.1
+            use_weight_norm: True
+            use_spectral_norm: False
+        follow_official_norm: False
+        periods: [2, 3, 5, 7, 11]
+        period_discriminator_params:
+            in_channels: 1
+            out_channels: 1
+            kernel_sizes: [5, 3]
+            channels: 32
+            downsample_scales: [3, 3, 3, 3, 1]
+            max_downsample_channels: 1024
+            bias: True
+            nonlinear_activation: "leakyrelu"
+            nonlinear_activation_params:
+                negative_slope: 0.1
+            use_weight_norm: True
+            use_spectral_norm: False
+    # others
+    sampling_rate: 22050          # needed in the inference for saving wav
+    cache_generator_outputs: True # whether to cache generator outputs in the training
+          
+###########################################################
+#                        LOSS SETTING                     #
+###########################################################
+# loss function related
+generator_adv_loss_params:
+    average_by_discriminators: False # whether to average loss value by #discriminators
+    loss_type: mse                   # loss type, "mse" or "hinge"
+discriminator_adv_loss_params:
+    average_by_discriminators: False # whether to average loss value by #discriminators
+    loss_type: mse                   # loss type, "mse" or "hinge"
+feat_match_loss_params:
+    average_by_discriminators: False # whether to average loss value by #discriminators
+    average_by_layers: False         # whether to average loss value by #layers of each discriminator
+    include_final_outputs: True      # whether to include final outputs for loss calculation
+mel_loss_params:
+    fs: 22050          # must be the same as the training data
+    fft_size: 1024        # fft points
+    hop_size: 256    # hop size
+    win_length: null   # window length
+    window: hann       # window type
+    num_mels: 80         # number of Mel basis
+    fmin: 0            # minimum frequency for Mel basis
+    fmax: null         # maximum frequency for Mel basis
+    log_base: null     # null represent natural log
+
+###########################################################
+#               ADVERSARIAL LOSS SETTING                  #
+###########################################################
+lambda_adv: 1.0        # loss scaling coefficient for adversarial loss
+lambda_mel: 45.0       # loss scaling coefficient for Mel loss
+lambda_feat_match: 2.0 # loss scaling coefficient for feat match loss
+lambda_dur: 1.0        # loss scaling coefficient for duration loss
+lambda_kl: 1.0         # loss scaling coefficient for KL divergence loss
+# others
+sampling_rate: 22050          # needed in the inference for saving wav
+cache_generator_outputs: True # whether to cache generator outputs in the training
+
+
+###########################################################
+#                  DATA LOADER SETTING                    #
+###########################################################
+batch_size: 64              # Batch size.
+num_workers: 4              # Number of workers in DataLoader.
+
+##########################################################
+#            OPTIMIZER & SCHEDULER SETTING               #
+##########################################################
+# optimizer setting for generator
+generator_optimizer_params:
+    beta1: 0.8
+    beta2: 0.99
+    epsilon: 1.0e-9
+    weight_decay: 0.0
+generator_scheduler: exponential_decay
+generator_scheduler_params:
+    learning_rate: 2.0e-4
+    gamma: 0.999875                   
+
+# optimizer setting for discriminator
+discriminator_optimizer_params:
+    beta1: 0.8
+    beta2: 0.99
+    epsilon: 1.0e-9
+    weight_decay: 0.0
+discriminator_scheduler: exponential_decay
+discriminator_scheduler_params:
+    learning_rate: 2.0e-4          
+    gamma: 0.999875
+generator_first: False # whether to start updating generator first
+
+##########################################################
+#                OTHER TRAINING SETTING                  #
+##########################################################
+num_snapshots: 30            # max number of snapshots to keep while training
+train_max_steps: 150000      # Number of training steps. == total_iters / ngpus, total_iters = 1000000
+save_interval_steps: 1000    # Interval steps to save checkpoint.
+eval_interval_steps: 250     # Interval steps to evaluate the network.
+seed: 777                    # random seed number

assets/phone_id_map.txt

@@ -0,0 +1,268 @@
+<pad> 0
+<unk> 1
+a1 2
+a2 3
+a3 4
+a4 5
+a5 6
+ai1 7
+ai2 8
+ai3 9
+ai4 10
+ai5 11
+air2 12
+air4 13
+an1 14
+an2 15
+an3 16
+an4 17
+an5 18
+ang1 19
+ang2 20
+ang3 21
+ang4 22
+ang5 23
+anr1 24
+anr3 25
+anr4 26
+ao1 27
+ao2 28
+ao3 29
+ao4 30
+ao5 31
+aor3 32
+aor4 33
+ar2 34
+ar3 35
+ar4 36
+b 37
+c 38
+ch 39
+d 40
+e1 41
+e2 42
+e3 43
+e4 44
+e5 45
+ei1 46
+ei2 47
+ei3 48
+ei4 49
+ei5 50
+en1 51
+en2 52
+en3 53
+en4 54
+en5 55
+eng1 56
+eng2 57
+eng3 58
+eng4 59
+eng5 60
+enr1 61
+enr2 62
+enr4 63
+enr5 64
+er2 65
+er3 66
+er4 67
+er5 68
+f 69
+g 70
+h 71
+i1 72
+i2 73
+i3 74
+i4 75
+i5 76
+ia1 77
+ia2 78
+ia3 79
+ia4 80
+ia5 81
+ian1 82
+ian2 83
+ian3 84
+ian4 85
+ian5 86
+iang1 87
+iang2 88
+iang3 89
+iang4 90
+iang5 91
+iangr4 92
+ianr1 93
+ianr2 94
+ianr3 95
+iao1 96
+iao2 97
+iao3 98
+iao4 99
+iao5 100
+iar1 101
+iar3 102
+ie1 103
+ie2 104
+ie3 105
+ie4 106
+ie5 107
+ii1 108
+ii2 109
+ii3 110
+ii4 111
+ii5 112
+iii1 113
+iii2 114
+iii3 115
+iii4 116
+iii5 117
+iiir4 118
+iir2 119
+in1 120
+in2 121
+in3 122
+in4 123
+in5 124
+ing1 125
+ing2 126
+ing3 127
+ing4 128
+ing5 129
+ingr2 130
+ingr3 131
+inr4 132
+io1 133
+io5 134
+iong1 135
+iong2 136
+iong3 137
+iong4 138
+iong5 139
+iou1 140
+iou2 141
+iou3 142
+iou4 143
+iou5 144
+iour1 145
+ir1 146
+ir2 147
+ir3 148
+ir4 149
+ir5 150
+j 151
+k 152
+l 153
+m 154
+n 155
+o1 156
+o2 157
+o3 158
+o4 159
+o5 160
+ong1 161
+ong2 162
+ong3 163
+ong4 164
+ong5 165
+ongr4 166
+ou1 167
+ou2 168
+ou3 169
+ou4 170
+ou5 171
+our2 172
+p 173
+q 174
+r 175
+s 176
+sh 177
+sil 178
+sp 179
+spl 180
+spn 181
+t 182
+u1 183
+u2 184
+u3 185
+u4 186
+u5 187
+ua1 188
+ua2 189
+ua3 190
+ua4 191
+ua5 192
+uai1 193
+uai2 194
+uai3 195
+uai4 196
+uai5 197
+uair4 198
+uan1 199
+uan2 200
+uan3 201
+uan4 202
+uan5 203
+uang1 204
+uang2 205
+uang3 206
+uang4 207
+uang5 208
+uanr1 209
+uanr2 210
+uei1 211
+uei2 212
+uei3 213
+uei4 214
+uei5 215
+ueir1 216
+ueir3 217
+ueir4 218
+uen1 219
+uen2 220
+uen3 221
+uen4 222
+uen5 223
+ueng1 224
+ueng2 225
+ueng3 226
+ueng4 227
+uenr3 228
+uenr4 229
+uo1 230
+uo2 231
+uo3 232
+uo4 233
+uo5 234
+uor2 235
+uor3 236
+ur3 237
+ur4 238
+v1 239
+v2 240
+v3 241
+v4 242
+v5 243
+van1 244
+van2 245
+van3 246
+van4 247
+van5 248
+vanr4 249
+ve1 250
+ve2 251
+ve3 252
+ve4 253
+ve5 254
+vn1 255
+vn2 256
+vn3 257
+vn4 258
+vn5 259
+x 260
+z 261
+zh 262
+， 263
+。 264
+？ 265
+！ 266
+<eos> 267

assets/reference_impl/run_vits_hybrid_310p.py

@@ -0,0 +1,126 @@
+#!/usr/bin/env python3
+import json
+import time
+from pathlib import Path
+
+import numpy as np
+import onnxruntime as ort
+import soundfile as sf
+from ais_bench.infer.interface import InferSession
+
+
+ROOT = Path("/root/workspace/VITS")
+ENCODER = ROOT / "output/onnx/vits_text_encoder_clean.onnx"
+DURATION = ROOT / "output/onnx/vits_duration.onnx"
+FLOW_OM = ROOT / "output/om/vits_flow_decoder_t639_fp16.om"
+GOLDEN = ROOT / "golden/hybrid_chain_golden.npz"
+WAV = ROOT / "output/audio/vits_hybrid_310p_noise_dur0_seed1234.wav"
+REPORT = ROOT / "manifest/vits_hybrid_310p_report.json"
+
+
+def generate_path(durations):
+    durations = np.asarray(durations, dtype=np.int64)
+    total = int(durations.sum())
+    path = np.zeros((1, 1, total, durations.shape[-1]), dtype=np.float32)
+    offset = 0
+    for index, count in enumerate(durations.reshape(-1)):
+        next_offset = offset + int(count)
+        path[0, 0, offset:next_offset, index] = 1.0
+        offset = next_offset
+    return path
+
+
+def percentile(values, q):
+    return float(np.percentile(np.asarray(values, dtype=np.float64), q))
+
+
+golden = np.load(GOLDEN)
+phone_ids = np.ascontiguousarray(golden["phone_ids"].astype(np.int64))
+text_lengths = np.ascontiguousarray(golden["text_lengths"].astype(np.int64))
+load_started = time.perf_counter()
+encoder = ort.InferenceSession(str(ENCODER), providers=["CPUExecutionProvider"])
+duration = ort.InferenceSession(str(DURATION), providers=["CPUExecutionProvider"])
+flow = InferSession(0, str(FLOW_OM))
+load_seconds = time.perf_counter() - load_started
+
+
+def infer_once():
+    stage = {}
+    started = time.perf_counter()
+    x, m_p, logs_p, x_mask = encoder.run(
+        None, {"text": phone_ids, "text_lengths": text_lengths}
+    )
+    stage["encoder"] = time.perf_counter() - started
+    started = time.perf_counter()
+    logw = duration.run(
+        None, {"x": x.astype(np.float32), "x_mask": x_mask.astype(bool)}
+    )[0]
+    stage["duration"] = time.perf_counter() - started
+    started = time.perf_counter()
+    dur = np.ceil(np.exp(logw) * x_mask.astype(np.float32)).astype(np.int64)
+    path = generate_path(dur)
+    m_expanded = np.matmul(path[:, 0], np.transpose(m_p, (0, 2, 1)))
+    m_expanded = np.transpose(m_expanded, (0, 2, 1)).astype(np.float32)
+    logs_expanded = np.matmul(path[:, 0], np.transpose(logs_p, (0, 2, 1)))
+    logs_expanded = np.transpose(logs_expanded, (0, 2, 1)).astype(np.float32)
+    y_mask = np.ones((1, 1, m_expanded.shape[-1]), dtype=np.float32)
+    noise = np.random.RandomState(1234).standard_normal(m_expanded.shape).astype(np.float32)
+    z_p = np.ascontiguousarray(
+        m_expanded + noise * np.exp(logs_expanded) * np.float32(0.667)
+    )
+    stage["host"] = time.perf_counter() - started
+    if z_p.shape[-1] != 639:
+        raise RuntimeError(f"Expected 639 frames for this fixed OM, got {z_p.shape[-1]}")
+    started = time.perf_counter()
+    wav = flow.infer([z_p, y_mask], out_array=True)[0]
+    stage["flow_om"] = time.perf_counter() - started
+    return np.asarray(wav, dtype=np.float32).reshape(-1), stage
+
+
+for _ in range(3):
+    infer_once()
+elapsed = []
+stages = {"encoder": [], "duration": [], "host": [], "flow_om": []}
+wav = None
+for _ in range(20):
+    started = time.perf_counter()
+    wav, stage = infer_once()
+    elapsed.append(time.perf_counter() - started)
+    for name, value in stage.items():
+        stages[name].append(value)
+
+sf.write(WAV, wav, 22050)
+audio_seconds = wav.size / 22050.0
+reference = golden["wav"].reshape(-1).astype(np.float64)
+candidate = wav.astype(np.float64)
+diff = np.abs(reference - candidate)
+report = {
+    "python": "/root/IB_Robot/venv/bin/python3",
+    "models": {
+        "text_encoder": str(ENCODER),
+        "duration": str(DURATION),
+        "flow_om": str(FLOW_OM),
+    },
+    "load_seconds": load_seconds,
+    "runs": len(elapsed),
+    "audio_seconds": audio_seconds,
+    "duration_frames": int(wav.size // 256),
+    "latency_ms_mean": float(np.mean(elapsed) * 1000),
+    "latency_ms_p50": percentile(elapsed, 50) * 1000,
+    "latency_ms_p95": percentile(elapsed, 95) * 1000,
+    "rtf_mean": float(np.mean(elapsed) / audio_seconds),
+    "stages_ms_mean": {
+        name: float(np.mean(values) * 1000) for name, values in stages.items()
+    },
+    "finite": bool(np.isfinite(wav).all()),
+    "cosine_vs_onnx": float(
+        np.dot(reference, candidate)
+        / (np.linalg.norm(reference) * np.linalg.norm(candidate))
+    ),
+    "max_abs_vs_onnx": float(diff.max()),
+    "mean_abs_vs_onnx": float(diff.mean()),
+    "rms": float(np.sqrt(np.mean(candidate * candidate))),
+    "wav": str(WAV),
+}
+REPORT.write_text(json.dumps(report, indent=2), encoding="utf-8")
+print(json.dumps(report, indent=2))

assets/reference_impl/synthesize_vits_310p.py

@@ -0,0 +1,154 @@
+#!/usr/bin/env python3
+import argparse
+import json
+import os
+import sys
+import time
+from pathlib import Path
+
+ROOT = Path("/root/workspace/VITS")
+os.environ.setdefault("TMPDIR", str(ROOT / "tmp"))
+sys.path.insert(0, str(ROOT / "vendor"))
+
+import numpy as np
+import onnxruntime as ort
+import soundfile as sf
+from ais_bench.infer.interface import InferSession
+from paddlespeech.t2s.frontend.zh_frontend import Frontend
+
+
+def generate_path(durations):
+    durations = np.asarray(durations, dtype=np.int64)
+    total = int(durations.sum())
+    path = np.zeros((1, 1, total, durations.shape[-1]), dtype=np.float32)
+    offset = 0
+    for index, count in enumerate(durations.reshape(-1)):
+        next_offset = offset + int(count)
+        path[0, 0, offset:next_offset, index] = 1.0
+        offset = next_offset
+    return path
+
+
+def main():
+    parser = argparse.ArgumentParser(description="PaddleSpeech VITS-CSMSC hybrid inference on Ascend 310P")
+    parser.add_argument("--text", required=True)
+    parser.add_argument("--output", default=str(ROOT / "output/audio/vits_310p.wav"))
+    parser.add_argument("--seed", type=int, default=1234)
+    parser.add_argument("--runs", type=int, default=5)
+    args = parser.parse_args()
+    encoder_path = ROOT / "output/onnx/vits_text_encoder_clean.onnx"
+    duration_path = ROOT / "output/onnx/vits_duration.onnx"
+    dynamic_om = ROOT / "output/om/vits_flow_decoder_dynamic_fp16_linux_aarch64.om"
+    fixed_om = ROOT / "output/om/vits_flow_decoder_t639_fp16.om"
+    om_path = dynamic_om if dynamic_om.exists() else fixed_om
+
+    frontend_started = time.perf_counter()
+    frontend = Frontend(
+        g2p_model="pypinyin",
+        phone_vocab_path=str(ROOT / "models/phone_id_map.txt"),
+    )
+    phone_ids = frontend.get_input_ids(
+        args.text, merge_sentences=True, add_blank=True
+    )["phone_ids"][0].astype(np.int64)[None]
+    frontend_seconds = time.perf_counter() - frontend_started
+    text_lengths = np.array([phone_ids.shape[1]], dtype=np.int64)
+
+    load_started = time.perf_counter()
+    encoder = ort.InferenceSession(str(encoder_path), providers=["CPUExecutionProvider"])
+    duration = ort.InferenceSession(str(duration_path), providers=["CPUExecutionProvider"])
+    flow = InferSession(0, str(om_path))
+    load_seconds = time.perf_counter() - load_started
+
+    def infer_once():
+        stage = {}
+        started = time.perf_counter()
+        x, m_p, logs_p, x_mask = encoder.run(
+            None, {"text": phone_ids, "text_lengths": text_lengths}
+        )
+        stage["text_encoder"] = time.perf_counter() - started
+        started = time.perf_counter()
+        logw = duration.run(
+            None, {"x": x.astype(np.float32), "x_mask": x_mask.astype(bool)}
+        )[0]
+        stage["duration"] = time.perf_counter() - started
+        started = time.perf_counter()
+        dur = np.ceil(np.exp(logw) * x_mask.astype(np.float32)).astype(np.int64)
+        path = generate_path(dur)
+        m_expanded = np.matmul(path[:, 0], np.transpose(m_p, (0, 2, 1)))
+        m_expanded = np.transpose(m_expanded, (0, 2, 1)).astype(np.float32)
+        logs_expanded = np.matmul(path[:, 0], np.transpose(logs_p, (0, 2, 1)))
+        logs_expanded = np.transpose(logs_expanded, (0, 2, 1)).astype(np.float32)
+        y_mask = np.ones((1, 1, m_expanded.shape[-1]), dtype=np.float32)
+        noise = np.random.RandomState(args.seed).standard_normal(m_expanded.shape).astype(np.float32)
+        z_p = np.ascontiguousarray(
+            m_expanded + noise * np.exp(logs_expanded) * np.float32(0.667)
+        )
+        y_mask = np.ascontiguousarray(y_mask)
+        stage["host"] = time.perf_counter() - started
+        if om_path == fixed_om and z_p.shape[-1] != 639:
+            raise RuntimeError(
+                f"Dynamic OM is not ready; fixed OM accepts 639 frames, this text produced {z_p.shape[-1]}"
+            )
+        started = time.perf_counter()
+        if om_path == dynamic_om:
+            wav = flow.infer(
+                [z_p, y_mask], mode="dymshape", custom_sizes=20_000_000, out_array=True
+            )[0]
+        else:
+            wav = flow.infer([z_p, y_mask], out_array=True)[0]
+        stage["flow_om"] = time.perf_counter() - started
+        return np.asarray(wav, dtype=np.float32).reshape(-1), int(dur.sum()), stage
+
+    for _ in range(2):
+        infer_once()
+    totals = []
+    stages = {"text_encoder": [], "duration": [], "host": [], "flow_om": []}
+    wav = None
+    frames = None
+    for _ in range(args.runs):
+        started = time.perf_counter()
+        wav, frames, stage = infer_once()
+        totals.append(time.perf_counter() - started)
+        for name, value in stage.items():
+            stages[name].append(value)
+
+    output = Path(args.output)
+    if not output.is_absolute():
+        output = ROOT / "output/audio" / output
+    output = output.resolve()
+    allowed = (ROOT / "output/audio").resolve()
+    if output != allowed and allowed not in output.parents:
+        raise ValueError(f"Output must stay under {allowed}")
+    output.parent.mkdir(parents=True, exist_ok=True)
+    sf.write(output, wav, 22050)
+    audio_seconds = wav.size / 22050.0
+    report = {
+        "text": args.text,
+        "phone_count": int(phone_ids.shape[1]),
+        "duration_frames": frames,
+        "sample_rate": 22050,
+        "audio_seconds": audio_seconds,
+        "frontend_seconds": frontend_seconds,
+        "load_seconds": load_seconds,
+        "warm_runs": args.runs,
+        "latency_ms_mean": float(np.mean(totals) * 1000),
+        "latency_ms_p50": float(np.percentile(totals, 50) * 1000),
+        "latency_ms_p95": float(np.percentile(totals, 95) * 1000),
+        "rtf_mean": float(np.mean(totals) / audio_seconds),
+        "stages_ms_mean": {
+            name: float(np.mean(values) * 1000) for name, values in stages.items()
+        },
+        "finite": bool(np.isfinite(wav).all()),
+        "rms": float(np.sqrt(np.mean(wav.astype(np.float64) ** 2))),
+        "min": float(wav.min()),
+        "max": float(wav.max()),
+        "om": str(om_path),
+        "output": str(output),
+    }
+    report_path = ROOT / "manifest" / f"{output.stem}_report.json"
+    report_path.write_text(json.dumps(report, ensure_ascii=False, indent=2), encoding="utf-8")
+    print(json.dumps(report, ensure_ascii=False, indent=2))
+
+
+if __name__ == "__main__":
+    main()

assets/snapshot_iter_150000.pdz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ef0faad8186cdd1aa278b61e0b2a3cb7dee9216d1b84f1bc760ec887bcd935c7
+size 1116898380

inference_manifest.json

@@ -0,0 +1,508 @@
+{
+  "bundle": {
+    "digest": {
+      "algorithm": "sha256",
+      "scope": "structure",
+      "value": "0d44f6715fb1c717c40707cf727096a7bb41642044ce232edea2200b92c46773"
+    },
+    "files": [
+      {
+        "path": "assets/adapter.json"
+      },
+      {
+        "path": "assets/phone_id_map.txt"
+      },
+      {
+        "path": "assets/snapshot_iter_150000.pdz"
+      },
+      {
+        "path": "assets/default.yaml"
+      },
+      {
+        "path": "artifacts/ascend_310p/vits_flow_decoder_t639_fp16.om"
+      },
+      {
+        "path": "artifacts/ascend_310p/vits_flow_decoder_t681_fp16.om"
+      }
+    ],
+    "name": "vits-csmsc",
+    "revision": 1,
+    "uuid": "ddac9629-06a2-5bc4-87ce-6ae97c4ebd24"
+  },
+  "deployments": {
+    "ascend_310p": {
+      "artifacts": {
+        "duration": {
+          "format": "onnx",
+          "path": "artifacts/onnx/vits_duration.onnx",
+          "sha256": "f75fd325801a62976ecea0f52ba03a384197f574fedf0486598e5c0fc840c89c"
+        },
+        "flow_decoder": {
+          "format": "om",
+          "path": "artifacts/ascend_310p/vits_flow_decoder_dynamic_fp16.om",
+          "sha256": "c8b26160a0bc7e4de987424fab228bcc8dfd3fb053bd910737443999412c40d8"
+        },
+        "text_encoder": {
+          "format": "onnx",
+          "path": "artifacts/onnx/vits_text_encoder_clean.onnx",
+          "sha256": "15c5fb01538ce38c4748c9bc5892fc0af9bee7c65b228cd116651237a3f72536"
+        }
+      },
+      "bindings": {
+        "duration": {
+          "inputs": [
+            {
+              "dtype": "float32",
+              "index": 0,
+              "runtime_name": "x",
+              "semantic": "host.vits.x",
+              "shape": [
+                1,
+                192,
+                -1
+              ]
+            },
+            {
+              "dtype": "bool",
+              "index": 1,
+              "runtime_name": "x_mask",
+              "semantic": "host.vits.x_mask",
+              "shape": [
+                1,
+                1,
+                -1
+              ]
+            }
+          ],
+          "outputs": [
+            {
+              "dtype": "float32",
+              "index": 0,
+              "runtime_name": "save_infer_model/scale_0.tmp_0",
+              "semantic": "host.vits.logw",
+              "shape": [
+                1,
+                1,
+                -1
+              ]
+            }
+          ]
+        },
+        "flow_decoder": {
+          "inputs": [
+            {
+              "dtype": "float32",
+              "index": 0,
+              "runtime_name": "z_p",
+              "semantic": "host.vits.z_p",
+              "shape": [
+                1,
+                192,
+                -1
+              ]
+            },
+            {
+              "dtype": "float32",
+              "index": 1,
+              "runtime_name": "y_mask",
+              "semantic": "host.vits.y_mask",
+              "shape": [
+                1,
+                1,
+                -1
+              ]
+            }
+          ],
+          "outputs": [
+            {
+              "dtype": "float32",
+              "index": 0,
+              "runtime_name": "p2o.Squeeze.118:0:save_infer_model/scale_0.tmp_0",
+              "semantic": "tts.audio",
+              "shape": [
+                1,
+                -1
+              ]
+            }
+          ]
+        },
+        "text_encoder": {
+          "inputs": [
+            {
+              "dtype": "int64",
+              "index": 0,
+              "runtime_name": "text",
+              "semantic": "host.vits.phone_ids",
+              "shape": [
+                1,
+                -1
+              ]
+            },
+            {
+              "dtype": "int64",
+              "index": 1,
+              "runtime_name": "text_lengths",
+              "semantic": "host.vits.phone_count",
+              "shape": [
+                1
+              ]
+            }
+          ],
+          "outputs": [
+            {
+              "dtype": "float32",
+              "index": 0,
+              "runtime_name": "save_infer_model/scale_0.tmp_0",
+              "semantic": "host.vits.x",
+              "shape": [
+                1,
+                192,
+                -1
+              ]
+            },
+            {
+              "dtype": "float32",
+              "index": 1,
+              "runtime_name": "save_infer_model/scale_1.tmp_0",
+              "semantic": "host.vits.m_p",
+              "shape": [
+                1,
+                192,
+                -1
+              ]
+            },
+            {
+              "dtype": "float32",
+              "index": 2,
+              "runtime_name": "save_infer_model/scale_2.tmp_0",
+              "semantic": "host.vits.logs_p",
+              "shape": [
+                1,
+                192,
+                -1
+              ]
+            },
+            {
+              "dtype": "bool",
+              "index": 3,
+              "runtime_name": "unsqueeze2_2.tmp_0",
+              "semantic": "host.vits.x_mask",
+              "shape": [
+                1,
+                1,
+                -1
+              ]
+            }
+          ]
+        }
+      },
+      "execution": [
+        "text_encoder",
+        "duration",
+        "flow_decoder"
+      ],
+      "execution_contract": {
+        "cancellation_granularity": "stage",
+        "execution_structure": "direct",
+        "state_scope": "request",
+        "stateful": false
+      },
+      "metadata": {
+        "audio_contract": {
+          "channel_semantics": "mono",
+          "channels": 1,
+          "execution_mode": "offline",
+          "sample_dtype": "float32",
+          "sample_rate_hz": 22050
+        },
+        "host_stages": [
+          "duration_expand_path",
+          "sample_z_p"
+        ]
+      },
+      "revision": 1,
+      "role_identities": {
+        "duration": {
+          "interface": "tensor_model",
+          "model_type": "vits_csmsc_duration_predictor",
+          "operation": "predict_duration"
+        },
+        "flow_decoder": {
+          "interface": "tensor_model",
+          "model_type": "vits_csmsc_flow_decoder",
+          "operation": "decode_waveform"
+        },
+        "text_encoder": {
+          "interface": "tensor_model",
+          "model_type": "vits_csmsc_text_encoder",
+          "operation": "encode_text"
+        }
+      },
+      "role_runtime_profiles": {
+        "duration": {
+          "backend": "onnx",
+          "profile": {
+            "device": "cpu"
+          },
+          "target": {
+            "runtime": "onnx"
+          }
+        },
+        "flow_decoder": {
+          "backend": "ascend",
+          "profile": {
+            "device_id": 0
+          },
+          "target": {
+            "runtime": "acl",
+            "runtime_abi": "cann-8.1.RC1",
+            "soc": "Ascend310P1"
+          }
+        },
+        "text_encoder": {
+          "backend": "onnx",
+          "profile": {
+            "device": "cpu"
+          },
+          "target": {
+            "runtime": "onnx"
+          }
+        }
+      },
+      "uuid": "f93e0d28-a973-54bd-a51e-f7ee8a868f2e"
+    },
+    "torch_cpu": {
+      "artifacts": {
+        "duration": {
+          "format": "onnx",
+          "path": "artifacts/onnx/vits_duration.onnx",
+          "sha256": "f75fd325801a62976ecea0f52ba03a384197f574fedf0486598e5c0fc840c89c"
+        },
+        "flow_decoder": {
+          "format": "onnx",
+          "path": "artifacts/onnx/vits_flow_decoder.onnx",
+          "sha256": "3cb9dfe036b843abce71b867626f23d60745c0f0f35f29787ecec7fbfaf0f7fa"
+        },
+        "text_encoder": {
+          "format": "onnx",
+          "path": "artifacts/onnx/vits_text_encoder_clean.onnx",
+          "sha256": "15c5fb01538ce38c4748c9bc5892fc0af9bee7c65b228cd116651237a3f72536"
+        }
+      },
+      "bindings": {
+        "duration": {
+          "inputs": [
+            {
+              "dtype": "float32",
+              "index": 0,
+              "runtime_name": "x",
+              "semantic": "host.vits.x",
+              "shape": [
+                1,
+                192,
+                -1
+              ]
+            },
+            {
+              "dtype": "bool",
+              "index": 1,
+              "runtime_name": "x_mask",
+              "semantic": "host.vits.x_mask",
+              "shape": [
+                1,
+                1,
+                -1
+              ]
+            }
+          ],
+          "outputs": [
+            {
+              "dtype": "float32",
+              "index": 0,
+              "runtime_name": "save_infer_model/scale_0.tmp_0",
+              "semantic": "host.vits.logw",
+              "shape": [
+                1,
+                1,
+                -1
+              ]
+            }
+          ]
+        },
+        "flow_decoder": {
+          "inputs": [
+            {
+              "dtype": "float32",
+              "index": 0,
+              "runtime_name": "z_p",
+              "semantic": "host.vits.z_p",
+              "shape": [
+                1,
+                192,
+                -1
+              ]
+            },
+            {
+              "dtype": "float32",
+              "index": 1,
+              "runtime_name": "y_mask",
+              "semantic": "host.vits.y_mask",
+              "shape": [
+                1,
+                1,
+                -1
+              ]
+            }
+          ],
+          "outputs": [
+            {
+              "dtype": "float32",
+              "index": 0,
+              "runtime_name": "p2o.Squeeze.118:0:save_infer_model/scale_0.tmp_0",
+              "semantic": "tts.audio",
+              "shape": [
+                1,
+                -1
+              ]
+            }
+          ]
+        },
+        "text_encoder": {
+          "inputs": [
+            {
+              "dtype": "int64",
+              "index": 0,
+              "runtime_name": "text",
+              "semantic": "host.vits.phone_ids",
+              "shape": [
+                1,
+                -1
+              ]
+            },
+            {
+              "dtype": "int64",
+              "index": 1,
+              "runtime_name": "text_lengths",
+              "semantic": "host.vits.phone_count",
+              "shape": [
+                1
+              ]
+            }
+          ],
+          "outputs": [
+            {
+              "dtype": "float32",
+              "index": 0,
+              "runtime_name": "save_infer_model/scale_0.tmp_0",
+              "semantic": "host.vits.x",
+              "shape": [
+                1,
+                192,
+                -1
+              ]
+            },
+            {
+              "dtype": "float32",
+              "index": 1,
+              "runtime_name": "save_infer_model/scale_1.tmp_0",
+              "semantic": "host.vits.m_p",
+              "shape": [
+                1,
+                192,
+                -1
+              ]
+            },
+            {
+              "dtype": "float32",
+              "index": 2,
+              "runtime_name": "save_infer_model/scale_2.tmp_0",
+              "semantic": "host.vits.logs_p",
+              "shape": [
+                1,
+                192,
+                -1
+              ]
+            },
+            {
+              "dtype": "bool",
+              "index": 3,
+              "runtime_name": "unsqueeze2_2.tmp_0",
+              "semantic": "host.vits.x_mask",
+              "shape": [
+                1,
+                1,
+                -1
+              ]
+            }
+          ]
+        }
+      },
+      "execution": [
+        "text_encoder",
+        "duration",
+        "flow_decoder"
+      ],
+      "execution_contract": {
+        "cancellation_granularity": "stage",
+        "execution_structure": "direct",
+        "state_scope": "request",
+        "stateful": false
+      },
+      "metadata": {
+        "audio_contract": {
+          "channel_semantics": "mono",
+          "channels": 1,
+          "execution_mode": "offline",
+          "sample_dtype": "float32",
+          "sample_rate_hz": 22050
+        },
+        "host_stages": [
+          "duration_expand_path",
+          "sample_z_p"
+        ]
+      },
+      "revision": 1,
+      "runtime_profile": {
+        "backend": "onnx",
+        "profile": {
+          "device": "cpu"
+        },
+        "target": {
+          "runtime": "onnx"
+        }
+      },
+      "uuid": "aac245e5-989a-599a-8c90-173e9f055cfd"
+    }
+  },
+  "model": {
+    "inputs": [
+      {
+        "dtype": "int64",
+        "semantic": "tts.phone_ids",
+        "shape": [
+          1,
+          -1
+        ]
+      }
+    ],
+    "interface": "tensor_model",
+    "model_type": "vits",
+    "operation": "synthesize",
+    "outputs": [
+      {
+        "dtype": "float32",
+        "semantic": "tts.audio",
+        "shape": [
+          1,
+          -1
+        ]
+      }
+    ],
+    "semantic_identity": {
+      "logical_model_revision": "paddlespeech-vits-csmsc@onnx-export-v1",
+      "output_semantics": "mono-float32-pcm-22050hz-v1",
+      "preprocessing_contract": "paddlespeech-zh-frontend-jieba-pypinyin-phoneid-v1"
+    }
+  },
+  "schema_version": 3
+}