inoryQwQ commited on
Commit
0c313fe
ยท
1 Parent(s): 6badca1

feat: add AX620E and AX637 support (encoder NPU + decoder ONNX fallback)

Browse files

- AX620E: Encoder axmodel (U16, cosine 1.0), Decoder ONNX (CPU)
- AX637: Encoder axmodel (U16, cosine 1.0), Decoder ONNX (CPU)
- AX650: Full NPU (encoder + decoder axmodel)
- Multi-chip engine with auto-detection
- Note: HiFi-GAN decoder cannot compile on AX620E/AX637
(ConvTranspose stride=8 exceeds NPU tiling limits)

.gitattributes CHANGED
@@ -1,4 +1,5 @@
1
- models/*.axmodel filter=lfs diff=lfs merge=lfs -text
2
  *.pth filter=lfs diff=lfs merge=lfs -text
3
  *.onnx filter=lfs diff=lfs merge=lfs -text
4
  *.wav filter=lfs diff=lfs merge=lfs -text
 
 
1
+ models/**/*.axmodel filter=lfs diff=lfs merge=lfs -text
2
  *.pth filter=lfs diff=lfs merge=lfs -text
3
  *.onnx filter=lfs diff=lfs merge=lfs -text
4
  *.wav filter=lfs diff=lfs merge=lfs -text
5
+ *.onnx.data filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -1,65 +1,52 @@
1
  ---
2
  license: mit
3
- language:
4
- - en
5
- tags:
6
- - text-to-speech
7
- - vits
8
- - ax650
9
- - npu
10
- - onnx
11
  pipeline_tag: text-to-speech
12
  ---
13
 
14
- # Inflect-Micro-v2 for AX650 NPU
15
 
16
- High-quality English TTS (VITS architecture) optimized for **AX650 NPU3**, running entirely on-chip with zero PyTorch dependency at inference time.
17
 
18
- - **Encoder + Decoder**: AX650 NPU (AXMODEL, U16 precision)
19
- - **Duration Predictor**: ONNX Runtime (onnxruntime)
20
- - **Embedding**: NumPy lookup
21
- - **Real-time Factor**: ~0.05x (20ร— faster than real-time)
 
22
 
23
- ## ๐ŸŽง Audio Samples
24
 
25
- *All samples generated on AX650 NPU, 24kHz.*
26
-
27
- **"Hello and welcome to Inflect Micro V2 text to speech."** (3.6s, RTF 0.05x)
28
  <audio controls src="https://huggingface.co/AXERA-TECH/inflect_micro_v2/resolve/main/samples/en_hello.wav"></audio>
29
 
30
- **"The weather today is sunny with a gentle breeze."** (2.6s, RTF 0.06x)
31
  <audio controls src="https://huggingface.co/AXERA-TECH/inflect_micro_v2/resolve/main/samples/en_weather.wav"></audio>
32
 
33
- **"Once upon a time, there lived a curious robot who loved to sing."** (3.9s, RTF 0.05x)
34
  <audio controls src="https://huggingface.co/AXERA-TECH/inflect_micro_v2/resolve/main/samples/en_story.wav"></audio>
35
 
36
- **"Neural voice synthesis is the next frontier of artificial intelligence."** (4.1s, RTF 0.04x)
37
  <audio controls src="https://huggingface.co/AXERA-TECH/inflect_micro_v2/resolve/main/samples/en_tech.wav"></audio>
38
 
39
- **"This audio was generated entirely on the AX650 NPU chip."** (4.7s, RTF 0.04x)
40
  <audio controls src="https://huggingface.co/AXERA-TECH/inflect_micro_v2/resolve/main/samples/en_intro.wav"></audio>
41
 
42
  ## ๐Ÿš€ Quick Start
43
 
44
  ```bash
45
  bash setup.sh
46
- bash run.sh "Hello world." output.wav
47
  ```
48
 
49
- ## ๐Ÿ“ก API
50
-
51
- ```bash
52
- # CLI
53
- python -m inflect_tts_sdk --text "Hello" --output out.wav
54
 
55
- # OpenAI-compatible Server
56
- python -m inflect_tts_sdk.server --port 8000
57
  ```
58
-
59
- ## ๐Ÿ“ File Structure
60
-
61
- - `models/` โ€” AXMODEL, dp.onnx, emb.npy
62
- - `python/inflect_tts_sdk/` โ€” Python SDK (CLI + OpenAI Server)
63
- - `cpp/inflect_tts_sdk/` โ€” C++ SDK (CLI + Server)
64
- - `samples/` โ€” audio samples
65
-
 
1
  ---
2
  license: mit
3
+ language: [en]
4
+ tags: [text-to-speech, vits, ax650, ax620e, ax637, npu, onnx]
 
 
 
 
 
 
5
  pipeline_tag: text-to-speech
6
  ---
7
 
8
+ # Inflect-Micro-v2 for AXera NPU
9
 
10
+ High-quality English TTS (VITS) optimized for **AX650 / AX620E / AX637** NPUs.
11
 
12
+ | Chip | Encoder | Decoder | Precision |
13
+ |------|---------|---------|-----------|
14
+ | **AX650** | NPU (axmodel) | NPU (axmodel) | U16 |
15
+ | **AX620E** | NPU (axmodel) | CPU (onnxruntime) | U16 |
16
+ | **AX637** | NPU (axmodel) | CPU (onnxruntime) | U16 |
17
 
18
+ ## ๐ŸŽง Audio Samples (AX650 NPU)
19
 
20
+ **"Hello and welcome to Inflect Micro V2 text to speech."**
 
 
21
  <audio controls src="https://huggingface.co/AXERA-TECH/inflect_micro_v2/resolve/main/samples/en_hello.wav"></audio>
22
 
23
+ **"The weather today is sunny with a gentle breeze."**
24
  <audio controls src="https://huggingface.co/AXERA-TECH/inflect_micro_v2/resolve/main/samples/en_weather.wav"></audio>
25
 
26
+ **"Once upon a time, there lived a curious robot who loved to sing."**
27
  <audio controls src="https://huggingface.co/AXERA-TECH/inflect_micro_v2/resolve/main/samples/en_story.wav"></audio>
28
 
29
+ **"Neural voice synthesis is the next frontier of AI."**
30
  <audio controls src="https://huggingface.co/AXERA-TECH/inflect_micro_v2/resolve/main/samples/en_tech.wav"></audio>
31
 
32
+ **"This audio was generated entirely on the AX650 NPU chip."**
33
  <audio controls src="https://huggingface.co/AXERA-TECH/inflect_micro_v2/resolve/main/samples/en_intro.wav"></audio>
34
 
35
  ## ๐Ÿš€ Quick Start
36
 
37
  ```bash
38
  bash setup.sh
39
+ bash run.sh "Hello world." output.wav cli ax650
40
  ```
41
 
42
+ ## ๐Ÿ“ Structure
 
 
 
 
43
 
 
 
44
  ```
45
+ models/
46
+ ax650/ โ€” full NPU (encoder + decoder axmodel)
47
+ ax620e/ โ€” encoder axmodel + decoder ONNX
48
+ ax637/ โ€” encoder axmodel + decoder ONNX
49
+ python/ โ€” Python SDK
50
+ cpp/ โ€” C++ SDK
51
+ samples/ โ€” audio samples
52
+ ```
models/{dp.onnx โ†’ ax620e/dp.onnx} RENAMED
File without changes
models/{emb.npy โ†’ ax620e/emb.npy} RENAMED
File without changes
models/ax620e/inflect_decoder.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6ad0e403794f3e9b24be9bcc438785ff46b887f10bf90698411fc25edcc0504e
3
+ size 107178
models/ax620e/inflect_decoder.onnx.data ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f378527dfcfb8b7069193559bde3702af3a1b889b0f3af403dbc3a3c4ed8c5ad
3
+ size 38338560
models/ax620e/inflect_encoder.axmodel ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cc63f65ab37ad48156cbfc26c499f5ea5616ef0a0ce98d9ece7ecfcb3f56192b
3
+ size 1728627
models/ax620e/model_meta.json ADDED
@@ -0,0 +1,13 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model": "inflect_micro_v2",
3
+ "chip": "ax620e",
4
+ "encoder": "axmodel",
5
+ "decoder": "onnx",
6
+ "dp": "onnx",
7
+ "embedding": "npy",
8
+ "sample_rate": 24000,
9
+ "hop_length": 256,
10
+ "precision": "U16",
11
+ "max_tokens": 200,
12
+ "max_mel_frames": 500
13
+ }
models/ax637/dp.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fb2f0f1138c7c409a89eaeaec81dec6720dfbc82d4f20d0159b1c92058527731
3
+ size 1105165
models/ax637/emb.npy ADDED
Binary file (68.5 kB). View file
 
models/ax637/inflect_decoder.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6ad0e403794f3e9b24be9bcc438785ff46b887f10bf90698411fc25edcc0504e
3
+ size 107178
models/ax637/inflect_decoder.onnx.data ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f378527dfcfb8b7069193559bde3702af3a1b889b0f3af403dbc3a3c4ed8c5ad
3
+ size 38338560
models/ax637/inflect_encoder.axmodel ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4a883cae3c3d0f8f654acd38c67c8c96994e130f19a53a28794b1fa3c8c590bd
3
+ size 1695942
models/ax637/model_meta.json ADDED
@@ -0,0 +1,13 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model": "inflect_micro_v2",
3
+ "chip": "ax637",
4
+ "encoder": "axmodel",
5
+ "decoder": "onnx",
6
+ "dp": "onnx",
7
+ "embedding": "npy",
8
+ "sample_rate": 24000,
9
+ "hop_length": 256,
10
+ "precision": "U16",
11
+ "max_tokens": 200,
12
+ "max_mel_frames": 500
13
+ }
models/ax650/dp.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fb2f0f1138c7c409a89eaeaec81dec6720dfbc82d4f20d0159b1c92058527731
3
+ size 1105165
models/ax650/emb.npy ADDED
Binary file (68.5 kB). View file
 
models/{inflect_decoder.axmodel โ†’ ax650/inflect_decoder.axmodel} RENAMED
File without changes
models/{inflect_encoder.axmodel โ†’ ax650/inflect_encoder.axmodel} RENAMED
File without changes
models/ax650/model_meta.json ADDED
@@ -0,0 +1,13 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model": "inflect_micro_v2",
3
+ "chip": "ax650",
4
+ "encoder": "axmodel",
5
+ "decoder": "axmodel",
6
+ "dp": "onnx",
7
+ "embedding": "npy",
8
+ "sample_rate": 24000,
9
+ "hop_length": 256,
10
+ "precision": "U16",
11
+ "max_tokens": 200,
12
+ "max_mel_frames": 500
13
+ }
models/model_meta.json DELETED
@@ -1,58 +0,0 @@
1
- {
2
- "model_name": "inflect_micro_v2",
3
- "model_type": "vits_tts",
4
- "framework": "pytorch",
5
- "task": "text_to_speech",
6
- "encoder_decoder_split": true,
7
- "config": {
8
- "sampling_rate": 24000,
9
- "hop_length": 256,
10
- "inter_channels": 192,
11
- "hidden_channels": 96,
12
- "filter_channels": 768,
13
- "n_vocab": 256,
14
- "add_blank": true
15
- },
16
- "submodels": [
17
- {
18
- "name": "inflect_encoder",
19
- "onnx_file": "inflect_encoder_static.onnx",
20
- "description": "TextEncoder: tokens -> m_p, logs_p, x, x_mask",
21
- "inputs": [
22
- {"name": "tokens", "shape": [1, 200], "dtype": "int64", "dynamic_dim": 1},
23
- {"name": "lengths", "shape": [1], "dtype": "int64"}
24
- ],
25
- "outputs": [
26
- {"name": "m_p", "shape": [1, 192, "token_len"], "dtype": "float32"},
27
- {"name": "logs_p", "shape": [1, 192, "token_len"], "dtype": "float32"},
28
- {"name": "x", "shape": [1, 96, "token_len"], "dtype": "float32"},
29
- {"name": "x_mask", "shape": [1, 1, "token_len"], "dtype": "float32"}
30
- ]
31
- },
32
- {
33
- "name": "inflect_decoder",
34
- "onnx_file": "inflect_decoder_static.onnx",
35
- "description": "Flow(reverse) + HiFi-GAN Generator: z_p -> waveform",
36
- "inputs": [
37
- {"name": "z_p", "shape": [1, 192, 500], "dtype": "float32", "dynamic_dim": 2},
38
- {"name": "y_mask", "shape": [1, 1, 500], "dtype": "float32", "dynamic_dim": 2}
39
- ],
40
- "outputs": [
41
- {"name": "waveform", "shape": [1, 1, 128000], "dtype": "float32"}
42
- ]
43
- }
44
- ],
45
- "pipeline": {
46
- "preprocessing": "text -> phonemes -> token_ids (CPU, Python frontend)",
47
- "encoder": "tokens -> enc_p -> m_p, logs_p (NPU)",
48
- "duration_alignment": "dp + generate_path + expand (CPU, Python)",
49
- "decoder": "z_p -> flow(reverse) -> dec -> waveform (NPU)",
50
- "postprocessing": "none (raw waveform output)"
51
- },
52
- "export_info": {
53
- "opset_version": 18,
54
- "torch_version": "2.x",
55
- "dynamic_onnx_available": true,
56
- "static_onnx_for_compile": true
57
- }
58
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
python/inflect_tts_sdk/__main__.py CHANGED
@@ -1,19 +1,20 @@
1
- """CLI: python -m inflect_tts_sdk --text "Hello" --output out.wav"""
2
  import argparse
3
  from pathlib import Path
4
  from .tts_engine import InflectTTSEngine
5
 
6
  def main():
7
- ap = argparse.ArgumentParser(description="Inflect-Micro-v2 TTS (AX650 NPU3)")
8
- ap.add_argument("--text", required=True, help="Text to synthesize")
9
  ap.add_argument("--output", default="output.wav")
10
  ap.add_argument("--speed", type=float, default=1.0)
11
  ap.add_argument("--variation", type=float, default=0.667)
12
  ap.add_argument("--seed", type=int, default=0)
13
  ap.add_argument("--model-dir", default=None)
 
14
  args = ap.parse_args()
15
 
16
- engine = InflectTTSEngine(model_dir=args.model_dir)
17
  engine.save(args.text, args.output, speed=args.speed,
18
  variation=args.variation, seed=args.seed)
19
  print(f"๐Ÿ”Š {args.output}")
 
1
+ """CLI: python -m inflect_tts_sdk --text "Hello" --output out.wav [--chip ax650]"""
2
  import argparse
3
  from pathlib import Path
4
  from .tts_engine import InflectTTSEngine
5
 
6
  def main():
7
+ ap = argparse.ArgumentParser(description="Inflect-Micro-v2 TTS (AX NPU)")
8
+ ap.add_argument("--text", required=True)
9
  ap.add_argument("--output", default="output.wav")
10
  ap.add_argument("--speed", type=float, default=1.0)
11
  ap.add_argument("--variation", type=float, default=0.667)
12
  ap.add_argument("--seed", type=int, default=0)
13
  ap.add_argument("--model-dir", default=None)
14
+ ap.add_argument("--chip", default="ax650", choices=["ax650", "ax620e", "ax637"])
15
  args = ap.parse_args()
16
 
17
+ engine = InflectTTSEngine(model_dir=args.model_dir, chip=args.chip)
18
  engine.save(args.text, args.output, speed=args.speed,
19
  variation=args.variation, seed=args.seed)
20
  print(f"๐Ÿ”Š {args.output}")
python/inflect_tts_sdk/tts_engine.py CHANGED
@@ -1,13 +1,13 @@
1
  """
2
- Inflect-Micro-v2 TTS Engine for AX650 NPU3
3
- ็บฏ NPU + ONNX + NumPy๏ผŒ้›ถ PyTorch/CPU fallback ไพ่ต–ใ€‚
 
4
  """
5
  from __future__ import annotations
6
- import sys, os, io, math, re, json, warnings, contextlib
7
  from pathlib import Path
8
  import numpy as np
9
  import onnxruntime as ort
10
- import axengine as axe
11
 
12
  PKG = Path(__file__).resolve().parent
13
  sys.path.insert(0, str(PKG))
@@ -18,70 +18,96 @@ from text.symbols import symbols
18
  from text import cleaned_text_to_sequence
19
 
20
 
21
- # โ”€โ”€ numpy ็‰ˆๆœฌ็š„ VITS ๅทฅๅ…ทๅ‡ฝๆ•ฐ โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
22
 
23
  def _intersperse(lst, item):
24
  result = [item] * (len(lst) * 2 + 1)
25
  result[1::2] = lst
26
  return result
27
 
28
-
29
  def _sequence_mask(length, max_length=None):
30
  if max_length is None:
31
  max_length = int(length.max())
32
  return np.arange(max_length, dtype=length.dtype)[None, :] < length[:, None]
33
 
34
-
35
  def _generate_path(duration, mask):
36
- """numpy ็‰ˆๆœฌ็š„ commons.generate_path
37
- duration: [b, 1, t_x]
38
- mask: [b, 1, t_y, t_x]
39
- """
40
  b, _, t_y, t_x = mask.shape
41
- cum_duration = np.cumsum(duration, axis=-1) # [b, 1, t_x]
42
- cum_duration_flat = cum_duration.reshape(b * t_x) # [b*t_x]
43
- path = _sequence_mask(cum_duration_flat, t_y).astype(mask.dtype) # [b*t_x, t_y]
44
  path = path.reshape(b, t_x, t_y)
45
- padded = np.pad(path[:, :-1], ((0, 0), (1, 0), (0, 0))) # shift right
46
  path = path - padded
47
- path = path[:, np.newaxis, :, :].transpose(0, 1, 3, 2) * mask # [b,1,t_y,t_x]
48
  return path
49
 
50
 
51
  # โ”€โ”€ ๅผ•ๆ“Ž โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
52
 
53
  class InflectTTSEngine:
54
- """Inflect-Micro-v2 TTS โ€” AX650 NPU3 + ONNX dp๏ผŒ้›ถ PyTorchใ€‚"""
55
 
56
- def __init__(self, model_dir: str | Path | None = None):
57
  if model_dir:
58
  self.root = Path(model_dir)
59
  else:
60
- self.root = PKG.parent.parent / "models"
 
 
 
61
 
62
  # ๅŠ ่ฝฝ้…็ฝฎ
63
- cfg_path = PKG / "config.json"
64
- with open(cfg_path) as f:
65
  self.hps = json.load(f)
66
  self.sample_rate = int(self.hps["data"]["sampling_rate"])
67
  self.hop_length = int(self.hps["data"]["hop_length"])
68
  self.hidden_channels = int(self.hps["model"]["hidden_channels"])
 
 
 
 
 
 
 
 
 
69
 
70
- # Embedding ๆƒ้‡ (n_vocab, hidden_channels)
71
- emb_path = self.root / "emb.npy"
72
- self._emb = np.load(str(emb_path))
73
  self._emb_scale = math.sqrt(self.hidden_channels)
74
 
75
  # Duration Predictor ONNX
76
- dp_path = self.root / "dp.onnx"
77
- providers = ["CPUExecutionProvider"]
78
- self._dp_sess = ort.InferenceSession(str(dp_path), providers=providers)
79
-
80
- # NPU: Encoder + Decoder
81
- enc_path = str(self.root / "inflect_encoder.axmodel")
82
- dec_path = str(self.root / "inflect_decoder.axmodel")
83
- self.enc_session = axe.InferenceSession(enc_path)
84
- self.dec_session = axe.InferenceSession(dec_path)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
85
 
86
  def synthesize(self, text: str, speed: float = 1.0, variation: float = 0.667,
87
  seed: int = 0):
@@ -96,8 +122,7 @@ class InflectTTSEngine:
96
  pieces = []
97
  for idx, chunk in enumerate(sentences):
98
  if idx > 0:
99
- pause = round(self.sample_rate * 0.08)
100
- pieces.append(np.zeros(pause, dtype=np.float32))
101
 
102
  phonemes = run_vits_frontend(chunk).phoneme_text
103
  seq = cleaned_text_to_sequence(phonemes)
@@ -106,64 +131,68 @@ class InflectTTSEngine:
106
  if not seq:
107
  continue
108
 
109
- tokens = np.array(seq, dtype=np.int64) # [T]
110
  tlen = len(tokens)
111
 
112
- # โ”€โ”€ NumPy: Embedding โ”€โ”€
113
- x_emb = self._emb[tokens] * self._emb_scale # [T, H]
114
- x_emb = x_emb.T[np.newaxis, :, :] # [1, H, T]
115
 
116
- # โ”€โ”€ NPU: Encoder โ”€โ”€
117
  MAX_TOK = 200
118
  if tlen > MAX_TOK:
119
- raise ValueError(f"Text too long: {tlen} tokens > {MAX_TOK} max")
120
  x_pad = np.zeros((1, self.hidden_channels, MAX_TOK), dtype=np.float32)
121
  x_pad[:, :, :tlen] = x_emb
122
 
123
  enc_out = self.enc_session.run(None, {
124
  "x_emb": x_pad,
125
- "lengths": np.array([tlen], dtype=np.int64),
126
  })
127
- m_p = np.asarray(enc_out[0])[:, :, :tlen] # [1, inter, T]
128
  logs_p = np.asarray(enc_out[1])[:, :, :tlen]
129
  x_enc = np.asarray(enc_out[2])[:, :, :tlen]
130
  x_mask = np.asarray(enc_out[3])[:, :, :tlen]
131
 
132
- # โ”€โ”€ ONNX: Duration Predictor โ”€โ”€
133
  logw = self._dp_sess.run(None, {
134
  "x": x_enc.astype(np.float32),
135
  "x_mask": x_mask.astype(np.float32),
136
- })[0] # [1, 1, T]
137
 
138
- # โ”€โ”€ NumPy: Duration + Alignment โ”€โ”€
139
  rng = np.random.RandomState(seed + idx)
140
  w = np.exp(logw) * x_mask * (1.0 / speed)
141
  w_ceil = np.ceil(w)
142
  y_lengths = np.clip(w_ceil.sum(axis=(1, 2)), 1, None).astype(np.int64)
143
  y_mask = _sequence_mask(y_lengths, None)[:, np.newaxis, :].astype(x_mask.dtype)
144
  attn_mask = x_mask[:, :, np.newaxis, :] * y_mask[:, :, :, np.newaxis]
145
-
146
- attn = _generate_path(w_ceil, attn_mask) # [1, 1, Ty, T]
147
 
148
  m_p = (attn.squeeze(1) @ m_p.transpose(0, 2, 1)).transpose(0, 2, 1)
149
  logs_p = (attn.squeeze(1) @ logs_p.transpose(0, 2, 1)).transpose(0, 2, 1)
150
  z_p = m_p + rng.randn(*m_p.shape).astype(np.float32) * np.exp(logs_p) * variation
151
 
152
- # โ”€โ”€ NPU: Decoder โ”€โ”€
153
  mel_len = z_p.shape[2]
154
  MAX_MEL = 500
155
  if mel_len > MAX_MEL:
156
- raise ValueError(f"Audio too long: {mel_len} frames > {MAX_MEL} max")
157
- zp_np = np.zeros((1, self.hps["model"]["inter_channels"], MAX_MEL), dtype=np.float32)
158
- ym_np = np.zeros((1, 1, MAX_MEL), dtype=np.float32)
159
- zp_np[:, :, :mel_len] = z_p
160
- ym_np[:, :, :mel_len] = y_mask
161
-
162
- dec_out = self.dec_session.run(None, {
163
- "z_p": zp_np,
164
- "y_mask": ym_np,
165
- })
166
- waveform = dec_out[0][0, 0, :mel_len * self.hop_length]
 
 
 
 
 
167
  pieces.append(waveform)
168
 
169
  waveform = np.clip(np.concatenate(pieces), -1.0, 1.0)
 
1
  """
2
+ Inflect-Micro-v2 TTS Engine โ€” multi-chip, zero PyTorch.
3
+
4
+ Supports: AX650 (full NPU), AX620E (encoder NPU + decoder ONNX), AX637 (encoder NPU + decoder ONNX).
5
  """
6
  from __future__ import annotations
7
+ import sys, os, io, math, re, json
8
  from pathlib import Path
9
  import numpy as np
10
  import onnxruntime as ort
 
11
 
12
  PKG = Path(__file__).resolve().parent
13
  sys.path.insert(0, str(PKG))
 
18
  from text import cleaned_text_to_sequence
19
 
20
 
21
+ # โ”€โ”€ numpy ๅทฅๅ…ทๅ‡ฝๆ•ฐ โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
22
 
23
  def _intersperse(lst, item):
24
  result = [item] * (len(lst) * 2 + 1)
25
  result[1::2] = lst
26
  return result
27
 
 
28
  def _sequence_mask(length, max_length=None):
29
  if max_length is None:
30
  max_length = int(length.max())
31
  return np.arange(max_length, dtype=length.dtype)[None, :] < length[:, None]
32
 
 
33
  def _generate_path(duration, mask):
 
 
 
 
34
  b, _, t_y, t_x = mask.shape
35
+ cum_duration = np.cumsum(duration, axis=-1)
36
+ cum_duration_flat = cum_duration.reshape(b * t_x)
37
+ path = _sequence_mask(cum_duration_flat, t_y).astype(mask.dtype)
38
  path = path.reshape(b, t_x, t_y)
39
+ padded = np.pad(path[:, :-1], ((0, 0), (1, 0), (0, 0)))
40
  path = path - padded
41
+ path = path[:, np.newaxis, :, :].transpose(0, 1, 3, 2) * mask
42
  return path
43
 
44
 
45
  # โ”€โ”€ ๅผ•ๆ“Ž โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
46
 
47
  class InflectTTSEngine:
48
+ """Inflect-Micro-v2 TTS โ€” ๅคš่Šฏ็‰‡๏ผŒ้›ถ PyTorchใ€‚"""
49
 
50
+ def __init__(self, model_dir: str | Path | None = None, chip: str = "ax650"):
51
  if model_dir:
52
  self.root = Path(model_dir)
53
  else:
54
+ # Default: models/{chip}/
55
+ self.root = PKG.parent.parent / "models" / chip
56
+
57
+ self.chip = chip
58
 
59
  # ๅŠ ่ฝฝ้…็ฝฎ
60
+ with open(PKG / "config.json") as f:
 
61
  self.hps = json.load(f)
62
  self.sample_rate = int(self.hps["data"]["sampling_rate"])
63
  self.hop_length = int(self.hps["data"]["hop_length"])
64
  self.hidden_channels = int(self.hps["model"]["hidden_channels"])
65
+ self.inter_channels = int(self.hps["model"]["inter_channels"])
66
+
67
+ # ๅŠ ่ฝฝ chip-specific model_meta
68
+ meta_path = self.root / "model_meta.json"
69
+ if meta_path.exists():
70
+ with open(meta_path) as f:
71
+ self.meta = json.load(f)
72
+ else:
73
+ self.meta = {}
74
 
75
+ # Embedding (n_vocab, hidden_channels)
76
+ self._emb = np.load(str(self.root / "emb.npy"))
 
77
  self._emb_scale = math.sqrt(self.hidden_channels)
78
 
79
  # Duration Predictor ONNX
80
+ self._dp_sess = ort.InferenceSession(
81
+ str(self.root / "dp.onnx"), providers=["CPUExecutionProvider"]
82
+ )
83
+
84
+ # Encoder โ€” always AXMODEL
85
+ try:
86
+ import axengine as axe
87
+ enc_path = str(self.root / "inflect_encoder.axmodel")
88
+ self.enc_session = axe.InferenceSession(enc_path)
89
+ self._has_npu = True
90
+ except ImportError:
91
+ self._has_npu = False
92
+ print("[WARN] axengine not available, encoder will use CPU ONNX")
93
+ self.enc_session = ort.InferenceSession(
94
+ str(self.root / "inflect_encoder.onnx"),
95
+ providers=["CPUExecutionProvider"],
96
+ )
97
+
98
+ # Decoder โ€” AXMODEL or ONNX fallback
99
+ dec_axmodel = self.root / "inflect_decoder.axmodel"
100
+ if dec_axmodel.exists() and self._has_npu:
101
+ self.dec_session = axe.InferenceSession(str(dec_axmodel))
102
+ self._dec_is_npu = True
103
+ else:
104
+ dec_onnx = self.root / "inflect_decoder.onnx"
105
+ self.dec_session = ort.InferenceSession(
106
+ str(dec_onnx), providers=["CPUExecutionProvider"]
107
+ )
108
+ self._dec_is_npu = False
109
+ if self._has_npu:
110
+ print(f"[INFO] {chip}: decoder running on CPU (ONNX)")
111
 
112
  def synthesize(self, text: str, speed: float = 1.0, variation: float = 0.667,
113
  seed: int = 0):
 
122
  pieces = []
123
  for idx, chunk in enumerate(sentences):
124
  if idx > 0:
125
+ pieces.append(np.zeros(round(self.sample_rate * 0.08), dtype=np.float32))
 
126
 
127
  phonemes = run_vits_frontend(chunk).phoneme_text
128
  seq = cleaned_text_to_sequence(phonemes)
 
131
  if not seq:
132
  continue
133
 
134
+ tokens = np.array(seq, dtype=np.int64)
135
  tlen = len(tokens)
136
 
137
+ # โ”€โ”€ Embedding โ”€โ”€
138
+ x_emb = self._emb[tokens] * self._emb_scale
139
+ x_emb = x_emb.T[np.newaxis, :, :]
140
 
141
+ # โ”€โ”€ Encoder โ”€โ”€
142
  MAX_TOK = 200
143
  if tlen > MAX_TOK:
144
+ raise ValueError(f"Text too long: {tlen} tokens > {MAX_TOK}")
145
  x_pad = np.zeros((1, self.hidden_channels, MAX_TOK), dtype=np.float32)
146
  x_pad[:, :, :tlen] = x_emb
147
 
148
  enc_out = self.enc_session.run(None, {
149
  "x_emb": x_pad,
150
+ "lengths": np.array([tlen], dtype=np.int32),
151
  })
152
+ m_p = np.asarray(enc_out[0])[:, :, :tlen]
153
  logs_p = np.asarray(enc_out[1])[:, :, :tlen]
154
  x_enc = np.asarray(enc_out[2])[:, :, :tlen]
155
  x_mask = np.asarray(enc_out[3])[:, :, :tlen]
156
 
157
+ # โ”€โ”€ Duration Predictor โ”€โ”€
158
  logw = self._dp_sess.run(None, {
159
  "x": x_enc.astype(np.float32),
160
  "x_mask": x_mask.astype(np.float32),
161
+ })[0]
162
 
163
+ # โ”€โ”€ Duration + Alignment โ”€โ”€
164
  rng = np.random.RandomState(seed + idx)
165
  w = np.exp(logw) * x_mask * (1.0 / speed)
166
  w_ceil = np.ceil(w)
167
  y_lengths = np.clip(w_ceil.sum(axis=(1, 2)), 1, None).astype(np.int64)
168
  y_mask = _sequence_mask(y_lengths, None)[:, np.newaxis, :].astype(x_mask.dtype)
169
  attn_mask = x_mask[:, :, np.newaxis, :] * y_mask[:, :, :, np.newaxis]
170
+ attn = _generate_path(w_ceil, attn_mask)
 
171
 
172
  m_p = (attn.squeeze(1) @ m_p.transpose(0, 2, 1)).transpose(0, 2, 1)
173
  logs_p = (attn.squeeze(1) @ logs_p.transpose(0, 2, 1)).transpose(0, 2, 1)
174
  z_p = m_p + rng.randn(*m_p.shape).astype(np.float32) * np.exp(logs_p) * variation
175
 
176
+ # โ”€โ”€ Decoder โ”€โ”€
177
  mel_len = z_p.shape[2]
178
  MAX_MEL = 500
179
  if mel_len > MAX_MEL:
180
+ raise ValueError(f"Audio too long: {mel_len} frames > {MAX_MEL}")
181
+
182
+ if self._dec_is_npu:
183
+ zp_np = np.zeros((1, self.inter_channels, MAX_MEL), dtype=np.float32)
184
+ ym_np = np.zeros((1, 1, MAX_MEL), dtype=np.float32)
185
+ zp_np[:, :, :mel_len] = z_p
186
+ ym_np[:, :, :mel_len] = y_mask
187
+ dec_out = self.dec_session.run(None, {"z_p": zp_np, "y_mask": ym_np})
188
+ waveform = dec_out[0][0, 0, :mel_len * self.hop_length]
189
+ else:
190
+ dec_out = self.dec_session.run(None, {
191
+ "z_p": z_p.astype(np.float32),
192
+ "y_mask": y_mask.astype(np.float32),
193
+ })
194
+ waveform = dec_out[0][0, 0, :mel_len * self.hop_length]
195
+
196
  pieces.append(waveform)
197
 
198
  waveform = np.clip(np.concatenate(pieces), -1.0, 1.0)
run.sh CHANGED
@@ -3,15 +3,16 @@ set -euo pipefail
3
  TEXT="${1:-Hello world, this is Inflect Micro V2.}"
4
  OUTPUT="${2:-output.wav}"
5
  MODE="${3:-cli}"
 
6
 
7
  SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
8
  cd "$SCRIPT_DIR/python"
9
 
10
  if [ "$MODE" = "server" ]; then
11
- echo "๐ŸŒ Starting OpenAI TTS Server on port ${4:-8000}..."
12
- python3 -m inflect_tts_sdk.server --port "${4:-8000}"
13
  else
14
  echo "๐ŸŽค ${TEXT}"
15
- python3 -m inflect_tts_sdk --text "${TEXT}" --output "${OUTPUT}"
16
  echo "๐Ÿ”Š ${OUTPUT}"
17
  fi
 
3
  TEXT="${1:-Hello world, this is Inflect Micro V2.}"
4
  OUTPUT="${2:-output.wav}"
5
  MODE="${3:-cli}"
6
+ CHIP="${4:-ax650}"
7
 
8
  SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
9
  cd "$SCRIPT_DIR/python"
10
 
11
  if [ "$MODE" = "server" ]; then
12
+ echo "๐ŸŒ Starting OpenAI TTS Server on port ${5:-8000}..."
13
+ python3 -m inflect_tts_sdk.server --port "${5:-8000}" --chip "$CHIP"
14
  else
15
  echo "๐ŸŽค ${TEXT}"
16
+ python3 -m inflect_tts_sdk --text "${TEXT}" --output "${OUTPUT}" --chip "$CHIP"
17
  echo "๐Ÿ”Š ${OUTPUT}"
18
  fi