jonpark0/supertonic-3-AX650

Supertonic 3 — AX650 / AX8850 (LLM-8850) NPU

Supertone/supertonic-3 converted with Axera Pulsar2 7.0-patch1 to run on the AX650 / AX8850 NPU, tested on the M5Stack LLM-8850 PCIe card (AXCL). The flow-matching estimator (8 steps) and the vocoder run on the card. Text processing, the duration predictor, the text encoder and the Euler update stay on the host CPU.

On the card it reads Korean, Japanese and English almost as accurately as the original (table below). There are two static sizes ("buckets"): a message of up to about 6.7 s takes about 0.18 s, and one of up to about 13 s takes about 0.35 s. Card memory is about 220 MiB with both buckets loaded, or 99 MiB with the short one only. The host runtime is the HTTP server in JonPark0/TTSBot (server/), written for a Discord TTS bot.

한국어 안내는 아래에 있습니다.

Accuracy and speed

60 sentences (Korean standard 20, Korean chat 20, Japanese 10, English 10) read by whisper-large-v3-turbo. Korean and Japanese are scored by CER (Japanese in kana), English by WER. UTMOS is utmos22_strong, an English-trained proxy. On human Korean speech the same ASR gives 1.7% CER.

	Korean standard CER	Korean chat CER	Japanese CER	English WER	UTMOS	Time per sentence
Original ONNX (CPU)	0.0%	3.9%	1.7%	1.4%	4.00	–
Static graph + Erf approximation (CPU float, same noise)	0.3%	5.5%	1.7%	1.4%	3.99	–
Card estimator + CPU vocoder	0.3%	5.5%	2.2%	1.4%	4.00	0.53 s
Card estimator + card vocoder (this repo)	0.3%	6.5%	2.2%	1.4%	3.95	0.18 s

Only two sentences are read differently by the card and by the float model with the same noise.

• Per call on the card: estimator about 18 ms per step (× 8), vocoder about 17 ms.
• Through the TTSBot HTTP server: 0.175–0.178 s for a short message.

Long messages (192 bucket)

30 messages of 7–14 s (20 Korean, 5 Japanese, 5 English) on the card, voice F1. With the short bucket only, the server has to split each message into 2–4 pieces. With both buckets, 29 of 30 messages are read in one piece.

Card	Korean CER	Japanese CER	English WER	Korean UTMOS	Median time per message	Pieces
96 bucket only	0.1%	1.5%	0%	3.85	0.39 s	70
96 + 192 buckets	0.0%	0.8%	0%	3.91	0.35 s	31

• One 192 call: estimator about 34 ms per step (× 8), vocoder about 46 ms.
• Compiled vs float (pulsar2 run): cosine 0.9999 and 0.9995 at estimator steps 0 and 7, 0.994 for the vocoder.
• The gain is fewer breaks inside a sentence rather than accuracy. In CPU float, Korean UTMOS was 4.06 for the original, 3.85 with the 96 bucket only, and 4.01 with both buckets.
• These runs are before the server's edge-quiet trimming (below). With trimming, the kept speech samples are identical, but Korean UTMOS reads about 0.05 lower (3.91 → 3.86). The ASR error is the same.

Voices

All 10 preset voices on the card, each reading the same 20 Korean sentences (10 standard, 10 chat). With 10 sentences per group, one or two misread sentences move the numbers a lot. All voices are usable.

Voice	Standard CER	Chat CER	UTMOS
F1	0.6%	3.7%	3.85
F2	0.0%	8.1%	3.66
F3	0.0%	2.0%	4.03
F4	0.0%	13.0%	3.74
F5	0.0%	3.3%	3.72
M1	0.0%	2.7%	3.67
M2	0.0%	0.0%	3.55
M3	0.6%	7.4%	3.54
M4	0.0%	2.5%	3.84
M5	0.0%	2.0%	3.68

Contents

File	Description
st_est.axmodel	Flow-matching estimator, static text length T = 96 and latent length L = 96 (about 6.7 s of audio)
st_voc.axmodel	Vocoder, L = 96 → 44.1 kHz audio
st_est_T192_L192.axmodel	Estimator, T = L = 192 (about 13 s)
st_voc_L192.axmodel	Vocoder, L = 192
st_rope_T96_L96.onnx, st_rope_T192_L192.onnx	Rotary sin/cos tables for the actual text and latent lengths, one per bucket (host, CPU)
time_table.npy	Time embeddings for the 8 flow steps (host)
LICENSE, NOTICE	Original license (BigScience OpenRAIL-M) and the list of changes

File	sha256
st_est.axmodel	5fec2e6d05f608cecfcdadf4de198916ba810473c73034508788ad450226a561
st_voc.axmodel	124620f4104678623b112f936306cf0644592315c0b422767ed3240a616696da
st_est_T192_L192.axmodel	2005389fe06fa131fbbf3fa66777856ea82718a6aaf0f7db89ed049c2e815fdb
st_voc_L192.axmodel	2f7a3635dc676d6402400ca291f34be9e41223e702a15c46d519718b9d29d4af

The duration predictor, the text encoder, the text processor and the voice styles are not in this repo. The host runtime downloads them from Supertone/supertonic-3 through the supertonic Python package on first run.

Quick start

Requirements: an AX650 / AX8850 AXCL card with the AXCL driver and runtime installed (libaxcl_rt.dll on Windows, libaxcl_rt.so on Linux), Python 3.10+.

git lfs install
git clone https://huggingface.co/jonpark0/supertonic-3-AX650
git clone https://github.com/JonPark0/TTSBot && cd TTSBot
pip install -r server/requirements.txt          # numpy, onnxruntime, supertonic

python -m server.say --models ../supertonic-3-AX650 --out out/say "안녕하세요" "ㅇㅋ 10분 뒤에 들어갈게"
python -m server.tts_server --models ../supertonic-3-AX650 --port 8850
curl -s localhost:8850/tts -H 'Content-Type: application/json' \
  -d '{"text": "오늘 오후 3시 30분에 회의가 있습니다.", "lang": "ko", "voice": "F1"}' -o out.wav

• lang: ko, ja or en. Voices: F1–F5, M1–M5.
• The server loads every bucket in the folder (--buckets 96 keeps only the short one). Messages are split with the duration predictor to fit the largest bucket: by sentence first, then by clause, then into balanced word groups that prefer to end after a connective ending (after particles for Japanese). Each piece runs on the smallest bucket it fits.
• The model pads each piece with about 0.5 s of quiet before speech and 0.6 s after. The server trims this to 0.06 s and 0.1 s and joins pieces with 0.2–0.3 s pauses.
• See server/README.md in TTSBot for the API, error codes, text normalization for chat messages, and a systemd example.

Limitations

• Static shapes: at most 192 text tokens and 192 latent frames (about 13 s) per card call, or 96 / 6.7 s with the short bucket. Longer text has to be split, as the TTSBot server does.
• Tested with Korean, Japanese and English and the 10 preset voices. The other languages of Supertonic 3 were not measured. Custom voice styles were not tried.
• One request at a time per card.
• Tested on Windows 11 with the AXCL Windows driver. Linux hosts use the same code with libaxcl_rt.so but have not been tested on hardware.

How it was converted

• Pulsar2 7.0-patch1, target AX650 (NPU3), U16 quantization, MinMax calibration on 256 estimator inputs and 32 vocoder inputs. Calibration used voices F1 and M1, with short sentences for the 96 bucket and 7–13 s sentences for the 192 bucket.
• Graph changes for Pulsar2 and the NPU:
  • 3D Pad(mode=edge) replaced by Slice + Tile + Concat. Padded frames are first filled with the last valid frame so the static graph matches the dynamic original.
  • Rotary sin/cos and the time embedding depend on the actual lengths and the step, so they are computed on the host and fed as inputs.
  • Vocoder 1D convolutions rewritten as 1×K 2D convolutions (the 1D form failed to compile).
  • The compiled Erf (in GELU) gives wrong results on the NPU in this Pulsar2 version, so it is replaced by erf(u) ≈ tanh(1.1283792·u·(1 + 0.08943·u²)) with u clipped to ±4 (maximum error 3.6e-4).
• pulsar2 run on the compiled models matched the card output bit for bit.
• Scripts: server/convert/ in JonPark0/TTSBot.

License

These files are Derivatives of the Model under the BigScience OpenRAIL-M license of Supertonic 3. See LICENSE and NOTICE. The use restrictions in Attachment A of the license apply to you and to anyone you pass the model to. For example, restriction (e) requires saying that content is machine generated when you publish it, so a bot using this model should make clear its voice is synthetic.

---

한국어

Supertone의 Supertonic 3를 Axera Pulsar2 7.0-patch1로 변환해 AX650 / AX8850 NPU에서 돌리는 모델입니다. M5Stack LLM-8850 PCIe 카드(AXCL)에서 시험했습니다. 카드에서는 추정기(8단계)와 보코더가 돌고, 텍스트 처리·길이 예측기·텍스트 인코더·오일러 갱신은 호스트 CPU에서 돕니다.

• 정확도: 60문장 받아쓰기 오류율이 한국어 표준 0.3%, 채팅 6.5%, 일본어 2.2%, 영어 1.4%입니다. 원본은 0.0%, 3.9%, 1.7%, 1.4%입니다. 같은 잡음의 float 결과와 다르게 읽힌 문장은 두 개뿐입니다.
• 속도: 약 6.7초까지의 메시지는 약 0.18초, 약 13초까지는 긴 버킷(T=L=192)으로 한 번에 약 0.35초입니다. 카드 메모리는 두 버킷을 다 올리면 약 220 MiB, 짧은 버킷만이면 약 99 MiB입니다.
• 긴 메시지: 7–14초 메시지 30개 중 29개를 한 번에 읽어, 문장 중간에서 끊기는 곳이 사라졌습니다. 받아쓰기 정확도는 두 방식 모두 거의 완벽합니다.
• 목소리: F1–F5, M1–M5 10종 모두 카드에서 시험했고 모두 쓸 만합니다. 목소리별 결과는 위 표에 있습니다.
• 사용: JonPark0/TTSBot의 server/가 이 저장소 폴더를 --models로 받습니다.

  git lfs install
  git clone https://huggingface.co/jonpark0/supertonic-3-AX650
  git clone https://github.com/JonPark0/TTSBot && cd TTSBot
  pip install -r server/requirements.txt
  python -m server.tts_server --models ../supertonic-3-AX650
  

  길이 예측기, 텍스트 인코더, 목소리는 처음 실행할 때 supertonic 패키지가 원본 저장소에서 내려받습니다.
• 제한: 카드 호출 한 번에 텍스트 192토큰, 음성 약 13초까지입니다(짧은 버킷은 96, 6.7초). 더 긴 메시지는 TTSBot 서버가 문장, 쉼표, 띄어쓰기 순서로 나눠 읽습니다. 한국어·일본어·영어와 기본 목소리 10종만 시험했습니다. 요청은 카드당 한 번에 하나입니다. Windows 11에서 시험했고, Linux는 같은 코드지만 실제 카드로는 아직 시험하지 않았습니다.
• 라이선스: 원본과 같은 BigScience OpenRAIL-M입니다(LICENSE, NOTICE). 부록 A의 사용 제한이 그대로 적용됩니다. 예를 들어 (e)항에 따라 생성한 음성을 내보일 때 기계가 만든 것임을 밝혀야 하니, 봇에 쓸 때는 합성 음성이라는 점을 알려 주세요.