Text-to-Speech
LiteRT
LiteRT
ONNX
Core ML
PyTorch
tts
singing
singing-voice-synthesis
multilingual
expressive-tts
duration-flow
Instructions to use spybyscript/scyllasband with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- LiteRT
How to use spybyscript/scyllasband with LiteRT:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Publish inference weights only and add third-party notices
Browse files- README.md +2 -2
- SHA256SUMS.json +10 -10
- THIRD_PARTY_NOTICES.md +62 -0
- pytorch/autoencoder.pt +0 -3
- pytorch/g2p.pt +2 -2
- pytorch/release_config.json +1 -2
README.md
CHANGED
|
@@ -94,7 +94,7 @@ All ten have Spanish (`es`), Italian (`it`), French (`fr`), German (`de`) and Vi
|
|
| 94 |
- `coreai/` (155 MB): Core AI bundle for apps on iOS, iPadOS and visionOS 27 (`.aimodel` assets); it also runs on macOS 27, where the runtime uses the Core ML bundle by default. INT8 weights per output channel with FP16 compute; size buckets are functions of one asset sharing weights. The flow also runs accurately on the Neural Engine (`controls.coreai.neural_engine_assets`), which is the placement for apps that speak in the background; the GPU is the default.
|
| 95 |
- `coreml/` (142 MB): Core ML bundle for Apple silicon with iOS 18 / macOS 15 and later (compiled `.mlmodelc` assets), with INT8 weights and FP16 compute; GPU by default, CPU in background apps.
|
| 96 |
- `onnx/`: ONNX Runtime bundle with dynamic INT8 transformer weights; the vocoder stays in full precision.
|
| 97 |
-
- `pytorch/`: duration, vector, adapter/vocoder
|
| 98 |
- `SHA256SUMS.json`: published-file hashes.
|
| 99 |
|
| 100 |
Every runtime bundle includes the G2P model in a matching precision. The Apple bundles also give the G2P narrower input widths (64–512 tokens), so short phrases are converted faster.
|
|
@@ -116,4 +116,4 @@ The system supports managed voices, not arbitrary speaker cloning. Intended uses
|
|
| 116 |
|
| 117 |
## License and acknowledgments
|
| 118 |
|
| 119 |
-
Apache 2.0. LiteRT and ONNX Runtime provide inference; Vocos (`charactr/vocos-mel-24khz`) is the base of the 24 kHz waveform decoder. DeepPhonemizer-derived tooling, Wiktionary-derived vocabulary, eSpeak phonemization and Montreal Forced Aligner contributed to the frontend and alignment workflow.
|
|
|
|
| 94 |
- `coreai/` (155 MB): Core AI bundle for apps on iOS, iPadOS and visionOS 27 (`.aimodel` assets); it also runs on macOS 27, where the runtime uses the Core ML bundle by default. INT8 weights per output channel with FP16 compute; size buckets are functions of one asset sharing weights. The flow also runs accurately on the Neural Engine (`controls.coreai.neural_engine_assets`), which is the placement for apps that speak in the background; the GPU is the default.
|
| 95 |
- `coreml/` (142 MB): Core ML bundle for Apple silicon with iOS 18 / macOS 15 and later (compiled `.mlmodelc` assets), with INT8 weights and FP16 compute; GPU by default, CPU in background apps.
|
| 96 |
- `onnx/`: ONNX Runtime bundle with dynamic INT8 transformer weights; the vocoder stays in full precision.
|
| 97 |
+
- `pytorch/`: duration, vector, adapter/vocoder and G2P weights (no training state), with the release configuration assets.
|
| 98 |
- `SHA256SUMS.json`: published-file hashes.
|
| 99 |
|
| 100 |
Every runtime bundle includes the G2P model in a matching precision. The Apple bundles also give the G2P narrower input widths (64–512 tokens), so short phrases are converted faster.
|
|
|
|
| 116 |
|
| 117 |
## License and acknowledgments
|
| 118 |
|
| 119 |
+
Apache 2.0. LiteRT and ONNX Runtime provide inference; Vocos (`charactr/vocos-mel-24khz`) is the base of the 24 kHz waveform decoder. DeepPhonemizer-derived tooling, Wiktionary-derived vocabulary, eSpeak phonemization and Montreal Forced Aligner contributed to the frontend and alignment workflow. Vocos and DeepPhonemizer are MIT-licensed; their notices are in `THIRD_PARTY_NOTICES.md`.
|
SHA256SUMS.json
CHANGED
|
@@ -4,8 +4,12 @@
|
|
| 4 |
"bytes": 11357
|
| 5 |
},
|
| 6 |
"README.md": {
|
| 7 |
-
"sha256": "
|
| 8 |
-
"bytes":
|
|
|
|
|
|
|
|
|
|
|
|
|
| 9 |
},
|
| 10 |
"coreai/assets/fixed_references.json": {
|
| 11 |
"sha256": "d3c26e7e90d90fa719254ff1ff05dc4b391cd47231b7556c3c2ed5f63547925b",
|
|
@@ -419,10 +423,6 @@
|
|
| 419 |
"sha256": "4843d57a422b0a6045bbc6fc9a99afa329126a6742ff95ebcefcda660f3ee474",
|
| 420 |
"bytes": 7769
|
| 421 |
},
|
| 422 |
-
"pytorch/autoencoder.pt": {
|
| 423 |
-
"sha256": "09b837f52b5faa572d5cfff50476d9c9c449dd3b33dacbc21c95d4cbcf76d10d",
|
| 424 |
-
"bytes": 176762621
|
| 425 |
-
},
|
| 426 |
"pytorch/condition_index.json": {
|
| 427 |
"sha256": "995750fb8ff573f087f75df851fd38dd7d574e3a47773e8f150bb3b928cf251f",
|
| 428 |
"bytes": 612
|
|
@@ -436,16 +436,16 @@
|
|
| 436 |
"bytes": 786473
|
| 437 |
},
|
| 438 |
"pytorch/g2p.pt": {
|
| 439 |
-
"sha256": "
|
| 440 |
-
"bytes":
|
| 441 |
},
|
| 442 |
"pytorch/phone_vocab.json": {
|
| 443 |
"sha256": "83e6457a69948edbd9a94b404c22fd408c263eb0d0b5acaafa4926e994587721",
|
| 444 |
"bytes": 1793
|
| 445 |
},
|
| 446 |
"pytorch/release_config.json": {
|
| 447 |
-
"sha256": "
|
| 448 |
-
"bytes":
|
| 449 |
},
|
| 450 |
"pytorch/vector_estimator.pt": {
|
| 451 |
"sha256": "b5e0e57e4aae30d7d877121fa8b526f3f5105e5e97d5a0bab55e4bcba2a67b49",
|
|
|
|
| 4 |
"bytes": 11357
|
| 5 |
},
|
| 6 |
"README.md": {
|
| 7 |
+
"sha256": "4f2d6c847adb1718666fca546eb0f6730b4177a2b7ca117aa165a2a83ca3b572",
|
| 8 |
+
"bytes": 9591
|
| 9 |
+
},
|
| 10 |
+
"THIRD_PARTY_NOTICES.md": {
|
| 11 |
+
"sha256": "3b793b8ee250cc711e4012ee42e9625a8a7e788bf12cab85a1abbd17b17644ca",
|
| 12 |
+
"bytes": 2818
|
| 13 |
},
|
| 14 |
"coreai/assets/fixed_references.json": {
|
| 15 |
"sha256": "d3c26e7e90d90fa719254ff1ff05dc4b391cd47231b7556c3c2ed5f63547925b",
|
|
|
|
| 423 |
"sha256": "4843d57a422b0a6045bbc6fc9a99afa329126a6742ff95ebcefcda660f3ee474",
|
| 424 |
"bytes": 7769
|
| 425 |
},
|
|
|
|
|
|
|
|
|
|
|
|
|
| 426 |
"pytorch/condition_index.json": {
|
| 427 |
"sha256": "995750fb8ff573f087f75df851fd38dd7d574e3a47773e8f150bb3b928cf251f",
|
| 428 |
"bytes": 612
|
|
|
|
| 436 |
"bytes": 786473
|
| 437 |
},
|
| 438 |
"pytorch/g2p.pt": {
|
| 439 |
+
"sha256": "0085f558837c85cdf570337f7af37742b6158551a2c7f5078edcd35c6e84f7db",
|
| 440 |
+
"bytes": 119207485
|
| 441 |
},
|
| 442 |
"pytorch/phone_vocab.json": {
|
| 443 |
"sha256": "83e6457a69948edbd9a94b404c22fd408c263eb0d0b5acaafa4926e994587721",
|
| 444 |
"bytes": 1793
|
| 445 |
},
|
| 446 |
"pytorch/release_config.json": {
|
| 447 |
+
"sha256": "08612d7403dcb42e5061508acf59cd16e9ac1855d69a1b665fcf388ba01d84fb",
|
| 448 |
+
"bytes": 841
|
| 449 |
},
|
| 450 |
"pytorch/vector_estimator.pt": {
|
| 451 |
"sha256": "b5e0e57e4aae30d7d877121fa8b526f3f5105e5e97d5a0bab55e4bcba2a67b49",
|
THIRD_PARTY_NOTICES.md
ADDED
|
@@ -0,0 +1,62 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Third-party notices
|
| 2 |
+
|
| 3 |
+
Scylla's Band is released under the Apache License 2.0 (see `LICENSE`). Parts of the models are derived from, or
|
| 4 |
+
store objects defined by, the following MIT-licensed projects.
|
| 5 |
+
|
| 6 |
+
## Vocos
|
| 7 |
+
|
| 8 |
+
The 24 kHz waveform decoder in every bundle (`litert/`, `onnx/`, `coreml/`, `coreai/`) and in `pytorch/vocoder.pt` is
|
| 9 |
+
fine-tuned from the Vocos model `charactr/vocos-mel-24khz` (https://github.com/gemelo-ai/vocos).
|
| 10 |
+
|
| 11 |
+
```
|
| 12 |
+
MIT License
|
| 13 |
+
|
| 14 |
+
Copyright (c) 2023 Charactr Inc.
|
| 15 |
+
|
| 16 |
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
| 17 |
+
of this software and associated documentation files (the "Software"), to deal
|
| 18 |
+
in the Software without restriction, including without limitation the rights
|
| 19 |
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
| 20 |
+
copies of the Software, and to permit persons to whom the Software is
|
| 21 |
+
furnished to do so, subject to the following conditions:
|
| 22 |
+
|
| 23 |
+
The above copyright notice and this permission notice shall be included in all
|
| 24 |
+
copies or substantial portions of the Software.
|
| 25 |
+
|
| 26 |
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
| 27 |
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
| 28 |
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
| 29 |
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
| 30 |
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
| 31 |
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
| 32 |
+
SOFTWARE.
|
| 33 |
+
```
|
| 34 |
+
|
| 35 |
+
## DeepPhonemizer
|
| 36 |
+
|
| 37 |
+
The G2P model uses the DeepPhonemizer architecture, and `pytorch/g2p.pt` stores DeepPhonemizer preprocessing objects
|
| 38 |
+
(https://github.com/as-ideas/DeepPhonemizer).
|
| 39 |
+
|
| 40 |
+
```
|
| 41 |
+
MIT License
|
| 42 |
+
|
| 43 |
+
Copyright (c) 2021 Axel Springer News Media & Tech GmbH & Co. KG - Ideas Engineering
|
| 44 |
+
|
| 45 |
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
| 46 |
+
of this software and associated documentation files (the "Software"), to deal
|
| 47 |
+
in the Software without restriction, including without limitation the rights
|
| 48 |
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
| 49 |
+
copies of the Software, and to permit persons to whom the Software is
|
| 50 |
+
furnished to do so, subject to the following conditions:
|
| 51 |
+
|
| 52 |
+
The above copyright notice and this permission notice shall be included in all
|
| 53 |
+
copies or substantial portions of the Software.
|
| 54 |
+
|
| 55 |
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
| 56 |
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
| 57 |
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
| 58 |
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
| 59 |
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
| 60 |
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
| 61 |
+
SOFTWARE.
|
| 62 |
+
```
|
pytorch/autoencoder.pt
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:09b837f52b5faa572d5cfff50476d9c9c449dd3b33dacbc21c95d4cbcf76d10d
|
| 3 |
-
size 176762621
|
|
|
|
|
|
|
|
|
|
|
|
pytorch/g2p.pt
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:0085f558837c85cdf570337f7af37742b6158551a2c7f5078edcd35c6e84f7db
|
| 3 |
+
size 119207485
|
pytorch/release_config.json
CHANGED
|
@@ -1,11 +1,10 @@
|
|
| 1 |
{
|
| 2 |
"schema": "scyllasband_pytorch_release_v2",
|
| 3 |
"files": {
|
| 4 |
-
"autoencoder.pt": "09b837f52b5faa572d5cfff50476d9c9c449dd3b33dacbc21c95d4cbcf76d10d",
|
| 5 |
"condition_index.json": "995750fb8ff573f087f75df851fd38dd7d574e3a47773e8f150bb3b928cf251f",
|
| 6 |
"duration.pt": "765e8970c5c014be982a0645129682b47338656ec407373683da78318a77a5dd",
|
| 7 |
"fixed_references.json": "0120e1cf1ef9b3ee19ed3d269c4dc6f6c47e694a59bcce5bba89abe101cd20e4",
|
| 8 |
-
"g2p.pt": "
|
| 9 |
"phone_vocab.json": "83e6457a69948edbd9a94b404c22fd408c263eb0d0b5acaafa4926e994587721",
|
| 10 |
"vector_estimator.pt": "b5e0e57e4aae30d7d877121fa8b526f3f5105e5e97d5a0bab55e4bcba2a67b49",
|
| 11 |
"vocoder.pt": "e31e681d8c524f149efc2a0ea55474c28c1e1c44a5188378ba9d14918db2a15d"
|
|
|
|
| 1 |
{
|
| 2 |
"schema": "scyllasband_pytorch_release_v2",
|
| 3 |
"files": {
|
|
|
|
| 4 |
"condition_index.json": "995750fb8ff573f087f75df851fd38dd7d574e3a47773e8f150bb3b928cf251f",
|
| 5 |
"duration.pt": "765e8970c5c014be982a0645129682b47338656ec407373683da78318a77a5dd",
|
| 6 |
"fixed_references.json": "0120e1cf1ef9b3ee19ed3d269c4dc6f6c47e694a59bcce5bba89abe101cd20e4",
|
| 7 |
+
"g2p.pt": "0085f558837c85cdf570337f7af37742b6158551a2c7f5078edcd35c6e84f7db",
|
| 8 |
"phone_vocab.json": "83e6457a69948edbd9a94b404c22fd408c263eb0d0b5acaafa4926e994587721",
|
| 9 |
"vector_estimator.pt": "b5e0e57e4aae30d7d877121fa8b526f3f5105e5e97d5a0bab55e4bcba2a67b49",
|
| 10 |
"vocoder.pt": "e31e681d8c524f149efc2a0ea55474c28c1e1c44a5188378ba9d14918db2a15d"
|