spybyscript commited on
Commit
c12adda
·
verified ·
1 Parent(s): 42d06ab

LiteRT bundle exported again from the v2-20261020 weights, with faster first audio

Browse files

Same model and release_id (v2-20261020). The litert/ bundle is re-exported with lowkeytea/datasetGen full_retrain/export_litert.py --quantize int8: INT8 weights only on fully connected, convolution and embedding layers, latent buckets every 64 frames, prefix-less first-segment flow graphs, narrower span and G2P widths. Model card: the LiteRT rows, the bundle description and its validation. Needs the runtime at lowkeytea/scyllasband b2c85d0 or later for the new graphs; earlier runtimes use its full-size graphs.

README.md CHANGED
@@ -8,7 +8,7 @@ tags: [text-to-speech, tts, litert, onnx, coreml, coreai, pytorch, multilingual,
8
 
9
  # Scylla's Band v2
10
 
11
- Release **`v2-20261020`**. The [Scylla's Band runtime](https://github.com/lowkeytea/scyllasband) supports this release. Tag `v2-20261020-apple` adds Core ML and Core AI bundles of the same model; the runtime downloads from that tag.
12
 
13
  The model exposes **energy, tension, valence and assertiveness** on **0–4 scales with neutral at 2**. Combining the controls changes the generated delivery; a request is not a guarantee of a particular perceived emotion or exact external scorer value.
14
 
@@ -88,7 +88,7 @@ Long text is spoken as passages within the trained target lengths (64 to 420 lat
88
 
89
  Measured with the default sampler (Euler, 4 steps) through the native runtime: `scyllasband_speak` from [libscyllasband](https://github.com/lowkeytea/scyllasband/tree/main/libscyllasband) on the Mac and Android, and the [iOS sample](https://github.com/lowkeytea/scyllasband/tree/main/examples/ios)'s benchmark mode on the iPhone. The text is three sentences, 11.9 s of audio: "The last train leaves at midnight, so pack light. Bring a warm coat, and don't forget the map. If we miss it, we walk, and nobody wants to walk that far in the rain." (voice Scylla, seed 7).
90
 
91
- - **First run**: from launch to first audio on the first launch after installing the bundle. Core AI specializes its graphs for the device then and caches them until the next OS update.
92
  - **Cold start**: from launch to first audio on a later launch, including loading the bundle.
93
  - **Warm first audio**: from a request to its first audio in a process that has already spoken (median of three requests).
94
  - **Real-time speed**: seconds of audio generated per second when warm, rounded.
@@ -100,14 +100,14 @@ Measured with the default sampler (Euler, 4 steps) through the native runtime: `
100
  | | Core AI | GPU (default) | 5.88 s | 0.39 s | 27 ms | ~200× |
101
  | | Core AI | Neural Engine | 38.9 s | 0.52 s | 320 ms | ~17× |
102
  | | Core AI | CPU | 3.48 s | 1.13 s | 1036 ms | ~3.5× |
103
- | | LiteRT | CPU | 1.00 s | 1.04 s | 192 ms | ~22× |
104
  | | ONNX Runtime | CPU | 0.92 s | 0.94 s | 262 ms | ~17× |
105
  | iPhone 17 Pro Max (iOS 27.0.1) | Core ML | GPU (default) | 3.82 s | 1.49 s | 92 ms | ~53× |
106
  | | Core ML | CPU | 2.71 s | 0.51 s | 359 ms | ~10× |
107
  | | Core AI | GPU (default) | 7.68 s | 0.48 s | 84 ms | ~57× |
108
  | | Core AI | Neural Engine | 50.1 s | 0.67 s | 401 ms | ~14× |
109
  | | Core AI | CPU | 5.19 s | 1.42 s | 1368 ms | ~2.7× |
110
- | Galaxy Z Fold 8, Snapdragon SM8850 (Android 17) | LiteRT | CPU | 2.26 s | 1.87 s | 250 ms | ~16× |
111
  | | ONNX Runtime | CPU | 2.35 s | 1.72 s | 535 ms | ~7.7× |
112
 
113
  The GPU is the default on Apple platforms. The Neural Engine placement runs the flow on the Neural Engine and the other graphs on the CPU, for apps that speak in the background, where iOS does not allow GPU work. Core ML reloads its graphs on every launch, which accounts for most of its cold start on the GPU.
@@ -120,7 +120,7 @@ All fourteen have Spanish (`es`), Italian (`it`), French (`fr`), German (`de`) a
120
 
121
  ## Artifacts
122
 
123
- - `litert/` (228 MB): LiteRT bundle; default download. Transformer and vocoder weights are dynamic-range INT8; each size bucket is a signature of one model file, sharing weights.
124
  - `coreai/` (213 MB): Core AI bundle for apps on iOS, iPadOS and visionOS 27 (`.aimodel` assets); it also runs on macOS 27, where the runtime uses the Core ML bundle by default. INT8 weights per output channel with FP16 compute; size buckets are functions of one asset sharing weights. The flow also runs accurately on the Neural Engine (`controls.coreai.neural_engine_assets`), which is the placement for apps that speak in the background; the GPU is the default.
125
  - `coreml/` (200 MB): Core ML bundle for Apple silicon with iOS 18 / visionOS 2 / macOS 15 and later (compiled `.mlmodelc` assets), with INT8 weights and FP16 compute; GPU by default, CPU in background apps.
126
  - `onnx/` (323 MB): ONNX Runtime bundle with dynamic INT8 transformer weights; the vocoder stays in full precision.
@@ -128,7 +128,7 @@ All fourteen have Spanish (`es`), Italian (`it`), French (`fr`), German (`de`) a
128
  - `pytorch/`: duration, vector, adapter/vocoder and G2P weights (no training state), with the release configuration assets.
129
  - `SHA256SUMS.json`: published-file hashes.
130
 
131
- Every runtime bundle includes the G2P model in a matching precision. The Apple bundles also give the G2P narrower input widths (64–512 tokens), so short phrases are converted faster.
132
 
133
  ## Validation and limitations
134
 
@@ -136,7 +136,8 @@ Release validation separates graph accuracy, frontend/timing behaviour and perce
136
 
137
  - Component comparisons against PyTorch during export, and end-to-end comparison of full passages against the PyTorch reference path.
138
  - The distilled flow at 4 Euler steps compared with the base flow on the same passages, seeds and noise: closer to the base flow's 32-step output than the base flow's own 8 Heun steps, and compared blind with the 32-step base flow by listening across all fourteen voices.
139
- - Quantized bundles compared with full precision on real sentences (48 passages, six voices): identical phonemes; identical phone durations for 97.7% (ONNX), 97.9% (LiteRT) and 99.6% (WebGPU) of phones, the rest one frame apart; generated latents within 2% median relative error; followed by listening across all fourteen voices.
 
140
  - G2P conversions compared with the full-precision G2P on multilingual text.
141
  - Core ML and Core AI bundles compared with the PyTorch model on every graph and on real passages, on each compute unit they are used with (GPU, CPU, and for Core AI the Neural Engine): identical phonemes apart from two near-tie stress marks also seen in FP16, fewer changed phone durations than the LiteRT bundle, and smaller latent and spectral differences. The Apple exports rewrite three computations into equivalent forms that FP16 and the Neural Engine evaluate exactly (span pooling, the modifier bit decoding, the duration model's combined phone/segment index), and scale the flow's residual stream; in FP32 the rewritten graphs are bit-identical to the trained ones.
142
 
 
8
 
9
  # Scylla's Band v2
10
 
11
+ Release **`v2-20261020`**. The [Scylla's Band runtime](https://github.com/lowkeytea/scyllasband) supports this release. Tag `v2-20261020-apple` adds Core ML and Core AI bundles of the same model, and tag `v2-20261020-litert` adds those and a faster LiteRT bundle, exported again from the same weights; the runtime downloads from `v2-20261020-litert`.
12
 
13
  The model exposes **energy, tension, valence and assertiveness** on **0–4 scales with neutral at 2**. Combining the controls changes the generated delivery; a request is not a guarantee of a particular perceived emotion or exact external scorer value.
14
 
 
88
 
89
  Measured with the default sampler (Euler, 4 steps) through the native runtime: `scyllasband_speak` from [libscyllasband](https://github.com/lowkeytea/scyllasband/tree/main/libscyllasband) on the Mac and Android, and the [iOS sample](https://github.com/lowkeytea/scyllasband/tree/main/examples/ios)'s benchmark mode on the iPhone. The text is three sentences, 11.9 s of audio: "The last train leaves at midnight, so pack light. Bring a warm coat, and don't forget the map. If we miss it, we walk, and nobody wants to walk that far in the rain." (voice Scylla, seed 7).
90
 
91
+ - **First run**: from launch to first audio on the first launch after installing the bundle. Core AI specializes its graphs for the device then and caches them until the next OS update; LiteRT writes its packed weights to a cache file beside each graph.
92
  - **Cold start**: from launch to first audio on a later launch, including loading the bundle.
93
  - **Warm first audio**: from a request to its first audio in a process that has already spoken (median of three requests).
94
  - **Real-time speed**: seconds of audio generated per second when warm, rounded.
 
100
  | | Core AI | GPU (default) | 5.88 s | 0.39 s | 27 ms | ~200× |
101
  | | Core AI | Neural Engine | 38.9 s | 0.52 s | 320 ms | ~17× |
102
  | | Core AI | CPU | 3.48 s | 1.13 s | 1036 ms | ~3.5× |
103
+ | | LiteRT | CPU | 0.47 s | 0.35 s | 121 ms | ~29× |
104
  | | ONNX Runtime | CPU | 0.92 s | 0.94 s | 262 ms | ~17× |
105
  | iPhone 17 Pro Max (iOS 27.0.1) | Core ML | GPU (default) | 3.82 s | 1.49 s | 92 ms | ~53× |
106
  | | Core ML | CPU | 2.71 s | 0.51 s | 359 ms | ~10× |
107
  | | Core AI | GPU (default) | 7.68 s | 0.48 s | 84 ms | ~57× |
108
  | | Core AI | Neural Engine | 50.1 s | 0.67 s | 401 ms | ~14× |
109
  | | Core AI | CPU | 5.19 s | 1.42 s | 1368 ms | ~2.7× |
110
+ | Galaxy Z Fold 8, Snapdragon SM8850 (Android 17) | LiteRT | CPU | 1.47 s | 0.48 s | 132 ms | ~23× |
111
  | | ONNX Runtime | CPU | 2.35 s | 1.72 s | 535 ms | ~7.7× |
112
 
113
  The GPU is the default on Apple platforms. The Neural Engine placement runs the flow on the Neural Engine and the other graphs on the CPU, for apps that speak in the background, where iOS does not allow GPU work. Core ML reloads its graphs on every launch, which accounts for most of its cold start on the GPU.
 
120
 
121
  ## Artifacts
122
 
123
+ - `litert/` (267 MB): LiteRT bundle; default download. Dynamic-range INT8 weights on the fully connected, convolution and embedding layers; the vocoder's depthwise convolutions stay in full precision, which keeps them on XNNPACK. Each size is a signature of one model file, sharing weights: latent buckets every 64 frames up to 512 (then 768 and 1024), flow graphs without the 96 prefix frames for a request's first segment (up to 448 frames), and narrower duration and span-context (128 and 256 phones) and G2P (32–256 tokens) widths beside the full ones. (Tag `v2-20261020`'s LiteRT bundle, 228 MB, has six latent buckets and full-width graphs only.)
124
  - `coreai/` (213 MB): Core AI bundle for apps on iOS, iPadOS and visionOS 27 (`.aimodel` assets); it also runs on macOS 27, where the runtime uses the Core ML bundle by default. INT8 weights per output channel with FP16 compute; size buckets are functions of one asset sharing weights. The flow also runs accurately on the Neural Engine (`controls.coreai.neural_engine_assets`), which is the placement for apps that speak in the background; the GPU is the default.
125
  - `coreml/` (200 MB): Core ML bundle for Apple silicon with iOS 18 / visionOS 2 / macOS 15 and later (compiled `.mlmodelc` assets), with INT8 weights and FP16 compute; GPU by default, CPU in background apps.
126
  - `onnx/` (323 MB): ONNX Runtime bundle with dynamic INT8 transformer weights; the vocoder stays in full precision.
 
128
  - `pytorch/`: duration, vector, adapter/vocoder and G2P weights (no training state), with the release configuration assets.
129
  - `SHA256SUMS.json`: published-file hashes.
130
 
131
+ Every runtime bundle includes the G2P model in a matching precision. The Apple and LiteRT bundles also give the G2P narrower input widths (64–512 and 32–512 tokens), so short phrases are converted faster.
132
 
133
  ## Validation and limitations
134
 
 
136
 
137
  - Component comparisons against PyTorch during export, and end-to-end comparison of full passages against the PyTorch reference path.
138
  - The distilled flow at 4 Euler steps compared with the base flow on the same passages, seeds and noise: closer to the base flow's 32-step output than the base flow's own 8 Heun steps, and compared blind with the 32-step base flow by listening across all fourteen voices.
139
+ - Quantized bundles compared with full precision on real sentences (48 passages, six voices): identical phonemes; identical phone durations for 97.7% (ONNX), 97.9% (LiteRT, the bundle of tag `v2-20261020`) and 99.6% (WebGPU) of phones, the rest one frame apart; generated latents within 2% median relative error; followed by listening across all fourteen voices.
140
+ - The LiteRT bundle of tag `v2-20261020-litert` compared with the full-precision PyTorch graphs on every graph call the runtime makes speaking the runtime repository's sample text (`samples/gallery_script.json`'s line for each of the 84 voice/locale pairs and its delivery lines in three voices, and the first paragraph of `data/test_document.txt`: 103 requests, 1,091 graph calls): identical phonemes in all 136 G2P calls (the `v2-20261020` LiteRT G2P differs in 2); phone durations rounding to the same frame count for 97.9% of phones, at most 0.62 frames apart; flow velocity within 1.3% median relative error, latents after the 4 Euler steps within 2.1% (median); vocoder output 12.3 dB median SNR against the full-precision vocoder (6.9 dB for the `v2-20261020` bundle); and every graph identical with and without XNNPACK's weight cache.
141
  - G2P conversions compared with the full-precision G2P on multilingual text.
142
  - Core ML and Core AI bundles compared with the PyTorch model on every graph and on real passages, on each compute unit they are used with (GPU, CPU, and for Core AI the Neural Engine): identical phonemes apart from two near-tie stress marks also seen in FP16, fewer changed phone durations than the LiteRT bundle, and smaller latent and spectral differences. The Apple exports rewrite three computations into equivalent forms that FP16 and the Neural Engine evaluate exactly (span pooling, the modifier bit decoding, the duration model's combined phone/segment index), and scale the flow's residual stream; in FP32 the rewritten graphs are bit-identical to the trained ones.
143
 
SHA256SUMS.json CHANGED
@@ -4,8 +4,8 @@
4
  "bytes": 11357
5
  },
6
  "README.md": {
7
- "sha256": "20c2a808876a082ecf1916ca0708fc7d555bf1d86f9de8b596932c0747ecf06d",
8
- "bytes": 13098
9
  },
10
  "THIRD_PARTY_NOTICES.md": {
11
  "sha256": "757a3eca5f8284352d471887af2c31b65c5d23d2a642f0ae77e6a11c150dba39",
@@ -280,28 +280,28 @@
280
  "bytes": 613
281
  },
282
  "litert/litert/duration_predictor.tflite": {
283
- "sha256": "02d815abcae4b3915f7569a1796364a5b7d70adc8e928fc2b7beb33566f6b444",
284
- "bytes": 11668736
285
  },
286
  "litert/litert/g2p.tflite": {
287
- "sha256": "a416666d1088ccb6fe59b91c7c1c343aad4374f08d4c8e21726fc7aecb79bb0c",
288
- "bytes": 28626144
289
  },
290
  "litert/litert/vector_context_encoder.tflite": {
291
- "sha256": "de37ff6548202785b9a7c3edac609b9d94d11f437cc959bcfff61fc909272e5e",
292
- "bytes": 11894656
293
  },
294
  "litert/litert/vector_estimator.tflite": {
295
- "sha256": "cebf54a5b5009c4bf47982cf370a61b36bf3279ea8c16f7c0d974a4e5534ed56",
296
- "bytes": 146540928
297
  },
298
  "litert/litert/vocoder.tflite": {
299
- "sha256": "bc04828ad6575713d597585b117081c2f0af6185b6ea9eee6c979dd6b94cbf74",
300
- "bytes": 38615216
301
  },
302
  "litert/manifest.json": {
303
- "sha256": "e1c4791b0c9b24e5d3b97763c72b95922617402278e5c959736b3aa8f33c703c",
304
- "bytes": 21826
305
  },
306
  "onnx/assets/fixed_references.json": {
307
  "sha256": "f227c55cab5f6b8cffa33c3dd743e69b26c7e446be5b9cd9a286095cabaa249e",
 
4
  "bytes": 11357
5
  },
6
  "README.md": {
7
+ "sha256": "7e29373e185bf4964d8ff5b505c4c83d9ce9ccd52894112aa87aa5652fda217c",
8
+ "bytes": 14660
9
  },
10
  "THIRD_PARTY_NOTICES.md": {
11
  "sha256": "757a3eca5f8284352d471887af2c31b65c5d23d2a642f0ae77e6a11c150dba39",
 
280
  "bytes": 613
281
  },
282
  "litert/litert/duration_predictor.tflite": {
283
+ "sha256": "124182d4d58e5b7bd94d39797f4e6c445d401dd66966c5dfe925f27995e84490",
284
+ "bytes": 13195440
285
  },
286
  "litert/litert/g2p.tflite": {
287
+ "sha256": "557127105fed3041845d7bd1e36d6aeea66ecb9e329fa29eb8d0d048b4c0fad9",
288
+ "bytes": 33979216
289
  },
290
  "litert/litert/vector_context_encoder.tflite": {
291
+ "sha256": "b7a647f7fee08d7080f46473290c2452b6f51dd09a0b28eaa8e10d51ed14fd58",
292
+ "bytes": 12391616
293
  },
294
  "litert/litert/vector_estimator.tflite": {
295
+ "sha256": "689c832e078f20df4c088a0230e369767871fc92ca72aae0e18be2ad7cfb25ca",
296
+ "bytes": 174492064
297
  },
298
  "litert/litert/vocoder.tflite": {
299
+ "sha256": "1356078d3631e6a5b27f8e5584abd73a5f483bc4b4364a94f3dfd094423bdf77",
300
+ "bytes": 44714640
301
  },
302
  "litert/manifest.json": {
303
+ "sha256": "ab0f9e35a92b307c7d3d089ecd0f00b39242a5806ac4191a38284e24f3437a4b",
304
+ "bytes": 35746
305
  },
306
  "onnx/assets/fixed_references.json": {
307
  "sha256": "f227c55cab5f6b8cffa33c3dd743e69b26c7e446be5b9cd9a286095cabaa249e",
litert/litert/duration_predictor.tflite CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:02d815abcae4b3915f7569a1796364a5b7d70adc8e928fc2b7beb33566f6b444
3
- size 11668736
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:124182d4d58e5b7bd94d39797f4e6c445d401dd66966c5dfe925f27995e84490
3
+ size 13195440
litert/litert/g2p.tflite CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:a416666d1088ccb6fe59b91c7c1c343aad4374f08d4c8e21726fc7aecb79bb0c
3
- size 28626144
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:557127105fed3041845d7bd1e36d6aeea66ecb9e329fa29eb8d0d048b4c0fad9
3
+ size 33979216
litert/litert/vector_context_encoder.tflite CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:de37ff6548202785b9a7c3edac609b9d94d11f437cc959bcfff61fc909272e5e
3
- size 11894656
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b7a647f7fee08d7080f46473290c2452b6f51dd09a0b28eaa8e10d51ed14fd58
3
+ size 12391616
litert/litert/vector_estimator.tflite CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:cebf54a5b5009c4bf47982cf370a61b36bf3279ea8c16f7c0d974a4e5534ed56
3
- size 146540928
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:689c832e078f20df4c088a0230e369767871fc92ca72aae0e18be2ad7cfb25ca
3
+ size 174492064
litert/litert/vocoder.tflite CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:bc04828ad6575713d597585b117081c2f0af6185b6ea9eee6c979dd6b94cbf74
3
- size 38615216
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1356078d3631e6a5b27f8e5584abd73a5f483bc4b4364a94f3dfd094423bdf77
3
+ size 44714640
litert/manifest.json CHANGED
@@ -42,12 +42,56 @@
42
  ],
43
  "required": true
44
  },
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
45
  "g2p": {
46
  "artifacts": {
47
  "litert": {
48
  "format": "litert",
49
  "path": "litert/g2p.tflite",
50
- "signature": "serving_default"
51
  }
52
  },
53
  "inputs": [
@@ -58,6 +102,102 @@
58
  ],
59
  "required": true
60
  },
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
61
  "vector_context_encoder": {
62
  "artifacts": {
63
  "litert": {
@@ -76,6 +216,42 @@
76
  ],
77
  "required": true
78
  },
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
79
  "vector_estimator": {
80
  "artifacts": {
81
  "litert": {
@@ -138,6 +314,37 @@
138
  ],
139
  "required": false
140
  },
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
141
  "vector_estimator_256": {
142
  "artifacts": {
143
  "litert": {
@@ -169,6 +376,37 @@
169
  ],
170
  "required": false
171
  },
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
172
  "vector_estimator_384": {
173
  "artifacts": {
174
  "litert": {
@@ -200,6 +438,37 @@
200
  ],
201
  "required": false
202
  },
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
203
  "vector_estimator_512": {
204
  "artifacts": {
205
  "litert": {
@@ -262,6 +531,192 @@
262
  ],
263
  "required": false
264
  },
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
265
  "vocoder": {
266
  "artifacts": {
267
  "litert": {
@@ -302,6 +757,26 @@
302
  ],
303
  "required": false
304
  },
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
305
  "vocoder_256": {
306
  "artifacts": {
307
  "litert": {
@@ -322,6 +797,26 @@
322
  ],
323
  "required": false
324
  },
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
325
  "vocoder_384": {
326
  "artifacts": {
327
  "litert": {
@@ -342,6 +837,26 @@
342
  ],
343
  "required": false
344
  },
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
345
  "vocoder_512": {
346
  "artifacts": {
347
  "litert": {
@@ -584,29 +1099,70 @@
584
  "latent_frames": 1024,
585
  "phone_frames": 512
586
  },
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
587
  "graph_input_contract": "scyllasband_measured_delivery_v2",
588
  "litert": {
 
 
 
 
 
 
 
 
 
 
589
  "quantization": "dynamic_wi8_afp32",
590
  "report": {
591
  "duration_predictor": {
592
- "bytes": 11668736,
593
  "quantized": "dynamic_wi8_afp32"
594
  },
595
  "vector_context_encoder": {
596
- "bytes": 11894656,
597
  "quantized": "dynamic_wi8_afp32"
598
  },
599
  "vector_estimator": {
600
- "bytes": 146540928,
601
  "quantized": "dynamic_wi8_afp32"
602
  },
603
  "vocoder": {
604
- "bytes": 38615216,
605
  "quantized": "dynamic_wi8_afp32"
606
  }
607
  },
608
  "shared_bucket_weights": true,
609
  "signatures_per_component": true,
 
610
  "vocoder": "dynamic_wi8_afp32"
611
  },
612
  "pitch": {
@@ -637,6 +1193,18 @@
637
  ],
638
  "default": "euler"
639
  },
 
 
 
 
 
 
 
 
 
 
 
 
640
  "span_conditioning": {
641
  "context_max_phones": 512,
642
  "context_phones_each_side": 180,
@@ -678,20 +1246,44 @@
678
  "default": false,
679
  "latent_frames": 128,
680
  "vector_estimator": "vector_estimator_128",
 
681
  "vocoder": "vocoder_128"
682
  },
 
 
 
 
 
 
 
683
  {
684
  "default": false,
685
  "latent_frames": 256,
686
  "vector_estimator": "vector_estimator_256",
 
687
  "vocoder": "vocoder_256"
688
  },
 
 
 
 
 
 
 
689
  {
690
  "default": false,
691
  "latent_frames": 384,
692
  "vector_estimator": "vector_estimator_384",
 
693
  "vocoder": "vocoder_384"
694
  },
 
 
 
 
 
 
 
695
  {
696
  "default": false,
697
  "latent_frames": 512,
 
42
  ],
43
  "required": true
44
  },
45
+ "duration_predictor_128": {
46
+ "artifacts": {
47
+ "litert": {
48
+ "format": "litert",
49
+ "path": "litert/duration_predictor.tflite",
50
+ "signature": "duration_predictor_128"
51
+ }
52
+ },
53
+ "inputs": [
54
+ "span_phone_ids",
55
+ "span_segment_ids",
56
+ "span_mask",
57
+ "voice_id",
58
+ "language_id",
59
+ "delivery_values",
60
+ "delivery_present"
61
+ ],
62
+ "outputs": [
63
+ "durations"
64
+ ],
65
+ "required": false
66
+ },
67
+ "duration_predictor_256": {
68
+ "artifacts": {
69
+ "litert": {
70
+ "format": "litert",
71
+ "path": "litert/duration_predictor.tflite",
72
+ "signature": "duration_predictor_256"
73
+ }
74
+ },
75
+ "inputs": [
76
+ "span_phone_ids",
77
+ "span_segment_ids",
78
+ "span_mask",
79
+ "voice_id",
80
+ "language_id",
81
+ "delivery_values",
82
+ "delivery_present"
83
+ ],
84
+ "outputs": [
85
+ "durations"
86
+ ],
87
+ "required": false
88
+ },
89
  "g2p": {
90
  "artifacts": {
91
  "litert": {
92
  "format": "litert",
93
  "path": "litert/g2p.tflite",
94
+ "signature": "g2p"
95
  }
96
  },
97
  "inputs": [
 
102
  ],
103
  "required": true
104
  },
105
+ "g2p_128": {
106
+ "artifacts": {
107
+ "litert": {
108
+ "format": "litert",
109
+ "path": "litert/g2p.tflite",
110
+ "signature": "g2p_128"
111
+ }
112
+ },
113
+ "inputs": [
114
+ "text"
115
+ ],
116
+ "outputs": [
117
+ "logits"
118
+ ],
119
+ "required": false
120
+ },
121
+ "g2p_192": {
122
+ "artifacts": {
123
+ "litert": {
124
+ "format": "litert",
125
+ "path": "litert/g2p.tflite",
126
+ "signature": "g2p_192"
127
+ }
128
+ },
129
+ "inputs": [
130
+ "text"
131
+ ],
132
+ "outputs": [
133
+ "logits"
134
+ ],
135
+ "required": false
136
+ },
137
+ "g2p_256": {
138
+ "artifacts": {
139
+ "litert": {
140
+ "format": "litert",
141
+ "path": "litert/g2p.tflite",
142
+ "signature": "g2p_256"
143
+ }
144
+ },
145
+ "inputs": [
146
+ "text"
147
+ ],
148
+ "outputs": [
149
+ "logits"
150
+ ],
151
+ "required": false
152
+ },
153
+ "g2p_32": {
154
+ "artifacts": {
155
+ "litert": {
156
+ "format": "litert",
157
+ "path": "litert/g2p.tflite",
158
+ "signature": "g2p_32"
159
+ }
160
+ },
161
+ "inputs": [
162
+ "text"
163
+ ],
164
+ "outputs": [
165
+ "logits"
166
+ ],
167
+ "required": false
168
+ },
169
+ "g2p_64": {
170
+ "artifacts": {
171
+ "litert": {
172
+ "format": "litert",
173
+ "path": "litert/g2p.tflite",
174
+ "signature": "g2p_64"
175
+ }
176
+ },
177
+ "inputs": [
178
+ "text"
179
+ ],
180
+ "outputs": [
181
+ "logits"
182
+ ],
183
+ "required": false
184
+ },
185
+ "g2p_96": {
186
+ "artifacts": {
187
+ "litert": {
188
+ "format": "litert",
189
+ "path": "litert/g2p.tflite",
190
+ "signature": "g2p_96"
191
+ }
192
+ },
193
+ "inputs": [
194
+ "text"
195
+ ],
196
+ "outputs": [
197
+ "logits"
198
+ ],
199
+ "required": false
200
+ },
201
  "vector_context_encoder": {
202
  "artifacts": {
203
  "litert": {
 
216
  ],
217
  "required": true
218
  },
219
+ "vector_context_encoder_128": {
220
+ "artifacts": {
221
+ "litert": {
222
+ "format": "litert",
223
+ "path": "litert/vector_context_encoder.tflite",
224
+ "signature": "vector_context_encoder_128"
225
+ }
226
+ },
227
+ "inputs": [
228
+ "span_context_phone_ids",
229
+ "span_context_segment_ids",
230
+ "span_context_mask"
231
+ ],
232
+ "outputs": [
233
+ "span_context_hidden"
234
+ ],
235
+ "required": false
236
+ },
237
+ "vector_context_encoder_256": {
238
+ "artifacts": {
239
+ "litert": {
240
+ "format": "litert",
241
+ "path": "litert/vector_context_encoder.tflite",
242
+ "signature": "vector_context_encoder_256"
243
+ }
244
+ },
245
+ "inputs": [
246
+ "span_context_phone_ids",
247
+ "span_context_segment_ids",
248
+ "span_context_mask"
249
+ ],
250
+ "outputs": [
251
+ "span_context_hidden"
252
+ ],
253
+ "required": false
254
+ },
255
  "vector_estimator": {
256
  "artifacts": {
257
  "litert": {
 
314
  ],
315
  "required": false
316
  },
317
+ "vector_estimator_192": {
318
+ "artifacts": {
319
+ "litert": {
320
+ "format": "litert",
321
+ "path": "litert/vector_estimator.tflite",
322
+ "signature": "vector_estimator_192"
323
+ }
324
+ },
325
+ "inputs": [
326
+ "noise",
327
+ "time",
328
+ "expanded_phone_ids",
329
+ "voice_id",
330
+ "language_id",
331
+ "delivery_values",
332
+ "delivery_present",
333
+ "latent_mask",
334
+ "span_context_hidden",
335
+ "prefix_latents",
336
+ "prefix_mask",
337
+ "expanded_boundary_event_ids",
338
+ "expanded_modifier_event_ids",
339
+ "expanded_phone_phase",
340
+ "expanded_phone_log_duration",
341
+ "expanded_sentence_type_ids"
342
+ ],
343
+ "outputs": [
344
+ "velocity"
345
+ ],
346
+ "required": false
347
+ },
348
  "vector_estimator_256": {
349
  "artifacts": {
350
  "litert": {
 
376
  ],
377
  "required": false
378
  },
379
+ "vector_estimator_320": {
380
+ "artifacts": {
381
+ "litert": {
382
+ "format": "litert",
383
+ "path": "litert/vector_estimator.tflite",
384
+ "signature": "vector_estimator_320"
385
+ }
386
+ },
387
+ "inputs": [
388
+ "noise",
389
+ "time",
390
+ "expanded_phone_ids",
391
+ "voice_id",
392
+ "language_id",
393
+ "delivery_values",
394
+ "delivery_present",
395
+ "latent_mask",
396
+ "span_context_hidden",
397
+ "prefix_latents",
398
+ "prefix_mask",
399
+ "expanded_boundary_event_ids",
400
+ "expanded_modifier_event_ids",
401
+ "expanded_phone_phase",
402
+ "expanded_phone_log_duration",
403
+ "expanded_sentence_type_ids"
404
+ ],
405
+ "outputs": [
406
+ "velocity"
407
+ ],
408
+ "required": false
409
+ },
410
  "vector_estimator_384": {
411
  "artifacts": {
412
  "litert": {
 
438
  ],
439
  "required": false
440
  },
441
+ "vector_estimator_448": {
442
+ "artifacts": {
443
+ "litert": {
444
+ "format": "litert",
445
+ "path": "litert/vector_estimator.tflite",
446
+ "signature": "vector_estimator_448"
447
+ }
448
+ },
449
+ "inputs": [
450
+ "noise",
451
+ "time",
452
+ "expanded_phone_ids",
453
+ "voice_id",
454
+ "language_id",
455
+ "delivery_values",
456
+ "delivery_present",
457
+ "latent_mask",
458
+ "span_context_hidden",
459
+ "prefix_latents",
460
+ "prefix_mask",
461
+ "expanded_boundary_event_ids",
462
+ "expanded_modifier_event_ids",
463
+ "expanded_phone_phase",
464
+ "expanded_phone_log_duration",
465
+ "expanded_sentence_type_ids"
466
+ ],
467
+ "outputs": [
468
+ "velocity"
469
+ ],
470
+ "required": false
471
+ },
472
  "vector_estimator_512": {
473
  "artifacts": {
474
  "litert": {
 
531
  ],
532
  "required": false
533
  },
534
+ "vector_estimator_first_128": {
535
+ "artifacts": {
536
+ "litert": {
537
+ "format": "litert",
538
+ "path": "litert/vector_estimator.tflite",
539
+ "signature": "vector_estimator_first_128"
540
+ }
541
+ },
542
+ "inputs": [
543
+ "noise",
544
+ "time",
545
+ "expanded_phone_ids",
546
+ "voice_id",
547
+ "language_id",
548
+ "delivery_values",
549
+ "delivery_present",
550
+ "latent_mask",
551
+ "span_context_hidden",
552
+ "prefix_latents",
553
+ "prefix_mask",
554
+ "expanded_boundary_event_ids",
555
+ "expanded_modifier_event_ids",
556
+ "expanded_phone_phase",
557
+ "expanded_phone_log_duration",
558
+ "expanded_sentence_type_ids"
559
+ ],
560
+ "outputs": [
561
+ "velocity"
562
+ ],
563
+ "required": false
564
+ },
565
+ "vector_estimator_first_192": {
566
+ "artifacts": {
567
+ "litert": {
568
+ "format": "litert",
569
+ "path": "litert/vector_estimator.tflite",
570
+ "signature": "vector_estimator_first_192"
571
+ }
572
+ },
573
+ "inputs": [
574
+ "noise",
575
+ "time",
576
+ "expanded_phone_ids",
577
+ "voice_id",
578
+ "language_id",
579
+ "delivery_values",
580
+ "delivery_present",
581
+ "latent_mask",
582
+ "span_context_hidden",
583
+ "prefix_latents",
584
+ "prefix_mask",
585
+ "expanded_boundary_event_ids",
586
+ "expanded_modifier_event_ids",
587
+ "expanded_phone_phase",
588
+ "expanded_phone_log_duration",
589
+ "expanded_sentence_type_ids"
590
+ ],
591
+ "outputs": [
592
+ "velocity"
593
+ ],
594
+ "required": false
595
+ },
596
+ "vector_estimator_first_256": {
597
+ "artifacts": {
598
+ "litert": {
599
+ "format": "litert",
600
+ "path": "litert/vector_estimator.tflite",
601
+ "signature": "vector_estimator_first_256"
602
+ }
603
+ },
604
+ "inputs": [
605
+ "noise",
606
+ "time",
607
+ "expanded_phone_ids",
608
+ "voice_id",
609
+ "language_id",
610
+ "delivery_values",
611
+ "delivery_present",
612
+ "latent_mask",
613
+ "span_context_hidden",
614
+ "prefix_latents",
615
+ "prefix_mask",
616
+ "expanded_boundary_event_ids",
617
+ "expanded_modifier_event_ids",
618
+ "expanded_phone_phase",
619
+ "expanded_phone_log_duration",
620
+ "expanded_sentence_type_ids"
621
+ ],
622
+ "outputs": [
623
+ "velocity"
624
+ ],
625
+ "required": false
626
+ },
627
+ "vector_estimator_first_320": {
628
+ "artifacts": {
629
+ "litert": {
630
+ "format": "litert",
631
+ "path": "litert/vector_estimator.tflite",
632
+ "signature": "vector_estimator_first_320"
633
+ }
634
+ },
635
+ "inputs": [
636
+ "noise",
637
+ "time",
638
+ "expanded_phone_ids",
639
+ "voice_id",
640
+ "language_id",
641
+ "delivery_values",
642
+ "delivery_present",
643
+ "latent_mask",
644
+ "span_context_hidden",
645
+ "prefix_latents",
646
+ "prefix_mask",
647
+ "expanded_boundary_event_ids",
648
+ "expanded_modifier_event_ids",
649
+ "expanded_phone_phase",
650
+ "expanded_phone_log_duration",
651
+ "expanded_sentence_type_ids"
652
+ ],
653
+ "outputs": [
654
+ "velocity"
655
+ ],
656
+ "required": false
657
+ },
658
+ "vector_estimator_first_384": {
659
+ "artifacts": {
660
+ "litert": {
661
+ "format": "litert",
662
+ "path": "litert/vector_estimator.tflite",
663
+ "signature": "vector_estimator_first_384"
664
+ }
665
+ },
666
+ "inputs": [
667
+ "noise",
668
+ "time",
669
+ "expanded_phone_ids",
670
+ "voice_id",
671
+ "language_id",
672
+ "delivery_values",
673
+ "delivery_present",
674
+ "latent_mask",
675
+ "span_context_hidden",
676
+ "prefix_latents",
677
+ "prefix_mask",
678
+ "expanded_boundary_event_ids",
679
+ "expanded_modifier_event_ids",
680
+ "expanded_phone_phase",
681
+ "expanded_phone_log_duration",
682
+ "expanded_sentence_type_ids"
683
+ ],
684
+ "outputs": [
685
+ "velocity"
686
+ ],
687
+ "required": false
688
+ },
689
+ "vector_estimator_first_448": {
690
+ "artifacts": {
691
+ "litert": {
692
+ "format": "litert",
693
+ "path": "litert/vector_estimator.tflite",
694
+ "signature": "vector_estimator_first_448"
695
+ }
696
+ },
697
+ "inputs": [
698
+ "noise",
699
+ "time",
700
+ "expanded_phone_ids",
701
+ "voice_id",
702
+ "language_id",
703
+ "delivery_values",
704
+ "delivery_present",
705
+ "latent_mask",
706
+ "span_context_hidden",
707
+ "prefix_latents",
708
+ "prefix_mask",
709
+ "expanded_boundary_event_ids",
710
+ "expanded_modifier_event_ids",
711
+ "expanded_phone_phase",
712
+ "expanded_phone_log_duration",
713
+ "expanded_sentence_type_ids"
714
+ ],
715
+ "outputs": [
716
+ "velocity"
717
+ ],
718
+ "required": false
719
+ },
720
  "vocoder": {
721
  "artifacts": {
722
  "litert": {
 
757
  ],
758
  "required": false
759
  },
760
+ "vocoder_192": {
761
+ "artifacts": {
762
+ "litert": {
763
+ "format": "litert",
764
+ "path": "litert/vocoder.tflite",
765
+ "signature": "vocoder_192"
766
+ }
767
+ },
768
+ "inputs": [
769
+ "latents",
770
+ "latent_mask",
771
+ "voice_id",
772
+ "language_id",
773
+ "emotion_id"
774
+ ],
775
+ "outputs": [
776
+ "audio"
777
+ ],
778
+ "required": false
779
+ },
780
  "vocoder_256": {
781
  "artifacts": {
782
  "litert": {
 
797
  ],
798
  "required": false
799
  },
800
+ "vocoder_320": {
801
+ "artifacts": {
802
+ "litert": {
803
+ "format": "litert",
804
+ "path": "litert/vocoder.tflite",
805
+ "signature": "vocoder_320"
806
+ }
807
+ },
808
+ "inputs": [
809
+ "latents",
810
+ "latent_mask",
811
+ "voice_id",
812
+ "language_id",
813
+ "emotion_id"
814
+ ],
815
+ "outputs": [
816
+ "audio"
817
+ ],
818
+ "required": false
819
+ },
820
  "vocoder_384": {
821
  "artifacts": {
822
  "litert": {
 
837
  ],
838
  "required": false
839
  },
840
+ "vocoder_448": {
841
+ "artifacts": {
842
+ "litert": {
843
+ "format": "litert",
844
+ "path": "litert/vocoder.tflite",
845
+ "signature": "vocoder_448"
846
+ }
847
+ },
848
+ "inputs": [
849
+ "latents",
850
+ "latent_mask",
851
+ "voice_id",
852
+ "language_id",
853
+ "emotion_id"
854
+ ],
855
+ "outputs": [
856
+ "audio"
857
+ ],
858
+ "required": false
859
+ },
860
  "vocoder_512": {
861
  "artifacts": {
862
  "litert": {
 
1099
  "latent_frames": 1024,
1100
  "phone_frames": 512
1101
  },
1102
+ "g2p_buckets": [
1103
+ {
1104
+ "component": "g2p_32",
1105
+ "text_tokens": 32
1106
+ },
1107
+ {
1108
+ "component": "g2p_64",
1109
+ "text_tokens": 64
1110
+ },
1111
+ {
1112
+ "component": "g2p_96",
1113
+ "text_tokens": 96
1114
+ },
1115
+ {
1116
+ "component": "g2p_128",
1117
+ "text_tokens": 128
1118
+ },
1119
+ {
1120
+ "component": "g2p_192",
1121
+ "text_tokens": 192
1122
+ },
1123
+ {
1124
+ "component": "g2p_256",
1125
+ "text_tokens": 256
1126
+ },
1127
+ {
1128
+ "component": "g2p",
1129
+ "text_tokens": 512
1130
+ }
1131
+ ],
1132
  "graph_input_contract": "scyllasband_measured_delivery_v2",
1133
  "litert": {
1134
+ "int8_ops": [
1135
+ "FULLY_CONNECTED",
1136
+ "CONV_2D",
1137
+ "EMBEDDING_LOOKUP"
1138
+ ],
1139
+ "io": {
1140
+ "ids": "int32",
1141
+ "masks": "bool",
1142
+ "values": "float32"
1143
+ },
1144
  "quantization": "dynamic_wi8_afp32",
1145
  "report": {
1146
  "duration_predictor": {
1147
+ "bytes": 13195440,
1148
  "quantized": "dynamic_wi8_afp32"
1149
  },
1150
  "vector_context_encoder": {
1151
+ "bytes": 12391616,
1152
  "quantized": "dynamic_wi8_afp32"
1153
  },
1154
  "vector_estimator": {
1155
+ "bytes": 174492064,
1156
  "quantized": "dynamic_wi8_afp32"
1157
  },
1158
  "vocoder": {
1159
+ "bytes": 44714640,
1160
  "quantized": "dynamic_wi8_afp32"
1161
  }
1162
  },
1163
  "shared_bucket_weights": true,
1164
  "signatures_per_component": true,
1165
+ "target": "cpu",
1166
  "vocoder": "dynamic_wi8_afp32"
1167
  },
1168
  "pitch": {
 
1193
  ],
1194
  "default": "euler"
1195
  },
1196
+ "span_buckets": [
1197
+ {
1198
+ "duration_predictor": "duration_predictor_128",
1199
+ "phones": 128,
1200
+ "vector_context_encoder": "vector_context_encoder_128"
1201
+ },
1202
+ {
1203
+ "duration_predictor": "duration_predictor_256",
1204
+ "phones": 256,
1205
+ "vector_context_encoder": "vector_context_encoder_256"
1206
+ }
1207
+ ],
1208
  "span_conditioning": {
1209
  "context_max_phones": 512,
1210
  "context_phones_each_side": 180,
 
1246
  "default": false,
1247
  "latent_frames": 128,
1248
  "vector_estimator": "vector_estimator_128",
1249
+ "vector_estimator_first": "vector_estimator_first_128",
1250
  "vocoder": "vocoder_128"
1251
  },
1252
+ {
1253
+ "default": false,
1254
+ "latent_frames": 192,
1255
+ "vector_estimator": "vector_estimator_192",
1256
+ "vector_estimator_first": "vector_estimator_first_192",
1257
+ "vocoder": "vocoder_192"
1258
+ },
1259
  {
1260
  "default": false,
1261
  "latent_frames": 256,
1262
  "vector_estimator": "vector_estimator_256",
1263
+ "vector_estimator_first": "vector_estimator_first_256",
1264
  "vocoder": "vocoder_256"
1265
  },
1266
+ {
1267
+ "default": false,
1268
+ "latent_frames": 320,
1269
+ "vector_estimator": "vector_estimator_320",
1270
+ "vector_estimator_first": "vector_estimator_first_320",
1271
+ "vocoder": "vocoder_320"
1272
+ },
1273
  {
1274
  "default": false,
1275
  "latent_frames": 384,
1276
  "vector_estimator": "vector_estimator_384",
1277
+ "vector_estimator_first": "vector_estimator_first_384",
1278
  "vocoder": "vocoder_384"
1279
  },
1280
+ {
1281
+ "default": false,
1282
+ "latent_frames": 448,
1283
+ "vector_estimator": "vector_estimator_448",
1284
+ "vector_estimator_first": "vector_estimator_first_448",
1285
+ "vocoder": "vocoder_448"
1286
+ },
1287
  {
1288
  "default": false,
1289
  "latent_frames": 512,