Upload 27 files

ane_fp32/README.md

@@ -0,0 +1,115 @@
+---
+license: mit
+tags:
+- text-to-speech
+- styletts2
+- coreml
+- apple-silicon
+- ane
+library_name: coreml
+pipeline_tag: text-to-speech
+---
+
+# StyleTTS2 CoreML (LibriTTS) — fp32
+
+Apple CoreML conversion of [StyleTTS2](https://github.com/yl4579/StyleTTS2)
+(LibriTTS checkpoint), split into three fixed-shape mlpackages so each
+piece can target the compute unit it runs best on. Targets iOS 17+ /
+macOS 14+. fp32 across the board — fp16 / ANE placement is a separate
+phase (see Phase 6 in `coreml/PHASE6_FP16_DECODER.md` upstream).
+
+## Layout
+
+```
+styletts2_diffusion_step_256_fp32.mlpackage    (~95 MB)   one DDIM-style sampler step
+styletts2_decoder_noise_2048_fp32.mlpackage    (~9.4 MB)  HiFi-GAN SineGen + harmonic source
+styletts2_decoder_vocoder_2048_fp32.mlpackage  (~208 MB)  HiFi-GAN body (Snake → cos rewrite)
+voices/                                                    pre-extracted ref_s tensors (256-dim, fp32)
+```
+
+The three mlpackages are designed to be called in this order from a Swift
+host: text encoder + BERT + duration encoder run host-side, the diffusion
+step is invoked `num_steps` times by the host's ADPM2 sampler, then the
+noise + vocoder pair produce 24 kHz audio.
+
+## Bucket / shape contract
+
+| input | shape | notes |
+|---|---|---|
+| diffusion step | `noise (1, 1, 256)`, `embedding (1, T_tok, 768)`, `sigma (1,)`, `features (1, 256)` | Single fixed `T_tok=256` bucket |
+| noise | `F0_curve (1, 4096)`, `noise (1, 1228800, 9)` | `T_mel=2048` bucket; `noise` is fp32 Gaussian |
+| vocoder | `asr (1, 512, 2048)`, `F0_curve (1, 4096)`, `N (1, 4096)`, `s (1, 128)`, `har_source (1, 1, 1228800)` | `T_mel=2048` bucket |
+
+The `T_mel=2048` bucket caps text at ~6.4 s of audio per call (hop=300,
+sample_rate=24000). Short utterances pad with zeros up to the bucket on
+the host side.
+
+The noise mlpackage takes `noise` as a graph input (caller-supplied
+`torch.randn`); this is the Trial 35 fix that restored fricative
+broadband energy. Earlier exports baked a `sin(fn × 100)` substitute for
+the upstream `randn_like(sine_waves)` term to keep the trace
+deterministic — silent unvoiced regions destroyed /s/, /sh/, /f/.
+
+## Digests
+
+```
+diffusion_step_256:   563ed124a1ddbc3e4fb2374e53ef69966676d1d41bc07b0c25ba9f92a1902af2
+decoder_noise_2048:   a0b2778f35bbc81316ac5f15fea1e7cf68bf103ce635d590e498b935df7de50b
+decoder_vocoder_2048: 9cefdac41e6cc851774bcb21c8aa7ee635a3283d0e658a52f1e3242a6bf1612e
+```
+
+`package_digest` = `sha256(json([sorted (path, sha256(file)) for file in pkg]))`.
+See `scripts/phase5_lock_fp32.py` upstream for the locking workflow.
+
+## Voices
+
+`voices/ref_s_*.bin` — 256-dim `float32` style + prosody embeddings, raw
+little-endian (no header). One file per speaker; load with
+`Data(contentsOf:)` and copy into a `(1, 256)` MLMultiArray. Filenames
+mirror the LibriTTS demo speakers plus a few extras (Yinghao, Vinay,
+Nima, Gavin, sleepy, amused, anger, disgusted).
+
+## Compute units
+
+- diffusion step: `cpu_and_gpu` recommended (most ops Snake-rewritten;
+  ANE placement gated on Phase 6 fp16).
+- noise: `cpu_and_gpu` only — fp32 mlprogram cannot run on ANE; in
+  Phase 6 the SineGen phase accumulator saturates in fp16, so this
+  stage stays fp32 even after ANE migration.
+- vocoder: `cpu_and_gpu` (same fp32 → no ANE rule).
+
+Setting `compute_units = .cpuAndGPU` from Swift matches how the
+conversion harness validates parity.
+
+## Conversion source
+
+Conversion scripts and trial documentation live in the
+[mobius](https://github.com/your-org/mobius) repo under
+`models/tts/styletts2/`. Key files:
+
+- `scripts/04_export_diffusion_step_fp32.py` (diffusion step)
+- `scripts/04a_export_decoder_noise_fp32.py` (noise — Trial 35)
+- `scripts/04b_export_decoder_vocoder_fp32.py` (vocoder — Phase 3b
+  Snake → cos rewrite)
+- `scripts/phase4_chain_parity.py` (PT-vs-CoreML log-mel dB gate;
+  exit threshold 0.5 dB)
+- `scripts/phase5_lock_fp32.py` (digest lock for the three mlpackages)
+- `coreml/TRIALS.md` (full trial log including Bug C / Trial 35)
+- `coreml/PRECISION.md` (per-stage parity numbers)
+
+## License
+
+MIT. Upstream StyleTTS2 is also MIT-licensed; LibriTTS is CC BY 4.0.
+This export reproduces upstream weights without modification — only the
+graph layout changes (split + Snake → cos identity rewrite + constfold
+SineGen).
+
+## Limitations
+
+- Single fixed bucket per stage; very long inputs require host-side
+  chunking (not provided here).
+- fp32 only. ANE placement requires fp16; see Phase 6 notes upstream.
+- LibriTTS-only checkpoint — no LJSpeech / multilingual variants in
+  this folder.
+- Diffusion sampler must run host-side; the mlpackage exposes one step
+  at a time.

ane_fp32/styletts2_decoder_noise_2048_fp32.mlpackage/Data/com.apple.CoreML/model.mlmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:359949ba9d07ab754eceb53709769d3d7c8390395a6192aa3be5ade2acbea311
+size 12838

ane_fp32/styletts2_decoder_noise_2048_fp32.mlpackage/Data/com.apple.CoreML/weights/weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:89f32b66bbd82e57b1d6e61e2357780ca771196e1fbbefe5530c1cd05b40acb1
+size 9830592

ane_fp32/styletts2_decoder_noise_2048_fp32.mlpackage/Manifest.json

@@ -0,0 +1,18 @@
+{
+    "fileFormatVersion": "1.0.0",
+    "itemInfoEntries": {
+        "5D7D7039-E578-4193-85C9-C2D303886CEE": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Specification",
+            "name": "model.mlmodel",
+            "path": "com.apple.CoreML/model.mlmodel"
+        },
+        "A7A43036-469A-43AC-81C8-38A68C42CDEF": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Weights",
+            "name": "weights",
+            "path": "com.apple.CoreML/weights"
+        }
+    },
+    "rootModelIdentifier": "5D7D7039-E578-4193-85C9-C2D303886CEE"
+}

ane_fp32/styletts2_decoder_vocoder_2048_fp32.mlpackage/Data/com.apple.CoreML/model.mlmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:17ce364dee6a78bd6fce8e7013aa49804b4e5809b2ba68daa120f208312a1329
+size 451388

ane_fp32/styletts2_decoder_vocoder_2048_fp32.mlpackage/Data/com.apple.CoreML/weights/weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d48ca86b5a626c3e68f68128f557c7596210eac833f810bc4ff8fcb198aee581
+size 217140224

ane_fp32/styletts2_decoder_vocoder_2048_fp32.mlpackage/Manifest.json

@@ -0,0 +1,18 @@
+{
+    "fileFormatVersion": "1.0.0",
+    "itemInfoEntries": {
+        "3E9BF4B4-76A1-4CCF-8237-C467A80A1DB5": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Weights",
+            "name": "weights",
+            "path": "com.apple.CoreML/weights"
+        },
+        "B2CC0332-5A96-4669-BDE0-AA9E65B676D8": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Specification",
+            "name": "model.mlmodel",
+            "path": "com.apple.CoreML/model.mlmodel"
+        }
+    },
+    "rootModelIdentifier": "B2CC0332-5A96-4669-BDE0-AA9E65B676D8"
+}

ane_fp32/styletts2_diffusion_step_256_fp32.mlpackage/Data/com.apple.CoreML/model.mlmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ebea84b22ce6bb729d91a9fa7e8cc047b771b8785f002252540a7ec17714fdcc
+size 46991

ane_fp32/styletts2_diffusion_step_256_fp32.mlpackage/Data/com.apple.CoreML/weights/weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:acf1641c418eade9464e8f6ab04b986724865ecc8078c2a066bf50e78802a826
+size 99744384

ane_fp32/styletts2_diffusion_step_256_fp32.mlpackage/Manifest.json

@@ -0,0 +1,18 @@
+{
+    "fileFormatVersion": "1.0.0",
+    "itemInfoEntries": {
+        "19AC8D29-26A3-453B-B45C-6CFB76203362": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Specification",
+            "name": "model.mlmodel",
+            "path": "com.apple.CoreML/model.mlmodel"
+        },
+        "7F8A3CAC-B058-450A-B06C-CF827933236E": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Weights",
+            "name": "weights",
+            "path": "com.apple.CoreML/weights"
+        }
+    },
+    "rootModelIdentifier": "19AC8D29-26A3-453B-B45C-6CFB76203362"
+}

ane_fp32/voices/ref_s_1221-135767-0014.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7c8f34b18352203a90acc1620a8e6564cd76a76efaa3ca9f76c2e7f981925195
+size 1024

ane_fp32/voices/ref_s_1789_142896_000022_000005.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c0ae950fe25ce330dec76e2a312c7e9f08b3770f0c242ec39658c576173b2c53
+size 1024

ane_fp32/voices/ref_s_3.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8f5920cc5023efd2d0e65bfb1ce8a3ccaf505d00287401182685622cec62912a
+size 1024

ane_fp32/voices/ref_s_4.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:87b0125f709791b738f176f7d3d66db1278d12fd092ff71ce7603955907e70a7
+size 1024

ane_fp32/voices/ref_s_4077-13754-0000.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1bc6b4684b7dd023ab557014cf589931df7b6d6eb0fb45b2c9109e5a0b07b251
+size 1024

ane_fp32/voices/ref_s_5.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:09ac2a9de005df565e72236e4afc6a46facf5e0e9eaf143ca62b1d14e0edd5b3
+size 1024

ane_fp32/voices/ref_s_5639-40744-0020.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cc121e3357ca8e00bd5ef42f48c5b57c441ae0497aede091654ffae9a7fad93a
+size 1024

ane_fp32/voices/ref_s_696_92939_000016_000006.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:44a37a22ab2ffb1af297c9b1714e740ae9c481b3ee20afcac4e9fa8ed82185f6
+size 1024

ane_fp32/voices/ref_s_908-157963-0027.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:04026b56d755b3511f40c715465409a0f3fbbd3372de983db5b1b46a532dfc43
+size 1024

ane_fp32/voices/ref_s_Gavin.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2cd47901c6f8f83da48049fcf376480eb6c63b43e6f8359072f66c8fd6c3a9de
+size 1024

ane_fp32/voices/ref_s_Nima.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bc32e61a64bb8f5031d619902e60cf99113bd6d90abafbd3962b30c45a9aa1d4
+size 1024

ane_fp32/voices/ref_s_Vinay.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3c87703b18df2959e45e1df350e14640fa4be67397f05a716e80c80a8b8f2a13
+size 1024

ane_fp32/voices/ref_s_Yinghao.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b227bd8bb8e359150e43acef549720e7a8820c3afffe713c97707020e49e491b
+size 1024

ane_fp32/voices/ref_s_amused.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:58752cec26e778fc643d2b6b425ac7f6b88f466dfc7c90d5e27146c3e5aef03c
+size 1024

ane_fp32/voices/ref_s_anger.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bcda68be1f0016b0e8603cc804dfd18af525f533c398aef8221167688190722c
+size 1024

ane_fp32/voices/ref_s_disgusted.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:865912a9750e5d968b5c08606ea8beaf77f26f0f90a4a8ebf4ad105bf4596e97
+size 1024

ane_fp32/voices/ref_s_sleepy.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cb9e60a4fb5a9fd5a9e615bbef0b9f60d52ed0be3f23d1e0c8f56e8fb7f751f2
+size 1024