77ethers commited on
Commit
602ed72
·
verified ·
1 Parent(s): 023074c

v0.1.1: a cleaner voice from the same weights

Browse files

Same weights file as v0.1. The way the output is turned into sound is corrected while speaking (alpa/model.py, Polish); new browser file onnx/alpa-v0.1.1.onnx beside the v0.1 one; samples re-made; card updated. `--plain` gives the v0.1 sound.

.gitattributes CHANGED
@@ -40,3 +40,5 @@ samples/4_aspirates.wav filter=lfs diff=lfs merge=lfs -text
40
  samples/5_subhashita.wav filter=lfs diff=lfs merge=lfs -text
41
  assets/banner.png filter=lfs diff=lfs merge=lfs -text
42
  assets/how.png filter=lfs diff=lfs merge=lfs -text
 
 
 
40
  samples/5_subhashita.wav filter=lfs diff=lfs merge=lfs -text
41
  assets/banner.png filter=lfs diff=lfs merge=lfs -text
42
  assets/how.png filter=lfs diff=lfs merge=lfs -text
43
+ samples/after_v0.1.1.wav filter=lfs diff=lfs merge=lfs -text
44
+ samples/before_v0.1.wav filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -22,7 +22,9 @@ tags:
22
 
23
  **An English voice model, converted to Sanskrit: two tables edited by hand, then 100 minutes on a CPU. No GPU.**
24
 
25
- > **Early preview, v0.1.** One trained voice, 3.5 hours of speech, checked by one listener and one recogniser. See [Limits](#limits).
 
 
26
 
27
  ## Listen
28
 
@@ -36,6 +38,27 @@ tags:
36
 
37
  Each sample was generated once; nothing was re-rolled or selected. Verse is read aloud as text; Alpa does not chant.
38
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
39
  ## From English to Sanskrit, without a GPU
40
 
41
  ![Three steps. 1: start with tiny-voca, an English voice of 4.5 million parameters. 2: rewrite two tables by hand, 0.2% of its numbers, with no training; a Sanskrit recogniser then misreads 26.6% of phonemes. 3: 100 minutes on a CPU with 3.5 hours of Sanskrit speech; it then misreads 3.1%, against 3.3% for human recordings.](https://huggingface.co/77ethers/alpa/resolve/main/assets/how.png)
@@ -102,13 +125,15 @@ The whole path, from typed Devanagari to audio, runs inside the page: the text r
102
  | CPU slowed 4× | 2.9× real time | 2.7 s |
103
  | CPU slowed 6× | 1.9× real time | 4.3 s |
104
 
105
- The model file for the browser is 21 MB; with both runtimes a page needs 46 MB in total.
 
 
106
 
107
  The only other in-browser Sanskrit voice we know of is the port of Vāgdhenu, a chant system. By its own published benchmark it downloads 296 MB and, on a Pixel 10 Pro XL with six threads, needs about 15.5 seconds to chant a 7.5-second verse, with the first quarter-verse arriving after about 3.8 seconds.
108
 
109
  These two are not the same test. Theirs is a real flagship phone; ours is a desktop browser on a 2.1 GHz cloud CPU, slowed with Chrome's own CPU throttle to imitate slower hardware. **Alpa has not yet been timed on a real phone.**
110
 
111
- The file is `onnx/alpa-v0.1-preview.onnx`; `onnx/alpa-v0.1-preview.json` lists its inputs and the phoneme table.
112
 
113
  ```html
114
  <script src="https://cdn.jsdelivr.net/npm/onnxruntime-web@1.20.1/dist/ort.wasm.min.js"></script>
@@ -118,7 +143,7 @@ The file is `onnx/alpa-v0.1-preview.onnx`; `onnx/alpa-v0.1-preview.json` lists i
118
  const int64 = (v) => new ort.Tensor("int64", BigInt64Array.from(v.map(BigInt)), [1, v.length]);
119
  const one = (x) => new ort.Tensor("float32", Float32Array.from([x]), [1]);
120
  ort.env.wasm.numThreads = 1;
121
- const session = await ort.InferenceSession.create("alpa-v0.1-preview.onnx", { executionProviders: ["wasm"] });
122
  const out = await session.run({ tokens: int64(ids), weight: int64(weight),
123
  voice: new ort.Tensor("int64", BigInt64Array.from([1n]), [1]), // 1 = male
124
  speed: one(1), pitch_shift: one(0), pitch_range: one(1) });
@@ -130,14 +155,16 @@ The file is `onnx/alpa-v0.1-preview.onnx`; `onnx/alpa-v0.1-preview.json` lists i
130
 
131
  The judge is [Su-śrotā](https://huggingface.co/prathoshap/sushrota-sanskrit-asr), a Sanskrit speech recogniser. It listened to 148 sentences held out of training, spoken once by Alpa from text alone and once by the original voice artist.
132
 
133
- | | Alpa, `male` | Human recordings |
134
  |---|---:|---:|
135
- | Phonemes misread | **3.1%** | 3.3% |
136
- | ट heard as ट | 93% | 97% |
137
- | ड heard as ड | 89% | 94% |
138
- | ण heard as ण | 98% | 99% |
139
  | त, द, न heard correctly | 98–100% | 98–100% |
140
 
 
 
141
  Read these with two cautions. The recogniser knows Sanskrit words and partly guesses from context, which flatters both columns; the stricter between-vowels figures are in the table further up. And a recogniser is not an ear: one human listener has checked a handful of lines.
142
 
143
  **Against other systems.** There is no shared benchmark for Sanskrit speech synthesis yet, and the other systems have not been run through this judge, so we make no quality claim against them. Vāgdhenu's authors report a mean opinion score of about 4.6 from an expert listener, for chant. Alpa has no listener-panel score. On desktop speed the published figures are: Alpa about 100 times faster than real time on four CPU threads; Vāgdhenu about 3.5 times faster than real time on an M5 Max.
@@ -163,12 +190,12 @@ audio, phonemes, _ = speak(model, "सत्यमेव जयते नान
163
  sf.write("out.wav", audio.numpy(), 24000)
164
  ```
165
 
166
- `speak` also takes `speed`, `pitch_shift` and `pitch_range`. Text in Kannada, Telugu, Bengali or Malayalam script is read as well if you `pip install indic-transliteration`. The model file holds plain tensors and settings only and is loaded with `weights_only=True`.
167
 
168
  | | |
169
  |---|---|
170
  | Parameters | 4,531,723 |
171
- | Files | `alpa-v0.1-preview.pt` (18 MB), `onnx/alpa-v0.1-preview.onnx` (21 MB) |
172
  | Audio | 24 kHz, mono |
173
  | Input | Sanskrit in Devanagari |
174
  | Voices | `male` is trained. `female` is present but was never trained on Sanskrit |
@@ -185,8 +212,10 @@ sf.write("out.wav", audio.numpy(), 24000)
185
 
186
  ## Limits
187
 
188
- - **One trained voice.** With `female`, ट and ड are noticeably weaker.
189
- - **Little data and a short run.** It learned from 3.5 hours, and the adversarial stage of tiny-voca's recipe, which adds much of the naturalness, has not been run. The voice can sound flatter or buzzier than larger systems.
 
 
190
  - **ण and न** are not yet fully distinct to a human listener.
191
  - **Word-final visarga is sometimes weak.** The recogniser missed it in three of the five samples above.
192
  - **It speaks; it does not chant.** Vedic accent marks are ignored.
 
22
 
23
  **An English voice model, converted to Sanskrit: two tables edited by hand, then 100 minutes on a CPU. No GPU.**
24
 
25
+ > **Early preview, v0.1.1.** One trained voice, 3.5 hours of speech, checked by one listener and one recogniser. See [Limits](#limits).
26
+ >
27
+ > **New in v0.1.1:** a cleaner voice from the same weights. See [What changed in v0.1.1](#what-changed-in-v011).
28
 
29
  ## Listen
30
 
 
38
 
39
  Each sample was generated once; nothing was re-rolled or selected. Verse is read aloud as text; Alpa does not chant.
40
 
41
+ ## What changed in v0.1.1
42
+
43
+ The weights are the same file as v0.1. What changed is how the model's output is turned into sound, and again nothing was trained.
44
+
45
+ | | Audio |
46
+ |---|---|
47
+ | v0.1 | <audio controls src="https://huggingface.co/77ethers/alpa/resolve/main/samples/before_v0.1.wav"></audio> |
48
+ | v0.1.1 | <audio controls src="https://huggingface.co/77ethers/alpa/resolve/main/samples/after_v0.1.1.wav"></audio> |
49
+
50
+ *श्रीः स्त्री शास्त्रं स्पष्टं पश्यति। क्षत्रियः राष्ट्रं रक्षति।*
51
+
52
+ After the release, listeners reported a buzz. Our listener narrowed it down: in श, ष and क्ष, and then in ज, घ and ध, it sounded like "two metallic people saying at once". The cause was in the inherited sound generator. It makes the voice from pitched pulses, and the model had learnt to build its hiss, breath and bursts out of those same pulses, so every noisy sound carried a second, pitched voice inside it.
53
+
54
+ v0.1.1 corrects this while speaking:
55
+
56
+ - **Each sound gets a frequency limit.** Below it the generator is fed the voice's pulses; above it, plain noise of the same loudness. Sibilants and च are noise throughout; stops, ज and ह keep the voice only below 1 kHz; vowels keep it up to 4.5 kHz, as real voices do.
57
+ - **Sharper vowels.** The resonances that tell one vowel from another were flatter than in the recordings; they are restored to the recordings' level.
58
+ - **Smaller touches:** a steadier pitch, less whisper inside vowels, and a slightly brighter top.
59
+
60
+ Measured against the recordings, the noisy sounds now have no pitch above 1 kHz, as in real speech, and the recogniser's score is unchanged (3.0% of phonemes misread, 3.1% before). On a side-by-side of six sentences our listener judged the new voice clearly better. `--plain` gives the earlier sound.
61
+
62
  ## From English to Sanskrit, without a GPU
63
 
64
  ![Three steps. 1: start with tiny-voca, an English voice of 4.5 million parameters. 2: rewrite two tables by hand, 0.2% of its numbers, with no training; a Sanskrit recogniser then misreads 26.6% of phonemes. 3: 100 minutes on a CPU with 3.5 hours of Sanskrit speech; it then misreads 3.1%, against 3.3% for human recordings.](https://huggingface.co/77ethers/alpa/resolve/main/assets/how.png)
 
125
  | CPU slowed 4× | 2.9× real time | 2.7 s |
126
  | CPU slowed 6× | 1.9× real time | 4.3 s |
127
 
128
+ These timings are of the v0.1 file. The v0.1.1 file, measured beside it on the same day, is about 5% slower (14.0× against 14.5× real time on that day's machine).
129
+
130
+ The model file for the browser is 23 MB; with both runtimes a page needs 47 MB in total.
131
 
132
  The only other in-browser Sanskrit voice we know of is the port of Vāgdhenu, a chant system. By its own published benchmark it downloads 296 MB and, on a Pixel 10 Pro XL with six threads, needs about 15.5 seconds to chant a 7.5-second verse, with the first quarter-verse arriving after about 3.8 seconds.
133
 
134
  These two are not the same test. Theirs is a real flagship phone; ours is a desktop browser on a 2.1 GHz cloud CPU, slowed with Chrome's own CPU throttle to imitate slower hardware. **Alpa has not yet been timed on a real phone.**
135
 
136
+ The file is `onnx/alpa-v0.1.1.onnx`; `onnx/alpa-v0.1.1.json` lists its inputs and the phoneme table. It takes the same inputs as the v0.1 file, `onnx/alpa-v0.1-preview.onnx`, which is still here.
137
 
138
  ```html
139
  <script src="https://cdn.jsdelivr.net/npm/onnxruntime-web@1.20.1/dist/ort.wasm.min.js"></script>
 
143
  const int64 = (v) => new ort.Tensor("int64", BigInt64Array.from(v.map(BigInt)), [1, v.length]);
144
  const one = (x) => new ort.Tensor("float32", Float32Array.from([x]), [1]);
145
  ort.env.wasm.numThreads = 1;
146
+ const session = await ort.InferenceSession.create("alpa-v0.1.1.onnx", { executionProviders: ["wasm"] });
147
  const out = await session.run({ tokens: int64(ids), weight: int64(weight),
148
  voice: new ort.Tensor("int64", BigInt64Array.from([1n]), [1]), // 1 = male
149
  speed: one(1), pitch_shift: one(0), pitch_range: one(1) });
 
155
 
156
  The judge is [Su-śrotā](https://huggingface.co/prathoshap/sushrota-sanskrit-asr), a Sanskrit speech recogniser. It listened to 148 sentences held out of training, spoken once by Alpa from text alone and once by the original voice artist.
157
 
158
+ | | Alpa v0.1.1, `male` | Human recordings |
159
  |---|---:|---:|
160
+ | Phonemes misread | **3.0%** | 3.3% |
161
+ | ट heard as ट | 94% | 97% |
162
+ | ड heard as ड | 87% | 94% |
163
+ | ण heard as ण | 97% | 99% |
164
  | त, द, न heard correctly | 98–100% | 98–100% |
165
 
166
+ v0.1, with the same weights, scored 3.1%, 93%, 89% and 98% on the first four rows: the cleaner voice left pronunciation where it was.
167
+
168
  Read these with two cautions. The recogniser knows Sanskrit words and partly guesses from context, which flatters both columns; the stricter between-vowels figures are in the table further up. And a recogniser is not an ear: one human listener has checked a handful of lines.
169
 
170
  **Against other systems.** There is no shared benchmark for Sanskrit speech synthesis yet, and the other systems have not been run through this judge, so we make no quality claim against them. Vāgdhenu's authors report a mean opinion score of about 4.6 from an expert listener, for chant. Alpa has no listener-panel score. On desktop speed the published figures are: Alpa about 100 times faster than real time on four CPU threads; Vāgdhenu about 3.5 times faster than real time on an M5 Max.
 
190
  sf.write("out.wav", audio.numpy(), 24000)
191
  ```
192
 
193
+ `speak` also takes `speed`, `pitch_shift` and `pitch_range`. For the sound of v0.1 as first published, add `--plain` to the command or pass `polish=None` to `speak`. `python -m alpa.export` writes the browser file (it needs `pip install onnx onnxruntime`). Text in Kannada, Telugu, Bengali or Malayalam script is read as well if you `pip install indic-transliteration`. The model file holds plain tensors and settings only and is loaded with `weights_only=True`.
194
 
195
  | | |
196
  |---|---|
197
  | Parameters | 4,531,723 |
198
+ | Files | `alpa-v0.1-preview.pt` (18 MB, unchanged since v0.1), `onnx/alpa-v0.1.1.onnx` (23 MB), `onnx/alpa-v0.1-preview.onnx` (21 MB, the v0.1 sound) |
199
  | Audio | 24 kHz, mono |
200
  | Input | Sanskrit in Devanagari |
201
  | Voices | `male` is trained. `female` is present but was never trained on Sanskrit |
 
212
 
213
  ## Limits
214
 
215
+ - **One trained voice.** With `female`, ट and ड are noticeably weaker, the visarga fades, and our listener judged the voice not good enough to use.
216
+ - **Little data and a short run.** It learned from 3.5 hours, and the adversarial stage of tiny-voca's recipe, which adds much of the naturalness, has not been run. The voice can sound flatter than larger systems.
217
+ - **The cleaner voice is a correction, not a cure.** v0.1.1 fixes the sound while speaking; the model itself still makes the faults, and `--plain` lets you hear them.
218
+ - **A few vowels come out hoarse.** The ए of ते in कर्मण्येवाधिकारस्ते… मा ते सङ्गोऽस्त्वकर्मणि is the known case; it is not fixed.
219
  - **ण and न** are not yet fully distinct to a human listener.
220
  - **Word-final visarga is sometimes weak.** The recogniser missed it in three of the five samples above.
221
  - **It speaks; it does not chant.** Vedic accent marks are ignored.
THIRD_PARTY_NOTICES.md CHANGED
@@ -8,8 +8,8 @@ licensed under the Apache License, Version 2.0. A copy of that licence is in `LI
8
 
9
  - **Weights.** `alpa-v0.1-preview.pt` was trained starting from tiny-voca's published weights. Its phoneme
10
  table and its stress table were rewritten for Sanskrit before training; every other layer started as a copy.
11
- - **Browser file.** `onnx/alpa-v0.1-preview.onnx` was exported with tiny-voca's exporter (`src/tiny_voca/export.py`), unchanged.
12
- - **Code.** `alpa/model.py`, `alpa/config.py` and `alpa/infer.py` are adapted from tiny-voca's files of the
13
  same names. Each states its changes at its top. `alpa/text.py` is original to Alpa.
14
 
15
  ## Vāgdhenu
 
8
 
9
  - **Weights.** `alpa-v0.1-preview.pt` was trained starting from tiny-voca's published weights. Its phoneme
10
  table and its stress table were rewritten for Sanskrit before training; every other layer started as a copy.
11
+ - **Browser files.** `onnx/alpa-v0.1-preview.onnx` was exported with tiny-voca's exporter (`src/tiny_voca/export.py`), unchanged. `onnx/alpa-v0.1.1.onnx` was exported with `alpa/export.py`, which is adapted from it.
12
+ - **Code.** `alpa/model.py`, `alpa/config.py`, `alpa/infer.py` and `alpa/export.py` are adapted from tiny-voca's files of the
13
  same names. Each states its changes at its top. `alpa/text.py` is original to Alpa.
14
 
15
  ## Vāgdhenu
alpa/export.py ADDED
@@ -0,0 +1,322 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Adapted from tiny-voca by Muhammed Nazeem, commit 28e0722 (Apache-2.0; see LICENSES/Apache-2.0.txt
2
+ # and THIRD_PARTY_NOTICES.md). File: src/tiny_voca/export.py. Changes for Alpa: Alpa's model and syllable weight; the
3
+ # second input is named ``weight``; the corrections made while speaking (model.Polish) are part of the graph.
4
+
5
+ """ONNX export of free-running synthesis.
6
+
7
+ python -m alpa.export --checkpoint inference.pt --output alpa.onnx
8
+
9
+ The exported graph speaks one utterance: phoneme ids and syllable weights in,
10
+ waveform out. Text is turned into ids outside the graph (``alpa.text.phonemize``).
11
+
12
+ ``Exportable`` computes the same audio as ``Alpa.synthesize`` for a single
13
+ utterance, written with operations ONNX has: the short-time Fourier transforms
14
+ of the source-filter head become real convolutions with fixed kernels, the
15
+ complex filter responses are kept as separate real and imaginary parts, and
16
+ the frame expansion uses a comparison instead of a sorted search.
17
+
18
+ Inputs
19
+ tokens int64 [1, N] phoneme ids, with the edge tokens
20
+ weight int64 [1, N] 0 none, 1 laghu, 2 guru
21
+ voice int64 [1] index into ``voices`` in the metadata
22
+ speed float32 [1] 1.0 = as predicted, larger is faster
23
+ pitch_shift float32 [1] added to log F0 (0.058 is about a semitone)
24
+ pitch_range float32 [1] scales pitch movement around the mean
25
+ Outputs
26
+ audio float32 [1, samples]
27
+ durations int64 [1, N] frames per token (``hop`` samples each)
28
+ """
29
+
30
+ import argparse
31
+ import dataclasses
32
+ import json
33
+ import math
34
+ from pathlib import Path
35
+
36
+ import torch
37
+ from torch import nn
38
+ import torch.nn.functional as F
39
+
40
+ from .infer import load_model
41
+ from .model import ALL_PULSES, NO_PULSES, Polish, treble_kernel
42
+ from .text import SYMBOLS, frontend_fingerprint
43
+
44
+ INPUTS = ("tokens", "weight", "voice", "speed", "pitch_shift", "pitch_range")
45
+ OUTPUTS = ("audio", "durations")
46
+
47
+
48
+ def _cepstrum_matrices(positions, fft):
49
+ """Real and imaginary parts of rfft for unit coefficients at ``positions``."""
50
+ buffer = torch.zeros(len(positions), fft, dtype=torch.float64)
51
+ buffer[torch.arange(len(positions)), torch.tensor(positions)] = 1
52
+ spectrum = torch.fft.rfft(buffer, dim=-1)
53
+ return spectrum.real.float(), spectrum.imag.float()
54
+
55
+
56
+ def _analysis_kernels(window):
57
+ """Convolution kernels [bins, 1, fft] giving the real and imaginary STFT."""
58
+ fft = len(window)
59
+ spectrum = torch.fft.rfft(torch.eye(fft, dtype=torch.float64) * window.double(), dim=-1)
60
+ return spectrum.real.T[:, None].float().contiguous(), spectrum.imag.T[:, None].float().contiguous()
61
+
62
+
63
+ def _synthesis_kernel(window):
64
+ """Transposed-convolution kernel [2 * bins, 1, fft]: windowed inverse rfft
65
+ of the real parts (first half of the channels) and imaginary parts."""
66
+ fft = len(window)
67
+ bins = fft // 2 + 1
68
+ unit = torch.eye(bins, dtype=torch.complex128)
69
+ real = torch.fft.irfft(unit, n=fft, dim=-1)
70
+ imaginary = torch.fft.irfft(unit * 1j, n=fft, dim=-1)
71
+ return (torch.cat((real, imaginary)) * window.double())[:, None].float().contiguous()
72
+
73
+
74
+ class Exportable(nn.Module):
75
+ """Single-utterance synthesis in ONNX-friendly operations. ``polish`` as in ``Alpa.synthesize``."""
76
+
77
+ def __init__(self, model, external_noise=False, polish=Polish()):
78
+ super().__init__()
79
+ self.model = model
80
+ self.external_noise = external_noise
81
+ self.polish = polish
82
+ c = model.config
83
+ v = model.vocoder
84
+ causal = list(range(c.cep_causal))
85
+ anti = [c.synth_fft - 1 - i for i in range(c.cep_anti)]
86
+ for name, (real, imaginary) in (("harmonic", _cepstrum_matrices(causal + anti, c.synth_fft)),
87
+ ("noise", _cepstrum_matrices(list(range(c.noise_cep)), c.noise_fft))):
88
+ self.register_buffer(f"{name}_real", real, persistent=False)
89
+ self.register_buffer(f"{name}_imaginary", imaginary, persistent=False)
90
+ for name, window in (("synth", v.synth_window), ("noise", v.noise_window)):
91
+ real, imaginary = _analysis_kernels(window)
92
+ self.register_buffer(f"{name}_analysis_real", real, persistent=False)
93
+ self.register_buffer(f"{name}_analysis_imaginary", imaginary, persistent=False)
94
+ self.register_buffer(f"{name}_synthesis", _synthesis_kernel(window), persistent=False)
95
+ self.register_buffer(f"{name}_envelope", (window.double() ** 2).float()[None, None].contiguous(), persistent=False)
96
+ self.register_buffer("ramp", torch.arange(c.hop, dtype=torch.float32) / c.hop, persistent=False)
97
+ self.phase_steps = 1 << 24
98
+ if polish is not None:
99
+ # Which cepstral coefficients sharpening stretches, in the order causal then anticausal.
100
+ chosen = torch.zeros(c.cep_causal)
101
+ chosen[polish.sharpen_from:polish.sharpen_to] = 1
102
+ mirror = torch.zeros(c.cep_anti)
103
+ mirror[max(polish.sharpen_from - 1, 0):max(polish.sharpen_to - 1, 0)] = 1
104
+ self.register_buffer("stretched", torch.cat((chosen, mirror)), persistent=False)
105
+ triangle = torch.cat((torch.arange(1, polish.pitch_smooth + 2), torch.arange(polish.pitch_smooth, 0, -1))).float()
106
+ self.register_buffer("pitch_kernel", (triangle / triangle.sum())[None, None], persistent=False)
107
+ self.register_buffer("rumble_cut", torch.sigmoid((v.synth_frequency - 90.0) / 12.0)[None, :, None], persistent=False)
108
+ self.register_buffer("treble", treble_kernel(polish.treble, c.sample_rate)[None, None], persistent=False)
109
+
110
+ def _analysis(self, source, name, fft, hop):
111
+ """Real and imaginary STFT [1, bins, frames] of ``source`` [1, samples]."""
112
+ padded = F.pad(source[:, None], (fft // 2, fft))
113
+ return (F.conv1d(padded, getattr(self, f"{name}_analysis_real"), stride=hop),
114
+ F.conv1d(padded, getattr(self, f"{name}_analysis_imaginary"), stride=hop))
115
+
116
+ def _synthesis(self, s_real, s_imaginary, real, imaginary, name, fft, hop, samples):
117
+ """Multiply an STFT by a response given as real and imaginary parts
118
+ [1, frames, bins], and resynthesize ``samples`` samples."""
119
+ real, imaginary = real.transpose(1, 2), imaginary.transpose(1, 2)
120
+ product = torch.cat((s_real * real - s_imaginary * imaginary, s_real * imaginary + s_imaginary * real), dim=1)
121
+ wave = F.conv_transpose1d(product, getattr(self, f"{name}_synthesis"), stride=hop)
122
+ envelope = F.conv_transpose1d(torch.ones_like(s_real[:, :1]), getattr(self, f"{name}_envelope"), stride=hop)
123
+ return (wave / envelope)[:, 0, fft // 2:fft // 2 + samples]
124
+
125
+ def _filter(self, source, real, imaginary, name, fft, hop):
126
+ s_real, s_imaginary = self._analysis(source, name, fft, hop)
127
+ return self._synthesis(s_real, s_imaginary, real, imaginary, name, fft, hop, source.shape[1])
128
+
129
+ @staticmethod
130
+ def _response(cepstrum, real, imaginary):
131
+ magnitude = torch.exp((cepstrum @ real).clamp(-14, 6))
132
+ angle = cepstrum @ imaginary
133
+ return magnitude * torch.cos(angle), magnitude * torch.sin(angle)
134
+
135
+ def _pulses(self, f0):
136
+ """``pulse_train`` for one utterance, without 64-bit floats (the web
137
+ runtime has none). The phase is summed inside each frame in float32,
138
+ where it stays below a few cycles, and across frames as 24-bit integer
139
+ fractions of a cycle, which wrap exactly however long the utterance is."""
140
+ c = self.model.config
141
+ knots = torch.log(f0.clamp_min(1.0))
142
+ knots = torch.cat((knots, knots[:, -1:]), dim=1)
143
+ low, high = knots[:, :-1, None], knots[:, 1:, None]
144
+ frequency = torch.exp(low * (1 - self.ramp) + high * self.ramp)
145
+ within = torch.cumsum(frequency / c.sample_rate, dim=2)
146
+ whole_frame = within[:, :, -1]
147
+ steps = torch.round((whole_frame - torch.floor(whole_frame)) * self.phase_steps).long()
148
+ start = torch.remainder(torch.cumsum(steps, dim=1) - steps, self.phase_steps).to(within.dtype) / self.phase_steps
149
+ cycles = within + start[:, :, None]
150
+ theta = (2 * math.pi * (cycles - torch.floor(cycles))).reshape(1, -1)
151
+ frequency = frequency.reshape(1, -1)
152
+ harmonics = (c.harmonic_limit * c.sample_rate / frequency).clamp_min(1.0)
153
+ whole = torch.floor(harmonics)
154
+ denominator = 2 * torch.sin(theta / 2)
155
+ tiny = denominator.abs() < 1e-6
156
+ core = torch.where(tiny, whole, torch.sin((whole + 0.5) * theta) / torch.where(tiny, torch.ones_like(denominator), denominator) - 0.5)
157
+ wave = core + (harmonics - whole) * torch.cos((whole + 1) * theta)
158
+ return wave * torch.sqrt(2.0 / harmonics)
159
+
160
+ def _vocoder(self, hidden, f0, total, noise, breath=None, limits=None, open_=None, vowel=None):
161
+ c = self.model.config
162
+ v = self.model.vocoder
163
+ p = self.polish
164
+ last = total - 1
165
+ # Harmonic branch.
166
+ h = v.harmonic(hidden)
167
+ split = c.cep_causal + c.cep_anti
168
+ steps = v.harmonic_steps
169
+ sub = torch.minimum(torch.arange(steps * total + 3, device=hidden.device), steps * total - 1)
170
+ here = torch.div(sub, steps, rounding_mode="floor")
171
+ following = torch.minimum(torch.div(sub + 1, steps, rounding_mode="floor"), last)
172
+ envelope = 0.5 * (h[..., :split].index_select(1, here) + h[..., :split].index_select(1, following))
173
+ envelope = envelope * torch.cat((v.causal_lifter, v.anti_lifter))
174
+ gain = h[..., split:].flatten(1).index_select(1, sub)[..., None]
175
+ envelope = torch.cat((envelope[..., :1] + gain, envelope[..., 1:]), dim=-1)
176
+ if p is not None and p.sharpen != 1.0:
177
+ # Sharper formants in vowels, nasals and semivowels, with the loudness of each broad band put back.
178
+ beta = (1.0 + (p.sharpen - 1.0) * open_.to(envelope.dtype)).index_select(1, here)[..., None]
179
+ sharp = envelope * (1 + self.stretched * (beta - 1))
180
+ flat_level, sharp_level = (envelope @ self.harmonic_real).clamp(-14, 6), (sharp @ self.harmonic_real).clamp(-14, 6)
181
+ power = lambda level: torch.exp(2 * level) @ v.band_mask.T + 1e-20
182
+ magnitude = torch.exp(sharp_level + (0.5 * torch.log(power(flat_level) / power(sharp_level))) @ v.band_weight)
183
+ angle = sharp @ self.harmonic_imaginary
184
+ harmonic_real, harmonic_imaginary = magnitude * torch.cos(angle), magnitude * torch.sin(angle)
185
+ else:
186
+ harmonic_real, harmonic_imaginary = self._response(envelope, self.harmonic_real, self.harmonic_imaginary)
187
+ # Noise branch.
188
+ n = v.noise(hidden)
189
+ steps = v.noise_steps
190
+ sub = torch.minimum(torch.arange(steps * total + 3, device=hidden.device), steps * total - 1)
191
+ shape = n[..., :c.noise_cep].index_select(1, torch.div(sub, steps, rounding_mode="floor")) * v.noise_lifter
192
+ gain = n[..., c.noise_cep:].flatten(1).index_select(1, sub)[..., None]
193
+ shape = torch.cat((shape[..., :1] + gain, shape[..., 1:]), dim=-1)
194
+ noise_real, noise_imaginary = self._response(shape, self.noise_real, self.noise_imaginary)
195
+ pulses = self._pulses(f0)
196
+ size = torch.arange(c.hop * total, device=hidden.device)
197
+ noise = torch.randn_like(pulses) if noise is None else noise.index_select(1, size)
198
+ hiss = self._filter(noise, noise_real, noise_imaginary, "noise", c.noise_fft, c.noise_hop)
199
+ if p is None:
200
+ return self._filter(pulses, harmonic_real, harmonic_imaginary, "synth", c.synth_fft, c.synth_hop) + hiss
201
+ # The mixed source: pulses below each sub-frame's limit, noise of the same power above it.
202
+ breath = torch.randn_like(pulses) if breath is None else breath.index_select(1, size)
203
+ limit = limits.index_select(1, here)
204
+ width = torch.where(limit > 2000, torch.full_like(limit, 350.0), torch.full_like(limit, 120.0))
205
+ keep = torch.sigmoid((limit[:, None] - v.synth_frequency[None, :, None]) / width[:, None])
206
+ keep = F.avg_pool1d(F.pad(keep, (1, 1), mode="replicate"), 3, 1)
207
+ other = torch.sqrt((1 - keep ** 2).clamp_min(0)) * self.rumble_cut
208
+ p_real, p_imaginary = self._analysis(pulses, "synth", c.synth_fft, c.synth_hop)
209
+ b_real, b_imaginary = self._analysis(breath * math.sqrt(0.5 / c.harmonic_limit), "synth", c.synth_fft, c.synth_hop)
210
+ harmonic = self._synthesis(p_real * keep + b_real * other, p_imaginary * keep + b_imaginary * other, harmonic_real, harmonic_imaginary,
211
+ "synth", c.synth_fft, c.synth_hop, pulses.shape[1])
212
+ if p.noise_cap is not None:
213
+ level = lambda wave: wave.reshape(1, -1, c.hop).pow(2).mean(-1)
214
+ quieter = (level(harmonic) * 10 ** (p.noise_cap / 10) / level(hiss).clamp_min(1e-20)).clamp(max=1.0).sqrt()
215
+ quieter = torch.where(vowel, quieter, torch.ones_like(quieter))
216
+ quieter = F.avg_pool1d(F.pad(quieter[:, None], (1, 1), mode="replicate"), 3, 1)
217
+ hiss = hiss * F.interpolate(quieter, scale_factor=float(c.hop), mode="linear", align_corners=False)[:, 0]
218
+ wave = harmonic + hiss
219
+ if p.treble:
220
+ wave = F.conv1d(wave[:, None], self.treble, padding=self.treble.shape[-1] // 2)[:, 0]
221
+ return wave
222
+
223
+ def forward(self, tokens, weight, voice, speed, pitch_shift, pitch_range, noise=None, breath=None):
224
+ m = self.model
225
+ c = m.config
226
+ p = self.polish
227
+ token_mask = torch.ones_like(tokens, dtype=torch.bool)
228
+ hidden = m.encoder(m.tokens(tokens) + m.syllable_weight(weight) + m.voices(voice)[:, None], token_mask)
229
+ # Durations.
230
+ output = m.duration(hidden, token_mask)
231
+ frames = torch.round((torch.exp(output[..., 0].clamp(0, math.log1p(400))) - 1) / speed).long().clamp_min(1)
232
+ present = m.phone_tokens[tokens] | (output[..., 1] > 0)
233
+ durations = torch.where(present, frames, torch.zeros_like(frames))
234
+ # Repeat token states by duration.
235
+ ends = durations.cumsum(1)
236
+ total = ends[0, -1]
237
+ frame = torch.arange(total, device=tokens.device)[None]
238
+ index = (ends[:, None, :] <= frame[:, :, None]).sum(-1)
239
+ owned = durations.gather(1, index).clamp_min(1)
240
+ offset = frame - (ends.gather(1, index) - durations.gather(1, index))
241
+ relative = (offset.to(hidden.dtype) + 0.5) / owned
242
+ features = torch.stack((relative, 1 - relative, torch.log1p(owned.to(hidden.dtype)) / 4), dim=-1)
243
+ expanded = hidden.gather(1, index[..., None].expand(-1, -1, hidden.shape[-1]))
244
+ expanded = expanded + m.frame_position(features) + m.frame_voices(voice)[:, None]
245
+ frame_mask = torch.ones_like(frame, dtype=torch.bool)
246
+ # Pitch, voicing, energy.
247
+ prosody_hidden = m.prosody_stack(expanded, frame_mask)
248
+ prosody = m.prosody_head(prosody_hidden)
249
+ predicted = prosody[..., 0]
250
+ if p is not None and p.pitch_smooth:
251
+ predicted = F.conv1d(F.pad(predicted[:, None], (p.pitch_smooth, p.pitch_smooth), mode="replicate"), self.pitch_kernel)[:, 0]
252
+ vowel = m.vowel_tokens[tokens].gather(1, index)
253
+ voicing = torch.where(vowel, prosody[..., 1].clamp_min(2.0), prosody[..., 1])
254
+ voiced = (voicing > 0).to(prosody.dtype)
255
+ mean = (predicted * voiced).sum(1, keepdim=True) / voiced.sum(1, keepdim=True).clamp_min(1)
256
+ pitch = mean + (predicted - mean) * pitch_range
257
+ condition = torch.stack((pitch.clamp(-2.0, 1.5), voiced, prosody[..., 2].clamp(-4.5, 3.0)), dim=-1)
258
+ acoustic = m.acoustic_stack(prosody_hidden + m.prosody_projection(condition), frame_mask)
259
+ f0 = (c.pitch_reference_hz * torch.exp(pitch + pitch_shift)).clamp(c.f0_min, c.f0_max)
260
+ if p is None:
261
+ return self._vocoder(acoustic, f0, total, noise), durations
262
+ # A frequency limit per frame, from the sound it belongs to (see ``Polish``).
263
+ kind = m.sound_kind[tokens].gather(1, index)
264
+ is_voiced = voicing > 0
265
+ fill = lambda value: torch.full_like(pitch, value)
266
+ limits = torch.where(kind == 1, fill(p.open_limit), fill(ALL_PULSES))
267
+ limits = torch.where(kind == 2, fill(p.consonant_limit), limits)
268
+ limits = torch.where(kind == 3, fill(NO_PULSES), limits)
269
+ limits = torch.where(kind == 4, torch.where(is_voiced, fill(p.consonant_limit), fill(NO_PULSES)), limits)
270
+ limits = torch.where((kind == 5) & ~is_voiced, fill(NO_PULSES), limits)
271
+ return self._vocoder(acoustic, f0, total, noise, breath, limits, kind == 1, vowel), durations
272
+
273
+
274
+ def example_inputs(length=24):
275
+ tokens = torch.randint(5, len(SYMBOLS), (1, length))
276
+ one = lambda value: torch.tensor([value], dtype=torch.float32)
277
+ return tokens, torch.zeros_like(tokens), torch.tensor([0]), one(1.0), one(0.0), one(1.0)
278
+
279
+
280
+ def export(model, path, external_noise=False, opset=17, polish=Polish()):
281
+ wrapper = Exportable(model, external_noise, polish).eval()
282
+ inputs = example_inputs()
283
+ names = list(INPUTS)
284
+ axes = {"tokens": {1: "tokens"}, "weight": {1: "tokens"}, "audio": {1: "samples"}, "durations": {1: "tokens"}}
285
+ if external_noise:
286
+ for name in ("noise",) if polish is None else ("noise", "breath"):
287
+ inputs = (*inputs, torch.randn(1, model.config.hop * model.config.max_frames))
288
+ names.append(name)
289
+ axes[name] = {1: f"{name}_samples"}
290
+ with torch.no_grad():
291
+ torch.onnx.export(wrapper, inputs, str(path), input_names=names, output_names=list(OUTPUTS), dynamic_axes=axes,
292
+ opset_version=opset, do_constant_folding=True, dynamo=False)
293
+ return path
294
+
295
+
296
+ def metadata(model, checkpoint, polish=Polish()):
297
+ c = model.config
298
+ state = torch.load(checkpoint, map_location="cpu", weights_only=True)
299
+ return {"sample_rate": c.sample_rate, "hop": c.hop, "voices": list(c.speakers), "symbols": list(SYMBOLS),
300
+ "max_tokens": c.max_tokens, "max_frames": c.max_frames, "step": state.get("step"),
301
+ "frontend_fingerprint": frontend_fingerprint(), "inputs": list(INPUTS), "outputs": list(OUTPUTS),
302
+ "polish": None if polish is None else dataclasses.asdict(polish)}
303
+
304
+
305
+ def main():
306
+ parser = argparse.ArgumentParser(description="Export an Alpa inference checkpoint to ONNX")
307
+ parser.add_argument("--checkpoint", type=Path, required=True)
308
+ parser.add_argument("--output", type=Path, required=True)
309
+ parser.add_argument("--plain", action="store_true", help="Leave out the corrections made while speaking (the sound of v0.1 as first published)")
310
+ parser.add_argument("--external-noise", action="store_true", help="Take the noise signals as inputs (for exact comparisons)")
311
+ parser.add_argument("--opset", type=int, default=17)
312
+ args = parser.parse_args()
313
+ model = load_model(args.checkpoint, "cpu")
314
+ polish = None if args.plain else Polish()
315
+ args.output.parent.mkdir(parents=True, exist_ok=True)
316
+ export(model, args.output, args.external_noise, args.opset, polish)
317
+ args.output.with_suffix(".json").write_text(json.dumps(metadata(model, args.checkpoint, polish), ensure_ascii=False, indent=1) + "\n", encoding="utf-8")
318
+ print(f"wrote {args.output} ({args.output.stat().st_size / 1e6:.1f} MB) and {args.output.with_suffix('.json').name}")
319
+
320
+
321
+ if __name__ == "__main__":
322
+ main()
alpa/infer.py CHANGED
@@ -1,12 +1,13 @@
1
  # Adapted from tiny-voca by Muhammed Nazeem, commit 28e0722 (Apache-2.0; see LICENSES/Apache-2.0.txt
2
  # and THIRD_PARTY_NOTICES.md). File: src/tiny_voca/infer.py. Changes for Alpa: Alpa's model and syllable weight; one
3
- # front end, so the options for choosing between two are removed; the voice is checked against the checkpoint's own list.
 
4
 
5
  import argparse
6
  from pathlib import Path
7
  import torch
8
  from .config import CHECKPOINT_FORMAT, Config
9
- from .model import Alpa
10
  from .text import frontend_fingerprint, phonemize
11
 
12
 
@@ -21,8 +22,9 @@ def load_model(path, device="cpu"):
21
  return model.to(device).eval()
22
 
23
 
24
- def speak(model, text, voice, speed=1.0, pitch_shift=0.0, seed=0, pitch_range=1.0):
25
- """Text -> (float waveform tensor, phonemized text). Same front end as training."""
 
26
  device = next(model.parameters()).device
27
  phones = phonemize(text)
28
  if len(phones) > model.config.max_tokens:
@@ -32,7 +34,8 @@ def speak(model, text, voice, speed=1.0, pitch_shift=0.0, seed=0, pitch_range=1.
32
  if voice not in model.config.speakers:
33
  raise ValueError(f"unknown voice {voice!r}; this model has {model.config.speakers}")
34
  speaker = torch.tensor([model.config.speakers.index(voice)], device=device)
35
- audio, lengths, details = model.synthesize(tokens, weight, torch.tensor([len(phones)], device=device), speaker, speed, pitch_shift, seed, pitch_range)
 
36
  if not torch.isfinite(audio).all():
37
  raise RuntimeError("model produced non-finite audio")
38
  return audio[0, :int(lengths[0])].cpu(), phones, details
@@ -47,11 +50,13 @@ def main():
47
  parser.add_argument("--speed", type=float, default=1)
48
  parser.add_argument("--pitch-shift", type=float, default=0, help="Added to log F0 (0.058 is about one semitone)")
49
  parser.add_argument("--pitch-range", type=float, default=1, help="Scale pitch movement around the utterance mean (1 = as predicted)")
 
50
  parser.add_argument("--device", default="cpu")
51
  args = parser.parse_args()
52
  import soundfile as sf
53
  model = load_model(args.checkpoint, args.device)
54
- wave, phones, _ = speak(model, args.text, args.voice or model.config.speakers[0], args.speed, args.pitch_shift, pitch_range=args.pitch_range)
 
55
  args.output.parent.mkdir(parents=True, exist_ok=True)
56
  # Floating WAV preserves output for inspection; no silent clipping or normalization.
57
  sf.write(args.output, wave.numpy(), model.config.sample_rate, subtype="FLOAT")
 
1
  # Adapted from tiny-voca by Muhammed Nazeem, commit 28e0722 (Apache-2.0; see LICENSES/Apache-2.0.txt
2
  # and THIRD_PARTY_NOTICES.md). File: src/tiny_voca/infer.py. Changes for Alpa: Alpa's model and syllable weight; one
3
+ # front end, so the options for choosing between two are removed; the voice is checked against the checkpoint's own list;
4
+ # speech is polished (model.Polish) unless asked otherwise.
5
 
6
  import argparse
7
  from pathlib import Path
8
  import torch
9
  from .config import CHECKPOINT_FORMAT, Config
10
+ from .model import Alpa, Polish
11
  from .text import frontend_fingerprint, phonemize
12
 
13
 
 
22
  return model.to(device).eval()
23
 
24
 
25
+ def speak(model, text, voice, speed=1.0, pitch_shift=0.0, seed=0, pitch_range=1.0, polish=Polish()):
26
+ """Text -> (float waveform tensor, phonemized text). Same front end as training. ``polish=None`` gives the
27
+ sound exactly as the network was trained to make it."""
28
  device = next(model.parameters()).device
29
  phones = phonemize(text)
30
  if len(phones) > model.config.max_tokens:
 
34
  if voice not in model.config.speakers:
35
  raise ValueError(f"unknown voice {voice!r}; this model has {model.config.speakers}")
36
  speaker = torch.tensor([model.config.speakers.index(voice)], device=device)
37
+ audio, lengths, details = model.synthesize(tokens, weight, torch.tensor([len(phones)], device=device), speaker, speed, pitch_shift, seed, pitch_range,
38
+ polish=polish)
39
  if not torch.isfinite(audio).all():
40
  raise RuntimeError("model produced non-finite audio")
41
  return audio[0, :int(lengths[0])].cpu(), phones, details
 
50
  parser.add_argument("--speed", type=float, default=1)
51
  parser.add_argument("--pitch-shift", type=float, default=0, help="Added to log F0 (0.058 is about one semitone)")
52
  parser.add_argument("--pitch-range", type=float, default=1, help="Scale pitch movement around the utterance mean (1 = as predicted)")
53
+ parser.add_argument("--plain", action="store_true", help="Leave out the corrections made while speaking (the sound of v0.1 as first published)")
54
  parser.add_argument("--device", default="cpu")
55
  args = parser.parse_args()
56
  import soundfile as sf
57
  model = load_model(args.checkpoint, args.device)
58
+ wave, phones, _ = speak(model, args.text, args.voice or model.config.speakers[0], args.speed, args.pitch_shift, pitch_range=args.pitch_range,
59
+ polish=None if args.plain else Polish())
60
  args.output.parent.mkdir(parents=True, exist_ok=True)
61
  # Floating WAV preserves output for inspection; no silent clipping or normalization.
62
  sf.write(args.output, wave.numpy(), model.config.sample_rate, subtype="FLOAT")
alpa/model.py CHANGED
@@ -1,6 +1,7 @@
1
  # Adapted from tiny-voca by Muhammed Nazeem, commit 28e0722 (Apache-2.0; see LICENSES/Apache-2.0.txt
2
  # and THIRD_PARTY_NOTICES.md). File: src/tiny_voca/model.py. Changes for Alpa: the model is named Alpa; the per-token
3
- # stress input is the syllable weight (0 none, 1 laghu, 2 guru); the docstring and one comment are reworded.
 
4
 
5
  """Alpa: feed-forward transformer with a pitch-driven source-filter head.
6
 
@@ -21,6 +22,8 @@ is what keeps the voice from sounding buzzy or unstable at this size.
21
  """
22
 
23
  import math
 
 
24
  import torch
25
  from torch import nn
26
  from torch.nn import functional as F
@@ -162,6 +165,61 @@ def cepstrum_response(causal, anticausal, fft):
162
  return torch.polar(torch.exp(spectrum.real.clamp(-14, 6)), spectrum.imag)
163
 
164
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
165
  class SourceFilter(nn.Module):
166
  """Harmonic-plus-noise synthesis controlled per frame by two linear heads."""
167
 
@@ -186,6 +244,19 @@ class SourceFilter(nn.Module):
186
  self.register_buffer("noise_lifter", lifter(c.noise_cep, 0), persistent=False)
187
  self.register_buffer("synth_window", torch.hann_window(c.synth_fft), persistent=False)
188
  self.register_buffer("noise_window", torch.hann_window(c.noise_fft), persistent=False)
 
 
 
 
 
 
 
 
 
 
 
 
 
189
 
190
  @staticmethod
191
  def _take(values, index):
@@ -204,7 +275,7 @@ class SourceFilter(nn.Module):
204
  # then the same whether it stands alone or is padded inside a batch.
205
  return filtered(F.pad(source, (0, fft // 2)), response, fft, hop, window)[:, :source.shape[-1]]
206
 
207
- def forward(self, hidden, f0, lengths, noise=None, phase=None):
208
  c = self.config
209
  frames = hidden.shape[1]
210
  samples = frames * c.hop
@@ -233,10 +304,58 @@ class SourceFilter(nn.Module):
233
  pulses = pulse_train(f0, c.sample_rate, c.hop, c.harmonic_limit, phase) * audible
234
  if noise is None:
235
  noise = torch.randn(hidden.shape[0], samples, device=hidden.device)
 
 
236
  wave = (self._shape(pulses, harmonic_response, c.synth_fft, c.synth_hop, self.synth_window)
237
  + self._shape(noise * audible, noise_response, c.noise_fft, c.noise_hop, self.noise_window))
238
  return wave * audible
239
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
240
 
241
  class Alpa(nn.Module):
242
  def __init__(self, config=Config()):
@@ -264,6 +383,10 @@ class Alpa(nn.Module):
264
  vowels = torch.zeros(c.vocab_size, dtype=torch.bool)
265
  vowels[[SYMBOL_TO_ID[v] for v in VOWELS]] = True
266
  self.register_buffer("vowel_tokens", vowels, persistent=False)
 
 
 
 
267
 
268
  def encode(self, tokens, weight, lengths, speakers):
269
  if tokens.ndim != 2 or weight.shape != tokens.shape or lengths.shape != (tokens.shape[0],) or speakers.shape != lengths.shape:
@@ -344,16 +467,50 @@ class Alpa(nn.Module):
344
  slow = slow / slow.std().clamp_min(1e-6)
345
  return ((2 * slow + fast) / 5 ** 0.5 * semitones * math.log(2) / 12).to(device)
346
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
347
  @torch.inference_mode()
348
  def synthesize(self, tokens, weight, lengths, speakers, speed=1.0, pitch_shift=0.0, seed=0, pitch_range=1.0, pitch_jitter=0.0,
349
- voiced_vowels=True):
350
- """Free-running synthesis: predicted durations, pitch, voicing and energy."""
 
351
  if not math.isfinite(speed) or not 0.25 <= speed <= 4 or not math.isfinite(pitch_shift) or not 0 <= pitch_range <= 3:
352
  raise ValueError("speed must be between 0.25 and 4, pitch_shift finite and pitch_range between 0 and 3")
353
  hidden, mask = self.encode(tokens, weight, lengths, speakers)
354
  durations = self.predict_durations(hidden, mask, tokens, speed)
355
  frames, frame_mask, frame_lengths = self.expand(hidden, durations, mask, speakers)
356
  prosody_hidden, prosody = self.predict_prosody(frames, frame_mask)
 
 
357
  if voiced_vowels:
358
  # A pitch tracker can miss creaky vowels in the recordings, and the
359
  # voicing predictor then learns to drop whole vowels (heard as a whispered
@@ -373,5 +530,6 @@ class Alpa(nn.Module):
373
  pitch = prosody[..., 0] + pitch_shift
374
  if pitch_jitter > 0:
375
  pitch = pitch + self.pitch_wobble(pitch.shape, pitch_jitter, seed, pitch.device)
376
- wave = self.vocoder(acoustic, self.frequency(pitch), frame_lengths, noise=noise.to(hidden.device))
 
377
  return wave, frame_lengths * self.config.hop, {"durations": durations, "prosody": prosody, "frame_mask": frame_mask}
 
1
  # Adapted from tiny-voca by Muhammed Nazeem, commit 28e0722 (Apache-2.0; see LICENSES/Apache-2.0.txt
2
  # and THIRD_PARTY_NOTICES.md). File: src/tiny_voca/model.py. Changes for Alpa: the model is named Alpa; the per-token
3
+ # stress input is the syllable weight (0 none, 1 laghu, 2 guru); the docstring and one comment are reworded; ``Polish``,
4
+ # ``Shaping`` and the code that applies them when speaking are Alpa's own.
5
 
6
  """Alpa: feed-forward transformer with a pitch-driven source-filter head.
7
 
 
22
  """
23
 
24
  import math
25
+ from dataclasses import dataclass
26
+ from typing import NamedTuple
27
  import torch
28
  from torch import nn
29
  from torch.nn import functional as F
 
165
  return torch.polar(torch.exp(spectrum.real.clamp(-14, 6)), spectrum.imag)
166
 
167
 
168
+ # Sounds by how they are voiced, for ``Polish``: vowels, nasals and semivowels; stops and ज; sibilants and च.
169
+ OPEN = VOWELS + tuple("NYRnmyrlvM")
170
+ STOPS = tuple("kgjwqtdpb")
171
+ HISS = tuple("cSzs")
172
+ NO_PULSES, ALL_PULSES = -1e5, 1e6 # frequency limits (Hz) that mean "noise everywhere" and "pulses everywhere"
173
+
174
+
175
+ @dataclass(frozen=True)
176
+ class Polish:
177
+ """Corrections made while speaking; the weights are not touched.
178
+
179
+ The harmonic branch was trained with a pulse train as its only source, and the network learnt to build hiss,
180
+ breath and bursts out of those pulses as well, which a listener hears as a second, metallic voice inside the
181
+ consonants. Here every frame gets a frequency limit from the sound it belongs to. Below the limit the branch
182
+ is fed the pulses, above it white noise of the same power:
183
+
184
+ * sibilants and च: noise at every frequency;
185
+ * stops and ज: pulses below ``consonant_limit`` only; ह the same where the frame is voiced, else noise;
186
+ * visarga: pulses where the frame is voiced, else noise;
187
+ * vowels, nasals and semivowels: pulses up to ``open_limit``, above which a real voice is breath.
188
+
189
+ The other settings: the predicted pitch is averaged over ``pitch_smooth`` frames either side (triangular
190
+ weights); in vowels, nasals and semivowels the peaks and valleys of the spectral envelope are stretched by
191
+ ``sharpen`` (cepstral coefficients ``sharpen_from`` up to ``sharpen_to``, with the loudness of each broad
192
+ band put back); in vowels the noise branch is held ``noise_cap`` dB under the harmonic branch; and the top
193
+ of the spectrum is lifted by ``treble`` dB (from 2 kHz, in full from 5 kHz). The values were chosen against
194
+ the recordings and by ear: docs/runs, runs 20 to 22."""
195
+ pitch_smooth: int = 3
196
+ consonant_limit: float = 1000.0
197
+ open_limit: float = 4500.0
198
+ sharpen: float = 1.6
199
+ sharpen_from: int = 8
200
+ sharpen_to: int = 40
201
+ noise_cap: float = -15.0
202
+ treble: float = 4.0
203
+
204
+
205
+ class Shaping(NamedTuple):
206
+ """What ``SourceFilter`` needs to apply a ``Polish``: per-frame controls and the replacement noise."""
207
+ polish: Polish
208
+ limits: torch.Tensor # [B, frames] Hz: pulses below, noise above
209
+ open: torch.Tensor # [B, frames] bool: vowel, nasal or semivowel
210
+ vowel: torch.Tensor # [B, frames] bool
211
+ breath: torch.Tensor # [B, samples] white noise, unit variance
212
+
213
+
214
+ def treble_kernel(gain_db, sample_rate, taps=129):
215
+ """Zero-phase filter that lifts the top of the spectrum: nothing below 2 kHz, ``gain_db`` from 5 kHz up."""
216
+ size = 1024
217
+ frequency = torch.fft.rfftfreq(size, 1 / sample_rate).clamp_min(1.0)
218
+ rise = (torch.log2(frequency / 2000.0) / math.log2(2.5)).clamp(0, 1)
219
+ kernel = torch.fft.irfft(10 ** (gain_db * rise / 20), n=size).roll(taps // 2)[:taps]
220
+ return kernel * torch.hann_window(taps, periodic=False)
221
+
222
+
223
  class SourceFilter(nn.Module):
224
  """Harmonic-plus-noise synthesis controlled per frame by two linear heads."""
225
 
 
244
  self.register_buffer("noise_lifter", lifter(c.noise_cep, 0), persistent=False)
245
  self.register_buffer("synth_window", torch.hann_window(c.synth_fft), persistent=False)
246
  self.register_buffer("noise_window", torch.hann_window(c.noise_fft), persistent=False)
247
+ # For ``Polish``: four broad bands of the harmonic filter, and weights that turn one gain per band into a
248
+ # smooth gain per frequency (straight lines between the band centres, level outside them).
249
+ frequency = torch.arange(c.synth_fft // 2 + 1) * (c.sample_rate / c.synth_fft)
250
+ edges, centres = (0.0, 1000.0, 2500.0, 5000.0, float("inf")), torch.tensor([500.0, 1750.0, 3750.0, 8500.0])
251
+ self.register_buffer("band_mask", torch.stack([((frequency >= lo) & (frequency < hi)).float() for lo, hi in zip(edges, edges[1:])]), persistent=False)
252
+ inside = frequency.clamp(float(centres[0]), float(centres[-1]))
253
+ upper = torch.bucketize(inside, centres).clamp(1, len(centres) - 1)
254
+ share = (inside - centres[upper - 1]) / (centres[upper] - centres[upper - 1])
255
+ weight = torch.zeros(len(centres), len(frequency))
256
+ weight[upper - 1, torch.arange(len(frequency))] = 1 - share
257
+ weight[upper, torch.arange(len(frequency))] += share
258
+ self.register_buffer("band_weight", weight, persistent=False)
259
+ self.register_buffer("synth_frequency", frequency, persistent=False)
260
 
261
  @staticmethod
262
  def _take(values, index):
 
275
  # then the same whether it stands alone or is padded inside a batch.
276
  return filtered(F.pad(source, (0, fft // 2)), response, fft, hop, window)[:, :source.shape[-1]]
277
 
278
+ def forward(self, hidden, f0, lengths, noise=None, phase=None, shaping=None):
279
  c = self.config
280
  frames = hidden.shape[1]
281
  samples = frames * c.hop
 
304
  pulses = pulse_train(f0, c.sample_rate, c.hop, c.harmonic_limit, phase) * audible
305
  if noise is None:
306
  noise = torch.randn(hidden.shape[0], samples, device=hidden.device)
307
+ if shaping is not None:
308
+ return self._polished(shaping, causal, anti, harmonic_response, here, pulses, noise * audible, noise_response, last, audible)
309
  wave = (self._shape(pulses, harmonic_response, c.synth_fft, c.synth_hop, self.synth_window)
310
  + self._shape(noise * audible, noise_response, c.noise_fft, c.noise_hop, self.noise_window))
311
  return wave * audible
312
 
313
+ def _polished(self, shaping, causal, anti, response, here, pulses, noise, noise_response, last, audible):
314
+ """The same synthesis with a ``Polish`` applied. ``here`` is the frame of each harmonic sub-frame."""
315
+ c, p = self.config, shaping.polish
316
+ frames = shaping.limits.shape[1]
317
+ samples = frames * c.hop
318
+ frequency = self.synth_frequency
319
+ if p.sharpen != 1.0:
320
+ # Stretch the envelope's detail by ``beta`` (in dB) where the frame is a vowel, nasal or semivowel. The lowest
321
+ # coefficients (loudness and overall slope) and the highest (finer than a formant) are left alone.
322
+ beta = (1.0 + (p.sharpen - 1.0) * shaping.open.float()).gather(1, here)[..., None]
323
+ chosen = torch.zeros(c.cep_causal, device=causal.device)
324
+ chosen[p.sharpen_from:p.sharpen_to] = 1
325
+ mirror = torch.zeros(c.cep_anti, device=causal.device)
326
+ mirror[max(p.sharpen_from - 1, 0):max(p.sharpen_to - 1, 0)] = 1
327
+ sharp = cepstrum_response(causal * (1 + chosen * (beta - 1)), anti * (1 + mirror * (beta - 1)), c.synth_fft)
328
+ # Sharper peaks would move loudness between broad bands; give each band back what the network predicted.
329
+ power = lambda r: r.abs().pow(2) @ self.band_mask.T + 1e-20
330
+ response = sharp * torch.exp(0.5 * torch.log(power(response) / power(sharp)) @ self.band_weight)
331
+ # The mixed source, built in the short-time frames the filter works in: pulses below each sub-frame's
332
+ # limit, noise above it. The switch is 120 Hz wide for a consonant's limit, 350 Hz for the higher one of open
333
+ # sounds, and is softened over three sub-frames in time.
334
+ limit = shaping.limits.gather(1, here)
335
+ keep = torch.sigmoid((limit[:, None] - frequency[None, :, None]) / torch.where(limit > 2000, 350.0, 120.0)[:, None])
336
+ keep = F.avg_pool1d(F.pad(keep, (1, 1), mode="replicate"), 3, 1)
337
+ fft, hop, window = c.synth_fft, c.synth_hop, self.synth_window
338
+ spectrum = lambda source: torch.stft(F.pad(source, (0, fft // 2)), fft, hop, window=window, center=True, pad_mode="constant", return_complex=True)
339
+ # White noise with the pulse train's power per frequency. The pulses carry nothing below the pitch, so the
340
+ # noise that replaces them is cut below 90 Hz, or it would rumble.
341
+ breath = spectrum(shaping.breath * audible * math.sqrt(0.5 / c.harmonic_limit)) * torch.sigmoid((frequency - 90.0) / 12.0)[:, None]
342
+ mixed = spectrum(pulses) * keep + breath * torch.sqrt((1 - keep ** 2).clamp_min(0))
343
+ harmonic = torch.istft(mixed * response.transpose(1, 2), fft, hop, window=window, center=True, length=samples + fft // 2)[:, :samples]
344
+ hiss = self._shape(noise, noise_response, c.noise_fft, c.noise_hop, self.noise_window)
345
+ if p.noise_cap is not None:
346
+ # In a vowel the noise branch sometimes whispers the vowel as loudly as the harmonic branch voices it.
347
+ level = lambda wave: wave.unflatten(1, (frames, c.hop)).pow(2).mean(-1)
348
+ gain = (level(harmonic) * 10 ** (p.noise_cap / 10) / level(hiss).clamp_min(1e-20)).clamp(max=1.0).sqrt()
349
+ gain = torch.where(shaping.vowel, gain, torch.ones_like(gain))
350
+ gain = gain.gather(1, torch.minimum(torch.arange(frames, device=gain.device)[None], last)) # padding repeats the last frame
351
+ gain = F.avg_pool1d(F.pad(gain[:, None], (1, 1), mode="replicate"), 3, 1)
352
+ hiss = hiss * F.interpolate(gain, size=samples, mode="linear", align_corners=False)[:, 0]
353
+ wave = (harmonic + hiss) * audible
354
+ if p.treble:
355
+ kernel = treble_kernel(p.treble, c.sample_rate).to(wave.device)
356
+ wave = F.conv1d(wave[:, None], kernel[None, None], padding=len(kernel) // 2)[:, 0] * audible
357
+ return wave
358
+
359
 
360
  class Alpa(nn.Module):
361
  def __init__(self, config=Config()):
 
383
  vowels = torch.zeros(c.vocab_size, dtype=torch.bool)
384
  vowels[[SYMBOL_TO_ID[v] for v in VOWELS]] = True
385
  self.register_buffer("vowel_tokens", vowels, persistent=False)
386
+ kinds = torch.zeros(c.vocab_size, dtype=torch.long) # 0 pause or mark, then the groups ``Polish`` treats alike
387
+ for kind, symbols in enumerate((OPEN, STOPS, HISS, ("h",), ("H",)), start=1):
388
+ kinds[[SYMBOL_TO_ID[symbol] for symbol in symbols]] = kind
389
+ self.register_buffer("sound_kind", kinds, persistent=False)
390
 
391
  def encode(self, tokens, weight, lengths, speakers):
392
  if tokens.ndim != 2 or weight.shape != tokens.shape or lengths.shape != (tokens.shape[0],) or speakers.shape != lengths.shape:
 
467
  slow = slow / slow.std().clamp_min(1e-6)
468
  return ((2 * slow + fast) / 5 ** 0.5 * semitones * math.log(2) / 12).to(device)
469
 
470
+ @staticmethod
471
+ def frame_owner(durations, size):
472
+ """Index of the token each of ``size`` frames belongs to, [B, size]."""
473
+ frame = torch.arange(size, device=durations.device)[None].expand(durations.shape[0], -1).contiguous()
474
+ return torch.searchsorted(durations.cumsum(1), frame, right=True).clamp(max=durations.shape[1] - 1)
475
+
476
+ @staticmethod
477
+ def steady_pitch(prosody, frame_lengths, frame_mask, reach):
478
+ """Triangular moving average of the predicted pitch, ``reach`` frames either side. The prediction flickers
479
+ from frame to frame more than a voice does. Each row is extended with its own last value, so padding
480
+ inside a batch never changes it."""
481
+ last = (frame_lengths - 1).clamp_min(0)[:, None]
482
+ pitch = prosody[..., 0].gather(1, torch.minimum(torch.arange(prosody.shape[1], device=prosody.device)[None], last))
483
+ kernel = torch.cat((torch.arange(1, reach + 2), torch.arange(reach, 0, -1))).to(pitch)
484
+ pitch = F.conv1d(F.pad(pitch[:, None], (reach, reach), mode="replicate"), (kernel / kernel.sum())[None, None])[:, 0]
485
+ return torch.cat((pitch[..., None], prosody[..., 1:]), dim=-1) * frame_mask[..., None]
486
+
487
+ def shaping(self, polish, tokens, durations, voiced, frame_lengths, seed):
488
+ """Per-frame controls for the waveform head, from the sound each frame belongs to (see ``Polish``)."""
489
+ kind = self.sound_kind[tokens].gather(1, self.frame_owner(durations, voiced.shape[1]))
490
+ limits = torch.full(kind.shape, ALL_PULSES, device=kind.device)
491
+ limits = torch.where(kind == 1, polish.open_limit, limits)
492
+ limits = torch.where(kind == 2, polish.consonant_limit, limits)
493
+ limits = torch.where(kind == 3, NO_PULSES, limits)
494
+ limits = torch.where(kind == 4, torch.where(voiced, polish.consonant_limit, NO_PULSES), limits)
495
+ limits = torch.where((kind == 5) & ~voiced, NO_PULSES, limits)
496
+ vowel = self.vowel_tokens[tokens].gather(1, self.frame_owner(durations, voiced.shape[1]))
497
+ breath = nn.utils.rnn.pad_sequence([torch.randn(int(n) * self.config.hop, generator=torch.Generator().manual_seed(seed + 7919 + row))
498
+ for row, n in enumerate(frame_lengths)], batch_first=True)
499
+ return Shaping(polish, limits, kind == 1, vowel, breath.to(limits.device))
500
+
501
  @torch.inference_mode()
502
  def synthesize(self, tokens, weight, lengths, speakers, speed=1.0, pitch_shift=0.0, seed=0, pitch_range=1.0, pitch_jitter=0.0,
503
+ voiced_vowels=True, polish=None):
504
+ """Free-running synthesis: predicted durations, pitch, voicing and energy. ``polish`` (a ``Polish``) applies
505
+ the corrections made while speaking; ``None`` gives the sound exactly as the network was trained to make it."""
506
  if not math.isfinite(speed) or not 0.25 <= speed <= 4 or not math.isfinite(pitch_shift) or not 0 <= pitch_range <= 3:
507
  raise ValueError("speed must be between 0.25 and 4, pitch_shift finite and pitch_range between 0 and 3")
508
  hidden, mask = self.encode(tokens, weight, lengths, speakers)
509
  durations = self.predict_durations(hidden, mask, tokens, speed)
510
  frames, frame_mask, frame_lengths = self.expand(hidden, durations, mask, speakers)
511
  prosody_hidden, prosody = self.predict_prosody(frames, frame_mask)
512
+ if polish is not None and polish.pitch_smooth:
513
+ prosody = self.steady_pitch(prosody, frame_lengths, frame_mask, polish.pitch_smooth)
514
  if voiced_vowels:
515
  # A pitch tracker can miss creaky vowels in the recordings, and the
516
  # voicing predictor then learns to drop whole vowels (heard as a whispered
 
530
  pitch = prosody[..., 0] + pitch_shift
531
  if pitch_jitter > 0:
532
  pitch = pitch + self.pitch_wobble(pitch.shape, pitch_jitter, seed, pitch.device)
533
+ shaping = None if polish is None else self.shaping(polish, tokens, durations, (prosody[..., 1] > 0) & frame_mask, frame_lengths, seed)
534
+ wave = self.vocoder(acoustic, self.frequency(pitch), frame_lengths, noise=noise.to(hidden.device), shaping=shaping)
535
  return wave, frame_lengths * self.config.hop, {"durations": durations, "prosody": prosody, "frame_mask": frame_mask}
onnx/alpa-v0.1.1.json ADDED
@@ -0,0 +1,83 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model": "alpa-v0.1.1",
3
+ "sample_rate": 24000,
4
+ "hop": 256,
5
+ "voices": [
6
+ "female",
7
+ "male"
8
+ ],
9
+ "symbols": [
10
+ "<pad>",
11
+ "<bos>",
12
+ "<eos>",
13
+ "<sp>",
14
+ "<unk>",
15
+ ",",
16
+ "?",
17
+ "!",
18
+ "|",
19
+ "||",
20
+ "<yati>",
21
+ "a",
22
+ "A",
23
+ "i",
24
+ "I",
25
+ "u",
26
+ "U",
27
+ "f",
28
+ "e",
29
+ "E",
30
+ "o",
31
+ "O",
32
+ "k",
33
+ "g",
34
+ "N",
35
+ "c",
36
+ "j",
37
+ "Y",
38
+ "w",
39
+ "q",
40
+ "R",
41
+ "t",
42
+ "d",
43
+ "n",
44
+ "p",
45
+ "b",
46
+ "m",
47
+ "y",
48
+ "r",
49
+ "l",
50
+ "v",
51
+ "S",
52
+ "z",
53
+ "s",
54
+ "h",
55
+ "M",
56
+ "H"
57
+ ],
58
+ "frontend_fingerprint": "605680d636eba1c6b48119392b97d43de2b71f8abfc52ead230d0f2074ef69f2",
59
+ "max_tokens": 512,
60
+ "inputs": {
61
+ "tokens": "int64 [1, N] phoneme ids from alpa/text.py, with the edge tokens",
62
+ "weight": "int64 [1, N] syllable weight: 0 none, 1 laghu, 2 guru",
63
+ "voice": "int64 [1] index into voices",
64
+ "speed": "float32 [1], 1.0 = as predicted",
65
+ "pitch_shift": "float32 [1], added to log F0",
66
+ "pitch_range": "float32 [1], 1.0 = as predicted"
67
+ },
68
+ "outputs": {
69
+ "audio": "float32 [1, samples]",
70
+ "durations": "int64 [1, N] frames per token"
71
+ },
72
+ "polish": {
73
+ "pitch_smooth": 3,
74
+ "consonant_limit": 1000.0,
75
+ "open_limit": 4500.0,
76
+ "sharpen": 1.6,
77
+ "sharpen_from": 8,
78
+ "sharpen_to": 40,
79
+ "noise_cap": -15.0,
80
+ "treble": 4.0
81
+ },
82
+ "note": "Same weights as alpa-v0.1-preview.onnx. The corrections made while speaking (alpa/model.py, Polish) are part of the graph."
83
+ }
onnx/alpa-v0.1.1.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:778b6cbbeea71093f23456c8a21548512dbbcf2d03a3fe7a3c8110464a202502
3
+ size 22550089
samples/1_gita_1_1.wav CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:cd23c66fcdf3a9aea44bef08f2b9f362007e0ed0e01003d1e09e28c51f864f9b
3
  size 375852
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f36dc9ccbd2450188b375b51b35391f3fed8b4acd1366b39b13c15d1a139560e
3
  size 375852
samples/2_retroflex_and_dental.wav CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:37eac779074966caf3cb610b8a854236b3914dce69ca49c592b1809ec5aa396a
3
  size 307756
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6e18aa19c9c881b8654a793bcb715c9faf1b051ac6e92e8c0887152ce6e75d54
3
  size 307756
samples/3_everyday.wav CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:8a29f8671224c32e000b2de03fe3d813bf7f2d5c442ad4dd435b3d77cdc7b9e0
3
  size 343084
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:75691854bd6de908bfb5749df1752c888f34ac2aa6e47921803e597b1fdf8cd9
3
  size 343084
samples/4_aspirates.wav CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:0e1c5e465dc8dac0dbd0e89ee959c67655f93deb51cb64ab35b202a0a7b7f3f6
3
  size 297516
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4e690cbbbf345ecb38520b3de3e54c092c33bbed33269c522b1243834b8c19ef
3
  size 297516
samples/5_subhashita.wav CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:6228c2b25903d184e411d79a5cc082f1a3470e0a1b48fecb35b9b76767ac6035
3
  size 200236
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:12ac7547d1e9906f846b76b2c9b061f98b9a019677e375a4be669a554a0cf2d7
3
  size 200236
samples/after_v0.1.1.wav ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:89b7deb71e7f10aafa0e9a877669b1b2e50ebde3ca60e336d709929466ff3287
3
+ size 248364
samples/before_v0.1.wav ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:929368b57b2292d92c01df98d81dab719e1535f9a21eb820f57c5b7df9e2d630
3
+ size 248364