FashionFlora commited on
Commit
4e7d05e
·
verified ·
1 Parent(s): ea51f37

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +136 -0
README.md ADDED
@@ -0,0 +1,136 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ - pl
5
+ - ar
6
+ - es
7
+ - hi
8
+ - de
9
+ tags:
10
+ - TTS
11
+ ---
12
+ # SlopTTS
13
+
14
+ SlopTTS is an experimental text-to-speech system designed around a custom FSQ-based speech codec and flow-based generation pipeline. The repository contains training and inference code for speech synthesis, codec training, duration modeling, speaker conditioning, and latent flow modeling.
15
+
16
+ The project is intended for research and experimentation in neural speech synthesis and speech representation learning.
17
+
18
+ ## Features
19
+
20
+ * Custom FSQ speech codec pipeline
21
+ * Flow-based latent generation
22
+ * Speaker-aware synthesis components
23
+ * Duration prediction modules
24
+ * End-to-end training scripts
25
+ * Inference utilities for speech generation
26
+ * Modular architecture for experimentation and research
27
+
28
+ ## Repository Structure
29
+
30
+ ```text
31
+ Configs/ Configuration files
32
+ Data/ Dataset-related resources
33
+ Modules/ Model components
34
+
35
+ train_codec_hybrid_temporal.py Codec training
36
+ train_codec_mel_speaker.py Mel + speaker codec training
37
+ train_codec_speaker.py Speaker codec training
38
+
39
+ train_duration_predictor_context.py
40
+ Duration prediction training
41
+
42
+ train_fsq_flow_convnext.py Flow model training
43
+
44
+ train_predictors_speaker_flow_context_temporal.py
45
+ Predictor training
46
+
47
+ infer_sloptts.py Main TTS inference
48
+ infer_fsq_flow.py Flow inference
49
+
50
+ models.py Core model definitions
51
+ models_speaker.py Speaker modules
52
+ models_mel_speaker.py Mel-speaker models
53
+
54
+ losses.py Training losses
55
+ optimizers.py Optimizer utilities
56
+ utils.py Helper functions
57
+ ```
58
+
59
+ ## Project Overview
60
+
61
+ SlopTTS follows a multi-stage speech generation pipeline. Text is processed into intermediate representations, which are transformed through duration and contextual prediction modules. A custom FSQ-based codec is used to represent speech efficiently, while flow-based models generate coherent latent representations for speech reconstruction.
62
+
63
+ The repository separates codec training, predictor training, and speech generation into independent stages, allowing each component to be improved or replaced individually.
64
+
65
+ ## Training
66
+
67
+ Training is organized into multiple stages:
68
+
69
+ 1. Codec training
70
+ 2. Speaker-aware representation learning
71
+ 3. Duration prediction
72
+ 4. Flow model training
73
+ 5. Predictor training
74
+ 6. End-to-end synthesis evaluation
75
+
76
+ Individual training scripts are provided for each stage.
77
+
78
+ Example:
79
+
80
+ ```bash
81
+ python train_codec_hybrid_temporal.py
82
+ ```
83
+
84
+ or
85
+
86
+ ```bash
87
+ python train_fsq_flow_convnext.py
88
+ ```
89
+
90
+ Configuration files can be adjusted according to dataset size, hardware resources, and training objectives.
91
+
92
+ ## Inference
93
+
94
+ To generate speech:
95
+
96
+ ```bash
97
+ python infer_sloptts.py
98
+ ```
99
+
100
+ Additional flow-based inference utilities are available:
101
+
102
+ ```bash
103
+ python infer_fsq_flow.py
104
+ ```
105
+
106
+ Refer to the configuration files for model paths and runtime settings.
107
+
108
+ ## Requirements
109
+
110
+ Recommended:
111
+
112
+ * Python 3.10+
113
+ * PyTorch
114
+ * CUDA-capable GPU
115
+ * NumPy
116
+ * SciPy
117
+
118
+ Additional dependencies may be listed in the project environment configuration.
119
+
120
+ ## Current Status
121
+
122
+ This repository is actively developed and may contain experimental components. Interfaces, training procedures, and model architectures can change between versions.
123
+
124
+ ## Intended Use
125
+
126
+ SlopTTS is intended for:
127
+
128
+ * Speech synthesis research
129
+ * Codec-based TTS experiments
130
+ * Representation learning research
131
+ * Multilingual speech generation studies
132
+ * Custom voice and speaker modeling research
133
+
134
+ ## Disclaimer
135
+
136
+ This project is provided for research and educational purposes. Generated speech quality depends on the training data, configuration, and model checkpoints used.