Add ChiReSSD v5 checkpoint (inference-only), config, manifest, model card

README.md

@@ -1,3 +1,156 @@
 ---
+library_name: chiressd
+pipeline_tag: text-to-speech
+language: en
 license: mit
+base_model: yl4579/StyleTTS2
+tags:
+  - styletts2
+  - speech-reconstruction
+  - disordered-speech
+  - child-speech
+  - clinical
 ---
+
+# ChiReSSD
+
+Speaker-preserving reconstruction of pediatric disordered speech.
+
+Given a target transcript and a short reference recording from a child with a speech sound
+disorder (SSD), ChiReSSD synthesizes the target utterance with canonical pronunciation while
+keeping the child's voice and prosody. Pronunciation enters through the text pathway; identity
+and prosody come from the style pathway. That separation is the point: ordinary style-preserving
+TTS treats disordered articulation as part of the speaker's style and so reproduces the
+mispronunciation it was meant to correct.
+
+- Code: <https://github.com/Lab-MSP/ChiReSSD>
+- Paper: *Generative Reconstruction of Pediatric Disordered Speech for Automated Clinical
+  Evaluation*
+
+## Model description
+
+StyleTTS2 fine-tuned on child disordered speech. Two 128-dimensional style vectors are extracted
+from the reference recording — acoustic (timbre) and prosodic — and interpolated with a style
+sampled from the adapted diffusion prior. `alpha` weights the acoustic side and `beta` the
+prosodic; 1.0 is fully diffusion-sampled, 0.0 fully reference-driven.
+
+It works on unseen speakers from a reference as short as a few seconds. No per-child model is
+trained, and no paired typical/atypical recordings are required.
+
+## Base model and license chain
+
+Fine-tuned from the StyleTTS2 LibriTTS second-stage checkpoint by
+[yl4579](https://github.com/yl4579/StyleTTS2) (MIT). **That base checkpoint is not redistributed
+here** — obtain it from the upstream release. The frozen helper models (ASR text aligner, JDCNet
+pitch extractor, PL-BERT) likewise ship inside the upstream repository and are not redistributed.
+
+ChiReSSD's own modification to StyleTTS2 is two lines, published as patch files in the code
+repository rather than as a fork.
+
+## Intended use
+
+Research on speech reconstruction and on automated clinical evaluation of pediatric speech sound
+disorders.
+
+## Out of scope
+
+- **Not a medical device.** No diagnostic or treatment decision should rest on its output.
+- Not a replacement for assessment by a licensed speech-language pathologist.
+- Not validated for populations outside the training corpus — it was adapted to children aged
+  5–12 with a Central Scottish accent, and the phonemizer default (`en-gb`) reflects that.
+- Not a general-purpose voice cloning system.
+
+## Training data
+
+Child speech from UltraSuite/CLP-derived recordings, obtained under their own data use
+agreements. **The corpus is not released here and is not redistributable**: it is identifiable
+child clinical speech. See `DATA.md` in the code repository for how to obtain comparable data;
+<https://huggingface.co/datasets/changelinglab/ultrasuite-benchmark> is a suitable starting point
+for fine-tuning your own model.
+
+## Training configuration
+
+| | |
+|---|---|
+| Epochs | 4 |
+| Batch size | 4 |
+| Max length | 600 frames |
+| Learning rate | 1e-5 (`lr`, `bert_lr`, `ft_lr`) |
+| `lambda_F0` | 20 (upstream: 1) |
+| `lambda_mel` | 5 |
+| Style diffusion from epoch | 2 |
+| Joint SLM-adversarial from epoch | 3 |
+| Decoder | HiFi-GAN, multispeaker |
+| Sample rate | 24 kHz |
+| LR schedule | OneCycleLR, `pct_start=0.1` (upstream: 0) |
+
+The heavy pitch weighting is deliberate: the pitch extractor was pretrained on adult voices, and
+children's F0 is both higher and more variable, so it needs the strongest adaptation of any
+component. Conversely, only four epochs — longer schedules start fitting the disordered
+articulation itself.
+
+## Inference
+
+| Preset | alpha | beta | steps | Purpose |
+|---|---|---|---|---|
+| `star_v5` | 0.8 | 0.5 | 10 | Released operating point |
+| `star_v5_alt` | 1.0 | 0.4 | 10 | Variant |
+| `star_oneshot_base` | 1.0 | 0.5 | 5 | One-shot baseline (no fine-tuning) |
+| `star_adult_tts` | 0.3 | 0.7 | 5 | Standard adult-TTS baseline |
+| `torgo_v5` | 1.0 | 0.5 | 15 | Adult dysarthric speech |
+
+`alpha` is high on purpose. A low `alpha` leans on the reference acoustics, which is exactly
+where the disordered articulation lives.
+
+**Synthesis is stochastic.** The initial style latent is zeros rather than a Gaussian draw, but
+the ADPM2 sampler is ancestral and adds fresh noise at every step, so repeated calls differ in
+waveform and in duration. Pass a `seed` for reproducible output.
+
+## Evaluation
+
+See the paper. **This model repository does not reproduce the paper's reported numbers**, and the
+code repository does not either: the evaluation corpora are not redistributable, so the numbers
+cannot be independently recomputed from what is published here.
+
+## Ethical considerations
+
+This model is trained on identifiable clinical recordings of children and is designed to
+reproduce a child's vocal identity accurately. High speaker similarity is the method's goal and
+therefore also its misuse vector: the same property that makes feedback usable in a child's own
+voice makes the model a voice-cloning tool for a minor. Use it only with appropriate consent and
+ethical oversight.
+
+## Usage
+
+```python
+from chiressd.model import ChiReSSD
+from chiressd.config import load_preset
+
+model = ChiReSSD.from_pretrained()  # downloads this checkpoint
+style = model.compute_style("child_reference.wav")
+wav = model.synthesize(
+    "butterfly butterfly butterfly", style, seed=1234, **load_preset("star_v5").as_kwargs()
+)
+```
+
+Run `chiressd-setup` first: it clones and patches the upstream StyleTTS2 checkout that supplies
+the architecture and frozen helper models.
+
+## Checkpoint provenance
+
+Derived from training checkpoint `epoch_2nd_00003.pth` of the v5 run by keeping `state['net']`
+only, removing the `module.` prefix that DataParallel added to ten of the thirteen submodules,
+and making every tensor detached, CPU-resident and contiguous. Precision is unchanged (float32;
+no fp16 cast, which would alter outputs).
+
+- 2258 MB → 767 MB; the 1491 MB removed is optimizer state, which inference never reads
+- `sha256` of `model.pth`: `6c5faf5b4967b4d26cb4c58d4517dcc477729a579039316741bd88b9bd0f16e0`
+- Verified bit-exact against the original: identical 256-d style vector and all 105 000 waveform
+  samples under a fixed seed (`scripts/verify_checkpoint.py`)
+
+Full details in `manifest.json`.
+
+## Citation
+
+Rosero, Yeo, Mortensen, Van't Slot, Hallac, and Busso. *Generative Reconstruction of Pediatric
+Disordered Speech for Automated Clinical Evaluation.*

config.yml

@@ -0,0 +1,66 @@
+ASR_config: Utils/ASR/config.yml
+ASR_path: Utils/ASR/epoch_00080.pth
+F0_path: Utils/JDC/bst.t7
+PLBERT_dir: Utils/PLBERT/
+model_params:
+  decoder:
+    resblock_dilation_sizes:
+    - - 1
+      - 3
+      - 5
+    - - 1
+      - 3
+      - 5
+    - - 1
+      - 3
+      - 5
+    resblock_kernel_sizes:
+    - 3
+    - 7
+    - 11
+    type: hifigan
+    upsample_initial_channel: 512
+    upsample_kernel_sizes:
+    - 20
+    - 10
+    - 6
+    - 4
+    upsample_rates:
+    - 10
+    - 5
+    - 3
+    - 2
+  diffusion:
+    dist:
+      estimate_sigma_data: true
+      mean: -3.0
+      sigma_data: 0.2740232650313019
+      std: 1.0
+    embedding_mask_proba: 0.1
+    transformer:
+      head_features: 64
+      multiplier: 2
+      num_heads: 8
+      num_layers: 3
+  dim_in: 64
+  dropout: 0.2
+  hidden_dim: 512
+  max_conv_dim: 512
+  max_dur: 50
+  multispeaker: true
+  n_layer: 3
+  n_mels: 80
+  n_token: 178
+  slm:
+    hidden: 768
+    initial_channel: 64
+    model: microsoft/wavlm-base-plus
+    nlayers: 13
+    sr: 16000
+  style_dim: 128
+preprocess_params:
+  spect_params:
+    hop_length: 300
+    n_fft: 2048
+    win_length: 1200
+  sr: 24000

manifest.json

@@ -0,0 +1,50 @@
+{
+  "sha256": "6c5faf5b4967b4d26cb4c58d4517dcc477729a579039316741bd88b9bd0f16e0",
+  "filename": "model.pth",
+  "created": "2026-09-14T00:48:03+00:00",
+  "chiressd_commit": "e1b8bbd68255229c55a0797c4a8f2cb0f02fdb84",
+  "upstream_styletts2_commit": "5cedc71c333f8d8b8551ca59378bdcc7af4c9529",
+  "source_checkpoint": "epoch_2nd_00003.pth",
+  "source_config": "config_ft_F0_v5.yml",
+  "strip_recipe": "keep state['net'] only; drop optimizer and scalar bookkeeping; remove the leading 'module.' that DataParallel added; detach/cpu/contiguous every tensor; keep float32 (no fp16 cast, which would change outputs); re-save with _use_new_zipfile_serialization=True",
+  "modules": [
+    "bert",
+    "bert_encoder",
+    "decoder",
+    "diffusion",
+    "mpd",
+    "msd",
+    "pitch_extractor",
+    "predictor",
+    "predictor_encoder",
+    "style_encoder",
+    "text_aligner",
+    "text_encoder",
+    "wd"
+  ],
+  "dataparallel_prefixed_modules": [
+    "bert",
+    "bert_encoder",
+    "decoder",
+    "diffusion",
+    "pitch_extractor",
+    "predictor",
+    "predictor_encoder",
+    "style_encoder",
+    "text_aligner",
+    "text_encoder"
+  ],
+  "dropped_keys": [
+    "epoch",
+    "iters",
+    "optimizer",
+    "val_loss"
+  ],
+  "source_metadata": {
+    "epoch": 3,
+    "iters": 1700,
+    "val_loss": 0.3068065345287323
+  },
+  "source_bytes": 2257595196,
+  "released_bytes": 766549479
+}

model.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6c5faf5b4967b4d26cb4c58d4517dcc477729a579039316741bd88b9bd0f16e0
+size 766549479