Upload bidirectional ASR-TTS flow matching model weights

.gitattributes

@@ -1,35 +1,4 @@
-*.7z filter=lfs diff=lfs merge=lfs -text
-*.arrow filter=lfs diff=lfs merge=lfs -text
-*.bin filter=lfs diff=lfs merge=lfs -text
-*.bz2 filter=lfs diff=lfs merge=lfs -text
-*.ckpt filter=lfs diff=lfs merge=lfs -text
-*.ftz filter=lfs diff=lfs merge=lfs -text
-*.gz filter=lfs diff=lfs merge=lfs -text
-*.h5 filter=lfs diff=lfs merge=lfs -text
-*.joblib filter=lfs diff=lfs merge=lfs -text
-*.lfs.* filter=lfs diff=lfs merge=lfs -text
-*.mlmodel filter=lfs diff=lfs merge=lfs -text
-*.model filter=lfs diff=lfs merge=lfs -text
-*.msgpack filter=lfs diff=lfs merge=lfs -text
-*.npy filter=lfs diff=lfs merge=lfs -text
-*.npz filter=lfs diff=lfs merge=lfs -text
-*.onnx filter=lfs diff=lfs merge=lfs -text
-*.ot filter=lfs diff=lfs merge=lfs -text
-*.parquet filter=lfs diff=lfs merge=lfs -text
-*.pb filter=lfs diff=lfs merge=lfs -text
-*.pickle filter=lfs diff=lfs merge=lfs -text
-*.pkl filter=lfs diff=lfs merge=lfs -text
 *.pt filter=lfs diff=lfs merge=lfs -text
 *.pth filter=lfs diff=lfs merge=lfs -text
-*.rar filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
 *.safetensors filter=lfs diff=lfs merge=lfs -text
-saved_model/**/* filter=lfs diff=lfs merge=lfs -text
-*.tar.* filter=lfs diff=lfs merge=lfs -text
-*.tar filter=lfs diff=lfs merge=lfs -text
-*.tflite filter=lfs diff=lfs merge=lfs -text
-*.tgz filter=lfs diff=lfs merge=lfs -text
-*.wasm filter=lfs diff=lfs merge=lfs -text
-*.xz filter=lfs diff=lfs merge=lfs -text
-*.zip filter=lfs diff=lfs merge=lfs -text
-*.zst filter=lfs diff=lfs merge=lfs -text
-*tfevents* filter=lfs diff=lfs merge=lfs -text

.gitignore

@@ -0,0 +1,9 @@
+__pycache__/
+*.py[cod]
+*.log
+*.out
+*.err
+logs/
+runs/
+outputs/
+*.tmp

README.md

@@ -0,0 +1,53 @@
+---
+library_name: pytorch
+tags:
+- bidirectional-flow-matching
+- text-to-image
+- image-to-text
+- text-to-speech
+- automatic-speech-recognition
+---
+
+# BVFM model weights
+
+This model repository contains the deployment weights for the image/text and
+speech/text branches of Bidirectional Variational Flow Matching (BVFM). Source
+code, training/inference entry points, and environment instructions live in
+the separate GitHub repository.
+
+## Layout
+
+```text
+image/
+  bvfm_image_step40000.pt
+  FlowTiTok_512.bin
+  README.md
+speech/
+  bvfm_speech_step299999_inference.pt
+  merged_config.json
+  semantic_vae_1000k/
+    config.json
+    metainfo.json
+    dac/ema_state_dict.pth
+  README.md
+```
+
+The repository intentionally excludes training optimizer states, periodic
+snapshots, logs, metrics, generated samples, datasets, and caches.
+
+Clone with Git LFS or use `huggingface-cli download`. Point the GitHub code at
+the downloaded root with:
+
+```bash
+export BVFM_WEIGHTS_ROOT=/path/to/this/model/repository
+```
+
+See the branch-specific READMEs for exact file roles and checksums.
+
+## Third-party components
+
+`image/FlowTiTok_512.bin` is the released FlowTiTok image tokenizer required
+by image encoding and decoding. `speech/semantic_vae_1000k` is the released
+Semantic-VAE decoder required to map the 64-D speech latent to waveform. See
+`THIRD_PARTY_NOTICES.md` and retain the corresponding upstream notices when
+redistributing these files.

THIRD_PARTY_NOTICES.md

@@ -0,0 +1,23 @@
+# Third-party model assets
+
+This repository includes two released inference dependencies in addition to
+the BVFM checkpoints.
+
+## FlowTiTok
+
+`image/FlowTiTok_512.bin` is the released FlowTiTok 512px image tokenizer.
+The associated image branch code is distributed under the Apache License 2.0;
+the GitHub source repository carries the license text and upstream attribution.
+
+## Semantic-VAE
+
+`speech/semantic_vae_1000k` is the released Semantic-VAE 1M-step EMA decoder
+from the Semantic-VAE project:
+
+- Source: https://github.com/ZhikangNiu/Semantic-VAE
+- Original model repository:
+  https://huggingface.co/zkniu/Semantic-VAE/tree/main/semantic_vae_1000k
+- Upstream code license: MIT
+
+Do not remove upstream copyright, citation, or license notices when
+redistributing these assets.

image/FlowTiTok_512.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9169d1e02b8e8fe1bff8a97796b3f1171415c44a1a125c050f7138805b13ebc8
+size 1560616638

image/README.md

@@ -0,0 +1,31 @@
+# BVFM image/text weights
+
+## Files
+
+- `bvfm_image_step40000.pt` (3,230,123,258 bytes): selected bidirectional BVFM
+  checkpoint at step 40,000. Format
+  `flowtok_bvfm_shared_variational_v2`; contains the shared vector field,
+  pair posterior, text/image inference priors, and AR caption decoder. It has
+  no optimizer state.
+- `FlowTiTok_512.bin` (1,560,616,638 bytes): released 512px FlowTiTok tokenizer
+  required by both T2I decoding and I2T image encoding.
+
+Training-only initializers `FlowTok-XL.pth` and `decoder_init.pt` are not part
+of this deployment package.
+
+## SHA-256
+
+```text
+4fe414c46bb1b25781178e0f01f71047194fcc3af42eebeaf596f72386f0b6a3  bvfm_image_step40000.pt
+9169d1e02b8e8fe1bff8a97796b3f1171415c44a1a125c050f7138805b13ebc8  FlowTiTok_512.bin
+```
+
+With the GitHub code, either set `BVFM_WEIGHTS_ROOT` to the downloaded model
+repository root or pass the files explicitly:
+
+```bash
+export FLOWTITOK_CKPT=/path/to/model-repo/image/FlowTiTok_512.bin
+python scripts/infer_t2i.py \
+  --checkpoint /path/to/model-repo/image/bvfm_image_step40000.pt \
+  --output-dir runs/demo
+```

image/bvfm_image_step40000.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4fe414c46bb1b25781178e0f01f71047194fcc3af42eebeaf596f72386f0b6a3
+size 3230123258

speech/README.md

@@ -0,0 +1,36 @@
+# BVFM speech/text weights
+
+## Files
+
+- `bvfm_speech_step299999_inference.pt` (947,317,708 bytes): deployment-only
+  checkpoint extracted from the selected step-299,999 training checkpoint. It
+  contains `inference_modules`, model config, normalization/tokenizer/speaker
+  metadata, and no optimizer, scaler, RNG, training-module duplicate, or EMA
+  training container.
+- `merged_config.json`: effective checkpoint configuration and training
+  provenance. Runtime data/model locations can be overridden by the GitHub
+  entry points.
+- `semantic_vae_1000k/`: released Semantic-VAE EMA decoder metadata and
+  weights used to decode the 40 Hz, 64-D speech latent.
+
+## SHA-256
+
+```text
+6c4f1974e0e29ce7c4d755f1c49875eda5a8e4a03663d4289f0cd97e52df3429  bvfm_speech_step299999_inference.pt
+7c455aa8ab3f7d576b4834f8342558894aafaa61a371b84a9bfa4d10a100e516  semantic_vae_1000k/dac/ema_state_dict.pth
+```
+
+With the GitHub code:
+
+```bash
+export BVFM_WEIGHTS_ROOT=/path/to/model-repo
+python scripts/infer_tts_one.py \
+  --ckpt-dir /path/to/model-repo/speech \
+  --checkpoint bvfm_speech_step299999_inference.pt \
+  --text "A short synthesis example." \
+  --ref-wav /path/to/reference.wav \
+  --out-dir runs/demo
+```
+
+The large resumable `latest.pt`, `final.pt`, and periodic step snapshots are
+deliberately excluded from the Hugging Face package.

speech/bvfm_speech_step299999_inference.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6c4f1974e0e29ce7c4d755f1c49875eda5a8e4a03663d4289f0cd97e52df3429
+size 947317708

speech/merged_config.json

@@ -0,0 +1,381 @@
+{
+  "tokenizer": {
+    "type": "char",
+    "model_path": null
+  },
+  "paths": {
+    "cut_manifest": null,
+    "aligned_manifest": null,
+    "max_cut_rows": null,
+    "use_processed_unified": false,
+    "processed_unified_dir": null,
+    "demo_aligned_manifest": null
+  },
+  "cache": {
+    "mel_dir": null,
+    "speecht5_dir": null,
+    "wav_cache_max_items": 64,
+    "mel_cache_max_items": 1024,
+    "text_hidden_cache_max_items": 2048,
+    "stats_max_unique_wavs": 8000,
+    "gpu_mel_cache": true,
+    "gpu_text_cache": true,
+    "gpu_mel_preload": false,
+    "gpu_text_preload": false,
+    "gpu_mel_cache_dtype": "float32",
+    "gpu_text_cache_dtype": "bfloat16",
+    "gpu_mel_cache_limit_gib": null,
+    "gpu_text_cache_limit_gib": 24,
+    "gpu_cache_reserve_gib": 64.0,
+    "speech_backend": "svae",
+    "svae_latent_dir": null,
+    "semantic_vae_root": null,
+    "semantic_vae_ckpt": null,
+    "semantic_vae_use_ema": true,
+    "svae_dim": 64,
+    "svae_sample_rate": 16000,
+    "svae_hop_size": 400,
+    "mel_stats_mode": "per_bin"
+  },
+  "data": {
+    "batch_size": 16,
+    "batch_size_tts": 16,
+    "batch_size_asr": 16,
+    "ds_factor": 1,
+    "ds_align": 1,
+    "top_k_spk": 1000,
+    "target_spks": null,
+    "force_text_normalize": true,
+    "use_dataloader": true,
+    "loader_num_workers": 8,
+    "loader_pin_memory": true,
+    "loader_persistent_workers": true,
+    "loader_prefetch_factor": 4,
+    "dataset_mel_worker_cache_max_items": 512,
+    "enable_length_bucket": true,
+    "num_length_buckets": 32,
+    "train_unit": "utterance",
+    "max_frames_per_batch": 16384,
+    "max_utts_per_batch": 24,
+    "core_loss_only": false,
+    "pad_to_multiple": 16
+  },
+  "model": {
+    "E_spk": 128,
+    "spk_scale": 0.3,
+    "spk_drop_rate": 0.1,
+    "speaker_cond_type": "ecapa",
+    "speaker_emb_path": null,
+    "speaker_emb_l2_normalize": true,
+    "speaker_emb_missing": "error",
+    "speaker_emb_trainable": false,
+    "speaker_delta_scale": 0.1,
+    "speaker_cond_layernorm": true,
+    "vf_use_speaker_cond": true,
+    "asr_vf_use_speaker_cond": false,
+    "asr_use_spk_cond": false,
+    "asr_spk_scale": 1.0,
+    "asr_spk_unknown": "zero",
+    "asr_use_style_cond": true,
+    "asr_style_use_mean": true,
+    "asr_style_temp": 0.0,
+    "asr_style_detach": false,
+    "use_tts_source_cond": false,
+    "tts_source_cond_hidden": 128,
+    "tts_source_cond_scale": 1.0,
+    "use_tts_style_latent": true,
+    "tts_style_dim": 128,
+    "tts_style_hidden": 256,
+    "tts_style_dropout": 0.05,
+    "tts_style_source_scale": 0.0,
+    "tts_style_post_mode": "speech",
+    "tts_style_prior_type": "speaker",
+    "tts_style_prior_canonical_detach": true,
+    "tts_style_prior_logvar_bias": 0.0,
+    "tts_style_into_source": false,
+    "use_true_canonical_latent": false,
+    "canonical_dim": 192,
+    "canonical_hidden": 256,
+    "canonical_dropout": 0.1,
+    "use_vf_canonical_text_cond": false,
+    "st5_layer_idx": -2,
+    "text_encoder_type": "speecht5",
+    "text_encoder_dim": 384,
+    "text_encoder_layers": 6,
+    "text_encoder_heads": 6,
+    "text_encoder_ff_mult": 4,
+    "text_encoder_conv_ksize": 5,
+    "text_encoder_dropout": 0.1,
+    "text_encoder_max_len": 1024,
+    "use_adapter": true,
+    "adapter_bottleneck": 256,
+    "adapter_dropout": 0.1,
+    "bigvgan_name": "nvidia/bigvgan_22khz_80band",
+    "vf_hidden": 768,
+    "vf_depth": 10,
+    "vf_heads": 12,
+    "vf_dropout": 0.0,
+    "vf_max_len": 2048,
+    "dur_hidden": 256,
+    "dur_dropout": 0.4,
+    "use_len_predictor": false,
+    "len_hidden": 192,
+    "ctc_head_type": "framectcconvhead",
+    "ctc_hidden": 512,
+    "ctc_layers": 6,
+    "ctc_ksize": 7,
+    "ctc_lstm_hidden": 384,
+    "ctc_lstm_layers": 2,
+    "ctc_dropout": 0.1,
+    "att_decoder_hidden": 384,
+    "att_decoder_layers": 4,
+    "att_decoder_heads": 6,
+    "att_decoder_ff_mult": 4,
+    "att_decoder_dropout": 0.1,
+    "att_decoder_max_len": 512,
+    "use_refiner": false,
+    "ref_hidden": 512,
+    "ref_blocks": 8,
+    "ref_ksize": 7,
+    "ref_dropout": 0.1,
+    "zero_shot": {
+      "enable": true,
+      "ref_speaker_emb_model": "speechbrain/spkrec-ecapa-voxceleb",
+      "ref_speaker_emb_savedir": null,
+      "ref_emb_cache_dir": null,
+      "ref_max_sec": 12.0,
+      "ref_l2_normalize": true,
+      "train_ref_source": "same_speaker",
+      "asr_ref_source": "self"
+    },
+    "use_tts_style_pair_posterior": true,
+    "adapter_type": "canonical_text_encoder",
+    "canonical_text_layers": 4,
+    "canonical_text_heads": 8,
+    "canonical_text_ff_mult": 4,
+    "canonical_text_conv_ksize": 5,
+    "canonical_text_residual_scale": 1.0,
+    "vf_condition_injection": "legacy",
+    "ctc_subsample_factor": 1,
+    "ctc_subsample_apply_to": "none",
+    "ctc_heads": 6,
+    "ctc_ff_mult": 4,
+    "ctc_zipformer_downsample": 2
+  },
+  "loss": {
+    "mel_floor": -20.0,
+    "mel_ceil": 20.0,
+    "w_end": 0.3,
+    "enable_fwd_end_loss": true,
+    "w_end_fwd": 0.0,
+    "w_end_bwd": 1.0,
+    "w_ctc_hat": 0.8,
+    "w_ctc_T": 0.05,
+    "w_ctc_start": 200,
+    "enable_ctc_dur": false,
+    "w_ctc_dur": 0.1,
+    "ctc_dur_start": 200,
+    "enable_att_decoder": true,
+    "w_att_decoder": 0.1,
+    "att_decoder_start": 1000,
+    "att_decoder_anneal_steps": 10000,
+    "att_decoder_label_smoothing": 0.1,
+    "att_decoder_detach_input": false,
+    "w_tts_mel": 1.0,
+    "w_ref": 0.5,
+    "w_delta": 0.02,
+    "w_mel_range": 0.0,
+    "w_align_start": 0,
+    "w_dur": 0.2,
+    "w_len": 0.05,
+    "mas_temp": 1.0,
+    "mas_mode": "hard",
+    "mas_mix_alpha": 0.3,
+    "fwd_prior_mode": "mas",
+    "fwd_prior_mix_alpha": 0.5,
+    "fwd_anchor_mode": "sample",
+    "fwd_anchor_mix_alpha": 0.5,
+    "use_full_tts_teacher": false,
+    "w_prior": 0.4,
+    "enable_acoustic_prior_nll": true,
+    "enable_canonical_nll": false,
+    "w_canonical_nll": 0.1,
+    "canonical_nll_start": 200,
+    "canonical_nll_anneal_steps": 10000,
+    "prior_loss_mode": "gaussian_nll",
+    "prior_mu_loss_type": "smooth_l1",
+    "w_prior_mu": 1.0,
+    "w_prior_var": 0.05,
+    "w_prior_nll": 0.1,
+    "prior_fixed_logvar": -2.0,
+    "prior_var_reg_target": -2.0,
+    "w_tts_style_kl": 0.005,
+    "tts_style_kl_start": 5000,
+    "tts_style_kl_anneal_steps": 100000,
+    "enable_full_asr_ctc_aux": false,
+    "w_ctc_full": 0.2,
+    "full_asr_ctc_aux_start": 1000,
+    "full_asr_ctc_aux_every": 4,
+    "full_asr_ctc_aux_batch_size": 1,
+    "use_stat_match": true,
+    "w_stat": 0.01,
+    "use_stft": false,
+    "w_stft": 0.2,
+    "enable_vf_lip": false,
+    "vf_lip_start": 1000,
+    "w_vf_lip": 0.0,
+    "vf_lip_L_hi": 1.0,
+    "vf_lip_sigma": 0.01,
+    "vf_lip_every": 1,
+    "vf_lip_print_every": 200,
+    "enable_bilip_diag": false,
+    "diag_every": 500,
+    "amp_sigma": 0.01,
+    "fwd_ode_grad": true,
+    "bwd_ode_grad": true,
+    "enable_source_ctc": false,
+    "w_ctc_source": 0.3,
+    "source_ctc_start": 200,
+    "source_ctc_lr_scale": 1.0,
+    "enable_zc_sample_ctc": true,
+    "w_ctc_sample": 0.05,
+    "ctc_sample_start": 200,
+    "ctc_sample_temp": 0.2,
+    "enable_dit_hidden_ctc": false,
+    "w_dit_hidden_ctc": 0.02,
+    "dit_hidden_ctc_start": 3000,
+    "dit_hidden_ctc_anneal_steps": 10000,
+    "dit_hidden_ctc_t_min": 0.0,
+    "dit_hidden_ctc_t_max": 0.3,
+    "dit_hidden_ctc_tap_index": 5,
+    "dit_hidden_ctc_anchor": "sample",
+    "dit_hidden_ctc_anchor_mix_alpha": 0.5,
+    "dit_hidden_ctc_apply_subsample": false,
+    "enable_bwd_fm": false,
+    "w_bwd_fm": 0.05,
+    "bwd_fm_start": 1000,
+    "bwd_fm_anneal_steps": 10000,
+    "bwd_fm_t_min": 0.6,
+    "bwd_fm_t_max": 1.0,
+    "bwd_fm_anchor": "mean",
+    "bwd_fm_anchor_mix_alpha": 0.5,
+    "duration_perturb_enable": false,
+    "duration_perturb_num": 1,
+    "duration_perturb_sigma": 0.0,
+    "duration_perturb_include_base": true,
+    "alignment_prior_mode": "mas",
+    "use_gt_alignment_prior": false,
+    "use_gt_duration_teacher": false,
+    "canonical_match_mode": "nll",
+    "canonical_align_candidates": 1,
+    "canonical_softmin_tau": 0.1,
+    "canonical_stopgrad_split": true,
+    "w_canonical_prior_nll": 0.01,
+    "w_canonical_bwd_nll": 0.01,
+    "w_tts_style_asr_kl": 0.02,
+    "tts_style_asr_kl_start": 5000,
+    "tts_style_asr_kl_anneal_steps": 100000,
+    "full_asr_ctc_aux_ode_grad": false,
+    "w_fm": 1.0
+  },
+  "train": {
+    "seed": 0,
+    "total_steps": 300000,
+    "lr_all": 0.0001,
+    "lr_schedule": "warmup_cosine",
+    "lr_warmup_steps": 2000,
+    "lr_min_scale": 0.04,
+    "save_every_steps": 5000,
+    "keep_last_k": 3,
+    "resume_from": null,
+    "use_ema": false,
+    "ema_decay": 0.9999,
+    "log_every": 200,
+    "perf_log_every": 200,
+    "debug_every": 1000,
+    "grad_clip": 1.0
+  },
+  "infer": {
+    "ode_steps_eval": 10,
+    "ode_steps_endloss": 5,
+    "full_asr_chunk_core": 256,
+    "full_asr_chunk_ctx": 96,
+    "full_asr_use_euler": true,
+    "demo_cfg_scale": 1.5,
+    "demo_prior_temp": 0.2,
+    "demo_plot_trajectory": true,
+    "demo_trajectory_asr_realization_plot": true,
+    "demo_trajectory_asr_realization_speakers": 4,
+    "demo_trajectory_asr_realization_styles": 8,
+    "demo_every": 1000,
+    "asr_demo_whole_utterance": true,
+    "asr_demo_steps": 5,
+    "asr_demo_decode_mode": "kenlm",
+    "asr_demo_kenlm": {
+      "lm": null,
+      "lexicon": null,
+      "beam_size": 100,
+      "beam_size_token": 30,
+      "lm_weight": 1.23,
+      "word_score": -0.26,
+      "allow_fallback": false
+    },
+    "disable_demo_cfg_guidance": true,
+    "demo_style_temp": 0,
+    "demo_rtf": true,
+    "demo_trajectory_export_csv": true,
+    "demo_trajectory_projection": "pca",
+    "demo_trajectory_dims": [
+      0,
+      1
+    ],
+    "demo_trajectory_pool": "utterance_mean",
+    "demo_trajectory_frames": 12,
+    "demo_trajectory_speakers": 3,
+    "demo_trajectory_samples": 32,
+    "demo_trajectory_reverse": false,
+    "demo_trajectory_paper_style": true,
+    "demo_trajectory_annotate_points": false,
+    "demo_trajectory_zu_fanout": true,
+    "demo_trajectory_zu_zc_samples": 3,
+    "demo_trajectory_zu_u_samples": 8,
+    "demo_trajectory_asr_many_to_one": true,
+    "demo_trajectory_display_x_scale": 0.65,
+    "demo_trajectory_display_y_scale": 1.0,
+    "tts_demo_text_source": "demo_manifest",
+    "spk_sim_enable": true,
+    "spk_sim_model": "speechbrain/spkrec-ecapa-voxceleb",
+    "spk_sim_savedir": null,
+    "demo_eval_utmos": true,
+    "demo_utmos_repo": "tarepan/SpeechMOS:v1.2.0",
+    "demo_utmos_model": "utmos22_strong",
+    "demo_eval_whisper": true,
+    "demo_whisper_model": "medium.en",
+    "demo_plot_generated_wav_mel": true,
+    "demo_generated_mel_frontend": "bigvgan",
+    "demo_mel_plot_cmap": "viridis",
+    "demo_mel_plot_top_percentile": 99.5,
+    "demo_mel_plot_dynamic_range_db": 65.0,
+    "demo_trajectory_canonical_color": "#1f77b4",
+    "demo_trajectory_speaker_colors": [
+      "#2ca02c",
+      "#ff7f0e"
+    ]
+  },
+  "runtime": {
+    "allow_tf32": true,
+    "cudnn_benchmark": true,
+    "matmul_precision": "high",
+    "compile_enable": false,
+    "compile_mode": "default",
+    "compile_dynamic": true,
+    "compile_vf": true,
+    "compile_refiner": false,
+    "compile_ctc_head": false,
+    "load_bigvgan_model": true
+  },
+  "io": {
+    "demo_dir": null,
+    "ckpt_dir": null
+  }
+}

speech/semantic_vae_1000k/config.json

@@ -0,0 +1,1069 @@
+{
+    "AdamW.betas": [
+        0.8,
+        0.99
+    ],
+    "generator/AdamW.betas": [
+        0.8,
+        0.99
+    ],
+    "discriminator/AdamW.betas": [
+        0.8,
+        0.99
+    ],
+    "AdamW.eps": 1e-08,
+    "generator/AdamW.eps": 1e-08,
+    "discriminator/AdamW.eps": 1e-08,
+    "AdamW.weight_decay": 0.01,
+    "generator/AdamW.weight_decay": 0.01,
+    "discriminator/AdamW.weight_decay": 0.01,
+    "AdamW.amsgrad": false,
+    "generator/AdamW.amsgrad": false,
+    "discriminator/AdamW.amsgrad": false,
+    "AdamW.maximize": false,
+    "generator/AdamW.maximize": false,
+    "discriminator/AdamW.maximize": false,
+    "AdamW.capturable": false,
+    "generator/AdamW.capturable": false,
+    "discriminator/AdamW.capturable": false,
+    "AdamW.differentiable": false,
+    "generator/AdamW.differentiable": false,
+    "discriminator/AdamW.differentiable": false,
+    "amp": false,
+    "ExponentialLR.gamma": 0.999996,
+    "generator/ExponentialLR.gamma": 0.999996,
+    "discriminator/ExponentialLR.gamma": 0.999996,
+    "DAC.encoder_dim": 64,
+    "DAC.encoder_rates": [
+        4,
+        4,
+        5,
+        5
+    ],
+    "DAC.latent_dim": null,
+    "DAC.decoder_dim": 1024,
+    "DAC.decoder_rates": [
+        5,
+        5,
+        4,
+        4
+    ],
+    "DAC.sample_rate": 16000,
+    "DAC.distill": true,
+    "DAC.distill_hidden_dim": 1024,
+    "DAC.decoder_type": "bigvgan",
+    "DAC.pre_vae_block": false,
+    "DAC.attn_proj": true,
+    "DAC.post_vae_block": false,
+    "DAC.bigvgan_conf": "conf/bigvgan_conf/bigvgan_v2_16khz_40hz_base_vae64.json",
+    "DAC.sampling_ratios": [
+        0,
+        1
+    ],
+    "DAC.masked_mean": true,
+    "DAC.distill_loss_dim": -1,
+    "DAC.align_space": "vae",
+    "DAC.align_loss": "cosine",
+    "DAC.use_l2_norm": false,
+    "Discriminator.rates": [],
+    "Discriminator.periods": [
+        2,
+        3,
+        5,
+        7,
+        11
+    ],
+    "Discriminator.fft_sizes": [
+        2048,
+        1024,
+        512
+    ],
+    "Discriminator.sample_rate": 16000,
+    "Discriminator.bands": [
+        [
+            0.0,
+            0.1
+        ],
+        [
+            0.1,
+            0.25
+        ],
+        [
+            0.25,
+            0.5
+        ],
+        [
+            0.5,
+            0.75
+        ],
+        [
+            0.75,
+            1.0
+        ]
+    ],
+    "Discriminator.use_msstft_replace_mrd": false,
+    "AudioDataset.n_examples": 1000,
+    "train/AudioDataset.n_examples": 10000000,
+    "val/AudioDataset.n_examples": 250,
+    "AudioDataset.duration": 0.5,
+    "train/AudioDataset.duration": 3.0,
+    "val/AudioDataset.duration": 5.0,
+    "AudioDataset.offset": null,
+    "train/AudioDataset.offset": null,
+    "val/AudioDataset.offset": null,
+    "AudioDataset.loudness_cutoff": -40,
+    "train/AudioDataset.loudness_cutoff": -40,
+    "val/AudioDataset.loudness_cutoff": -40,
+    "AudioDataset.num_channels": 1,
+    "train/AudioDataset.num_channels": 1,
+    "val/AudioDataset.num_channels": 1,
+    "AudioDataset.aligned": false,
+    "train/AudioDataset.aligned": false,
+    "val/AudioDataset.aligned": false,
+    "AudioDataset.shuffle_loaders": false,
+    "train/AudioDataset.shuffle_loaders": false,
+    "val/AudioDataset.shuffle_loaders": false,
+    "AudioDataset.without_replacement": true,
+    "train/AudioDataset.without_replacement": true,
+    "val/AudioDataset.without_replacement": true,
+    "AudioLoader.sources": null,
+    "train/AudioLoader.sources": null,
+    "val/AudioLoader.sources": null,
+    "AudioLoader.weights": null,
+    "train/AudioLoader.weights": null,
+    "val/AudioLoader.weights": null,
+    "AudioLoader.relative_path": "",
+    "train/AudioLoader.relative_path": "",
+    "val/AudioLoader.relative_path": "",
+    "AudioLoader.ext": [
+        ".wav",
+        ".flac",
+        ".mp3",
+        ".mp4"
+    ],
+    "train/AudioLoader.ext": [
+        ".wav",
+        ".flac",
+        ".mp3",
+        ".mp4"
+    ],
+    "val/AudioLoader.ext": [
+        ".wav",
+        ".flac",
+        ".mp3",
+        ".mp4"
+    ],
+    "AudioLoader.shuffle": true,
+    "train/AudioLoader.shuffle": true,
+    "val/AudioLoader.shuffle": true,
+    "AudioLoader.shuffle_state": 0,
+    "train/AudioLoader.shuffle_state": 0,
+    "val/AudioLoader.shuffle_state": 0,
+    "AudioLoader.guidance_frame_rate": 50,
+    "train/AudioLoader.guidance_frame_rate": 50,
+    "val/AudioLoader.guidance_frame_rate": 50,
+    "BackgroundNoise.snr": [
+        "uniform",
+        10.0,
+        30.0
+    ],
+    "train/BackgroundNoise.snr": [
+        "uniform",
+        10.0,
+        30.0
+    ],
+    "val/BackgroundNoise.snr": [
+        "uniform",
+        10.0,
+        30.0
+    ],
+    "BackgroundNoise.sources": null,
+    "train/BackgroundNoise.sources": null,
+    "val/BackgroundNoise.sources": null,
+    "BackgroundNoise.weights": null,
+    "train/BackgroundNoise.weights": null,
+    "val/BackgroundNoise.weights": null,
+    "BackgroundNoise.eq_amount": [
+        "const",
+        1.0
+    ],
+    "train/BackgroundNoise.eq_amount": [
+        "const",
+        1.0
+    ],
+    "val/BackgroundNoise.eq_amount": [
+        "const",
+        1.0
+    ],
+    "BackgroundNoise.n_bands": 3,
+    "train/BackgroundNoise.n_bands": 3,
+    "val/BackgroundNoise.n_bands": 3,
+    "BackgroundNoise.name": null,
+    "train/BackgroundNoise.name": null,
+    "val/BackgroundNoise.name": null,
+    "BackgroundNoise.prob": 1.0,
+    "train/BackgroundNoise.prob": 1.0,
+    "val/BackgroundNoise.prob": 1.0,
+    "BackgroundNoise.loudness_cutoff": null,
+    "train/BackgroundNoise.loudness_cutoff": null,
+    "val/BackgroundNoise.loudness_cutoff": null,
+    "ClippingDistortion.perc": [
+        "uniform",
+        0.0,
+        0.1
+    ],
+    "train/ClippingDistortion.perc": [
+        "uniform",
+        0.0,
+        0.1
+    ],
+    "val/ClippingDistortion.perc": [
+        "uniform",
+        0.0,
+        0.1
+    ],
+    "ClippingDistortion.name": null,
+    "train/ClippingDistortion.name": null,
+    "val/ClippingDistortion.name": null,
+    "ClippingDistortion.prob": 1.0,
+    "train/ClippingDistortion.prob": 1.0,
+    "val/ClippingDistortion.prob": 1.0,
+    "CorruptPhase.scale": [
+        "uniform",
+        0,
+        3.141592653589793
+    ],
+    "train/CorruptPhase.scale": [
+        "uniform",
+        0,
+        3.141592653589793
+    ],
+    "val/CorruptPhase.scale": [
+        "uniform",
+        0,
+        3.141592653589793
+    ],
+    "CorruptPhase.name": null,
+    "train/CorruptPhase.name": null,
+    "val/CorruptPhase.name": null,
+    "CorruptPhase.prob": 1,
+    "train/CorruptPhase.prob": 1,
+    "val/CorruptPhase.prob": 1,
+    "CrossTalk.snr": [
+        "uniform",
+        0.0,
+        10.0
+    ],
+    "train/CrossTalk.snr": [
+        "uniform",
+        0.0,
+        10.0
+    ],
+    "val/CrossTalk.snr": [
+        "uniform",
+        0.0,
+        10.0
+    ],
+    "CrossTalk.sources": null,
+    "train/CrossTalk.sources": null,
+    "val/CrossTalk.sources": null,
+    "CrossTalk.weights": null,
+    "train/CrossTalk.weights": null,
+    "val/CrossTalk.weights": null,
+    "CrossTalk.name": null,
+    "train/CrossTalk.name": null,
+    "val/CrossTalk.name": null,
+    "CrossTalk.prob": 1.0,
+    "train/CrossTalk.prob": 1.0,
+    "val/CrossTalk.prob": 1.0,
+    "CrossTalk.loudness_cutoff": -40,
+    "train/CrossTalk.loudness_cutoff": -40,
+    "val/CrossTalk.loudness_cutoff": -40,
+    "Equalizer.eq_amount": [
+        "const",
+        1.0
+    ],
+    "train/Equalizer.eq_amount": [
+        "const",
+        1.0
+    ],
+    "val/Equalizer.eq_amount": [
+        "const",
+        1.0
+    ],
+    "Equalizer.n_bands": 6,
+    "train/Equalizer.n_bands": 6,
+    "val/Equalizer.n_bands": 6,
+    "Equalizer.name": null,
+    "train/Equalizer.name": null,
+    "val/Equalizer.name": null,
+    "Equalizer.prob": 1.0,
+    "train/Equalizer.prob": 1.0,
+    "val/Equalizer.prob": 1.0,
+    "FrequencyMask.f_center": [
+        "uniform",
+        0.0,
+        1.0
+    ],
+    "train/FrequencyMask.f_center": [
+        "uniform",
+        0.0,
+        1.0
+    ],
+    "val/FrequencyMask.f_center": [
+        "uniform",
+        0.0,
+        1.0
+    ],
+    "FrequencyMask.f_width": [
+        "const",
+        0.1
+    ],
+    "train/FrequencyMask.f_width": [
+        "const",
+        0.1
+    ],
+    "val/FrequencyMask.f_width": [
+        "const",
+        0.1
+    ],
+    "FrequencyMask.name": null,
+    "train/FrequencyMask.name": null,
+    "val/FrequencyMask.name": null,
+    "FrequencyMask.prob": 1,
+    "train/FrequencyMask.prob": 1,
+    "val/FrequencyMask.prob": 1,
+    "FrequencyNoise.f_center": [
+        "uniform",
+        0.0,
+        1.0
+    ],
+    "train/FrequencyNoise.f_center": [
+        "uniform",
+        0.0,
+        1.0
+    ],
+    "val/FrequencyNoise.f_center": [
+        "uniform",
+        0.0,
+        1.0
+    ],
+    "FrequencyNoise.f_width": [
+        "const",
+        0.1
+    ],
+    "train/FrequencyNoise.f_width": [
+        "const",
+        0.1
+    ],
+    "val/FrequencyNoise.f_width": [
+        "const",
+        0.1
+    ],
+    "FrequencyNoise.name": null,
+    "train/FrequencyNoise.name": null,
+    "val/FrequencyNoise.name": null,
+    "FrequencyNoise.prob": 1,
+    "train/FrequencyNoise.prob": 1,
+    "val/FrequencyNoise.prob": 1,
+    "GlobalVolumeNorm.db": [
+        "const",
+        -24
+    ],
+    "train/GlobalVolumeNorm.db": [
+        "const",
+        -24
+    ],
+    "val/GlobalVolumeNorm.db": [
+        "const",
+        -24
+    ],
+    "GlobalVolumeNorm.name": null,
+    "train/GlobalVolumeNorm.name": null,
+    "val/GlobalVolumeNorm.name": null,
+    "GlobalVolumeNorm.prob": 1.0,
+    "train/GlobalVolumeNorm.prob": 1.0,
+    "val/GlobalVolumeNorm.prob": 1.0,
+    "HighPass.cutoff": [
+        "choice",
+        [
+            50,
+            100,
+            250,
+            500,
+            1000
+        ]
+    ],
+    "train/HighPass.cutoff": [
+        "choice",
+        [
+            50,
+            100,
+            250,
+            500,
+            1000
+        ]
+    ],
+    "val/HighPass.cutoff": [
+        "choice",
+        [
+            50,
+            100,
+            250,
+            500,
+            1000
+        ]
+    ],
+    "HighPass.zeros": 51,
+    "train/HighPass.zeros": 51,
+    "val/HighPass.zeros": 51,
+    "HighPass.name": null,
+    "train/HighPass.name": null,
+    "val/HighPass.name": null,
+    "HighPass.prob": 1,
+    "train/HighPass.prob": 1,
+    "val/HighPass.prob": 1,
+    "BaseTransform.keys": [],
+    "train/BaseTransform.keys": [],
+    "val/BaseTransform.keys": [],
+    "BaseTransform.name": null,
+    "train/BaseTransform.name": null,
+    "val/BaseTransform.name": null,
+    "BaseTransform.prob": 1.0,
+    "train/BaseTransform.prob": 1.0,
+    "val/BaseTransform.prob": 1.0,
+    "InvertPhase.name": null,
+    "train/InvertPhase.name": null,
+    "val/InvertPhase.name": null,
+    "InvertPhase.prob": 1,
+    "train/InvertPhase.prob": 1,
+    "val/InvertPhase.prob": 1,
+    "LowPass.cutoff": [
+        "choice",
+        [
+            4000,
+            8000,
+            16000
+        ]
+    ],
+    "train/LowPass.cutoff": [
+        "choice",
+        [
+            4000,
+            8000,
+            16000
+        ]
+    ],
+    "val/LowPass.cutoff": [
+        "choice",
+        [
+            4000,
+            8000,
+            16000
+        ]
+    ],
+    "LowPass.zeros": 51,
+    "train/LowPass.zeros": 51,
+    "val/LowPass.zeros": 51,
+    "LowPass.name": null,
+    "train/LowPass.name": null,
+    "val/LowPass.name": null,
+    "LowPass.prob": 1,
+    "train/LowPass.prob": 1,
+    "val/LowPass.prob": 1,
+    "MaskLowMagnitudes.db_cutoff": [
+        "uniform",
+        -10,
+        10
+    ],
+    "train/MaskLowMagnitudes.db_cutoff": [
+        "uniform",
+        -10,
+        10
+    ],
+    "val/MaskLowMagnitudes.db_cutoff": [
+        "uniform",
+        -10,
+        10
+    ],
+    "MaskLowMagnitudes.name": null,
+    "train/MaskLowMagnitudes.name": null,
+    "val/MaskLowMagnitudes.name": null,
+    "MaskLowMagnitudes.prob": 1,
+    "train/MaskLowMagnitudes.prob": 1,
+    "val/MaskLowMagnitudes.prob": 1,
+    "MuLawQuantization.channels": [
+        "choice",
+        [
+            8,
+            32,
+            128,
+            256,
+            1024
+        ]
+    ],
+    "train/MuLawQuantization.channels": [
+        "choice",
+        [
+            8,
+            32,
+            128,
+            256,
+            1024
+        ]
+    ],
+    "val/MuLawQuantization.channels": [
+        "choice",
+        [
+            8,
+            32,
+            128,
+            256,
+            1024
+        ]
+    ],
+    "MuLawQuantization.name": null,
+    "train/MuLawQuantization.name": null,
+    "val/MuLawQuantization.name": null,
+    "MuLawQuantization.prob": 1.0,
+    "train/MuLawQuantization.prob": 1.0,
+    "val/MuLawQuantization.prob": 1.0,
+    "NoiseFloor.db": [
+        "const",
+        -50.0
+    ],
+    "train/NoiseFloor.db": [
+        "const",
+        -50.0
+    ],
+    "val/NoiseFloor.db": [
+        "const",
+        -50.0
+    ],
+    "NoiseFloor.name": null,
+    "train/NoiseFloor.name": null,
+    "val/NoiseFloor.name": null,
+    "NoiseFloor.prob": 1.0,
+    "train/NoiseFloor.prob": 1.0,
+    "val/NoiseFloor.prob": 1.0,
+    "Quantization.channels": [
+        "choice",
+        [
+            8,
+            32,
+            128,
+            256,
+            1024
+        ]
+    ],
+    "train/Quantization.channels": [
+        "choice",
+        [
+            8,
+            32,
+            128,
+            256,
+            1024
+        ]
+    ],
+    "val/Quantization.channels": [
+        "choice",
+        [
+            8,
+            32,
+            128,
+            256,
+            1024
+        ]
+    ],
+    "Quantization.name": null,
+    "train/Quantization.name": null,
+    "val/Quantization.name": null,
+    "Quantization.prob": 1.0,
+    "train/Quantization.prob": 1.0,
+    "val/Quantization.prob": 1.0,
+    "Repeat.n_repeat": 1,
+    "train/Repeat.n_repeat": 1,
+    "val/Repeat.n_repeat": 1,
+    "Repeat.name": null,
+    "train/Repeat.name": null,
+    "val/Repeat.name": null,
+    "Repeat.prob": 1.0,
+    "train/Repeat.prob": 1.0,
+    "val/Repeat.prob": 1.0,
+    "RepeatUpTo.max_repeat": 5,
+    "train/RepeatUpTo.max_repeat": 5,
+    "val/RepeatUpTo.max_repeat": 5,
+    "RepeatUpTo.weights": null,
+    "train/RepeatUpTo.weights": null,
+    "val/RepeatUpTo.weights": null,
+    "RepeatUpTo.name": null,
+    "train/RepeatUpTo.name": null,
+    "val/RepeatUpTo.name": null,
+    "RepeatUpTo.prob": 1.0,
+    "train/RepeatUpTo.prob": 1.0,
+    "val/RepeatUpTo.prob": 1.0,
+    "RescaleAudio.val": 1.0,
+    "train/RescaleAudio.val": 1.0,
+    "val/RescaleAudio.val": 1.0,
+    "RescaleAudio.name": null,
+    "train/RescaleAudio.name": null,
+    "val/RescaleAudio.name": null,
+    "RescaleAudio.prob": 1,
+    "train/RescaleAudio.prob": 1,
+    "val/RescaleAudio.prob": 1,
+    "RoomImpulseResponse.drr": [
+        "uniform",
+        0.0,
+        30.0
+    ],
+    "train/RoomImpulseResponse.drr": [
+        "uniform",
+        0.0,
+        30.0
+    ],
+    "val/RoomImpulseResponse.drr": [
+        "uniform",
+        0.0,
+        30.0
+    ],
+    "RoomImpulseResponse.sources": null,
+    "train/RoomImpulseResponse.sources": null,
+    "val/RoomImpulseResponse.sources": null,
+    "RoomImpulseResponse.weights": null,
+    "train/RoomImpulseResponse.weights": null,
+    "val/RoomImpulseResponse.weights": null,
+    "RoomImpulseResponse.eq_amount": [
+        "const",
+        1.0
+    ],
+    "train/RoomImpulseResponse.eq_amount": [
+        "const",
+        1.0
+    ],
+    "val/RoomImpulseResponse.eq_amount": [
+        "const",
+        1.0
+    ],
+    "RoomImpulseResponse.n_bands": 6,
+    "train/RoomImpulseResponse.n_bands": 6,
+    "val/RoomImpulseResponse.n_bands": 6,
+    "RoomImpulseResponse.name": null,
+    "train/RoomImpulseResponse.name": null,
+    "val/RoomImpulseResponse.name": null,
+    "RoomImpulseResponse.prob": 1.0,
+    "train/RoomImpulseResponse.prob": 1.0,
+    "val/RoomImpulseResponse.prob": 1.0,
+    "RoomImpulseResponse.use_original_phase": false,
+    "train/RoomImpulseResponse.use_original_phase": false,
+    "val/RoomImpulseResponse.use_original_phase": false,
+    "RoomImpulseResponse.offset": 0.0,
+    "train/RoomImpulseResponse.offset": 0.0,
+    "val/RoomImpulseResponse.offset": 0.0,
+    "RoomImpulseResponse.duration": 1.0,
+    "train/RoomImpulseResponse.duration": 1.0,
+    "val/RoomImpulseResponse.duration": 1.0,
+    "ShiftPhase.shift": [
+        "uniform",
+        -3.141592653589793,
+        3.141592653589793
+    ],
+    "train/ShiftPhase.shift": [
+        "uniform",
+        -3.141592653589793,
+        3.141592653589793
+    ],
+    "val/ShiftPhase.shift": [
+        "uniform",
+        -3.141592653589793,
+        3.141592653589793
+    ],
+    "ShiftPhase.name": null,
+    "train/ShiftPhase.name": null,
+    "val/ShiftPhase.name": null,
+    "ShiftPhase.prob": 1,
+    "train/ShiftPhase.prob": 1,
+    "val/ShiftPhase.prob": 1,
+    "Silence.name": null,
+    "train/Silence.name": null,
+    "val/Silence.name": null,
+    "Silence.prob": 0.1,
+    "train/Silence.prob": 0.1,
+    "val/Silence.prob": 0.1,
+    "Smoothing.window_type": [
+        "const",
+        "average"
+    ],
+    "train/Smoothing.window_type": [
+        "const",
+        "average"
+    ],
+    "val/Smoothing.window_type": [
+        "const",
+        "average"
+    ],
+    "Smoothing.window_length": [
+        "choice",
+        [
+            8,
+            16,
+            32,
+            64,
+            128,
+            256,
+            512
+        ]
+    ],
+    "train/Smoothing.window_length": [
+        "choice",
+        [
+            8,
+            16,
+            32,
+            64,
+            128,
+            256,
+            512
+        ]
+    ],
+    "val/Smoothing.window_length": [
+        "choice",
+        [
+            8,
+            16,
+            32,
+            64,
+            128,
+            256,
+            512
+        ]
+    ],
+    "Smoothing.name": null,
+    "train/Smoothing.name": null,
+    "val/Smoothing.name": null,
+    "Smoothing.prob": 1,
+    "train/Smoothing.prob": 1,
+    "val/Smoothing.prob": 1,
+    "SpectralDenoising.eq_amount": [
+        "const",
+        1.0
+    ],
+    "train/SpectralDenoising.eq_amount": [
+        "const",
+        1.0
+    ],
+    "val/SpectralDenoising.eq_amount": [
+        "const",
+        1.0
+    ],
+    "SpectralDenoising.denoise_amount": [
+        "uniform",
+        0.8,
+        1.0
+    ],
+    "train/SpectralDenoising.denoise_amount": [
+        "uniform",
+        0.8,
+        1.0
+    ],
+    "val/SpectralDenoising.denoise_amount": [
+        "uniform",
+        0.8,
+        1.0
+    ],
+    "SpectralDenoising.nz_volume": -40,
+    "train/SpectralDenoising.nz_volume": -40,
+    "val/SpectralDenoising.nz_volume": -40,
+    "SpectralDenoising.n_bands": 6,
+    "train/SpectralDenoising.n_bands": 6,
+    "val/SpectralDenoising.n_bands": 6,
+    "SpectralDenoising.n_freq": 3,
+    "train/SpectralDenoising.n_freq": 3,
+    "val/SpectralDenoising.n_freq": 3,
+    "SpectralDenoising.n_time": 5,
+    "train/SpectralDenoising.n_time": 5,
+    "val/SpectralDenoising.n_time": 5,
+    "SpectralDenoising.name": null,
+    "train/SpectralDenoising.name": null,
+    "val/SpectralDenoising.name": null,
+    "SpectralDenoising.prob": 1,
+    "train/SpectralDenoising.prob": 1,
+    "val/SpectralDenoising.prob": 1,
+    "TimeMask.t_center": [
+        "uniform",
+        0.0,
+        1.0
+    ],
+    "train/TimeMask.t_center": [
+        "uniform",
+        0.0,
+        1.0
+    ],
+    "val/TimeMask.t_center": [
+        "uniform",
+        0.0,
+        1.0
+    ],
+    "TimeMask.t_width": [
+        "const",
+        0.025
+    ],
+    "train/TimeMask.t_width": [
+        "const",
+        0.025
+    ],
+    "val/TimeMask.t_width": [
+        "const",
+        0.025
+    ],
+    "TimeMask.name": null,
+    "train/TimeMask.name": null,
+    "val/TimeMask.name": null,
+    "TimeMask.prob": 1,
+    "train/TimeMask.prob": 1,
+    "val/TimeMask.prob": 1,
+    "TimeNoise.t_center": [
+        "uniform",
+        0.0,
+        1.0
+    ],
+    "train/TimeNoise.t_center": [
+        "uniform",
+        0.0,
+        1.0
+    ],
+    "val/TimeNoise.t_center": [
+        "uniform",
+        0.0,
+        1.0
+    ],
+    "TimeNoise.t_width": [
+        "const",
+        0.025
+    ],
+    "train/TimeNoise.t_width": [
+        "const",
+        0.025
+    ],
+    "val/TimeNoise.t_width": [
+        "const",
+        0.025
+    ],
+    "TimeNoise.name": null,
+    "train/TimeNoise.name": null,
+    "val/TimeNoise.name": null,
+    "TimeNoise.prob": 1,
+    "train/TimeNoise.prob": 1,
+    "val/TimeNoise.prob": 1,
+    "VolumeChange.db": [
+        "uniform",
+        -12.0,
+        0.0
+    ],
+    "train/VolumeChange.db": [
+        "uniform",
+        -12.0,
+        0.0
+    ],
+    "val/VolumeChange.db": [
+        "uniform",
+        -12.0,
+        0.0
+    ],
+    "VolumeChange.name": null,
+    "train/VolumeChange.name": null,
+    "val/VolumeChange.name": null,
+    "VolumeChange.prob": 1.0,
+    "train/VolumeChange.prob": 1.0,
+    "val/VolumeChange.prob": 1.0,
+    "VolumeNorm.db": [
+        "const",
+        -24
+    ],
+    "train/VolumeNorm.db": [
+        "const",
+        -24
+    ],
+    "val/VolumeNorm.db": [
+        "const",
+        -24
+    ],
+    "VolumeNorm.name": null,
+    "train/VolumeNorm.name": null,
+    "val/VolumeNorm.name": null,
+    "VolumeNorm.prob": 1.0,
+    "train/VolumeNorm.prob": 1.0,
+    "val/VolumeNorm.prob": 1.0,
+    "L1Loss.attribute": "audio_data",
+    "L1Loss.weight": 1.0,
+    "MelSpectrogramLoss.n_mels": [
+        5,
+        10,
+        20,
+        40,
+        80,
+        160,
+        320
+    ],
+    "MelSpectrogramLoss.window_lengths": [
+        32,
+        64,
+        128,
+        256,
+        512,
+        1024,
+        2048
+    ],
+    "MelSpectrogramLoss.clamp_eps": 1e-05,
+    "MelSpectrogramLoss.mag_weight": 0.0,
+    "MelSpectrogramLoss.log_weight": 1.0,
+    "MelSpectrogramLoss.pow": 1.0,
+    "MelSpectrogramLoss.weight": 1.0,
+    "MelSpectrogramLoss.match_stride": false,
+    "MelSpectrogramLoss.mel_fmin": [
+        0,
+        0,
+        0,
+        0,
+        0,
+        0,
+        0
+    ],
+    "MelSpectrogramLoss.mel_fmax": [
+        null,
+        null,
+        null,
+        null,
+        null,
+        null,
+        null
+    ],
+    "MelSpectrogramLoss.window_type": null,
+    "MultiScaleSTFTLoss.window_lengths": [
+        2048,
+        512
+    ],
+    "MultiScaleSTFTLoss.clamp_eps": 1e-05,
+    "MultiScaleSTFTLoss.mag_weight": 1.0,
+    "MultiScaleSTFTLoss.log_weight": 1.0,
+    "MultiScaleSTFTLoss.pow": 2.0,
+    "MultiScaleSTFTLoss.weight": 1.0,
+    "MultiScaleSTFTLoss.match_stride": false,
+    "MultiScaleSTFTLoss.window_type": null,
+    "SISDRLoss.scaling": true,
+    "SISDRLoss.reduction": "mean",
+    "SISDRLoss.zero_mean": true,
+    "SISDRLoss.clip_min": null,
+    "SISDRLoss.weight": 1.0,
+    "build_transform.augment_prob": 0.0,
+    "train/build_transform.augment_prob": 0.0,
+    "val/build_transform.augment_prob": 1.0,
+    "build_transform.preprocess": [
+        "Identity"
+    ],
+    "train/build_transform.preprocess": [
+        "Identity"
+    ],
+    "val/build_transform.preprocess": [
+        "Identity"
+    ],
+    "build_transform.augment": [
+        "Identity"
+    ],
+    "train/build_transform.augment": [
+        "Identity"
+    ],
+    "val/build_transform.augment": [
+        "Identity"
+    ],
+    "build_transform.postprocess": [
+        "Identity"
+    ],
+    "train/build_transform.postprocess": [
+        "Identity"
+    ],
+    "val/build_transform.postprocess": [
+        "Identity"
+    ],
+    "build_dataset.folders": null,
+    "train/build_dataset.folders": {
+        "speech": [
+            "/vepfs/group09/speech_data/libriheavy/small/",
+            "/vepfs/group09/speech_data/libriheavy/medium/",
+            "/vepfs/group09/speech_data/LibriTTS/train-clean-100",
+            "/vepfs/group09/speech_data/LibriTTS/train-clean-360",
+            "/vepfs/group09/speech_data/LibriTTS/train-other-500"
+        ]
+    },
+    "val/build_dataset.folders": {
+        "speech": [
+            "/vepfs/group09/speech_data/LibriTTS/test-clean"
+        ]
+    },
+    "test/build_dataset.folders": {
+        "speech": [
+            "/vepfs/group09/speech_data/LibriTTS/test-clean"
+        ]
+    },
+    "resume": true,
+    "tag": "latest",
+    "load_weights": false,
+    "seed": 0,
+    "save_path": "/vepfs/group09/descript-audio-codec/ckpts/svae/final_naive_16khz_40hz_5544_kl1e-2_vae64_7khr_bigvgan_base_no_past_bsz64_align_vae_wavlm_23",
+    "num_iters": 1000001,
+    "save_iters": [
+        100000,
+        200000,
+        300000,
+        400000,
+        500000,
+        600000,
+        700000,
+        800000,
+        900000,
+        1000000
+    ],
+    "sample_freq": 10000,
+    "valid_freq": 1000,
+    "batch_size": 64,
+    "val_batch_size": 8,
+    "num_workers": 96,
+    "val_idx": [
+        0,
+        1,
+        2,
+        3,
+        4,
+        5,
+        6,
+        7
+    ],
+    "lambdas": {
+        "mel/loss": 15.0,
+        "adv/feat_loss": 2.0,
+        "adv/gen_loss": 1.0,
+        "vae/kl_loss": 0.01,
+        "vae/proj_loss": 1.0
+    },
+    "use_kl_warmup": false,
+    "kl_warmup_ratio": 0.4,
+    "kl_start_weight": 1e-05,
+    "disc_warmup_step": 0,
+    "args.unknown": [],
+    "DAC.vae_dim": 64,
+    "AdamW.lr": 0.0001,
+    "device": "cuda",
+    "train/AudioDataset.guidance_path": [
+        "/vepfs/group09/descript-audio-codec/scripts/wavlm_large_23_layer/libriheavy/small",
+        "/vepfs/group09/descript-audio-codec/scripts/wavlm_large_23_layer/libriheavy/medium",
+        "/vepfs/group09/descript-audio-codec/scripts/wavlm_large_23_layer/LibriTTS/train-clean-100",
+        "/vepfs/group09/descript-audio-codec/scripts/wavlm_large_23_layer/LibriTTS/train-clean-360",
+        "/vepfs/group09/descript-audio-codec/scripts/wavlm_large_23_layer/LibriTTS/train-other-500"
+    ],
+    "test/AudioDataset.duration": 10.0,
+    "test/build_transform.augment_prob": 1.0,
+    "test/AudioDataset.n_examples": 1000,
+    "args.debug": 0,
+    "args.load": "conf/svae/final_16khz_40hz_5544_kl1e-2_vae64_7khr_bigvgan_base_ll_sm_attn_proj_wavlm_23_align_vae_no_past_finetune_300k.yml",
+    "args.save": null
+}

speech/semantic_vae_1000k/dac/ema_state_dict.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7c455aa8ab3f7d576b4834f8342558894aafaa61a371b84a9bfa4d10a100e516
+size 321925784

speech/semantic_vae_1000k/metainfo.json

@@ -0,0 +1 @@
+{"DAC": {"encoder_dim": 64, "encoder_rates": [4, 4, 5, 5], "latent_dim": null, "decoder_dim": 1024, "decoder_rates": [5, 5, 4, 4], "sample_rate": 16000, "distill": true, "distill_hidden_dim": 1024, "decoder_type": "bigvgan", "pre_vae_block": false, "attn_proj": true, "post_vae_block": false, "bigvgan_conf": "conf/bigvgan_conf/bigvgan_v2_16khz_40hz_base_vae64.json", "sampling_ratios": [0, 1], "masked_mean": true, "distill_loss_dim": -1, "align_space": "vae", "align_loss": "cosine", "use_l2_norm": false, "vae_dim": 64}, "Decoder": {"resblock": "1", "num_mels": 64, "upsample_rates": [5, 5, 2, 2, 2, 2], "upsample_kernel_sizes": [9, 9, 4, 4, 4, 4], "upsample_initial_channel": 1024, "resblock_kernel_sizes": [3, 7, 11], "resblock_dilation_sizes": [[1, 3, 5], [1, 3, 5], [1, 3, 5]], "use_tanh_at_final": false, "use_bias_at_final": false, "activation": "snakebeta", "snake_logscale": true}, "Discriminator": {"rates": [], "periods": [2, 3, 5, 7, 11], "fft_sizes": [2048, 1024, 512], "sample_rate": 16000, "bands": [[0.0, 0.1], [0.1, 0.25], [0.25, 0.5], [0.5, 0.75], [0.75, 1.0]], "use_msstft_replace_mrd": false}}