GeWu-Lab commited on
Commit
694a535
·
verified ·
1 Parent(s): e20ced8

Link both downstream fine-tuning workflows and update card checksum

Browse files
Files changed (2) hide show
  1. README.md +9 -0
  2. manifest.json +2 -2
README.md CHANGED
@@ -100,6 +100,15 @@ Input files are downmixed to mono, resampled to 16 kHz, and truncated or right-p
100
 
101
  **GPU memory for inference: approximately 8 GB** (BF16, batch size 1, 10-second audio).
102
 
 
 
 
 
 
 
 
 
 
103
  ## Model files
104
 
105
  The DiT and VAE weights are stored in BF16. The T5 encoder and vocoder files are stored in FP32 and cast to the runtime dtype when loaded. Architecture and audio settings are in `config.json`.
 
100
 
101
  **GPU memory for inference: approximately 8 GB** (BF16, batch size 1, 10-second audio).
102
 
103
+ ## Fine-tuning
104
+
105
+ Fine-tune the released SteerSound checkpoint for downstream audio editing with either workflow:
106
+
107
+ 1. **Your own audio–caption pairs:** follow the [data preparation guide](https://github.com/GeWu-Lab/SteerSound/blob/main/pretrain_and_standard_FM_finetune/dataset/README.md). Editing instructions and input/target audio are constructed online. Use [Emphasis](https://github.com/GeWu-Lab/SteerSound/tree/main/STEER_post_training/emphasis/complex) or [Recovery](https://github.com/GeWu-Lab/SteerSound/tree/main/STEER_post_training/recovery/complex), then validate and evaluate the fine-tuned editor on the Complex scene dataset.
108
+ 2. **Our released Selective dataset:** download its train/val/test splits from the [🤗 dataset](https://huggingface.co/datasets/GeWu-Lab/SteerSound) and follow [Emphasis / Selective](https://github.com/GeWu-Lab/SteerSound/tree/main/STEER_post_training/emphasis/selective) or [Recovery / Selective](https://github.com/GeWu-Lab/SteerSound/tree/main/STEER_post_training/recovery/selective).
109
+
110
+ See the [fine-tuning overview](https://github.com/GeWu-Lab/SteerSound/tree/main/STEER_post_training) for required assets, commands, and evaluation steps.
111
+
112
  ## Model files
113
 
114
  The DiT and VAE weights are stored in BF16. The T5 encoder and vocoder files are stored in FP32 and cast to the runtime dtype when loaded. Architecture and audio settings are in `config.json`.
manifest.json CHANGED
@@ -6,8 +6,8 @@
6
  "sha256": "1851bd54305514445591afe73dec20f7142d93892ae24d0fc62dc2855f4d0009"
7
  },
8
  "README.md": {
9
- "bytes": 4822,
10
- "sha256": "b643502cc6e085ebb5b6846d8944cbb82d0897db5ace90177ed2b7595531d468"
11
  },
12
  "dit/model.safetensors": {
13
  "bytes": 1346719328,
 
6
  "sha256": "1851bd54305514445591afe73dec20f7142d93892ae24d0fc62dc2855f4d0009"
7
  },
8
  "README.md": {
9
+ "bytes": 6178,
10
+ "sha256": "bcdf5fa8f0dd409827f72897dbe83d70ac39ce858dc3cf2b4fe33d9e71c870fe"
11
  },
12
  "dit/model.safetensors": {
13
  "bytes": 1346719328,