GeWu-Lab commited on
Commit
c10f277
·
verified ·
1 Parent(s): 694a535

Clarify standard FM Complex evaluation and STEER data preparation workflows

Browse files
Files changed (2) hide show
  1. README.md +4 -4
  2. manifest.json +2 -2
README.md CHANGED
@@ -102,12 +102,12 @@ Input files are downmixed to mono, resampled to 16 kHz, and truncated or right-p
102
 
103
  ## Fine-tuning
104
 
105
- Fine-tune the released SteerSound checkpoint for downstream audio editing with either workflow:
106
 
107
- 1. **Your own audio–caption pairs:** follow the [data preparation guide](https://github.com/GeWu-Lab/SteerSound/blob/main/pretrain_and_standard_FM_finetune/dataset/README.md). Editing instructions and input/target audio are constructed online. Use [Emphasis](https://github.com/GeWu-Lab/SteerSound/tree/main/STEER_post_training/emphasis/complex) or [Recovery](https://github.com/GeWu-Lab/SteerSound/tree/main/STEER_post_training/recovery/complex), then validate and evaluate the fine-tuned editor on the Complex scene dataset.
108
- 2. **Our released Selective dataset:** download its train/val/test splits from the [🤗 dataset](https://huggingface.co/datasets/GeWu-Lab/SteerSound) and follow [Emphasis / Selective](https://github.com/GeWu-Lab/SteerSound/tree/main/STEER_post_training/emphasis/selective) or [Recovery / Selective](https://github.com/GeWu-Lab/SteerSound/tree/main/STEER_post_training/recovery/selective).
109
 
110
- See the [fine-tuning overview](https://github.com/GeWu-Lab/SteerSound/tree/main/STEER_post_training) for required assets, commands, and evaluation steps.
 
111
 
112
  ## Model files
113
 
 
102
 
103
  ## Fine-tuning
104
 
105
+ **Pretraining and standard FM fine-tuning:** use the [original audio datasets used in our paper](https://github.com/GeWu-Lab/SteerSound/blob/main/pretrain_and_standard_FM_finetune/dataset/SOURCES.md) or your own audio–caption data. Obtain raw audio from its original datasets and follow the [format and processing guide](https://github.com/GeWu-Lab/SteerSound/blob/main/pretrain_and_standard_FM_finetune/dataset/README.md). Editing examples are constructed online. Use the [standard training guide](https://github.com/GeWu-Lab/SteerSound/tree/main/pretrain_and_standard_FM_finetune) to pretrain or fine-tune, then evaluate on Complex val/test.
106
 
107
+ **STEER post-training:** start from the released [SteerSound checkpoint](https://huggingface.co/GeWu-Lab/SteerSound) and choose one of these settings:
 
108
 
109
+ 1. **Complex audio editing data:** prepare the original audio datasets used in our paper, or your own audio–caption data, using the same guide. Apply [Emphasis](https://github.com/GeWu-Lab/SteerSound/tree/main/STEER_post_training/emphasis/complex) or [Recovery](https://github.com/GeWu-Lab/SteerSound/tree/main/STEER_post_training/recovery/complex) with online editing-example construction; validate and evaluate on `complex_val` / `complex_test`.
110
+ 2. **Selective dataset:** download the released `selective_train`, `selective_val`, and `selective_test` from the [🤗 dataset](https://huggingface.co/datasets/GeWu-Lab/SteerSound). Follow [Emphasis / Selective](https://github.com/GeWu-Lab/SteerSound/tree/main/STEER_post_training/emphasis/selective) or [Recovery / Selective](https://github.com/GeWu-Lab/SteerSound/tree/main/STEER_post_training/recovery/selective) to post-train on the supplied editing pairs and evaluate on the corresponding splits.
111
 
112
  ## Model files
113
 
manifest.json CHANGED
@@ -6,8 +6,8 @@
6
  "sha256": "1851bd54305514445591afe73dec20f7142d93892ae24d0fc62dc2855f4d0009"
7
  },
8
  "README.md": {
9
- "bytes": 6178,
10
- "sha256": "bcdf5fa8f0dd409827f72897dbe83d70ac39ce858dc3cf2b4fe33d9e71c870fe"
11
  },
12
  "dit/model.safetensors": {
13
  "bytes": 1346719328,
 
6
  "sha256": "1851bd54305514445591afe73dec20f7142d93892ae24d0fc62dc2855f4d0009"
7
  },
8
  "README.md": {
9
+ "bytes": 6759,
10
+ "sha256": "7224003900dd14f33a9cebb70acdf63594f56a674449212ba33789c74fdff54a"
11
  },
12
  "dit/model.safetensors": {
13
  "bytes": 1346719328,