Other
IN03X commited on
Commit
b7622c6
·
verified ·
1 Parent(s): 844622a

Show four MP4 demos and expand section-level documentation links

Browse files
README.md CHANGED
@@ -34,42 +34,42 @@ into a variable set of source maps. The release includes the complete inference,
34
  evaluation, training, and Online Scene Generation workflows described in the
35
  paper.
36
 
37
- ## Demos
38
 
39
- The following 20-second examples use the SAID (PaSST) paper checkpoint. Ground
40
- Truth is shown on the left and Prediction on the right with a shared timeline,
41
- class legend, colors, and display scale. Each MP4 includes synchronized audio;
42
- the inline animation previews the visual output.
43
 
44
- ### Demo 1 — TAU scene
 
 
 
45
 
46
- [![SAID Demo 1: Ground Truth and Prediction in a TAU scene](docs/media/said_demo_1.webp)](docs/media/said_demo_1.mp4)
47
 
48
- [Open Demo 1 as MP4](docs/media/said_demo_1.mp4)
49
 
50
- ### Demo 3 — Sony scene
51
 
52
- [![SAID Demo 3: Ground Truth and Prediction in a Sony scene](docs/media/said_demo_3.webp)](docs/media/said_demo_3.mp4)
53
 
54
- [Open Demo 3 as MP4](docs/media/said_demo_3.mp4)
55
 
56
- <details>
57
- <summary>Audio2Sph + Panoramic Decoder demonstrations</summary>
58
 
59
- These class-agnostic examples visualize the panoramic acoustic field used during
60
- Audio2Sph pretraining.
61
 
62
- - [TAU scene](docs/media/audio2sph_demo_1.mp4)
63
- - [Sony scene](docs/media/audio2sph_demo_3.mp4)
64
 
65
- </details>
66
 
67
  ## Quick start
68
 
69
- SAID supports Python 3.10 through 3.12. Install the project from the repository
70
- root:
71
 
72
  ```bash
 
 
 
73
  pip install -e .
74
  ```
75
 
@@ -90,8 +90,9 @@ The CLI downloads the selected checkpoint from
90
  `checkpoints/` directory on first use. `recording.wav` may contain the complete
91
  32-channel Eigenmike signal or the four selected capsules `[6, 10, 26, 22]` in
92
  that order. The default output is readable DCASE JSON. Select the AudioMAE
93
- variant with `--model said_audiomae`; complete options and the Python API are in
94
- [Inference](docs/inference.md).
 
95
 
96
  ## Results
97
 
@@ -120,8 +121,8 @@ The table evaluates all 78 full development-test recordings.
120
 
121
  The `(4.2)` AP columns reproduce the evaluator revision used by the paper;
122
  `(6.30)` applies the current official evaluator to the same predictions.
123
- [Evaluation](docs/evaluation.md#metric-versions) records the exact dates,
124
- commits, definitions, and output fields.
125
 
126
  ## Evaluation
127
 
@@ -140,6 +141,8 @@ The default metrics follow the current `(6.30)` official definition;
140
  The evaluator is acquired from the
141
  [official DCASE2026 Task 3 repository](https://github.com/iranroman/DCASE2026_Task3_SAISELD_baseline).
142
 
 
 
143
  ## Models and checkpoints
144
 
145
  | Model | Role and output | Checkpoint |
@@ -153,6 +156,10 @@ and SHA256 before loading. They are released for non-commercial research use onl
153
  under the [SAID Model Weights Non-Commercial Research License 1.0](LICENSES/SAID-Model-Weights-NonCommercial-1.0.txt)
154
  and all applicable upstream terms.
155
 
 
 
 
 
156
  ## Training and data
157
 
158
  The paper training workflow is:
@@ -179,9 +186,8 @@ the included adapters:
179
  - VCTK v0.80 for Audio2Sph pretraining;
180
  - an authorized SourceBank manifest for class-labeled Online Scene Generation.
181
 
182
- [Training](docs/training.md) gives the complete checkpoint flow and recipes.
183
- [Data preparation](docs/data.md) gives the required sources, directory layouts,
184
- manifest schema, and license-aware interfaces.
185
 
186
  ## Documentation
187
 
 
34
  evaluation, training, and Online Scene Generation workflows described in the
35
  paper.
36
 
37
+ Details: [Model architecture](docs/model_architecture.md).
38
 
39
+ ## Demos
 
 
 
40
 
41
+ The following 20-second examples show SAID (PaSST) and Audio2Sph + Panoramic
42
+ Decoder outputs. Ground Truth is shown on the left and Prediction on the right
43
+ with a shared timeline, colors, and display scale. The SAID videos also include
44
+ the sound-event class legend. All four MP4 files include synchronized audio.
45
 
46
+ ### SAID (PaSST) — TAU scene
47
 
48
+ <video src="https://huggingface.co/IN03X/SAID/resolve/main/docs/media/said_demo_1.mp4" controls playsinline preload="metadata" width="100%"></video>
49
 
50
+ ### SAID (PaSST) — Sony scene
51
 
52
+ <video src="https://huggingface.co/IN03X/SAID/resolve/main/docs/media/said_demo_3.mp4" controls playsinline preload="metadata" width="100%"></video>
53
 
54
+ ### Audio2Sph + Panoramic Decoder — TAU scene
55
 
56
+ <video src="https://huggingface.co/IN03X/SAID/resolve/main/docs/media/audio2sph_demo_1.mp4" controls playsinline preload="metadata" width="100%"></video>
 
57
 
58
+ ### Audio2Sph + Panoramic Decoder — Sony scene
 
59
 
60
+ <video src="https://huggingface.co/IN03X/SAID/resolve/main/docs/media/audio2sph_demo_3.mp4" controls playsinline preload="metadata" width="100%"></video>
 
61
 
62
+ Details: [Demos and inference](docs/inference.md).
63
 
64
  ## Quick start
65
 
66
+ SAID supports Python 3.10 through 3.12. Create an environment, enter the project
67
+ directory, and install the package:
68
 
69
  ```bash
70
+ conda create -n said python=3.10 -y
71
+ conda activate said
72
+ cd /path/to/SAID
73
  pip install -e .
74
  ```
75
 
 
90
  `checkpoints/` directory on first use. `recording.wav` may contain the complete
91
  32-channel Eigenmike signal or the four selected capsules `[6, 10, 26, 22]` in
92
  that order. The default output is readable DCASE JSON. Select the AudioMAE
93
+ variant with `--model said_audiomae`.
94
+
95
+ Details: [Inference and Python API](docs/inference.md).
96
 
97
  ## Results
98
 
 
121
 
122
  The `(4.2)` AP columns reproduce the evaluator revision used by the paper;
123
  `(6.30)` applies the current official evaluator to the same predictions.
124
+
125
+ Details: [Metric versions and definitions](docs/evaluation.md#metric-versions).
126
 
127
  ## Evaluation
128
 
 
141
  The evaluator is acquired from the
142
  [official DCASE2026 Task 3 repository](https://github.com/iranroman/DCASE2026_Task3_SAISELD_baseline).
143
 
144
+ Details: [DCASE evaluation and reproduction](docs/evaluation.md).
145
+
146
  ## Models and checkpoints
147
 
148
  | Model | Role and output | Checkpoint |
 
156
  under the [SAID Model Weights Non-Commercial Research License 1.0](LICENSES/SAID-Model-Weights-NonCommercial-1.0.txt)
157
  and all applicable upstream terms.
158
 
159
+ Details: [SAID (PaSST)](docs/checkpoints/said_passt.md),
160
+ [SAID (AudioMAE)](docs/checkpoints/said_audiomae.md), and
161
+ [Audio2Sph + Panoramic Decoder](docs/checkpoints/audio2sph.md).
162
+
163
  ## Training and data
164
 
165
  The paper training workflow is:
 
186
  - VCTK v0.80 for Audio2Sph pretraining;
187
  - an authorized SourceBank manifest for class-labeled Online Scene Generation.
188
 
189
+ Details: [Training](docs/training.md), [data preparation](docs/data.md), and
190
+ [configuration](docs/configuration.md).
 
191
 
192
  ## Documentation
193
 
docs/media/said_demo_1.webp DELETED

Git LFS Details

  • SHA256: 72f82d02cbfb9f495064e896a16d44ca2de1a133e80b8a31a03165662db81c9c
  • Pointer size: 131 Bytes
  • Size of remote file: 641 kB
docs/media/said_demo_3.webp DELETED

Git LFS Details

  • SHA256: 202ec87074dae22187fb38fa11d26e3d2b67bc558335272f3812ec6bd3272774
  • Pointer size: 131 Bytes
  • Size of remote file: 717 kB