afloven commited on
Commit
10ee84a
·
verified ·
1 Parent(s): 5913ad5

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +64 -1
README.md CHANGED
@@ -3,10 +3,73 @@ license: mit
3
  language:
4
  - en
5
  metrics:
 
6
  - accuracy
7
  base_model:
8
  - MIT/ast-finetuned-audioset-10-10-0.4593
9
  pipeline_tag: audio-classification
10
  tags:
11
  - music
12
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  language:
4
  - en
5
  metrics:
6
+ - f1
7
  - accuracy
8
  base_model:
9
  - MIT/ast-finetuned-audioset-10-10-0.4593
10
  pipeline_tag: audio-classification
11
  tags:
12
  - music
13
+ - audio-transformers
14
+ ---
15
+
16
+ # Audio Spectrogram Transformer (AST) for Music Genre Classification
17
+
18
+ This model was developed to identify musical genres in complex, noisy audio environments. It was built as part of a deep learning project focused on the **Messy Mashup** challenge, where the goal was to classify genres even when multiple instrument tracks and environmental noises overlap.
19
+
20
+ ---
21
+
22
+ ## Model Overview
23
+
24
+ This model is based on the **Audio Spectrogram Transformer (AST)**. Unlike traditional models that look at audio as a simple sequence of sounds, the AST treats an audio spectrogram like a high-resolution image. It uses an **Attention Mechanism** to look at the entire 10-second clip at once, allowing it to pick up on both subtle instrument textures and overall rhythmic patterns.
25
+
26
+ ### Technical Details
27
+ * **Architecture:** Audio Spectrogram Transformer (AST)
28
+ * **Sampling Rate:** 16,000 Hz
29
+ * **Input Length:** 10 Seconds
30
+ * **Feature Type:** Log-Mel Spectrogram (128 Mel bands)
31
+ * **Performance:** Achieved a **0.85 Macro F1-score** on the evaluation set.
32
+
33
+ ---
34
+
35
+ ## The Training Strategy: Cross-Song Recombination
36
+
37
+ The main reason this model performs well is a training technique called **Cross-Song Recombination**.
38
+
39
+ In most training setups, a model listens to a single song at a time. To make this model more resilient to noise and "messy" audio, we created synthetic mashups during training. We took individual "stems" (the separate recordings for bass, drums, vocals, and other instruments) from different songs within the same genre and mixed them together.
40
+
41
+ We also added environmental noise from the ESC-50 dataset. This forced the model to ignore the chaos and focus only on the core spectral patterns that define a genre—such as the specific frequency of a blues guitar or the tempo of a techno beat.
42
+
43
+ ---
44
+
45
+ ## Supported Genres
46
+
47
+ The model is trained to classify audio into one of the following 10 categories:
48
+ 1. **Blues**
49
+ 2. **Classical**
50
+ 3. **Country**
51
+ 4. **Disco**
52
+ 5. **Hiphop**
53
+ 6. **Jazz**
54
+ 7. **Metal**
55
+ 8. **Pop**
56
+ 9. **Reggae**
57
+ 10. **Rock**
58
+
59
+ ---
60
+
61
+ ## How to Use
62
+
63
+ You can use this model directly with the Hugging Face `transformers` library.
64
+
65
+ ```python
66
+ from transformers import pipeline
67
+
68
+ # Load the model and preprocessor
69
+ classifier = pipeline("audio-classification", model="afloven/messymashupclassifier")
70
+
71
+ # Predict the genre of an audio file
72
+ # Make sure your audio is sampled at 16,000 Hz
73
+ results = classifier("path_to_your_audio_file.wav")
74
+
75
+ print(results)