Update README
Browse files
README.md
CHANGED
|
@@ -198,21 +198,21 @@ Embeat is a music recommendation system built on Spotify acoustic feature data.
|
|
| 198 |
</details>
|
| 199 |
|
| 200 |
### LLM Blind Evaluation
|
| 201 |
-
|
| 202 |
Using the LLM-as-a-Judge method, Embeat was blindly evaluated against Netease Cloud Music in AB tests (Evaluation date: 2026-09-15)
|
| 203 |
-
|
| 204 |
-
|
| 205 |
-
|
| 206 |
| Judge Model | Embeat Wins | Netease Wins | Tie | Not rated |
|
| 207 |
|-------------|:-----------:|:------------:|:---:|:---------:|
|
| 208 |
| Claude Opus 5 | **122** | 14 | 1 | 20 |
|
| 209 |
| Kimi K3 | **117** | 23 | 2 | 15 |
|
| 210 |
| GPT 5.6 Sol | **109** | 20 | 0 | 28 |
|
| 211 |
| Gemini 3.1 Pro | **90** | 5 | 0 | 62 |
|
| 212 |
-
|
| 213 |
**Conclusions:**
|
| 214 |
-
|
| 215 |
-
- **Availability**: Embeat returned a full set of recommendations for all 176 seeds; Netease failed on 10.8% of them, returning nothing at all in 4 seeds
|
| 216 |
- **Diversity**: Embeat averages 4.88 distinct artists per list against Netease's 3.57; in 28% of Netease's lists, 4 or more of the 5 tracks come from a single artist
|
| 217 |
- **Robustness**: the lead holds across all four popularity bands and is in fact larger for popular seeds (90%) than for long-tail ones (77%), so this is not an advantage confined to obscure music
|
| 218 |
- **By language**: Embeat leads in all four buckets (Mandarin, Japanese, Korean, Others), including 81~88% in Mandarin, where Netease is strongest
|
|
|
|
| 198 |
</details>
|
| 199 |
|
| 200 |
### LLM Blind Evaluation
|
| 201 |
+
|
| 202 |
Using the LLM-as-a-Judge method, Embeat was blindly evaluated against Netease Cloud Music in AB tests (Evaluation date: 2026-09-15)
|
| 203 |
+
|
| 204 |
+
The seed tracks were selected from 6,291 representative songs across microgenres on EveryNoise. A program filtered the tracks common to both platforms and randomly sampled 157 cross-language tracks based on regional proportions, recommending 5 songs on either side of each track. After standardizing the metadata and randomly shuffling the order, an LLM from four different vendors independently scored each track
|
| 205 |
+
|
| 206 |
| Judge Model | Embeat Wins | Netease Wins | Tie | Not rated |
|
| 207 |
|-------------|:-----------:|:------------:|:---:|:---------:|
|
| 208 |
| Claude Opus 5 | **122** | 14 | 1 | 20 |
|
| 209 |
| Kimi K3 | **117** | 23 | 2 | 15 |
|
| 210 |
| GPT 5.6 Sol | **109** | 20 | 0 | 28 |
|
| 211 |
| Gemini 3.1 Pro | **90** | 5 | 0 | 62 |
|
| 212 |
+
|
| 213 |
**Conclusions:**
|
| 214 |
+
|
| 215 |
+
- **Availability**: Embeat returned a full set of recommendations for all 176 seeds; Netease failed on 10.8% of them, returning nothing at all in 4 seeds (176 - 19 = 157)
|
| 216 |
- **Diversity**: Embeat averages 4.88 distinct artists per list against Netease's 3.57; in 28% of Netease's lists, 4 or more of the 5 tracks come from a single artist
|
| 217 |
- **Robustness**: the lead holds across all four popularity bands and is in fact larger for popular seeds (90%) than for long-tail ones (77%), so this is not an advantage confined to obscure music
|
| 218 |
- **By language**: Embeat leads in all four buckets (Mandarin, Japanese, Korean, Others), including 81~88% in Mandarin, where Netease is strongest
|