File size: 2,675 Bytes
9681162 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 | ---
license: apache-2.0
library_name: coreml
tags:
- coreml
- yamnet
- audio-classification
- tensorflow
- apple
---
# YAMNet Core ML
This is a Core ML conversion of the YAMNet audio event classifier. The model was converted from the TensorFlow Hub YAMNet SavedModel.
Important: this Core ML package does not accept raw waveform audio. It accepts precomputed YAMNet log-mel feature patches shaped `[1, 96, 64]`.
## Model Interface
Input:
- `features`: `MLMultiArray` shaped `[1, 96, 64]`, `Float16`
- Represents one YAMNet log-mel patch: 96 frames x 64 mel bands.
Output:
- `Identity`: `MLMultiArray` shaped `[1, 521]`, `Float16`
- Contains class scores for the 521 YAMNet AudioSet classes.
Class labels are available in:
```text
yamnet_model/assets/yamnet_class_map.csv
```
## Swift Usage
```swift
import CoreML
import Foundation
let modelURL = URL(fileURLWithPath: "YAMNet.mlpackage")
let model = try MLModel(contentsOf: modelURL)
let features = try MLMultiArray(shape: [1, 96, 64], dataType: .float16)
// Fill `features` with YAMNet-compatible log-mel values.
// Layout is [batch, frame, melBand].
for frame in 0..<96 {
for band in 0..<64 {
let index = [0, NSNumber(value: frame), NSNumber(value: band)]
features[index] = 0.0
}
}
let input = try MLDictionaryFeatureProvider(dictionary: [
"features": MLFeatureValue(multiArray: features)
])
let output = try model.prediction(from: input)
guard let scores = output.featureValue(for: "Identity")?.multiArrayValue else {
throw NSError(domain: "YAMNet", code: 1, userInfo: [NSLocalizedDescriptionKey: "Missing scores output"])
}
var bestIndex = 0
var bestScore = -Double.infinity
for index in 0..<521 {
let score = scores[[0, NSNumber(value: index)]].doubleValue
if score > bestScore {
bestScore = score
bestIndex = index
}
}
print("Top class index: \(bestIndex), score: \(bestScore)")
```
## Feature Extraction
To use this model with raw audio, compute YAMNet-compatible features first:
- sample rate: 16 kHz mono
- STFT window: 25 ms
- STFT hop: 10 ms
- mel bands: 64
- mel range: 125 Hz to 7500 Hz
- log offset: `0.001`
- patch size: 96 frames x 64 mel bands
The original TensorFlow Hub YAMNet model includes waveform preprocessing, but that path uses TensorFlow ops that are not currently supported by Core ML conversion. This package therefore contains only the classifier network after feature extraction.
## Conversion
The package was generated with:
```bash
uv run main.py --model-path ./yamnet_model --output ./YAMNet.mlpackage
```
To download the TensorFlow Hub source model:
```bash
uv run main.py --download-only
```
|