Yehor/YAMNet-CoreML

1---2license: apache-2.03library_name: coreml4tags:5  - coreml6  - yamnet7  - audio-classification8  - tensorflow9  - apple10---11 12# YAMNet Core ML13 14This is a Core ML conversion of the YAMNet audio event classifier. The model was converted from the TensorFlow Hub YAMNet SavedModel.15 16Important: this Core ML package does not accept raw waveform audio. It accepts precomputed YAMNet log-mel feature patches shaped `[1, 96, 64]`.17 18## Model Interface19 20Input:21 22- `features`: `MLMultiArray` shaped `[1, 96, 64]`, `Float16`23- Represents one YAMNet log-mel patch: 96 frames x 64 mel bands.24 25Output:26 27- `Identity`: `MLMultiArray` shaped `[1, 521]`, `Float16`28- Contains class scores for the 521 YAMNet AudioSet classes.29 30Class labels are available in:31 32```text33yamnet_model/assets/yamnet_class_map.csv34```35 36## Swift Usage37 38```swift39import CoreML40import Foundation41 42let modelURL = URL(fileURLWithPath: "YAMNet.mlpackage")43let model = try MLModel(contentsOf: modelURL)44 45let features = try MLMultiArray(shape: [1, 96, 64], dataType: .float16)46 47// Fill `features` with YAMNet-compatible log-mel values.48// Layout is [batch, frame, melBand].49for frame in 0..<96 {50    for band in 0..<64 {51        let index = [0, NSNumber(value: frame), NSNumber(value: band)]52        features[index] = 0.053    }54}55 56let input = try MLDictionaryFeatureProvider(dictionary: [57    "features": MLFeatureValue(multiArray: features)58])59 60let output = try model.prediction(from: input)61guard let scores = output.featureValue(for: "Identity")?.multiArrayValue else {62    throw NSError(domain: "YAMNet", code: 1, userInfo: [NSLocalizedDescriptionKey: "Missing scores output"])63}64 65var bestIndex = 066var bestScore = -Double.infinity67for index in 0..<521 {68    let score = scores[[0, NSNumber(value: index)]].doubleValue69    if score > bestScore {70        bestScore = score71        bestIndex = index72    }73}74 75print("Top class index: \(bestIndex), score: \(bestScore)")76```77 78## Feature Extraction79 80To use this model with raw audio, compute YAMNet-compatible features first:81 82- sample rate: 16 kHz mono83- STFT window: 25 ms84- STFT hop: 10 ms85- mel bands: 6486- mel range: 125 Hz to 7500 Hz87- log offset: `0.001`88- patch size: 96 frames x 64 mel bands89 90The original TensorFlow Hub YAMNet model includes waveform preprocessing, but that path uses TensorFlow ops that are not currently supported by Core ML conversion. This package therefore contains only the classifier network after feature extraction.91 92## Conversion93 94The package was generated with:95 96```bash97uv run main.py --model-path ./yamnet_model --output ./YAMNet.mlpackage98```99 100To download the TensorFlow Hub source model:101 102```bash103uv run main.py --download-only104```105