unity/inference-engine-whisper-tiny

1---2license: apache-2.03library_name: unity-sentis4pipeline_tag: automatic-speech-recognition5tags:6  - unity-inference-engine7---8 9# Whisper-Tiny model in Unity 6 with Inference Engine10 11This is the [Whisper Tiny](https://huggingface.co/openai/whisper-tiny) model running in Unity 6 with Inference Engine. It is a speech-to-text model that transcribes 16kHz wav audio to text.12 13## How to Use14 15* Create a new scene in Unity 6;16* Install `com.unity.ai.inference` from the package manager;17* Add the `RunWhisper.cs` script to the Main Camera;18* Drag the `decoder_model.onnx` asset from the `models` folder into the `Audio Decoder 1` field;19* Drag the `decoder_with_past_model.onnx` asset from the `models` folder into the `Audio Decoder 2` field;20* Drag the `encoder_model.onnx` asset from the `models` folder into the `Audio Encoder` field;21* Drag the `logmel_spectrogram.onnx` asset from the `models` folder into the `Log Mel Spectro` field;22* Drag the `vocab.json` asset from the `data` folder into the `Vocab Asset` field;23* Drag an audio asset, e.g. `data/answering-machine16kHz.wav` to the `Audio Clip` field. Ensure the `Normalize` flag is set on asset import for best results.24 25## Preview26Enter play mode. If working correctly the transcribed audio will be logged to the console.27 28## Inference Engine29Inference Engine is a neural network inference library for Unity. Find out more [here](https://docs.unity3d.com/Packages/com.unity.ai.inference@latest).