Picovoice/speech-to-intent-benchmark
Speech-to-Intent Benchmark Made in Vancouver, Canada by Picovoice This framework benchmarks the accuracy of Picovoice's Speech-to-Intent engine, Rhino. It compares the accuracy of Rhino with: Amazon Lex Google Dialogflow IBM Watson Microsoft LUIS Results Command acceptance rate is the probability of an engine correctly understanding the spoken command. Below is the summary: The figure below depicts engines performance at each SNR: Data The… See the full description on the dataset page: https://huggingface.co/datasets/Picovoice/speech-to-intent-benchmark.
Speech-to-Intent Benchmark
Made in Vancouver, Canada by Picovoice
 
This framework benchmarks the accuracy of Picovoice's Speech-to-Intent engine, Rhino. It compares the accuracy of Rhino with:
Table of Contents
Results
Command acceptance rate is the probability of an engine correctly understanding the spoken command. Below is the summary:
The figure below depicts engines performance at each SNR:
Data
The speech data are crowd-sourced from more than 50 unique speakers. Each speaker contributed about ten different utterances. Collectively there are 619 commands used in this benchmark. We test the engines in noisy conditions to simulate real-world situations. Noise is from Freesound.
How to Reproduce?
Clone the Picovoice/speech-to-intent-benchmark repository on GitHub:
git clone https://github.com/Picovoice/speech-to-intent-benchmark.gitGet the usage message:
python3 src/bench.py --helpThen run the script for each engine.