SpeechAntiSpoofingBenchmarks/SONAR
SONAR Benchmark-ready packaging of the SONAR synthetic-audio detection evaluation set (arXiv 2410.04324), for speech anti-spoofing and synthetic / deepfake voice detection. Overview SONAR is a binary-classification benchmark: bonafide (genuine human speech) vs. spoof (AI-synthesized speech). The spoof side aggregates clips from eight modern speech-synthesis systems, deliberately spanning architectures and providers so that a detector cannot win by memorising one… See the full description on the dataset page: https://huggingface.co/datasets/SpeechAntiSpoofingBenchmarks/SONAR.
SONAR
Benchmark-ready packaging of the SONAR synthetic-audio detection evaluation set (arXiv 2410.04324), for speech anti-spoofing and synthetic / deepfake voice detection.
Overview
SONAR is a binary-classification benchmark: bonafide (genuine human speech) vs. spoof (AI-synthesized speech). The spoof side aggregates clips from eight modern speech-synthesis systems, deliberately spanning architectures and providers so that a detector cannot win by memorising one generator's artifacts. The bonafide side is real clips sampled from the LibriTTS clean-test set. Original code and data: https://github.com/Jessegator/SONAR.
Seed-TTS excluded (2026-06-07). The upstreamseedtts_testset/directory holds the SeedTTS reference prompts — real Common Voice audio (common_voice_en_<prompt>-<target>.wav), not synthesized speech (the actual SeedTTS-generated outputs are not redistributed on disk). Labeling those 600 real clips as spoof was incorrect, so they are dropped from this package. This matches the SONAR authors' own released eval set (3948 clips).
License & redistribution
This packaging is released under the Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) license — see LICENSE.txt. SONAR is an aggregate of audio from multiple synthesis systems and from LibriTTS, each carrying its own upstream terms; the original authors note that the constituent sources "may be subject to different distribution licenses and usage restrictions." This repackaging is provided for non-commercial research use only; the underlying sources retain their own terms, and downstream users should consult the respective upstream providers' usage policies. Audio is re-encoded to 16 kHz mono FLAC for a uniform schema; labels are unmodified.
Schema
notes example:
{"utterance_id": "SONAR_OpenAI_alloy_69", "system": "OpenAI", "language": "en"}Quick Start
from datasets import load_dataset
ds = load_dataset("SpeechAntiSpoofingBenchmarks/SONAR", split="test")
print(ds[0])Stats
Source provenance
- Paper: https://arxiv.org/abs/2410.04324
- Code / data: https://github.com/Jessegator/SONAR
- Labels derived from the source directory layout (
real_samples/= bonafide; the eight synthesis-system directories = spoof;seedtts_testset/excluded — real reference prompts).
Evaluation
For evaluation instructions and submission format, see `submissions/README.md`.
Citation
@article{li2024sonar,
title = {Where are we in audio deepfake detection? A systematic analysis over
generative and detection models},
author = {Li, Xiang and Chen, Pin-Yu and Wei, Wenqi},
journal = {arXiv preprint arXiv:2410.04324},
year = {2024}
}Maintainer
Maintained by Kirill Borodin (SpeechAntiSpoofingBenchmarks).
- Email: ~~k.n.borodin@mtuci.ru~~ (deprecated — use kborodin.research@gmail.com)
- Telegram: @korallll_ai