caa-speech-detection-asvspoof2019/wav2vec2

Improved version available: caa-speech-detection-asvspoof2019/wav2vec2-v2-unfrozen — wav2vec2_v2 (top-4 transformer blocks unfrozen) achieves 1.55% eval EER (vs 7.53% here) and 0.3966 tandem min t-DCF.

wav2vec2 — ASVspoof 2019 LA Binary Anti-Spoofing Detector

Binary bonafide-vs-spoof classifier built on facebook/wav2vec2-base with a frozen encoder and a lightweight classification head, trained on the ASVspoof 2019 Logical Access scenario.

Model Details

The encoder masking used during wav2vec 2.0 self-supervised pretraining (mask_time_prob, mask_feature_prob) is disabled at load time, following Tak et al. (2022) who showed masking hurts countermeasure performance.

Training Data

ASVspoof 2019 Logical Access

• —No data augmentation was applied (baseline experiment).
• —Class-weighted cross-entropy ([8.837, 1.0] for bonafide/spoof) compensates for the ~8.84:1 spoof-heavy imbalance in the training split.
• —Hyperparameters: seed=17245921, lr=1e-5, batch_size=8, 20 epochs, no early stopping.

Results

Baseline to beat: EER 8.09% (LFCC+GMM system from the ASVspoof 2019 paper).

Note on t-DCF: The eval t-DCF of ≈ 1 for this baseline indicates a calibration failure — the model's score distribution is not well aligned with the cost function's operating point despite achieving reasonable EER. t-DCF values use the normalized [0, 1] convention with tandem ASV scores.

For comparison, lcnn_v7_cqt achieves 3.26% eval EER with a t-DCF of 0.4930, making it the recommended model for applications that weight calibration alongside raw detection rate. The improved wav2vec2_v2 achieves the best eval EER (1.55%) and the best t-DCF (0.3966) across all models.

Usage

Download

huggingface-cli download caa-speech-detection-asvspoof2019/wav2vec2

Load

Clone the training repository and install dependencies, then:

import torch
from src.models.wav2vec2.model import Wav2Vec2Model

config = {
    "pretrained_model": "facebook/wav2vec2-base",
    "hidden_dim": 256,
    "dropout": 0.1,
    "freeze_encoder": True,
}

model = Wav2Vec2Model(config)
state = torch.load("best.pt", map_location="cpu")
model.load_state_dict(state)
model.eval()

Inference

from transformers import Wav2Vec2FeatureExtractor
from src.data.audio import load_audio, pad_or_trim
import torch

extractor = Wav2Vec2FeatureExtractor.from_pretrained("facebook/wav2vec2-base")
waveform = load_audio("audio.flac", target_sr=16000)
waveform = pad_or_trim(waveform, 64000)

inputs = extractor(waveform, sampling_rate=16000, return_tensors="pt")
frames = inputs.input_values.float()

with torch.no_grad():
    logits = model({"frames": frames})["logits"]
    pred = logits.argmax(dim=-1).item()
    # 0 = bonafide, 1 = spoof

Intended Use & Limitations

• —Intended use: detecting AI-generated / synthesised speech in the ASVspoof 2019 LA scenario as a countermeasure (CM) module.
• —Domain: Trained and evaluated on the LA (logical access) partition only. Not validated on physical access (PA) or in-the-wild recordings.
• —Unseen attacks: The eval set (A07–A19) contains attack types not seen during training. The 7.53% eval EER demonstrates reasonable generalisation, though the t-DCF ≈ 1 signals that score calibration is poor and the model should not be used directly as a calibrated likelihood ratio.
• —Deployment caveat: The 362 MB checkpoint size makes this unsuitable for edge or real-time deployment. For resource-constrained scenarios, lcnn_v4_cqt (3.5 MB, 3.03% eval EER) is recommended.

Citation

ASVspoof 2019 dataset and challenge:

@inproceedings{wang2020asvspoof,
  title     = {{ASVspoof} 2019: A Large-Scale Public Database of Synthesized,
               Converted and Replayed Speech},
  author    = {Wang, Xin and Yamagishi, Junichi and Todisco, Massimiliano and
               Delgado, H{\'e}ctor and Nautsch, Andreas and Evans, Nicholas and
               Shim, Md Sahidullah Hector and Kinnunen, Tomi and Lee, Kong Aik and
               Patino, Jose and others},
  booktitle = {Computer Speech \& Language},
  volume    = {64},
  year      = {2021},
}

wav2vec 2.0:

@inproceedings{baevski2020wav2vec,
  title     = {wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations},
  author    = {Baevski, Alexei and Zhou, Henry and Mohamed, Abdelrahman and Auli, Michael},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
  year      = {2020},
}