CoolFace
Apppublic

AbhayJaiswal/audio-forensics-detector

sourceHugging Facemitupdated 5mo agoView on Hugging Face
0likes
App README

🔊 Audio Forensics Module — Real-Time Voice Clone Detection

A real-time audio forensics module that detects AI-cloned voices during live calls within the first 10 seconds — before fraud can occur.

🎯 What It Does

  • —Streams incoming audio through a forensic analysis pipeline
  • —Converts chunks into Mel spectrograms for visual analysis
  • —Runs them through a fine-tuned CNN-Transformer hybrid model (WavLM)
  • —Detects synthetic artifacts: unnatural pitch consistency, GAN-generated frequency smoothness, missing micro-tremors
  • —Outputs a live Risk Score (0-100) with color-coded verdict
  • —Shows a highlighted spectrogram heatmap showing exactly which frequencies triggered the alert
  • —Delivers an instant HIGH RISK flag within 10 seconds

🧠 Architecture

Raw Audio → [CNN Feature Extractor (7 layers)] → [Transformer Encoder (12 layers)] → [Classification Head] → Risk Score
                     ↓                                        ↓
            Low-level acoustic features            Temporal patterns & synthetic artifacts

Model: WavLM-Base-Plus (94M parameters)

  • —Pre-trained on 94,000 hours of speech data
  • —Fine-tuned on multi-corpus deepfake dataset (ASVspoof + FakeOrReal + InTheWild)
  • —99.62% accuracy on held-out test set

🔬 Detection Targets

Artifact TypeDescriptionWhy AI Misses It
Pitch ConsistencyHuman pitch naturally varies 0.5-3Hz; AI voices are unnaturally stableNeural vocoders optimize for smooth output
Frequency SmoothnessGAN-generated audio has suspiciously smooth frequency transitionsDiscriminator loss encourages smoothness
Missing Micro-tremorsReal human vocal cords have involuntary micro-tremorsTTS systems don't model muscular micro-variations
Phase CoherenceAI-generated audio often has unnatural phase relationshipsMost TTS focuses on magnitude spectrogram only
Formant TransitionsAI voices show abrupt formant changes at phoneme boundariesConcatenative artifacts in neural TTS

📊 How to Use

  1. 1.Upload a WAV/MP3/FLAC audio file, or record from your microphone
  2. 2.Click Analyze Audio
  3. 3.View the Risk Score (0-100), spectrogram heatmap, and verdict

For real-time analysis, use the Streaming tab to analyze live microphone input.

🏗️ Technical Details

  • —Sliding Window: 4-second chunks with 2-second stride → first result at 4 seconds, updated every 2 seconds
  • —Calibration: Temperature-scaled softmax (T=1.8) prevents overconfident predictions
  • —Input: 16kHz mono audio, any duration (minimum 0.5 seconds)
  • —Inference: ~50ms per chunk on GPU

📄 References

🏆 Built for Hackathon

Problem Statement PS2: Audio Forensics for Voice Security Real-world impact: Protects banks, insurance firms, and telecom providers from voice cloning fraud.