Apppublic

vanisha-vc5/speech-confidence-analyzer

sourceHugging Faceupdated 3mo agoView on Hugging Face

0likes

App README

Speech Confidence Analysis — Hybrid Pipeline

Predict speaker confidence from audio using a hybrid approach: Mel Spectrogram CNN embeddings + handcrafted acoustic features.

Architecture

Audio
  │
  ├── Handcrafted Features (~42)    ├── Mel Spectrogram (128×128)
  │   MFCC × 13 (mean + std)        │
  │   Pitch, Volume, ZCR            └── CNN (4× Conv + Dense)
  │   Pause metrics × 5                  → Embedding (64-dim)
  │   Speech Rate, Stutter × 3      │
  │                                  │
  └──────────── Concatenate ─────────┘
                    │
              ~106 features
                    │
         RandomForest / XGBoost Classifier
                    │
       Confidence: Low / Medium / High

Expected Accuracy

Method	Accuracy
Handcrafted features only	~70–78%
Mel Spectrogram CNN only	~75–82%
Hybrid (this pipeline)	~80–90%

Usage

Upload a .wav file or record via microphone. Toggle between the Baseline (handcrafted features) and Hybrid (CNN + handcrafted features) models in the sidebar to see confidence level, pause/stutter/volume analysis, and the Mel spectrogram.

Project Structure

├── models/          ← pretrained baseline + hybrid models
├── src/
│   ├── data_loader.py
│   ├── feature_extractor.py
│   ├── pause_detection.py
│   ├── stutter_detection.py
│   ├── spectrogram_cnn.py
│   ├── augmentation.py
│   ├── train_model.py
│   ├── train_hybrid.py
│   └── predict.py
├── app/streamlit_app.py
└── requirements.txt

Training code and the full CREMA-D dataset are not included in this Space — see the [source repository] for training instructions.