Mrsmetamorphosis/wav2vec2-large-xls-r-300m-dm32

wav2vec2-large-xls-r-300m-dm32

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m for binary speech classification (dementia vs. no dementia). The objective is to detect cognitive impairment from spontaneous speech audio.

🧠 Model Description

The model leverages the multilingual XLS-R 300M wav2vec2 architecture and fine-tunes a classification head for dementia detection.

Key characteristics:

• —Pretrained speech representations from wav2vec2
• —Fine-tuned classification head
• —Random audio chunk sampling during training
• —Mixed precision training (Native AMP)
• —Feature encoder frozen during early training for stability

The model accepts 16 kHz audio input and outputs probabilities for two classes:

• —0 → Dementia
• —1 → No Dementia

📂 Training and Evaluation Data

Dataset composition:

• —227 training samples
• —48 validation samples
• —Two balanced class labels: dementia and nodementia

Audio preprocessing:

• —Resampled to 16 kHz
• —Random 32 second segments used during training
• —Standardized waveform normalization

Due to the relatively small dataset size, some variance across training runs is expected.

⚙️ Training Procedure

Fine-tuning was conducted using the Hugging Face Trainer API.

Hyperparameters

• —Learning rate: 1e-4
• —Train batch size: 8
• —Eval batch size: 8
• —Gradient accumulation steps: 2
• —Effective batch size: 16
• —Number of epochs: 22
• —Optimizer: AdamW (fused implementation)
• —Betas: (0.9, 0.999)
• —Epsilon: 1e-8
• —LR scheduler: Linear
• —Mixed precision: Native AMP
• —Seed: 42

🖥️ Hardware

Training was performed on:

• —NVIDIA A100 GPU
• —CUDA-enabled PyTorch 2.10

📊 Evaluation Results (Validation Set)

Final validation performance after 22 epochs:

• —Validation Loss: 0.5444
• —Accuracy: 0.7778
• —Precision: 0.6818
• —Recall: 0.9375
• —F1 Score: 0.7895

Confusion Matrix

[[13, 7], [ 1,15]]

Where:

• —True Negatives: 13
• —False Positives: 7
• —False Negatives: 1
• —True Positives: 15

📈 Training Progress Snapshot

🔎 Interpretation

The model demonstrates high recall (0.94), meaning it correctly identifies most dementia cases. There are moderate false positives, where some non-dementia samples are classified as dementia. Given the screening context, prioritizing recall is desirable to reduce missed positive cases.

🚨 Intended Use

This model is intended for:

• —Academic research
• —Proof-of-concept dementia screening experiments
• —Speech-based cognitive impairment studies

This model is not intended for clinical diagnosis and should not be used for medical decision-making.

⚠️ Limitations

• —Small dataset size may limit generalization.
• —Performance may vary across accents, languages, and recording environments.
• —Further validation on larger and more diverse datasets is required.

📚 Framework Versions

• —Transformers 5.0.0
• —PyTorch 2.10.0+cu128
• —Datasets 4.0.0
• —Tokenizers 0.22.2