MBJamshidi/speech-emotion-support

Speech Emotion Support

Early intervention for psychological distress through dual-modality screening.

Speech Emotion Support is a Streamlit application that combines the Australian Kessler Psychological Distress Scale (K10) with a voice recording workflow based on the Rainbow Passage. The project is designed as a transparent, non-diagnostic screening aid that can support early identification of psychological distress and prepare structured vocal biomarkers for future machine learning integration.

The app is deployed as a Hugging Face Space and can also be run locally from this GitHub repository.

Clinical Framework

Kessler Psychological Distress Scale (K10)

The K10 is a 10-item psychological distress screening questionnaire that asks users how often they experienced distress-related symptoms over the previous four weeks. This implementation presents one question at a time to reduce survey fatigue and uses five response levels:

• —1 - Not at all
• —2 - Occasionally
• —3 - Sometimes
• —4 - Most days
• —5 - All the time

Each response is mapped to a numeric score from 1 to 5. The total score is calculated as the sum of all 10 responses, producing a possible range from 10 to 50.

Australian Risk Scoring Levels

The app uses Australian K10 interpretation bands:

Results are displayed with color-coded visual feedback:

• —Green for low distress
• —Yellow for mild distress
• —Orange for moderate distress
• —Red for severe or high distress

Rainbow Passage Voice Assessment

The voice module asks users to read the Rainbow Passage:

The rainbow is a division of white light into many beautiful colors, which take the shape of a long, round arch.

The Rainbow Passage is commonly used in speech and voice assessment because it contains broad phonetic coverage across English sounds. In this app, it provides a consistent spoken prompt for extracting preliminary vocal biomarkers such as pitch, energy, and cepstral features. This consistency helps future models compare recordings against a stable text prompt rather than unconstrained speech.

Backend Architecture

Streamlit Frontend

The user interface is built with Streamlit and focuses on accessibility for users who may be experiencing distress:

• —soft neutral background to reduce eye strain
• —large, bold question text
• —card-style response controls using st.segmented_control
• —clear progress feedback across the 10 K10 questions
• —large centered score visualization
• —visible Australian crisis support contacts

Text-Based K10 Scoring Algorithm

The K10 scoring logic is deterministic:

1. 1.Present 10 K10 questions.
2. 2.Map each selected response to a score from 1 to 5.
3. 3.Sum all response values.
4. 4.Classify the total score into the Australian K10 bands.
5. 5.Display the final score, interpretation, and support guidance.

No large language model is used to infer the K10 score. The score is calculated directly from the user's selected responses.

Audio Pipeline

The current voice pipeline prepares acoustic features for future model integration:

Raw signal capture -> Feature extraction -> Potential model integration

Current implementation:

1. 1.Raw signal capture: st.audio_input records audio in the browser.
2. 2.Audio loading: soundfile reads the recorded bytes into a waveform.
3. 3.Preprocessing: stereo audio is converted to mono and normalized with librosa.
4. 4.Feature extraction:
5. 5.MFCCs for spectral and vocal tract characteristics
6. 6.Pitch/F0 using librosa.pyin
7. 7.RMS energy for intensity
8. 8.duration and sample rate metadata
9. 9.Potential model integration:
10. 10.Wav2Vec2 or similar speech representation models
11. 11.speech emotion recognition classifiers
12. 12.longitudinal vocal biomarker tracking

The current app does not diagnose emotion or mental illness from the voice recording. The extracted features are displayed for transparency and future model development.

Installation & Setup

1. Clone the repository

git clone https://github.com/MBJamshidi/speech-emotion-support.git
cd speech-emotion-support

2. Create and activate a virtual environment

On macOS or Linux:

python -m venv .venv
source .venv/bin/activate

On Windows PowerShell:

python -m venv .venv
.\.venv\Scripts\Activate.ps1

3. Install dependencies

pip install -r requirements.txt

4. Launch the app

streamlit run app.py

Hugging Face Space

This repository is designed to mirror cleanly to a Hugging Face Streamlit Space.

Required Space files:

• —app.py
• —requirements.txt
• —README.md

The Hugging Face Space metadata is included at the top of this README:

sdk: streamlit
app_file: app.py

Mirroring GitHub to Hugging Face Spaces

You can keep the GitHub repository and Hugging Face Space synchronized with GitHub Actions. Store a Hugging Face token in GitHub repository secrets, for example HF_TOKEN, then add a workflow such as:

name: Sync to Hugging Face Space

on:
  push:
    branches: [main]

jobs:
  sync:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Push to Hugging Face
        env:
          HF_TOKEN: ${{ secrets.HF_TOKEN }}
        run: |
          git remote add space https://MBJamshidi:${HF_TOKEN}@huggingface.co/spaces/MBJamshidi/speech-emotion-support
          git push --force space main

Use a Hugging Face token with write access to the Space. If you prefer not to force push, keep both repositories on the same commit history and use a standard git push space main.

Medical Disclaimer

This application is a screening tool, not a diagnostic device. It does not diagnose mental illness, determine clinical risk, or replace assessment by a qualified health professional. K10 results and vocal biomarkers should be interpreted only as supportive screening information.

If you or someone else may be in immediate danger in Australia, call 000.

Australian crisis and support contacts:

• —Lifeline: 13 11 14
• —Beyond Blue: 1300 22 4636

For non-urgent concerns, consider speaking with a GP, psychologist, counsellor, or another qualified health professional.

License

This project is released under the MIT License.