Apppublic

vivekjutture/affective-computing

sourceHugging Facecc0-1.0updated 2mo agoView on Hugging Face

0likes

App README

🎭 Affective Computing — Live Multimodal Emotion Recognition

A live, in-browser demo of a multimodal emotion-recognition system. It reads emotion from three signals and fuses them:

Modality	Signal	Model
Facial	A webcam photo	DeepFace emotion analysis
Speech	The tone of a short voice clip	scikit-learn pipeline trained on RAVDESS (neutral / happy / sad / angry / fearful)
Text	The words spoken (or typed)	`j-hartmann/emotion-english-distilroberta-base`

Speech is transcribed with Google's free speech-to-text; a typed-text box is offered as a fallback. The three predictions are combined with weighted late fusion into one overall emotion.

Use it

Give any one — or all — of the inputs, then press Analyze emotion:

1.Face — snap a webcam photo (front-facing, well-lit works best).
2.Voice — record a couple of clear seconds of speech.
3.Text — optionally type/paste a sentence (otherwise your speech is transcribed).

Works on phones and laptops. The camera and microphone run in your browser; nothing is stored.

Self-healing speech model

The trained speech model (speech_emotion_model.pkl) ships with the Space, so normally there's nothing to do. If that file is ever missing (e.g. you delete it), the app retrains it automatically at startup from the RAVDESS dataset provided a data/ folder (with the Actor_* sub-folders) is present on the Space. If neither the .pkl nor data/ is available, the app still runs on the face and text modalities and simply skips voice-tone until you restore one of them.

This is the hosted demo of the Affective Computing project. The full training/CLI code, datasets and per-modality experiments live in the main repository.