Modelpublic

ghy-cmd/micro-expression-weights

sourceHugging Facecc-by-nc-4.0updated 2mo agoView on Hugging Face

0likes

Model Card

Micro-Expression Spotting and Recognition

[中文文档](README_zh.md) | English

A PyTorch-based end-to-end framework for micro-expression recognition (short clips) and long-video micro-expression spotting, with a built-in web visualization platform.

Supports datasets: CASME2 / SAMM / SMIC-HS / CAS(ME)³

Quick Start

1. Set up environment

bash

conda create -n me-env python=3.10
conda activate me-env

# GPU (CUDA 12.1)
conda install pytorch torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia

# CPU only
# conda install pytorch torchvision torchaudio cpuonly -c pytorch

pip install -r requirements.txt

2. Download model weights

python

from huggingface_hub import snapshot_download

snapshot_download(
    repo_id="ghy-cmd/micro-expression-weights",
    local_dir=".",  # restores checkpoints/ and weights/ under project root
)

3. Start the web app

bash

python -m webapp.app
# open http://localhost:5001

Features

Feature	Description
Micro-Expression Recognition	Upload a short video clip, auto-preprocess and classify emotion via PGCAN model
Long-Video Spotting	Upload any long video, auto face-crop + sliding-window inference, locates all micro-expression intervals

Model Description

Task 1 · Micro-Expression Recognition

Classifies short clips into 3 categories: Positive / Negative / Surprise

—HRNet feature extractor on onset & apex frames → (2, 256, 64, 64)
—TV-L1 optical flow (u, v, strain) → (3, 256, 256)
—Transformer encoder with attention fusion (16 heads, 4 stages, full guidance)
—Trained with LOSO cross-validation on Composite dataset (CASME2 + SMIC-HS + SAMM)

Task 2 · Long-Video Micro-Expression Spotting

Detects all micro-expression intervals in arbitrary-length videos, classifies each into 4 categories: Positive / Negative / Surprise / Others

—Sliding-window inference (window = 0.4s, stride = 0.2s @ 30 fps)
—Multi-scale temporal features (6 scales)
—Dual-task head: detection score + emotion classification
—Trained on CAS(ME)³-LV dataset
—Evaluation metric: overlap-based F1 (any frame overlap = TP)

Preprocessing Pipeline

Both tasks share the same pipeline, consistent with dataset preprocessing:

1.Face crop (dlib 68-point landmarks) → 256×256
2.Apex frame detection (optical strain + UPC) — recognition only
3.HRNet feature extraction → (2, 256, 64, 64)
4.Optical flow (TV-L1) → (3, 256, 256)

Training

bash

# Recognition — LOSO training
python train_loso.py --config configs/train_loso.yaml --dataset composite

# Detection — LOSO training
python detection/train_detection.py --config configs/detection_config.yaml

See docs/TRAINING_GUIDE.md for full training guide.

Project Structure

├── webapp/                  # Web visualization platform (Flask)
│   ├── app.py               # Entry point → http://localhost:5000
│   ├── api/                 # REST API (recognition + detection)
│   └── core/                # Inference wrappers
├── models/                  # Model architectures
├── detection/               # Spotting training & evaluation
├── trainers/                # Recognition training utilities
├── configs/                 # YAML config files
├── datasets/                # Dataset loaders
├── utils/                   # Shared utilities
├── docs/                    # Documentation & visualization scripts
├── train_loso.py            # Recognition LOSO training script
└── requirements.txt

License

CC BY-NC 4.0 — free for academic & personal use, commercial use prohibited.