ghy-cmd/micro-expression-weights

Micro-Expression Spotting and Recognition

中文文档 | English

A PyTorch-based end-to-end framework for micro-expression recognition (short clips) and long-video micro-expression spotting, with a built-in web visualization platform.

Supports datasets: CASME2 / SAMM / SMIC-HS / CAS(ME)³

---

Quick Start

1. Set up environment

conda create -n me-env python=3.10
conda activate me-env

# GPU (CUDA 12.1)
conda install pytorch torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia

# CPU only
# conda install pytorch torchvision torchaudio cpuonly -c pytorch

pip install -r requirements.txt

2. Download model weights

from huggingface_hub import snapshot_download

snapshot_download(
    repo_id="ghy-cmd/micro-expression-weights",
    local_dir=".",  # restores checkpoints/ and weights/ under project root
)

3. Start the web app

python -m webapp.app
# open http://localhost:5001

---

Features

Feature	Description
Micro-Expression Recognition	Upload a short video clip, auto-preprocess and classify emotion via PGCAN model
Long-Video Spotting	Upload any long video, auto face-crop + sliding-window inference, locates all micro-expression intervals

---

Model Description

Task 1 · Micro-Expression Recognition

Classifies short clips into 3 categories: Positive / Negative / Surprise

• HRNet feature extractor on onset & apex frames → (2, 256, 64, 64)
• TV-L1 optical flow (u, v, strain) → (3, 256, 256)
• Transformer encoder with attention fusion (16 heads, 4 stages, full guidance)
• Trained with LOSO cross-validation on Composite dataset (CASME2 + SMIC-HS + SAMM)

Task 2 · Long-Video Micro-Expression Spotting

Detects all micro-expression intervals in arbitrary-length videos, classifies each into 4 categories: Positive / Negative / Surprise / Others

• Sliding-window inference (window = 0.4s, stride = 0.2s @ 30 fps)
• Multi-scale temporal features (6 scales)
• Dual-task head: detection score + emotion classification
• Trained on CAS(ME)³-LV dataset
• Evaluation metric: overlap-based F1 (any frame overlap = TP)

---

Preprocessing Pipeline

Both tasks share the same pipeline, consistent with dataset preprocessing:

1. Face crop (dlib 68-point landmarks) → 256×256
2. Apex frame detection (optical strain + UPC) — recognition only
3. HRNet feature extraction → (2, 256, 64, 64)
4. Optical flow (TV-L1) → (3, 256, 256)

---

Training

# Recognition — LOSO training
python train_loso.py --config configs/train_loso.yaml --dataset composite

# Detection — LOSO training
python detection/train_detection.py --config configs/detection_config.yaml

See docs/TRAINING_GUIDE.md for full training guide.

---

Project Structure

├── webapp/                  # Web visualization platform (Flask)
│   ├── app.py               # Entry point → http://localhost:5000
│   ├── api/                 # REST API (recognition + detection)
│   └── core/                # Inference wrappers
├── models/                  # Model architectures
├── detection/               # Spotting training & evaluation
├── trainers/                # Recognition training utilities
├── configs/                 # YAML config files
├── datasets/                # Dataset loaders
├── utils/                   # Shared utilities
├── docs/                    # Documentation & visualization scripts
├── train_loso.py            # Recognition LOSO training script
└── requirements.txt

---

License

CC BY-NC 4.0 — free for academic & personal use, commercial use prohibited.