AMERS - Adaptive Multimodal Emotion Recognition System
This repository contains PyTorch checkpoints from AMERS (Adaptive Multimodal Emotion Recognition System), a research project combining EEG and speech signals for emotion recognition.
Model summary
The project combines:
- EEG preprocessing with differential-entropy features
- A conditional GAN for synthetic EEG feature augmentation
- A PPO reinforcement-learning agent for adaptive augmentation control
- A CNN-LSTM speech encoder using MFCC features
- Late fusion of EEG and speech embeddings
The target emotion setup includes happy, sad, angry, and neutral classes. Some checkpoints are from later v3 and v5 research variants and may use different architectures or training objectives than the baseline checkpoints.
Checkpoint layout
| Directory | Contents |
|---|---|
checkpoints/eeg |
Baseline EEG encoder |
checkpoints/speech |
Baseline speech encoder and intermediate epochs |
checkpoints/gan |
GAN checkpoints and epoch snapshots |
checkpoints/fusion |
Baseline multimodal fusion checkpoint |
checkpoints/rl |
PPO agent and RL fusion checkpoint |
checkpoints/v3 |
Contrastive, DANN, transformer-fusion, and PPO v3 checkpoints |
checkpoints/v5 |
CMMA v5 fusion checkpoint |
Intended use
These files are intended for academic research, reproducibility, evaluation, and continued experimentation. They are not validated for clinical diagnosis, employment decisions, education decisions, or other high-impact use.
Usage
The checkpoints are intended to be loaded with the corresponding model classes and configuration from the AMERS source repository. A checkpoint from one variant should not be assumed to be compatible with another variant.
import torch
checkpoint = torch.load(
"checkpoints/eeg/eeg_encoder_final.pt",
map_location="cpu",
weights_only=False,
)
Inspect the project training and evaluation scripts for the exact model class, configuration, preprocessing, and state-dict loading procedure for each file.
Data and limitations
The DEAP and IEMOCAP datasets are not included in this repository. They are subject to their own access terms and license agreements. Results depend on the original preprocessing, subject splits, labels, random seeds, and model variant. The checkpoint files alone are not sufficient to reproduce a result.
Source code
Source repository: https://github.com/RAVINDRA8008/Adaptive-GAN-Augmented-EEG-Speech-Emotion-Recognition-via-Deep-Reinforcement-Learning/
Citation
If you use these checkpoints, cite the source project and identify the exact checkpoint path and model variant used in your work.
License and dataset notice
The source project describes the code and checkpoints as intended for academic use. Confirm the applicable project license and the separate DEAP and IEMOCAP dataset terms before redistribution or publication.