File size: 8,778 Bytes
f8f4554
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
---
title: Emotion Recognition API
emoji: ๐Ÿง 
colorFrom: blue
colorTo: purple
sdk: docker
sdk_version: latest
app_file: app.py
pinned: false
---

# ๐Ÿ˜Š Emotion Recognition โ€“ Realโ€‘Time Facial Emotion Detection

![Python](https://img.shields.io/badge/Python-3.8+-blue.svg)
![PyTorch](https://img.shields.io/badge/PyTorch-Latest-orange.svg)
![FastAPI](https://img.shields.io/badge/FastAPI-Latest-green.svg)
![License](https://img.shields.io/badge/License-MIT-yellow.svg)

<div align="center">
  <img src="https://via.placeholder.com/800x400/4A90E2/FFFFFF?text=Emotion+Recognition+Demo" alt="Emotion Recognition Demo" width="800"/>
  
  *Real-time facial emotion detection using deep learning*
</div>

---

This project is a **realโ€‘time facial emotion recognition system** built around a **fineโ€‘tuned MobileNetV2 model**.  
We start from an **ImageNetโ€‘pretrained MobileNetV2**, fineโ€‘tune it on a custom facial emotion dataset, save the weights as `emotion_recognition_model.pth`, and then use that model behind a **FastAPI backend** and **browser-based webcam UI** to detect emotions such as ๐Ÿ˜  *Angry*, ๐Ÿ˜จ *Fear*, ๐Ÿ˜Š *Happy*, ๐Ÿ˜ข *Sad,* and ๐Ÿ˜ฒ *Surprise* from a live camera feed.

---

## โœจ Features

- ๐Ÿค– **Fineโ€‘tuned deep learning model** โ€“ MobileNetV2 is fineโ€‘tuned on a labeled facial emotion dataset to specialize it for emotion recognition.
- ๐Ÿ“น **Realโ€‘time detection** from your webcam (browser or local OpenCV).
- ๐Ÿš€ **FastAPI REST API** endpoint for image-based emotion prediction.
- ๐ŸŽจ **Simple HTML/JS frontend** that streams frames to the backend and displays the predicted emotion.
- ๐Ÿ’ป **Standalone webcam script** (`webcam.py`) if you prefer running everything locally without the browser.

---

## ๐Ÿ“ Project Structure

```
๐Ÿ“ฆ Emotion Recognition Project
โ”œโ”€โ”€ ๐Ÿ“‚ frontend/
โ”‚   โ”œโ”€โ”€ ๐Ÿ“„ index.html          # Minimal UI with a video element and live emotion text
โ”‚   โ”œโ”€โ”€ ๐Ÿ“„ script.js            # Grabs webcam frames, sends them to the backend
โ”‚   โ””โ”€โ”€ ๐Ÿ“„ style.css            # Basic styling for the page and video element
โ”œโ”€โ”€ ๐Ÿ“‚ model/
โ”‚   โ”œโ”€โ”€ ๐Ÿ api.py               # FastAPI app exposing POST /api/predict
โ”‚   โ”œโ”€โ”€ ๐Ÿ inference.py         # Loads model and defines predict(image) function
โ”‚   โ”œโ”€โ”€ ๐Ÿ model.py             # MobileNetV2 architecture definition
โ”‚   โ”œโ”€โ”€ ๐Ÿ webcam.py            # OpenCV-based realโ€‘time emotion recognition
โ”‚   โ”œโ”€โ”€ ๐Ÿ““ working.ipynb        # Jupyter notebook for fineโ€‘tuning MobileNetV2
โ”‚   โ”œโ”€โ”€ ๐Ÿ’พ emotion_recognition_model.pth  # Fineโ€‘tuned PyTorch checkpoint
โ”‚   โ””โ”€โ”€ ๐Ÿ“‚ Data/                # Dataset folders (Angry, Fear, Happy, Sad, Surprise)
โ”œโ”€โ”€ ๐Ÿณ DockerFile               # Docker image definition for deployment
โ””โ”€โ”€ ๐Ÿ“‹ requirements.txt         # Python dependencies
```

---

## ๐Ÿ› ๏ธ Requirements

### System Requirements
- ๐Ÿ **Python** 3.8+ (recommended)
- ๐Ÿ“ฆ **pip** for installing dependencies
- ๐Ÿ“ท A **webcam**
- โšก (Optional) **GPU with CUDA** for faster inference, otherwise CPU will be used.

### Python Packages
Python packages (also listed in `requirements.txt`):

- `torch`, `torchvision` ๐Ÿ”ฅ
- `numpy` ๐Ÿ”ข
- `opencv-python` ๐Ÿ“น
- `matplotlib` ๐Ÿ“Š
- `pillow` ๐Ÿ–ผ๏ธ
- `tqdm` ๐Ÿ“ˆ
- `requests` ๐ŸŒ
- `fastapi`, `uvicorn` โšก (install explicitly if missing)

### Installation

You can install everything with:

```bash
pip install -r requirements.txt fastapi uvicorn
```

---

## ๐Ÿš€ How to Run

### ๐ŸŒ Web API + Frontend

1. **Navigate to the model folder**:

   ```bash
   cd model
   ```

2. **Start the FastAPI server** (with Uvicorn):

   ```bash
   uvicorn api:app --host 0.0.0.0 --port 8000 --reload
   ```

3. **Open the frontend**:
   - Option A: The API mounts the `frontend` folder as static files.  
     Open your browser and go to:

     ```text
     http://localhost:8000/
     ```

   - If needed, ensure the working directory is such that `../frontend` (from `model/api.py`) points to the `frontend` folder in this repo.

4. **Allow camera access** ๐Ÿ“ท in the browser when prompted.

5. You should see the **live video** and the **predicted emotion** updating underneath! ๐ŸŽ‰

---

### ๐Ÿ’ป Local Webcam Script (No Browser)

If you prefer a pure Python / OpenCV pipeline:

1. Make sure dependencies are installed:

   ```bash
   pip install -r requirements.txt
   ```

2. From the `model` directory, run:

   ```bash
   python webcam.py
   ```

3. A window called **"Emotion Recognition"** will appear:
   - ๐Ÿ‘ค Detected faces will be highlighted with a bounding box.
   - ๐Ÿท๏ธ The predicted emotion label will be shown next to each face.
   - Press **`q`** to quit.

---

### ๐Ÿณ Run with Docker (Optional)

If you prefer not to manage Python and system dependencies manually, you can use the provided `DockerFile`:

1. **Build the image** from the project root:

   ```bash
   docker build -t emotion-recognition -f DockerFile .
   ```

2. **Run the container** and expose the API on port `8000`:

   ```bash
   docker run --rm -p 8000:8000 emotion-recognition
   ```

3. Open your browser at:

   ```text
   http://localhost:8000/
   ```

   and allow camera access. The behavior is the same as in the "Web API + Frontend" section above.

---

## ๐Ÿง  Model & Inference Details

- The model is a **MobileNetV2** classifier whose final layer is adapted to the number of emotion classes (e.g. ๐Ÿ˜  Angry, ๐Ÿ˜จ Fear, ๐Ÿ˜Š Happy, ๐Ÿ˜ข Sad, ๐Ÿ˜ฒ Surprise).
- The **fineโ€‘tuned weights and class names** are stored in `emotion_recognition_model.pth`, which is produced by the training notebook (`working.ipynb`).
- Images are preprocessed with:
  - ๐Ÿ“ Resize to \(224 \times 224\)
  - ๐Ÿ”„ Conversion to tensor
  - ๐Ÿ“Š Normalization with ImageNet mean and std
- Inference is done by `inference.py` via:
  - `predict(pil_image)` โ†’ returns a string label, e.g. `"Happy"`.

### Testing the Model

You can test the model directly with a static image (from the `model` directory):

```bash
python inference.py
```

This will load `Image.jpg` and print the predicted emotion.

---

## ๐Ÿ“ก API Reference

### Endpoints

- **`GET /`**
  - Returns a simple JSON status: `{"status": "API running"}`.

- **`POST /api/predict`**
  - **Body**: `multipart/form-data` with a single field:
    - `file`: image file (e.g. JPEG/PNG).
  - **Response**:
    ```json
    { "emotion": "Happy" }
    ```

The frontend uses this endpoint to send frames from your webcam (as blobs) approximately once per second.

---

## ๐Ÿ“š Dataset & Training (Fineโ€‘Tuning Overview)

- The (optional) `Data` directory is expected to contain labeled images organized by emotion:
  - ๐Ÿ˜  `Angry/`, ๐Ÿ˜จ `Fear/`, ๐Ÿ˜Š `Happy/`, ๐Ÿ˜ข `Sad/`, ๐Ÿ˜ฒ `Suprise/`.
- The **fineโ€‘tuning process is captured in `working.ipynb`**:
  - Load an **ImageNetโ€‘pretrained MobileNetV2** from `torchvision`.
  - Replace the final classification layer so that its output dimension matches the number of emotion classes.
  - Create PyTorch `Dataset`/`DataLoader` objects from the `Data` folders with standard augmentations and preprocessing.
  - Train the network (typically using crossโ€‘entropy loss and an optimizer like Adam/SGD) while monitoring validation accuracy.
  - Save the best model checkpoint and class mapping to `emotion_recognition_model.pth`.
- You can adapt the model for new datasets by:
  - Updating the `Data` folders and class list.
  - Reโ€‘running or modifying `working.ipynb` and, if needed, adjusting `model.py` for a different backbone or number of classes.

**Dataset Source**: [Kaggle - Human Face Emotions](https://www.kaggle.com/datasets/samithsachidanandan/human-face-emotions)

---

## ๐Ÿ”ง Troubleshooting

- **๐Ÿ“ท Camera access denied (browser)**  
  - Check browser permissions and ensure you're using `http://localhost` (not `file://`).

- **๐Ÿ”Œ "Backend not reachable" in the frontend**  
  - Confirm the FastAPI server is running on the same host/port that `script.js` expects (`/api/predict` โ†’ default `http://localhost:8000/api/predict`).
  - Check for CORS issues or port conflicts.

- **๐Ÿ“ฆ Model file not found**  
  - Ensure `emotion_recognition_model.pth` is present in the `model` directory when running any Python scripts there.

---

## ๐Ÿ“„ License & Credits

- The project uses **PyTorch**, **FastAPI**, **OpenCV**, and **PIL** under their respective licenses.
- Dataset images in `Data/` should respect their original source licenses (not provided here).
- Feel free to modify or extend this project for research, learning, or personal use.

---

<div align="center">
  <p>Made with โค๏ธ using PyTorch, FastAPI, and OpenCV</p>
  <p>โญ Star this repo if you find it helpful!</p>
</div>