Spaces:
Sleeping
Sleeping
File size: 8,778 Bytes
f8f4554 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 | ---
title: Emotion Recognition API
emoji: ๐ง
colorFrom: blue
colorTo: purple
sdk: docker
sdk_version: latest
app_file: app.py
pinned: false
---
# ๐ Emotion Recognition โ RealโTime Facial Emotion Detection




<div align="center">
<img src="https://via.placeholder.com/800x400/4A90E2/FFFFFF?text=Emotion+Recognition+Demo" alt="Emotion Recognition Demo" width="800"/>
*Real-time facial emotion detection using deep learning*
</div>
---
This project is a **realโtime facial emotion recognition system** built around a **fineโtuned MobileNetV2 model**.
We start from an **ImageNetโpretrained MobileNetV2**, fineโtune it on a custom facial emotion dataset, save the weights as `emotion_recognition_model.pth`, and then use that model behind a **FastAPI backend** and **browser-based webcam UI** to detect emotions such as ๐ *Angry*, ๐จ *Fear*, ๐ *Happy*, ๐ข *Sad,* and ๐ฒ *Surprise* from a live camera feed.
---
## โจ Features
- ๐ค **Fineโtuned deep learning model** โ MobileNetV2 is fineโtuned on a labeled facial emotion dataset to specialize it for emotion recognition.
- ๐น **Realโtime detection** from your webcam (browser or local OpenCV).
- ๐ **FastAPI REST API** endpoint for image-based emotion prediction.
- ๐จ **Simple HTML/JS frontend** that streams frames to the backend and displays the predicted emotion.
- ๐ป **Standalone webcam script** (`webcam.py`) if you prefer running everything locally without the browser.
---
## ๐ Project Structure
```
๐ฆ Emotion Recognition Project
โโโ ๐ frontend/
โ โโโ ๐ index.html # Minimal UI with a video element and live emotion text
โ โโโ ๐ script.js # Grabs webcam frames, sends them to the backend
โ โโโ ๐ style.css # Basic styling for the page and video element
โโโ ๐ model/
โ โโโ ๐ api.py # FastAPI app exposing POST /api/predict
โ โโโ ๐ inference.py # Loads model and defines predict(image) function
โ โโโ ๐ model.py # MobileNetV2 architecture definition
โ โโโ ๐ webcam.py # OpenCV-based realโtime emotion recognition
โ โโโ ๐ working.ipynb # Jupyter notebook for fineโtuning MobileNetV2
โ โโโ ๐พ emotion_recognition_model.pth # Fineโtuned PyTorch checkpoint
โ โโโ ๐ Data/ # Dataset folders (Angry, Fear, Happy, Sad, Surprise)
โโโ ๐ณ DockerFile # Docker image definition for deployment
โโโ ๐ requirements.txt # Python dependencies
```
---
## ๐ ๏ธ Requirements
### System Requirements
- ๐ **Python** 3.8+ (recommended)
- ๐ฆ **pip** for installing dependencies
- ๐ท A **webcam**
- โก (Optional) **GPU with CUDA** for faster inference, otherwise CPU will be used.
### Python Packages
Python packages (also listed in `requirements.txt`):
- `torch`, `torchvision` ๐ฅ
- `numpy` ๐ข
- `opencv-python` ๐น
- `matplotlib` ๐
- `pillow` ๐ผ๏ธ
- `tqdm` ๐
- `requests` ๐
- `fastapi`, `uvicorn` โก (install explicitly if missing)
### Installation
You can install everything with:
```bash
pip install -r requirements.txt fastapi uvicorn
```
---
## ๐ How to Run
### ๐ Web API + Frontend
1. **Navigate to the model folder**:
```bash
cd model
```
2. **Start the FastAPI server** (with Uvicorn):
```bash
uvicorn api:app --host 0.0.0.0 --port 8000 --reload
```
3. **Open the frontend**:
- Option A: The API mounts the `frontend` folder as static files.
Open your browser and go to:
```text
http://localhost:8000/
```
- If needed, ensure the working directory is such that `../frontend` (from `model/api.py`) points to the `frontend` folder in this repo.
4. **Allow camera access** ๐ท in the browser when prompted.
5. You should see the **live video** and the **predicted emotion** updating underneath! ๐
---
### ๐ป Local Webcam Script (No Browser)
If you prefer a pure Python / OpenCV pipeline:
1. Make sure dependencies are installed:
```bash
pip install -r requirements.txt
```
2. From the `model` directory, run:
```bash
python webcam.py
```
3. A window called **"Emotion Recognition"** will appear:
- ๐ค Detected faces will be highlighted with a bounding box.
- ๐ท๏ธ The predicted emotion label will be shown next to each face.
- Press **`q`** to quit.
---
### ๐ณ Run with Docker (Optional)
If you prefer not to manage Python and system dependencies manually, you can use the provided `DockerFile`:
1. **Build the image** from the project root:
```bash
docker build -t emotion-recognition -f DockerFile .
```
2. **Run the container** and expose the API on port `8000`:
```bash
docker run --rm -p 8000:8000 emotion-recognition
```
3. Open your browser at:
```text
http://localhost:8000/
```
and allow camera access. The behavior is the same as in the "Web API + Frontend" section above.
---
## ๐ง Model & Inference Details
- The model is a **MobileNetV2** classifier whose final layer is adapted to the number of emotion classes (e.g. ๐ Angry, ๐จ Fear, ๐ Happy, ๐ข Sad, ๐ฒ Surprise).
- The **fineโtuned weights and class names** are stored in `emotion_recognition_model.pth`, which is produced by the training notebook (`working.ipynb`).
- Images are preprocessed with:
- ๐ Resize to \(224 \times 224\)
- ๐ Conversion to tensor
- ๐ Normalization with ImageNet mean and std
- Inference is done by `inference.py` via:
- `predict(pil_image)` โ returns a string label, e.g. `"Happy"`.
### Testing the Model
You can test the model directly with a static image (from the `model` directory):
```bash
python inference.py
```
This will load `Image.jpg` and print the predicted emotion.
---
## ๐ก API Reference
### Endpoints
- **`GET /`**
- Returns a simple JSON status: `{"status": "API running"}`.
- **`POST /api/predict`**
- **Body**: `multipart/form-data` with a single field:
- `file`: image file (e.g. JPEG/PNG).
- **Response**:
```json
{ "emotion": "Happy" }
```
The frontend uses this endpoint to send frames from your webcam (as blobs) approximately once per second.
---
## ๐ Dataset & Training (FineโTuning Overview)
- The (optional) `Data` directory is expected to contain labeled images organized by emotion:
- ๐ `Angry/`, ๐จ `Fear/`, ๐ `Happy/`, ๐ข `Sad/`, ๐ฒ `Suprise/`.
- The **fineโtuning process is captured in `working.ipynb`**:
- Load an **ImageNetโpretrained MobileNetV2** from `torchvision`.
- Replace the final classification layer so that its output dimension matches the number of emotion classes.
- Create PyTorch `Dataset`/`DataLoader` objects from the `Data` folders with standard augmentations and preprocessing.
- Train the network (typically using crossโentropy loss and an optimizer like Adam/SGD) while monitoring validation accuracy.
- Save the best model checkpoint and class mapping to `emotion_recognition_model.pth`.
- You can adapt the model for new datasets by:
- Updating the `Data` folders and class list.
- Reโrunning or modifying `working.ipynb` and, if needed, adjusting `model.py` for a different backbone or number of classes.
**Dataset Source**: [Kaggle - Human Face Emotions](https://www.kaggle.com/datasets/samithsachidanandan/human-face-emotions)
---
## ๐ง Troubleshooting
- **๐ท Camera access denied (browser)**
- Check browser permissions and ensure you're using `http://localhost` (not `file://`).
- **๐ "Backend not reachable" in the frontend**
- Confirm the FastAPI server is running on the same host/port that `script.js` expects (`/api/predict` โ default `http://localhost:8000/api/predict`).
- Check for CORS issues or port conflicts.
- **๐ฆ Model file not found**
- Ensure `emotion_recognition_model.pth` is present in the `model` directory when running any Python scripts there.
---
## ๐ License & Credits
- The project uses **PyTorch**, **FastAPI**, **OpenCV**, and **PIL** under their respective licenses.
- Dataset images in `Data/` should respect their original source licenses (not provided here).
- Feel free to modify or extend this project for research, learning, or personal use.
---
<div align="center">
<p>Made with โค๏ธ using PyTorch, FastAPI, and OpenCV</p>
<p>โญ Star this repo if you find it helpful!</p>
</div>
|