---
title: AudioSummarizer
emoji: 📚
colorFrom: blue
colorTo: green
sdk: gradio
sdk_version: 5.46.1
app_file: app.py
pinned: false
license: mit
---

# AudioSummarizer

## What’s New (May 3, 2026)
- **Gradio upgraded to 5.46.1** to resolve a pip dependency conflict between `gradio==5.45.0` and `gradio==5.46.1`.
- **GitHub Actions CI/CD workflow added** — every push to `main` automatically syncs the repo to the Hugging Face Space (`samir72/AudioChatTranscriber`). Requires an `HF_TOKEN` secret configured in the GitHub repository settings.
- **Model switched to gpt-4o-mini** — replaced Phi-4-multimodal-instruct with `gpt-4o-mini` via Azure OpenAI to resolve `DeploymentNotFound` errors. Set `AC_MODEL_DEPLOYMENT=gpt-4o-mini` in your environment or HF Space secrets.
- **Audio pipeline updated for gpt-4o-mini compatibility** — `gpt-4o-mini` does not support native audio content blocks. All audio inputs (upload, microphone, direct MP3 URL) are now transcribed locally via **faster-whisper** before being sent to the model as text, eliminating the `400 invalid_request_error`.

---

## What’s New (Sep 26–28, 2025)
- **YouTube cookie refresh & expiry handling** added to avoid sign-in/download failures.  
- **DNS lookup improvements**: automatically skip DNS failures on Hugging Face Spaces to reduce false negatives.  
- **Azure Container App (ACA) integration**: bypasses YouTube blocking by offloading audio download to Azure, storing audio in Blob Storage, and feeding it into the HF pipeline.  
- **Docker / ACA enhancements**: uses Microsoft slim base image in ACR for faster builds, with trade-off that the base must be regularly refreshed.  
- **Repo restructuring**: renamed the app entry folder to `extract/` to resolve a Hugging Face build conflict.

---

## Overview
AudioSummarizer is a web app (deployed on Hugging Face Spaces) that summarizes audio from multiple sources — file upload, microphone, or URL (YouTube / direct MP3) — using **gpt-4o-mini** via Azure OpenAI for structured summarization. The app uses **faster‑whisper** for transcription and **yt-dlp** + **ffmpeg** for audio extraction, with a clean **Gradio** UI. Prompts are loaded from `metadata.json` to ensure replies include **Summary**, **Key Details**, and **Insights**.

Because Hugging Face often cannot directly fetch YouTube audio (due to network restrictions or blocking), we now route YouTube downloads through an **Azure Container App** which:

1. Fetches the YouTube audio independently.  
2. Stores the processed 16 kHz mono WAV file in **Azure Blob Storage**.  
3. Serves that file into the usual transcription/summarization pipeline in the HF app.

Thus, the HF interface remains unchanged to users, but YouTube support is restored reliably via Azure.

---

## Features
- Upload a local MP3 file, record via microphone, or enter a YouTube / MP3 URL.  
- **Azure Container App support** so YouTube content is reliably processed even if Hugging Face cannot fetch it.  
- Prompts fully customizable: you may define system and user prompts stored in `metadata.json`.  
- Transcription using **faster-whisper**, summarization through **gpt-4o-mini** (Azure OpenAI).  
- Clean and minimal **Gradio** UI for intuitive interaction.  
- Configuration via environment variables (`.env`) for Azure endpoint, deployment name, API key, etc.  
- YouTube audio extraction to **16 kHz mono WAV** (via yt-dlp + ffmpeg).  
- DNS‑based URL validation, with automatic skip of DNS errors in HF Spaces to reduce false rejections.

---

## Architecture / Data Flow

```
User Input (YouTube) ──▶ Hugging Face UI
   │
   └── If URL is YouTube:
         ─▶ forwarded to Azure Container App
               ├── ACA downloads YouTube audio (yt-dlp)
               └── Converts/stores WAV in Azure Blob Storage
         ─▶ HF app fetches WAV from Blob Storage
               ├── Transcribe via faster-whisper
               └── Summarize via Azure gpt-4o-mini
               

 ┌───────────────┐ file/mic/url ┌───────────────────────────┐
 │   Gradio UI   │─────────────▶│ process_audio(...)         │
 └──────┬────────┘              └──────────┬─────────────────┘
        │ validates/reads                  │
        ▼                                  ▼
 ┌───────────────────────────┐   ┌─────────────────────────────┐
 │ summarize_input(audio,...)│──▶│ Azure gpt-4o-mini │
 └───────────────────────────┘   │ Chat Completions (text+audio)│
                                 └─────────────────────────────┘

 YouTube Path (via ACA):
 ┌───────────────┐  YouTube URL ┌──────────────────────────────┐
 │   Gradio UI   │────────────▶ │ Azure Container App (yt-dlp)  │
 └───────────────┘              └──────────┬───────────────────┘
                                           │ uploads audio
                                           ▼
                                ┌──────────────────────────────┐
                                │ Azure Blob Storage (WAV 16k) │
                                └──────────┬───────────────────┘
                                           │
                                           ▼
                              ┌──────────────────────────────┐
                              │ faster-whisper transcription │
                              └──────────┬───────────────────┘
                                           │ text
                                           ▼
                              ┌──────────────────────────────┐
                              │ Azure gpt-4o-mini │
                              │ summarization                │
                              └──────────────────────────────┘

```

For non-YouTube inputs (local upload, mic, direct MP3 URL), the flow remains internal to the HF space: download/convert → transcription → summarization.

---

## CI/CD — GitHub Actions

A workflow at `.github/workflows/main.yml` runs on every push to `main` (and can be triggered manually via `workflow_dispatch`).

**What it does:**
1. Checks out the repo with a shallow clone (no LFS, no full history).
2. Creates a clean orphan branch — only the current file state, no large-file history.
3. Force-pushes that branch to the `main` branch of the Hugging Face Space `samir72/AudioChatTranscriber`.

**Setup requirement:** Add an `HF_TOKEN` secret in **GitHub → Settings → Secrets and variables → Actions** with a Hugging Face token that has write access to the Space.

---

## Docker & Azure Container Apps

### Optimization: Microsoft Slim Base in ACR
The Docker image now uses a **Microsoft slim base image** hosted in **Azure Container Registry (ACR)** to speed up builds (less reliance on external pulls).  
- ✅ **Advantage**: faster, more predictable builds in Azure / CI.  
- ⚠️ **Caveat**: you must **refresh the slim base in ACR routinely** to catch upstream security patches, updates, or bug fixes.

**Best Practice Recommendation:**  
Set up a scheduled job (e.g. via ACR Task or Azure DevOps pipeline) to pull the latest Microsoft slim base and update your ACR copy on a regular cadence (e.g. weekly) so your deployed containers remain current.

### Build & Run Example
```bash
# Build locally
docker build -t audiosummarizer:latest .

# Run container
docker run --rm -p 7860:7860   -e AC_OPENAI_ENDPOINT=...   -e AC_MODEL_DEPLOYMENT=...   -e AC_OPENAI_API_KEY=...   -e AC_OPENAI_API_VERSION=...   audiosummarizer:latest
```

For ACA deployment:
1. Push the Docker image to your ACR.
2. Deploy the image via **Azure Container Apps** with necessary environment variables.
3. The ACA will serve as the YouTube‐to‑Blob “fetcher” component, supporting the main HF app.

---

## Prerequisites
- Python **3.10+**  
- Azure subscription with deployment of **gpt-4o-mini**  
- `ffmpeg` installed and in `$PATH`  
- A valid `metadata.json` containing default prompts  
- For HF spaces: `packages.txt` including `ffmpeg`  

---

## Python Dependencies
Add to `requirements.txt`:
```
azure-identity>=1.17.1
openai>=1.0.0
gradio>=4.44.0
python-dotenv>=1.0.1
requests>=2.32.3
yt-dlp>=2024.8.6
faster-whisper>=0.10.0
beautifulsoup4>=4.12.2   # optional, for fallback scraping
```

Install as usual:
```bash
python -m venv .venv
source .venv/bin/activate  # on Windows: .venv\Scripts\activate
pip install -r requirements.txt
```

---

## Installation
```bash
git clone https://github.com/samir72/AudioSummarizer.git
cd AudioSummarizer
```
Install dependencies and make sure `ffmpeg` is available (or included via `packages.txt` in HF deployment).

---

## Configuration
Create a `.env` file at the project root:
```env
AC_OPENAI_ENDPOINT=https://<your-azure-resource>.openai.azure.com/
AC_MODEL_DEPLOYMENT=<your‑phi‑4 deployment name>
AC_OPENAI_API_KEY=<your azure openai api key>
AC_OPENAI_API_VERSION=<api version e.g. 2024-10-01>

GRADIO_SERVER_NAME=127.0.0.1
GRADIO_SERVER_PORT=7860
```

If you’re running the Azure Container App, ensure it is configured with:
- Proper role / access to write to Azure Blob Storage  
- Environment variables for any keys or connection strings it needs  
- Networking/firewall settings so the HF app can fetch from the blob store

---

## Usage
Run the app:
```bash
python app.py
```
Then open your browser to [http://127.0.0.1:7860](http://127.0.0.1:7860) or use your HF Space URL.

### Input options
- Upload MP3 file  
- Record via microphone  
- Enter a YouTube / direct MP3 URL  
- Modify system/user prompts (via `metadata.json`)  
- Click **Summarize** → get structured output (Summary, Key Details, Insights)

---

## Contributing
We welcome your improvements—especially around cloud integration, performance, and reliability.

**Suggested contribution areas:**
- Better error handling for cookie expiry, fallback strategies 
- Enhancements to the Azure Container App + Blob Storage pipeline  
- Caching / sync between ACA and the HF app  
- Automation of **ACR slim base refresh**  
 

**How to contribute:**
1. Fork the repository  
2. Create a feature branch (e.g. `git checkout -b feat/xyz`)  
3. Commit changes with meaningful messages  
4. Push and open a Pull Request  

Please reference this `README.md` when describing how the YouTube → ACA → Blob → HF flow works.

---

## License
This project is licensed under the **MIT License** — see [LICENSE](./LICENSE) for details.

---

## Acknowledgments
- Built with **Gradio** for UI 
- Application deployed on **Hugging Face Spaces**
- ACA deployed on **Azure**
- Application layer on ACA served by **FastAPI** 
- Intelligence by **Azure gpt-4o-mini**  
- YouTube audio extraction with **yt-dlp**  
- Transcription enabled by **faster-whisper**

---


## Contact
For questions or feedback, reach out to **Sayed Amir Rizvi**  
Email: syedamirhusain@gmail.com