Hugging Face's logo

Spaces:
samir72
/
AudioChatTranscriber
Sleeping

App Files Community
1
AudioChatTranscriber / README.md
GitHub Actions
Clean sync from GitHub - no large files in history
92ddce4
preview code
|
Raw
History Blame Contribute Delete
11.9 kB

A newer version of the Gradio SDK is available: 6.23.1

Upgrade
metadata 
title: AudioSummarizer
emoji: 📚
colorFrom: blue
colorTo: green
sdk: gradio
sdk_version: 5.46.1
app_file: app.py
pinned: false
license: mit

AudioSummarizer

What’s New (May 3, 2026)

  • Gradio upgraded to 5.46.1 to resolve a pip dependency conflict between gradio==5.45.0 and gradio==5.46.1.
  • GitHub Actions CI/CD workflow added — every push to main automatically syncs the repo to the Hugging Face Space (samir72/AudioChatTranscriber). Requires an HF_TOKEN secret configured in the GitHub repository settings.
  • Model switched to gpt-4o-mini — replaced Phi-4-multimodal-instruct with gpt-4o-mini via Azure OpenAI to resolve DeploymentNotFound errors. Set AC_MODEL_DEPLOYMENT=gpt-4o-mini in your environment or HF Space secrets.
  • Audio pipeline updated for gpt-4o-mini compatibility — gpt-4o-mini does not support native audio content blocks. All audio inputs (upload, microphone, direct MP3 URL) are now transcribed locally via faster-whisper before being sent to the model as text, eliminating the 400 invalid_request_error.

What’s New (Sep 26–28, 2025)

  • YouTube cookie refresh & expiry handling added to avoid sign-in/download failures.
  • DNS lookup improvements: automatically skip DNS failures on Hugging Face Spaces to reduce false negatives.
  • Azure Container App (ACA) integration: bypasses YouTube blocking by offloading audio download to Azure, storing audio in Blob Storage, and feeding it into the HF pipeline.
  • Docker / ACA enhancements: uses Microsoft slim base image in ACR for faster builds, with trade-off that the base must be regularly refreshed.
  • Repo restructuring: renamed the app entry folder to extract/ to resolve a Hugging Face build conflict.

Overview

AudioSummarizer is a web app (deployed on Hugging Face Spaces) that summarizes audio from multiple sources — file upload, microphone, or URL (YouTube / direct MP3) — using gpt-4o-mini via Azure OpenAI for structured summarization. The app uses faster‑whisper for transcription and yt-dlp + ffmpeg for audio extraction, with a clean Gradio UI. Prompts are loaded from metadata.json to ensure replies include Summary, Key Details, and Insights.

Because Hugging Face often cannot directly fetch YouTube audio (due to network restrictions or blocking), we now route YouTube downloads through an Azure Container App which:

  1. Fetches the YouTube audio independently.
  2. Stores the processed 16 kHz mono WAV file in Azure Blob Storage.
  3. Serves that file into the usual transcription/summarization pipeline in the HF app.

Thus, the HF interface remains unchanged to users, but YouTube support is restored reliably via Azure.

Features

  • Upload a local MP3 file, record via microphone, or enter a YouTube / MP3 URL.
  • Azure Container App support so YouTube content is reliably processed even if Hugging Face cannot fetch it.
  • Prompts fully customizable: you may define system and user prompts stored in metadata.json.
  • Transcription using faster-whisper, summarization through gpt-4o-mini (Azure OpenAI).
  • Clean and minimal Gradio UI for intuitive interaction.
  • Configuration via environment variables (.env) for Azure endpoint, deployment name, API key, etc.
  • YouTube audio extraction to 16 kHz mono WAV (via yt-dlp + ffmpeg).
  • DNS‑based URL validation, with automatic skip of DNS errors in HF Spaces to reduce false rejections.

Architecture / Data Flow 

User Input (YouTube) ──▶ Hugging Face UI
   │
   └── If URL is YouTube:
         ─▶ forwarded to Azure Container App
               ├── ACA downloads YouTube audio (yt-dlp)
               └── Converts/stores WAV in Azure Blob Storage
         ─▶ HF app fetches WAV from Blob Storage
               ├── Transcribe via faster-whisper
               └── Summarize via Azure gpt-4o-mini
               

 ┌───────────────┐ file/mic/url ┌───────────────────────────┐
 │   Gradio UI   │─────────────▶│ process_audio(...)         │
 └──────┬────────┘              └──────────┬─────────────────┘
        │ validates/reads                  │
        ▼                                  ▼
 ┌───────────────────────────┐   ┌─────────────────────────────┐
 │ summarize_input(audio,...)│──▶│ Azure gpt-4o-mini │
 └───────────────────────────┘   │ Chat Completions (text+audio)│
                                 └─────────────────────────────┘

 YouTube Path (via ACA):
 ┌───────────────┐  YouTube URL ┌──────────────────────────────┐
 │   Gradio UI   │────────────▶ │ Azure Container App (yt-dlp)  │
 └───────────────┘              └──────────┬───────────────────┘
                                           │ uploads audio
                                           ▼
                                ┌──────────────────────────────┐
                                │ Azure Blob Storage (WAV 16k) │
                                └──────────┬───────────────────┘
                                           │
                                           ▼
                              ┌──────────────────────────────┐
                              │ faster-whisper transcription │
                              └──────────┬───────────────────┘
                                           │ text
                                           ▼
                              ┌──────────────────────────────┐
                              │ Azure gpt-4o-mini │
                              │ summarization                │
                              └──────────────────────────────┘

For non-YouTube inputs (local upload, mic, direct MP3 URL), the flow remains internal to the HF space: download/convert → transcription → summarization.

CI/CD — GitHub Actions

A workflow at .github/workflows/main.yml runs on every push to main (and can be triggered manually via workflow_dispatch).

What it does:

  1. Checks out the repo with a shallow clone (no LFS, no full history).
  2. Creates a clean orphan branch — only the current file state, no large-file history.
  3. Force-pushes that branch to the main branch of the Hugging Face Space samir72/AudioChatTranscriber.

Setup requirement: Add an HF_TOKEN secret in GitHub → Settings → Secrets and variables → Actions with a Hugging Face token that has write access to the Space.

Docker & Azure Container Apps

Optimization: Microsoft Slim Base in ACR

The Docker image now uses a Microsoft slim base image hosted in Azure Container Registry (ACR) to speed up builds (less reliance on external pulls).

  • ✅ Advantage: faster, more predictable builds in Azure / CI.
  • ⚠️ Caveat: you must refresh the slim base in ACR routinely to catch upstream security patches, updates, or bug fixes.

Best Practice Recommendation:
Set up a scheduled job (e.g. via ACR Task or Azure DevOps pipeline) to pull the latest Microsoft slim base and update your ACR copy on a regular cadence (e.g. weekly) so your deployed containers remain current.

Build & Run Example 

# Build locally
docker build -t audiosummarizer:latest .

# Run container
docker run --rm -p 7860:7860   -e AC_OPENAI_ENDPOINT=...   -e AC_MODEL_DEPLOYMENT=...   -e AC_OPENAI_API_KEY=...   -e AC_OPENAI_API_VERSION=...   audiosummarizer:latest

For ACA deployment:

  1. Push the Docker image to your ACR.
  2. Deploy the image via Azure Container Apps with necessary environment variables.
  3. The ACA will serve as the YouTube‐to‑Blob “fetcher” component, supporting the main HF app.

Prerequisites

  • Python 3.10+
  • Azure subscription with deployment of gpt-4o-mini
  • ffmpeg installed and in $PATH
  • A valid metadata.json containing default prompts
  • For HF spaces: packages.txt including ffmpeg

Python Dependencies

Add to requirements.txt:

azure-identity>=1.17.1
openai>=1.0.0
gradio>=4.44.0
python-dotenv>=1.0.1
requests>=2.32.3
yt-dlp>=2024.8.6
faster-whisper>=0.10.0
beautifulsoup4>=4.12.2   # optional, for fallback scraping

Install as usual:

python -m venv .venv
source .venv/bin/activate  # on Windows: .venv\Scripts\activate
pip install -r requirements.txt

Installation 

git clone https://github.com/samir72/AudioSummarizer.git
cd AudioSummarizer

Install dependencies and make sure ffmpeg is available (or included via packages.txt in HF deployment).

Configuration

Create a .env file at the project root:

AC_OPENAI_ENDPOINT=https://<your-azure-resource>.openai.azure.com/
AC_MODEL_DEPLOYMENT=<your‑phi‑4 deployment name>
AC_OPENAI_API_KEY=<your azure openai api key>
AC_OPENAI_API_VERSION=<api version e.g. 2024-10-01>

GRADIO_SERVER_NAME=127.0.0.1
GRADIO_SERVER_PORT=7860

If you’re running the Azure Container App, ensure it is configured with:

  • Proper role / access to write to Azure Blob Storage
  • Environment variables for any keys or connection strings it needs
  • Networking/firewall settings so the HF app can fetch from the blob store

Usage

Run the app:

python app.py

Then open your browser to http://127.0.0.1:7860 or use your HF Space URL.

Input options

  • Upload MP3 file
  • Record via microphone
  • Enter a YouTube / direct MP3 URL
  • Modify system/user prompts (via metadata.json)
  • Click Summarize → get structured output (Summary, Key Details, Insights)

Contributing

We welcome your improvements—especially around cloud integration, performance, and reliability.

Suggested contribution areas:

  • Better error handling for cookie expiry, fallback strategies
  • Enhancements to the Azure Container App + Blob Storage pipeline
  • Caching / sync between ACA and the HF app
  • Automation of ACR slim base refresh

How to contribute:

  1. Fork the repository
  2. Create a feature branch (e.g. git checkout -b feat/xyz)
  3. Commit changes with meaningful messages
  4. Push and open a Pull Request

Please reference this README.md when describing how the YouTube → ACA → Blob → HF flow works.

License

This project is licensed under the MIT License — see LICENSE for details.

Acknowledgments

  • Built with Gradio for UI
  • Application deployed on Hugging Face Spaces
  • ACA deployed on Azure
  • Application layer on ACA served by FastAPI
  • Intelligence by Azure gpt-4o-mini
  • YouTube audio extraction with yt-dlp
  • Transcription enabled by faster-whisper

Contact

For questions or feedback, reach out to Sayed Amir Rizvi
Email: syedamirhusain@gmail.com