README.md

metadata

title: Audio Insight
emoji: 🎧
colorFrom: blue
colorTo: indigo
sdk: gradio
sdk_version: 6.28.0
python_version: '3.10'
app_file: app.py
pinned: false
short_description: Upload audio files; get an AI report and cross-file patterns

Audio Insight

Upload one or more audio files and get an analysis report: a transcript, speakers, themes, sentiment and recording quality for each file, and the patterns across files. The AI parts use Google Gemini; the acoustic measurements are computed locally.

What the report contains

For each recording

• A summary, content type, language and title (Gemini).
• A verbatim transcript with speaker labels and timestamps. It downloads as TXT and SRT.
• Themes with supporting quotes, key points, questions raised, decisions and follow-ups, notable quotes, tone moments, keywords and names, and a music profile when music is present.
• Answers to your own questions, with evidence from the recording.
• Measured audio values: integrated loudness (LUFS), noise floor, estimated SNR, sound activity, pauses, voice pitch, spectral balance, effective bandwidth, and clipping, hum, DC-offset and channel checks. Each recording also gets a quality verdict.
• Charts: loudness over time with pauses marked, who speaks when, talk-time share, and a spectrogram.
• Speech statistics computed from the transcript: words per minute, turns, questions, fillers and vocabulary variety.

Across recordings (two or more files)

• Shared themes, shown as a theme-by-file matrix with the strength in each file. Each theme has quotes.
• Differences, what each recording adds, speaker patterns, outliers and suggested next steps.
• Answers to your questions across all the files.
• A measured comparison of every metric, plus acoustic similarity (timbre, spectral balance, voice pitch) and vocabulary similarity (TF-IDF) between each pair of files.
• Shared and distinctive terms. With 3+ dated files, the trends over recording dates. With 4+ files, outliers and similarity groups.

Checks on the AI output

• Gemini listens in clips of about 10 minutes, cut at pauses, because its timestamps drift on long inputs. Its times are re-based, clamped to each clip and rescaled if they run past the clip's end.
• Every quote the model cites is looked up in the transcript and marked exact, close or not found. Its time is taken from the matching transcript line.
• Speaking rate, talk share, turns and fillers are computed from the transcript, not guessed by the model.
• Each AI-generated section is labelled, and so is each measured section.

Downloads: a self-contained HTML report (open it in any browser, or print it to PDF), a JSON file of all the data, a CSV of the metrics, the transcripts, and a ZIP with all of these.

Deploy on Hugging Face

1. Create a new Space at https://huggingface.co/new-space and choose Gradio as the SDK.
   • Free accounts can only run Gradio Spaces on ZeroGPU hardware. The app includes the stub @spaces.GPU function that ZeroGPU needs, but it never uses the GPU. On a paid plan, CPU basic works too.
2. Upload every file in this folder to the Space (web upload, or git push). Leave tests/ out if you like. Upload README.md as well, because its header configures the Space.
3. In Settings → Variables and secrets, add a secret:
   • GEMINI_API_KEY: your key from https://aistudio.google.com/apikey, or
   • GEMINI_API_KEYS: several keys separated by commas. When one key runs out of quota, the app switches to the next.
4. The Space builds and starts. If no key is configured, users can paste their own key under Options, or run the measured analysis only.

Optional variables (all have defaults):

Variable	Default	Meaning
GEMINI_MODEL	gemini-3.8-flash,gemini-flash-latest,gemini-3.5-flash,gemini-2.5-flash	Listening/analysis models, tried in order
GEMINI_DEEP_MODEL	gemini-pro-latest,gemini-3.1-pro-preview,gemini-2.5-pro	Used for the analysis when "Deeper reasoning" is ticked
MAX_FILES	20	Files per run
MAX_MINUTES_PER_FILE	180	Longest file accepted
MAX_TOTAL_MINUTES	600	Total audio per run
MAX_UPLOAD_MB	500	Upload size limit per file
PARALLEL_FILES	3	Files processed at the same time
CHUNK_SECONDS	600	Length of the clips Gemini listens to

Run locally

python3.10 -m venv .venv && source .venv/bin/activate
pip install gradio==6.28.0 -r requirements.txt
export GEMINI_API_KEY=your-key
python app.py                       # opens http://127.0.0.1:7860

Batch mode without the web UI writes the same report:

python pipeline.py interview1.m4a interview2.m4a --out results \
  --context "Research interviews about AI in nursing" \
  --question "What pain points do participants describe?"

Run the offline tests (Gemini is replaced by a stub, so they need no key):

pip install pytest && python -m pytest -q tests

Formats and limits

WAV, MP3, M4A/AAC, FLAC, OGG/Opus, WMA, AIFF and AMR are supported, as is the audio track of MP4/MOV/WebM/MKV video. Decoding uses PyAV's bundled FFmpeg, so the system needs no ffmpeg. The analysis runs on a 16 kHz mono mix, so content above 8 kHz is not measured.

Privacy

Files are processed in memory. Uploads and reports are removed from the server within about two hours. For the AI parts, the audio is sent to the Gemini API. With a free-tier key, Google may use the content to improve its products, and human reviewers may see it. For confidential or research-participant recordings, use a paid-tier (billing-enabled) key and follow your ethics approval.

Files

File	Role
app.py	Gradio interface (the Space entry point)
pipeline.py	Orchestration, and the command-line batch mode
audio_io.py	Decoding, metadata, recording dates and Opus encoding (PyAV)
acoustics.py	Measured analysis, quality checks and chunk planning
gemini_service.py	Gemini client: key rotation, model fallback, retries and JSON validation
schemas.py	Structured-output schemas for the three Gemini steps
ai_analysis.py	Listening, per-file analysis, cross-file synthesis and quote verification
patterns.py	Measured cross-file comparison, similarity, trends and outliers
charts.py, report.py	SVG charts, the HTML report, and the JSON/CSV/TXT/SRT exports