Spaces:
Sleeping
Sleeping
| title: Audio Insight | |
| emoji: 🎧 | |
| colorFrom: blue | |
| colorTo: indigo | |
| sdk: gradio | |
| sdk_version: 6.28.0 | |
| python_version: "3.10" | |
| app_file: app.py | |
| pinned: false | |
| short_description: Upload audio files; get an AI report and cross-file patterns | |
| # Audio Insight | |
| Upload one or more audio files and get an analysis report: a transcript, speakers, | |
| themes, sentiment and recording quality for each file, and the patterns across files. | |
| The AI parts use Google Gemini; the acoustic measurements are computed locally. | |
| ## What the report contains | |
| **For each recording** | |
| - A summary, content type, language and title (Gemini). | |
| - A verbatim transcript with speaker labels and timestamps. It downloads as TXT and SRT. | |
| - Themes with supporting quotes, key points, questions raised, decisions and follow-ups, | |
| notable quotes, tone moments, keywords and names, and a music profile when music is present. | |
| - Answers to your own questions, with evidence from the recording. | |
| - Measured audio values: integrated loudness (LUFS), noise floor, estimated SNR, sound activity, | |
| pauses, voice pitch, spectral balance, effective bandwidth, and clipping, hum, DC-offset and | |
| channel checks. Each recording also gets a quality verdict. | |
| - Charts: loudness over time with pauses marked, who speaks when, talk-time share, and a spectrogram. | |
| - Speech statistics computed from the transcript: words per minute, turns, questions, fillers and | |
| vocabulary variety. | |
| **Across recordings (two or more files)** | |
| - Shared themes, shown as a theme-by-file matrix with the strength in each file. Each theme has quotes. | |
| - Differences, what each recording adds, speaker patterns, outliers and suggested next steps. | |
| - Answers to your questions across all the files. | |
| - A measured comparison of every metric, plus acoustic similarity (timbre, spectral balance, | |
| voice pitch) and vocabulary similarity (TF-IDF) between each pair of files. | |
| - Shared and distinctive terms. With 3+ dated files, the trends over recording dates. | |
| With 4+ files, outliers and similarity groups. | |
| **Checks on the AI output** | |
| - Gemini listens in clips of about 10 minutes, cut at pauses, because its timestamps drift on long | |
| inputs. Its times are re-based, clamped to each clip and rescaled if they run past the clip's end. | |
| - Every quote the model cites is looked up in the transcript and marked *exact*, *close* or | |
| *not found*. Its time is taken from the matching transcript line. | |
| - Speaking rate, talk share, turns and fillers are computed from the transcript, not guessed by the model. | |
| - Each AI-generated section is labelled, and so is each measured section. | |
| **Downloads:** a self-contained HTML report (open it in any browser, or print it to PDF), a JSON file | |
| of all the data, a CSV of the metrics, the transcripts, and a ZIP with all of these. | |
| ## Deploy on Hugging Face | |
| 1. Create a new Space at <https://huggingface.co/new-space> and choose **Gradio** as the SDK. | |
| - Free accounts can only run Gradio Spaces on **ZeroGPU** hardware. The app includes the stub | |
| `@spaces.GPU` function that ZeroGPU needs, but it never uses the GPU. On a paid plan, | |
| **CPU basic** works too. | |
| 2. Upload every file in this folder to the Space (web upload, or `git push`). Leave `tests/` out if | |
| you like. Upload `README.md` as well, because its header configures the Space. | |
| 3. In **Settings → Variables and secrets**, add a secret: | |
| - `GEMINI_API_KEY`: your key from <https://aistudio.google.com/apikey>, or | |
| - `GEMINI_API_KEYS`: several keys separated by commas. When one key runs out of quota, the app | |
| switches to the next. | |
| 4. The Space builds and starts. If no key is configured, users can paste their own key under | |
| **Options**, or run the measured analysis only. | |
| Optional variables (all have defaults): | |
| | Variable | Default | Meaning | | |
| |---|---|---| | |
| | `GEMINI_MODEL` | `gemini-3.8-flash,gemini-flash-latest,gemini-3.5-flash,gemini-2.5-flash` | Listening/analysis models, tried in order | | |
| | `GEMINI_DEEP_MODEL` | `gemini-pro-latest,gemini-3.1-pro-preview,gemini-2.5-pro` | Used for the analysis when "Deeper reasoning" is ticked | | |
| | `MAX_FILES` | 20 | Files per run | | |
| | `MAX_MINUTES_PER_FILE` | 180 | Longest file accepted | | |
| | `MAX_TOTAL_MINUTES` | 600 | Total audio per run | | |
| | `MAX_UPLOAD_MB` | 500 | Upload size limit per file | | |
| | `PARALLEL_FILES` | 3 | Files processed at the same time | | |
| | `CHUNK_SECONDS` | 600 | Length of the clips Gemini listens to | | |
| ## Run locally | |
| ```bash | |
| python3.10 -m venv .venv && source .venv/bin/activate | |
| pip install gradio==6.28.0 -r requirements.txt | |
| export GEMINI_API_KEY=your-key | |
| python app.py # opens http://127.0.0.1:7860 | |
| ``` | |
| Batch mode without the web UI writes the same report: | |
| ```bash | |
| python pipeline.py interview1.m4a interview2.m4a --out results \ | |
| --context "Research interviews about AI in nursing" \ | |
| --question "What pain points do participants describe?" | |
| ``` | |
| Run the offline tests (Gemini is replaced by a stub, so they need no key): | |
| ```bash | |
| pip install pytest && python -m pytest -q tests | |
| ``` | |
| ## Formats and limits | |
| WAV, MP3, M4A/AAC, FLAC, OGG/Opus, WMA, AIFF and AMR are supported, as is the audio track of | |
| MP4/MOV/WebM/MKV video. Decoding uses PyAV's bundled FFmpeg, so the system needs no ffmpeg. | |
| The analysis runs on a 16 kHz mono mix, so content above 8 kHz is not measured. | |
| ## Privacy | |
| Files are processed in memory. Uploads and reports are removed from the server within about | |
| two hours. For the AI parts, the audio is sent to the Gemini API. With a free-tier key, Google may | |
| use the content to improve its products, and human reviewers may see it. For confidential or | |
| research-participant recordings, use a paid-tier (billing-enabled) key and follow your ethics | |
| approval. | |
| ## Files | |
| | File | Role | | |
| |---|---| | |
| | `app.py` | Gradio interface (the Space entry point) | | |
| | `pipeline.py` | Orchestration, and the command-line batch mode | | |
| | `audio_io.py` | Decoding, metadata, recording dates and Opus encoding (PyAV) | | |
| | `acoustics.py` | Measured analysis, quality checks and chunk planning | | |
| | `gemini_service.py` | Gemini client: key rotation, model fallback, retries and JSON validation | | |
| | `schemas.py` | Structured-output schemas for the three Gemini steps | | |
| | `ai_analysis.py` | Listening, per-file analysis, cross-file synthesis and quote verification | | |
| | `patterns.py` | Measured cross-file comparison, similarity, trends and outliers | | |
| | `charts.py`, `report.py` | SVG charts, the HTML report, and the JSON/CSV/TXT/SRT exports | | |