Spaces:
Paused
Download README.md from Phoenix21/lecturelens-demo: direct link, hf CLI and curl.
- Browser
- Download file 6.69 kB
-
https://huggingface.co/spaces/Phoenix21/lecturelens-demo/resolve/main/README.md
- Command line
-
hf download hf://spaces/Phoenix21/lecturelens-demo/README.md
-
curl -L -o README.md https://huggingface.co/spaces/Phoenix21/lecturelens-demo/resolve/main/README.md
A newer version of the Gradio SDK is available: 6.32.0
title: LectureLens
emoji: 🎞️
colorFrom: green
colorTo: blue
sdk: gradio
sdk_version: 6.30.0
python_version: 3.12.12
app_file: app.py
pinned: false
short_description: Ask videos with hierarchical retrieval and frame citations
LectureLens
Upload a short video, import a YouTube clip, or try the bundled gradient descent demo. Ask your own question or choose an example, then inspect the cited evidence, original frames, and replay clips.
Answer engines and quotas
- Auto (Groq + local fallback): use Groq when available; fall back to a local Qwen2.5-1.5B-Instruct model if Groq is limited, unavailable, or unconfigured. Audio recognition falls back to local Whisper Tiny. Optional visual descriptions can be skipped while speech and OCR remain available.
- Local models (no Groq calls): generate answers with local Qwen and transcribe with local Whisper. This text model uses speech and OCR evidence; it does not inspect frame images. It can be less accurate than the larger Groq models.
- Repeated successful questions reuse a cache isolated to the current video, answer engine, and browser session. Known Groq cooldowns honor
Retry-Afterwithout repeatedly sending rejected requests.
Local inference removes Groq's API quota dependency, but free ZeroGPU still has its own per-user GPU allowance and queue. This Space is not unlimited. Running the app on hardware you control avoids external inference-provider quotas; CPU inference is slower.
YouTube clips
Open Import a YouTube clip, paste a public video link, choose the start time in seconds and a length of up to 120 seconds, then load it. Long videos are supported through a selected clip; the app does not analyze the entire long video. Citation links and displayed timestamps point into the original YouTube video; frame replay uses the downloaded clip's local timeline. No YouTube captions are assumed or fabricated: speech recognition runs on the actual clip audio.
The importer uses the official youtube-dl Python package, pinned to source commit 956b8c585591b401a543e409accb163eeaaa1193. It resolves the video/audio streams, then FFmpeg downloads and accurately cuts the requested interval. It does not rely on yt-dlp-specific clip options or fetch an entire long video before trimming it.
Only supported public, unrestricted videos are imported. YouTube can reject data-center downloads, require sign-in, or change its delivery system. The app reports blocked/timed-out imports and keeps file upload available. It does not use cookies, credential forwarding to YouTube, proxy rotation, or bypass access restrictions. Import only content you have permission to process.
This prototype splits contiguous video frame ranges in half until each leaf represents one original frame. Evidence comes from subtitles or Groq Whisper speech recognition, sampled Tesseract OCR, and sampled Groq visual descriptions. Upper groups get purpose labels. On ZeroGPU, a small open-source MiniLM encoder embeds evidence and questions on GPU; semantic scores combine with lexical scores to route the hierarchy. The CPU/local path uses lexical routing. Broad summaries use coverage across the timeline.
Limits and grounding
- Uploads: up to 2 minutes, 100 MB, and 9,000 decoded frames.
- Each session: up to 20 questions; server inference budget defaults to 240 calls/hour across sessions.
- Frames use exact decoded ordinals and original presentation timestamps, normalized to the first video frame. Speech timestamps are approximate, and uploaded subtitles must align with that timeline.
- Citation IDs and source intervals are validated in code. This cannot prove factual entailment. Sampled visual indexing can miss fast events, and a limited beam can miss relevant branches.
- Purpose labels are generated for upper groups; deeper groups use extractive labels. Every original frame is represented, although every frame is not sent to a vision model.
Models and privacy
The Space reads GROQ_API_KEY from its server-side secrets. Groq defaults are openai/gpt-oss-20b for text, qwen/qwen3.8-27b for vision, and whisper-large-v3-turbo for transcription. Override GROQ_TEXT_MODEL and GROQ_VISION_MODEL through Space variables if account availability changes. Local generation uses Qwen2.5-1.5B-Instruct; local speech recognition uses Faster Whisper Tiny on CPU. These model weights download at first use/startup, with no inference API key needed for local mode.
Uploads and imported clips stay in temporary session directories on the Space. Auto mode sends audio, selected frames, and evidence text to Groq when available. Local mode keeps inference on the hosting machine, using ZeroGPU for generation on this Space. Sessions expire after inactivity and Gradio clears expired temporary output files. Do not upload confidential material. No upload or runtime key is committed to either repository.
This is a Gradio SDK Space. Groq generates answers, transcribes speech, and describes selected frames. On ZeroGPU, all-MiniLM-L6-v2 runs locally for semantic retrieval; its weights download at startup. This uses the free ZeroGPU request quota. If semantic inference is temporarily unavailable, lexical routing remains available. Deployment first tries CPU Basic; if Hugging Face requires a paid plan, it tries the documented free personal ZeroGPU allocation. That tier requires a verified account older than 30 days and an available free slot. No paid hardware or subscription is requested. GitHub Actions publishes only the allowlisted app runtime from aniketqw/lecturelens-research.
Run locally
Install FFmpeg and Tesseract, install gradio/requirements.txt, then run python gradio/app.py from the research repository. In the Space, run python app.py.
Select Local models (no Groq calls) to work without a Groq key. On your own machine, local generation defaults to CPU; set LECTURELENS_LOCAL_DEVICE=cuda to use your NVIDIA GPU directly. On ZeroGPU, the default local generator uses its managed GPU allowance; setting LECTURELENS_LOCAL_DEVICE=cpu selects CPU generation. Initial model downloads require internet access. Model size can be changed with LECTURELENS_LOCAL_MODEL; supported instruction models must work with the Qwen-style chat template. Use smaller models for low-memory CPUs, accepting reduced answer quality.
For fixture-only development tests without downloading Torch/model weights, install gradio/requirements-test.txt and run python -m unittest discover -s tests -v.