Spaces:
Paused
Paused
Commit History
-
Lazy diarization: the first answer no longer waits for pyannote; speaker labels attach to the cached transcript when readyβ
Malina +1: Claude Fable 5.1Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>1d90084 -
v0.90: scoped single-use tokens, hardened link fetch, 45-minute cap, whole-video digest, one classification call, no OCRβ
Malina +1: Claude Fable 5.1- auth.py shared with LUNA: scope + iat checks, jti persisted across restarts, TOKEN_REQUIRE_SCOPE - fetch.py: known hosts only, public-address check, no generic extractor, cookies filtered by domain, 240 s deadline, 400 MB cap, temp dirs swept - pipeline: duration cap, every refusal prefixed with the error marker, stage timings, int16 audio cache, ingest locks pruned - chat: one classify call (OFF/ANALYZE/ANSWER), server-side size caps, fixed error sentences, NEED_VIDEO handshake removed - synthesis: whole-transcript digest for summaries; language detector moved to lang.py; fmt_time m:ss / h:mm:ss - visual: seek-based frame sampling past five minutes - OCR removed (rapidocr, onnxruntime, libgl1); yt-dlp pinned; max_file_size at the door; README rewritten - tests: auth, fetch validation, language and time formatting (55) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>1d697a7
-
Film bez sciezki dzwiekowej przestaje wywracac cala analizeβ
Malina +1: Claude Opus 5 (1M context)Kontener bez audio wysadzal odczyt: PyAV rzuca IndexError w srodku faster_whisper, kiedy prosi o strumien audio numer 0 w pliku, ktory go nie ma. Wyjatek szedl do slownika i ginal, a uzytkownik dostawal "Error reading video." w jedna sekunde. Zmierzone na zwyklym H.264 bez audio: kazda analiza takiego pliku konczyla sie tak samo, czyli nagranie ekranu bez narracji, najbardziej naturalny przypadek uzycia dla tego narzedzia, nie dzialalo w ogole. Brak dzwieku to teraz cisza, a nie blad: load_audio zwraca pusta fale, Whisper nie jest w ogole wolany, a synteza idzie sciezka wizualna, ktora ten kod od poczatku deklarowal ("No speech is fine"). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>3b71b6c -
Poprawka linii logu: pipeline.py nie importuje osβ
Malina +1: Claude Opus 5 (1M context)Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>ac0c9e0 -
Nieudany odczyt wideo zostawia slad w logachβ
Malina +1: Claude Opus 5 (1M context)"Error reading video." bylo jedynym, co zostawalo po calkowicie nieudanym wczytaniu: wyjatek szedl do slownika i ginal, wiec ani uzytkownik, ani logi nie mialy z czego zaczac. 2026-09-21 kazda analiza konczyla sie tym komunikatem w jedna sekunde, a w logach byla tylko linia o przyjeciu zadania. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>0c1c8d1
-
Wczytane wideo przezywa odejscie sluchaczaβ
2026-09-19 13:17 przegladarka zerwala polaczenie w trakcie analizy. Worker ma tylko 30 s po rozlaczeniu klienta (limit ctx.waitUntil w Cloudflare), potem zostal anulowany, a Gradio zamknelo generator _run_core. Watek wczytujacy skonczyl mimo to (o 13:18:58 "Ingest done"), ale zapis do pamieci i na dysk robil generator dopiero po petli oczekiwania, ktorej juz nikt nie krecil. Transkrypcja i CLIP poszly do kosza, a ponowna proba zaczynalaby od zera. Teraz watek sam zapisuje wynik i sam zwalnia blokade (generator oddaje mu ja w chwili startu). Skutki: - praca przezywa sluchacza; w logach "Cached transcript" pojawia sie zawsze - ponowna proba dla tego samego wideo czeka na blokadzie, az trwajace wczytywanie sie skonczy, i trafia w pamiec podreczna zamiast odpalac drugie rownolegle Sprawdzone na prawdziwym _run_core z atrapami ciezkich modulow: sluchacz odchodzi w trakcie (wynik w pamieci i na dysku, blokada zwolniona po wszystkim), ponowna proba w trakcie (jedno wczytanie, nie dwa), zwβ¦1a0d147
-
Visual-only answers, coreference + on-topic guard, 15-min cacheβ
Malina +1: Claude Opus 4.8- pipeline: drop the "No speech detected" dead-end β when there's no usable transcript, synthesize from the visuals alone (VLM on keyframes), so "what does the person do?" works on silent/music clips. - chat: resolve references to earlier turns ("explain more"); ask to clarify when a reference is ambiguous; gate out off-topic requests and steer back to the uploaded video; route follow-ups to context instead of re-analysis. - app: keep uploaded media 15 min (delete_cache 900) so follow-up turns reuse the already-uploaded video without re-uploading. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>9a43844 -
Stream real per-stage progress markers during analysisβ
Malina +1: Claude Opus 4.8Replace the static "Reading videoβ¦" / "β³ β¦" keepalives with genuine sequential stage markers (subtitles, vocal isolation, OCR, retrieval, re-check) via a thread-safe _Stage holder polled every 2s, so the chat UI can show what NINA is actually doing instead of one static line. All markers stay π¬-prefixed so clients keep them out of the bubble. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>710a0dd -
NINA: conversational chat endpoint (Phase 2)β
Malina +1: Claude Opus 4.8- app.py β gr.Blocks exposing two API endpoints: /run (legacy single-shot, the current tool page) and /chat (new). Keeps delete_cache privacy purge. - chat.py: per-turn router (LLM decides ANALYZE vs ANSWER), context answering from the stored transcript + conversation, manual force_analyze override. Streams [answer, analyzed, transcript]; the website stores the transcript text (no media). - pipeline: split run() into a token-checked wrapper + _run_core so chat can analyze without re-consuming the single-use token; add transcript_text() for storage. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>9510df1 -
NINA: VLM-centric synthesis β the model now actually sees the videoβ
Malina +1: Claude Opus 4.8Pivot the answer engine from CLIP-tags + text-LLM to a vision-language model (Qwen3-VL-8B via HF Inference) that consumes sampled keyframes + the transcript: - frames.py: pick representative timestamps (region midpoints + even context) and extract downscaled base64 keyframes. - synthesis.vlm_synthesize_stream: sends frames + speech transcript + speaker labels to the VLM; reads on-screen text natively, describes who/what is shown, fuses vision with speech. Falls back to the text LLM if the VLM is unavailable. One call per request β no extra inference cost beyond the image tokens. - pipeline: route synthesis through the VLM with keyframes; disable the OCR rung (the VLM reads text far better); add a per-video ingest lock to stop the duplicate concurrent ingestion seen in the logs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>33ec6e1 -
NINA Phase 3b: OCR burned-in captions (RapidOCR) as the last fallback rungβ
Malina +1: Claude Opus 4.8Final rung of the transcript ladder: when the audio stays unreliable, there's no embedded subtitle track, AND CLIP detects on-screen text, OCR the bottom strip of sampled frames and collapse repeated cues into timed segments. - ocr.py: RapidOCR (onnxruntime) over ffmpeg-sampled frames, best-effort. - visual.has_on_screen_text(): CLIP gate so OCR stays off for most videos. - pipeline: post-ingest OCR fallback, rebuilds chunks + re-attaches speakers. - deps: rapidocr-onnxruntime; packages.txt adds libgl1/libglib2.0-0 for OpenCV. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>729fbc4 -
NINA Phase 3a: embedded subtitle fallback (no new deps)β
Malina +1: Claude Opus 4.8When the ASR transcript is unreliable, prefer a clean human subtitle track if the upload carries one (text-based codecs extracted via ffmpeg) before falling back to vocal isolation. subtitles.py parses SRT/ASS/mov_text/WebVTT into segments. Bitmap subs / burned-in captions (OCR) are out of scope here β separate heavier step. Best-effort: any failure leaves the ASR transcript in place. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>f1e4c89 -
NINA Phase 2: vocal isolation + speaker diarization + name attributionβ
Malina +1: Claude Opus 4.8- enhance.py: torchaudio Hybrid Demucs vocal isolation (best-effort, chunked). In ingest, when the first ASR pass is unreliable (musical/distorted), isolate the voice and re-transcribe; keep it only if it scores better. - diarize.py: pyannote speaker diarization β per-segment speaker labels, friendly Speaker 1/2 display names, multilingual self-introduction β name roster, and region speaker annotation. All best-effort (no-op if pyannote/token unavailable). - pipeline: third parallel ingest branch (Whisper β₯ CLIP β₯ diarization); speakers attached after join; region speakers + speaker note threaded to synthesis. - synthesis: fragments tagged with speaker; system prompt attributes lines and uses a self-introduced name for all of that speaker's lines (handles the gamer 'I won' case) β done inside the single existing LLM call (no extra inference credits). - requirements: torchaudio==2.12.0, pyannote.audio==3.3.2. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>e2c96b8 -
NINA: anti-hallucination ASR + honest low-confidence handling (Phase 1)β
Malina +1: Claude Opus 4.8On noisy/musical/distorted audio Whisper fabricates fluent nonsense, and the LLM was reporting it as fact. Now: - transcribe_fast gets hallucination guards (no_speech/log_prob/compression-ratio thresholds, condition_on_previous_text=False) and records no_speech_prob + compression_ratio per segment - transcript_reliability() scores the transcript; an unreliable one (poor mean logprob / mostly weak / mostly no-speech) is flagged - synthesis: when unreliable, the system prompt tells the model the audio is hard to make out and to say so plainly instead of inventing meaning for garbled text Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>6acd603 -
NINA: fully parallel Whisper β₯ CLIP, fused at summaryβ
Malina +1: Claude Opus 4.8- ingest_video: both branches start at t=0 β move the audio decode into the audio branch so the visual (CLIP) branch no longer waits on it; wait for both before continuing - run(): precise re-transcription (Whisper) and visual tagging (CLIP) now run in parallel before synthesis, since they touch independent region fields - synthesis fuses audio + visual only once both branches complete Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>264efa2 -
NINA v0.82 β upload compression, natural multilingual answers, robust preciseβ
Malina +1: Claude Opus 4.8- compress.py + pipeline guard: reject uploads >200 MB, ffmpeg-downscale anything >20 MB to 720p before decode (faster, smaller) - synthesis: detect the prompt language (langdetect + diacritic fallback) and answer in it; natural prose + a final "Sources:" line; auto-sized length - precise re-transcription: loosen logprob gate -0.55 β -0.40 so it actually fires on weak audio; never crash (graceful fallback if the model can't load) - app.py: drop benign ClientDisconnect tracebacks from upload aborts Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>0695bec
-
NINA: SSE keepalive for mobile β prevent stuck 'Uploading & analyzing'β
Malina +1: Claude Sonnet 4.6Run ingest_video() in a background thread and yield 'β³ β¦' every 15 s while waiting. Mobile SSE connections (iOS Safari, carrier proxies) time out after ~60 s of silence; the 1-5 min transcription+CLIP phase was completely silent, permanently hanging mobile clients. Keepalives keep the connection alive without any visible UX change (client silences them). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>3e1b026 -
NINA v0.90 β anti-hallucination pass + DEBUG-gated outputβ
Malina +1: Claude Sonnet 4.6- synthesis: replace static system message with language-aware _make_system_msg(); detect prompt language (Polish/German/French/Spanish/English) via Unicode heuristic and inject the explicit language name β prevents LLM from drifting to wrong language - synthesis: remove weaker in-message "write in the same language" rule (now in system) - synthesis: DEBUG-gate the "N region(s) found" header and "Sources:" footer; production output now shows only the π§ label + LLM answer - config: lower LLM_TEMPERATURE 0.1 β 0.05 for more deterministic output - pipeline: apply cached precise (large-v3-turbo) text to candidates before cross-encoder reranking β improves accuracy on repeat queries for same video Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>45db8e1
-
NINA v0.81 β modular refactor + usage-token gate + 8-vCPU queueβ
Malina +1: Claude Opus 4.7- Split monolith into app.py + pipeline/audio/auth/cache/config/models/retrieval/ synthesis/utils/visual modules. - auth.verify_usage_token: HMAC (NINA_SIGNING_SECRET) + expiry + single-use; pipeline.run() refuses without a valid Worker-issued token (per-account 10/5h limit enforced server-side; the public Space can't be used directly). - run() takes a 3rd `token` input; Interface queue tuned for ~8 vCPU (concurrency 3, max 40); uploaded-media purge kept (delete_cache 5 min). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>05f095d