Spaces:
Sleeping
Sleeping
File size: 3,279 Bytes
264dea1 1f4b05d 264dea1 9ddf856 264dea1 1f4b05d 9ddf856 264dea1 1f4b05d ed4d016 1f4b05d 1667781 1f4b05d 1667781 1f4b05d 9ddf856 1f4b05d 1667781 1f4b05d 1667781 1f4b05d 9ddf856 1f4b05d | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 | ---
title: SceneSolver
emoji: 🕵️
colorFrom: yellow
colorTo: gray
sdk: gradio
sdk_version: 5.49.1
app_file: app.py
python_version: "3.12"
license: mit
short_description: Multimodal surveillance analysis and evidence synthesis
---
# SceneSolver — CrimeSceneSolver (Scene Solver v3)
SceneSolver is a multimodal surveillance-analysis system. This public Space runs the
real nine-stage pipeline from the original notebook, together with the Sherl0ck
agentic escalation layer (Cohere reasoning + LFM2.5-Audio voice) and the full
black/white/orange four-tab frontend (Home · Analyse · Sherl0ck Agent · Contact).
## Pipeline (nine stages, all real models)
0. **AutoEncoder (AE)** — primary anomaly signal (batched reconstruction error)
1. **TimeSformer Binary** — semantic gate (Normal / Suspicious)
2. **TimeSformer Multi-class** — fine-grained anomaly classification (7 classes)
3. **YOLO Weapon Detection** — hard safety trigger (melee / firearms)
4. **Audio Feature Extraction** — ZCR / MFCC variance / spectral flux / RMS / centroid
5. **Tracking & Behavior** — BYTETrack or optical-flow motion analysis
6. **Fusion Layer** — rule-based reasoning core with a full reasoning trace
7. **RL3033** — PPO temporal-reasoning agent (final risk)
8. **LLaVA** — optional 4-frame contextual narrative (checkbox on the Analyse tab)
Outputs: interactive risk report, peak anomaly frame, 11-page forensic PDF dossier,
and structured JSON.
## Sherl0ck Agent
Text chat and hands-free voice call (browser VAD → LFM2.5-Audio ASR → Cohere →
LFM2.5-Audio TTS), incoming-call overlay for HIGH/CRITICAL results, and evidence
dispatch to Telegram / Discord / email when those secrets are configured.
## Model sources (public)
- [`samxkevin/scenesolver-assets`](https://huggingface.co/samxkevin/scenesolver-assets) — AE, TimeSformer, RL3033, YOLO checkpoints + agent audio
- [`samxkevin/SceneDescriptionModel`](https://huggingface.co/samxkevin/SceneDescriptionModel) — LLaVA VLM (~14 GB, downloaded on first use of Stage 8)
- `facebook/timesformer-base-finetuned-k400` — TimeSformer base weights
- `LiquidAI/LFM2.5-Audio-1.5B` — Sherl0ck voice (ASR/TTS)
## Hardware — ZeroGPU
This Space is designed for **ZeroGPU** (select *ZeroGPU* under Settings → Space
hardware; ZeroGPU is not expressible in this README). All GPU-heavy inference runs
inside `@spaces.GPU` regions (full pipeline analysis with dynamic duration, and the
Sherl0ck voice ASR/TTS turns). Python is pinned to 3.12 (a ZeroGPU-supported
runtime).
## Secrets (Settings → Variables and secrets)
All integrations are optional and read from the environment — no credentials are
stored in this repository:
| Secret | Purpose |
| --- | --- |
| `COHERE_API_KEY` | Sherl0ck text/voice reasoning (offline fallback replies when absent) |
| `TELEGRAM_BOT_TOKEN`, `TELEGRAM_CHAT_ID` | Telegram evidence dispatch |
| `DISCORD_WEBHOOK_URL` | Discord evidence dispatch |
| `ALERT_EMAIL_FROM`, `ALERT_EMAIL_PASS`, `ALERT_EMAIL_TO` | Email alerts |
| `SENDER_EMAIL`, `SENDER_PASS`, `ADMIN_EMAIL` | Contact form delivery |
## Disclaimer
Experimental research project. ML outputs are investigative aids, not definitive
conclusions, and require human expert review before any operational action.
|