samxkevin commited on
Commit
1f4b05d
·
verified ·
1 Parent(s): bc04c5f

Upload 4 files

Browse files
Files changed (4) hide show
  1. README.md +57 -10
  2. app.py +0 -0
  3. requirements.txt +31 -13
  4. scenesolver_core.py +146 -966
README.md CHANGED
@@ -1,26 +1,73 @@
1
  ---
2
  title: SceneSolver
3
- author: samxkevin
 
 
4
  sdk: gradio
5
  sdk_version: 5.49.1
6
  app_file: app.py
 
7
  license: mit
8
  short_description: Multimodal surveillance analysis and evidence synthesis
9
  ---
10
 
11
- # SceneSolver
12
 
13
- SceneSolver is a multimodal surveillance-analysis system. The public demo exposes the inference pipeline while the trained model assets remain in the author's Hugging Face repositories.
 
 
 
14
 
15
- ## Pipeline
16
 
17
- AE anomaly signal → TimeSformer binary gate → TimeSformer multi-class classification → YOLO weapon detection → audio features → tracking/behavior extraction → fusion reasoning → RL3033 temporal reasoning → optional LLaVA contextual explanation.
 
 
 
 
 
 
 
 
18
 
19
- ## Model sources
 
20
 
21
- - `samxkevin/scenesolver-assets`
22
- - `samxkevin/SceneDescriptionModel`
23
 
24
- ## Notes
 
 
25
 
26
- The system is an experimental/research project. Results are not definitive conclusions and require human review.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  title: SceneSolver
3
+ emoji: 🕵️
4
+ colorFrom: yellow
5
+ colorTo: gray
6
  sdk: gradio
7
  sdk_version: 5.49.1
8
  app_file: app.py
9
+ python_version: "3.12"
10
  license: mit
11
  short_description: Multimodal surveillance analysis and evidence synthesis
12
  ---
13
 
14
+ # SceneSolver — CrimeSceneSolver (Scene Solver v3)
15
 
16
+ SceneSolver is a multimodal surveillance-analysis system. This public Space runs the
17
+ real nine-stage pipeline from the original notebook, together with the Sherl0ck
18
+ agentic escalation layer (Cohere reasoning + LFM2.5-Audio voice) and the full
19
+ black/white/orange four-tab frontend (Home · Analyse · Sherl0ck Agent · Contact).
20
 
21
+ ## Pipeline (nine stages, all real models)
22
 
23
+ 0. **AutoEncoder (AE)** — primary anomaly signal (batched reconstruction error)
24
+ 1. **TimeSformer Binary** — semantic gate (Normal / Suspicious)
25
+ 2. **TimeSformer Multi-class** — fine-grained anomaly classification (7 classes)
26
+ 3. **YOLO Weapon Detection** — hard safety trigger (melee / firearms)
27
+ 4. **Audio Feature Extraction** — ZCR / MFCC variance / spectral flux / RMS / centroid
28
+ 5. **Tracking & Behavior** — BYTETrack or optical-flow motion analysis
29
+ 6. **Fusion Layer** — rule-based reasoning core with a full reasoning trace
30
+ 7. **RL3033** — PPO temporal-reasoning agent (final risk)
31
+ 8. **LLaVA** — optional 4-frame contextual narrative (checkbox on the Analyse tab)
32
 
33
+ Outputs: interactive risk report, peak anomaly frame, 11-page forensic PDF dossier,
34
+ and structured JSON.
35
 
36
+ ## Sherl0ck Agent
 
37
 
38
+ Text chat and hands-free voice call (browser VAD → LFM2.5-Audio ASR → Cohere →
39
+ LFM2.5-Audio TTS), incoming-call overlay for HIGH/CRITICAL results, and evidence
40
+ dispatch to Telegram / Discord / email when those secrets are configured.
41
 
42
+ ## Model sources (public)
43
+
44
+ - [`samxkevin/scenesolver-assets`](https://huggingface.co/samxkevin/scenesolver-assets) — AE, TimeSformer, RL3033, YOLO checkpoints + agent audio
45
+ - [`samxkevin/SceneDescriptionModel`](https://huggingface.co/samxkevin/SceneDescriptionModel) — LLaVA VLM (~14 GB, downloaded on first use of Stage 8)
46
+ - `facebook/timesformer-base-finetuned-k400` — TimeSformer base weights
47
+ - `LiquidAI/LFM2.5-Audio-1.5B` — Sherl0ck voice (ASR/TTS)
48
+
49
+ ## Hardware — ZeroGPU
50
+
51
+ This Space is designed for **ZeroGPU** (select *ZeroGPU* under Settings → Space
52
+ hardware; ZeroGPU is not expressible in this README). All GPU-heavy inference runs
53
+ inside `@spaces.GPU` regions (full pipeline analysis with dynamic duration, and the
54
+ Sherl0ck voice ASR/TTS turns). Python is pinned to 3.12 (a ZeroGPU-supported
55
+ runtime).
56
+
57
+ ## Secrets (Settings → Variables and secrets)
58
+
59
+ All integrations are optional and read from the environment — no credentials are
60
+ stored in this repository:
61
+
62
+ | Secret | Purpose |
63
+ | --- | --- |
64
+ | `COHERE_API_KEY` | Sherl0ck text/voice reasoning (offline fallback replies when absent) |
65
+ | `TELEGRAM_BOT_TOKEN`, `TELEGRAM_CHAT_ID` | Telegram evidence dispatch |
66
+ | `DISCORD_WEBHOOK_URL` | Discord evidence dispatch |
67
+ | `ALERT_EMAIL_FROM`, `ALERT_EMAIL_PASS`, `ALERT_EMAIL_TO` | Email alerts |
68
+ | `SENDER_EMAIL`, `SENDER_PASS`, `ADMIN_EMAIL` | Contact form delivery |
69
+
70
+ ## Disclaimer
71
+
72
+ Experimental research project. ML outputs are investigative aids, not definitive
73
+ conclusions, and require human expert review before any operational action.
app.py CHANGED
The diff for this file is too large to render. See raw diff
 
requirements.txt CHANGED
@@ -1,20 +1,38 @@
1
- gradio>=5.0,<6.0
 
 
 
 
2
  spaces>=0.38.0
3
  huggingface_hub>=0.34.0
 
 
4
  numpy
5
  opencv-python-headless
6
- torch
7
- torchvision
8
- transformers>=4.45
9
- accelerate
10
- safetensors
11
  Pillow
12
  tqdm
13
  matplotlib
14
- reportlab
15
- stable-baselines3
16
- ultralytics
17
- lap
18
- librosa
19
- openai-whisper
20
- cohere>=5.0.0
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # ── SceneSolver Space dependencies ─────────────────────────────────────────────
2
+ # Gradio is pinned to exactly match sdk_version in README.md (no conflicting
3
+ # requirements anywhere). liquid-audio 1.3+ has no gradio dependency, so the
4
+ # historical gradio==4.44.0 / >=4.44.1 resolver conflict cannot recur.
5
+ gradio==5.49.1
6
  spaces>=0.38.0
7
  huggingface_hub>=0.34.0
8
+
9
+ # ── Core numerics / vision ────────────────────────────────────────────────────
10
  numpy
11
  opencv-python-headless
 
 
 
 
 
12
  Pillow
13
  tqdm
14
  matplotlib
15
+
16
+ # ── Deep learning (torch>=2.8 required by liquid-audio; PyPI linux wheels are
17
+ # CUDA-enabled, which is what ZeroGPU needs) ────────────────────────────────
18
+ torch>=2.8.0
19
+ torchvision>=0.23.0
20
+ torchaudio>=2.8.0
21
+ transformers>=4.55.4
22
+ accelerate>=1.10.1
23
+ safetensors
24
+
25
+ # ── Pipeline stage dependencies ───────────────────────────────────────────────
26
+ stable-baselines3 # Stage 7 — RL3033 (PPO)
27
+ ultralytics # Stage 3/5 — YOLO weapon detection + tracking
28
+ lap # ultralytics BYTETrack association
29
+ librosa>=0.11.0 # Stage 4 — audio feature extraction
30
+ reportlab # forensic PDF dossier
31
+ imageio-ffmpeg # bundled ffmpeg binary (no apt access on Spaces)
32
+
33
+ # ── Sherl0ck agent layer ──────────────────────────────────────────────────────
34
+ cohere>=5.0.0 # Sherl0ck reasoning (command-a-03-2025)
35
+ liquid-audio>=1.3.0 # LFM2.5-Audio-1.5B ASR/TTS bridge
36
+ soundfile
37
+ requests
38
+ anyio # used by the /ssvoice HTTP routes (also a starlette dep)
scenesolver_core.py CHANGED
@@ -1,52 +1,11 @@
1
  # -*- coding: utf-8 -*-
2
- """NewChecker.ipynb
3
-
4
- Automatically generated by Colab.
5
-
6
- Original file is located at
7
- https://colab.research.google.com/drive/
8
- Run the below code to download the Dataset into the Runtime.
9
- ```
10
- # ---- interactive prompt ----------------------------------------------------
11
- print("Kaggle dataset access, choose a mode:")
12
- print(" [1] Download a single video")
13
- print(" [2] Download the whole dataset (~103GB)")
14
- print(" [3] Skip download (use an existing local video)")
15
-
16
- import threading
17
-
18
- choice = {"value": None}
19
-
20
- def get_choice():
21
- choice["value"] = input("Enter 1, 2, or 3: ").strip()
22
-
23
- t = threading.Thread(target=get_choice, daemon=True)
24
- t.start()
25
- t.join(timeout=3)
26
-
27
- if choice["value"] is None:
28
- choice["value"] = "3"
29
- print("\n[Auto] 3 seconds elapsed : defaulting to Skip download.")
30
-
31
- choice = choice["value"]
32
-
33
- KAGGLE_MODE = {
34
- "1": "single_file",
35
- "2": "full",
36
- "3": "skip"
37
- }.get(choice)
38
-
39
- if KAGGLE_MODE is None:
40
- raise ValueError(f"Invalid choice: {choice!r}. Expected 1, 2, or 3.")
41
- # ----------------------------------------------------------------------------
42
- ```
43
  """
 
 
 
 
 
44
 
45
- # !pip install -qq reportlab stable-baselines3 ultralytics lap
46
-
47
- """
48
- Scene Solver v3 : Coherent Reasoning System
49
- ============================================
50
  Stage 0 : AutoEncoder (AE) : PRIMARY anomaly signal (batched)
51
  Stage 1 : TimeSformer Binary : Semantic gate (anomaly/normal)
52
  Stage 2 : TimeSformer Multi-class : Fine-grained anomaly classification
@@ -59,8 +18,22 @@ Stage 8 : LLaVA (EXPERIMENTAL) : Context explanation (multi-frame)
59
 
60
  LOGIC-NORMAL: When Stage 1 = Normal, status is overridden to SAFE,
61
  Stages 2 and 3 are skipped, and LLaVA runs with neutral prompts.
 
 
 
 
 
 
 
 
 
 
 
 
 
62
  """
63
 
 
64
  # ── Standard library ──────────────────────────────────────────────────────────
65
  import json
66
  import os
@@ -137,51 +110,113 @@ log = logging.getLogger("SceneSolver")
137
  # ══════════════════════════════════════════════════════════════════════════════
138
  # CONFIGURATION
139
  # ══════════════════════════════════════════════════════════════════════════════
140
- from huggingface_hub import hf_hub_download,snapshot_download
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
141
  HF_ASSET_REPO = "samxkevin/scenesolver-assets"
142
- AUDIOPATHS = {
143
- "CALLER_TUNE": hf_hub_download(
144
- repo_id=HF_ASSET_REPO,
145
- filename="audio/phone.mp3",
146
- ),
147
- "XTTS_SPEAKER_WAV": hf_hub_download(
148
- repo_id=HF_ASSET_REPO,
149
- filename="audio/DoctorVoice_22050.wav",
150
- ),
151
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
152
 
153
  class Config:
 
 
 
154
  DEVICE = torch.device("cuda" if torch.cuda.is_available() else "cpu")
155
- PATHS = {
156
- "AAE": hf_hub_download(
157
- repo_id=HF_ASSET_REPO,
158
- filename="AAE/best_model.pth",
159
- ),
160
- "AE": hf_hub_download(
161
- repo_id=HF_ASSET_REPO,
162
- filename="AE/best_autoencoder.pth",
163
- ),
164
- "RL_AGENT": hf_hub_download(
165
- repo_id=HF_ASSET_REPO,
166
- filename="RL/best_model.zip",
167
- ),
168
- "TS_BINARY": hf_hub_download(
169
- repo_id=HF_ASSET_REPO,
170
- filename="TFB/timesformer_best_binary.pth",
171
- ),
172
- "TS_MULTI": hf_hub_download(
173
- repo_id=HF_ASSET_REPO,
174
- filename="TFM/best_model.pth",
175
- ),
176
- "YOLO": hf_hub_download(
177
- repo_id=HF_ASSET_REPO,
178
- filename="YOLO/weapon_yolov8n.pt",
179
- ),
180
- "LLAVA": snapshot_download(
181
- repo_id="samxkevin/SceneDescriptionModel"
182
- ),
183
- }
184
- print("[HF] All SceneSolver assets are available.")
185
  SAFE_THRESHOLD_FLOOR = 0.030
186
  SAFE_THRESHOLD_LEGACY = 0.055
187
  AUDIO_ANOMALY_THRESHOLD = 0.1
@@ -1381,6 +1416,16 @@ class Stage8LLaVA:
1381
  self.model = None
1382
  return self
1383
 
 
 
 
 
 
 
 
 
 
 
1384
  def _build_prompt(self, role: str, frame_number: int, yolo_summary: str,
1385
  motion_behaviors: List[str], audio_status: str,
1386
  audio_score: float, binary_result: str) -> str:
@@ -2875,6 +2920,13 @@ class SceneSolverPipeline:
2875
  self.alert_system = AlertSystem(self.config)
2876
  self.reporter = EnhancedReportGenerator()
2877
 
 
 
 
 
 
 
 
2878
  def _make_output_dir(self, video_path: str, base: str) -> str:
2879
  name = os.path.splitext(os.path.basename(video_path))[0]
2880
  ts = datetime.now().strftime("%Y%m%d_%H%M%S")
@@ -3013,6 +3065,8 @@ class SceneSolverPipeline:
3013
 
3014
  self._save_outputs(results, out)
3015
  self._print_summary(results)
 
 
3016
  return results
3017
 
3018
  def _save_outputs(self, results: dict, out_dir: str):
@@ -3143,6 +3197,7 @@ class SceneSolverPipeline:
3143
  return stream
3144
 
3145
 
 
3146
  # ══════════════════════════════════════════════════════════════════════════════
3147
  # READY MESSAGE
3148
  # ══════════════════════════════════════════════════════════════════════════════
@@ -3159,886 +3214,11 @@ print(" pipeline = SceneSolverPipeline()")
3159
  print(" results = pipeline.analyze('/path/to/video.mp4')")
3160
  print("=" * 65)
3161
 
3162
- #from google.colab import drive
3163
- #drive.mount('/content/drive')
3164
-
3165
- # pipeline instantiated lazily by the Space app
3166
- # results = pipeline.analyze('/content/drive/MyDrive/solver1/AnomalyDetectionDataset/Explosion/Explosion001_x264.mp4')
3167
- # Without LLaVA (faster):
3168
- # results = pipeline.analyze('/path/to/video.mp4', enable_llava=False)
3169
-
3170
- # !pip install -q gradio
3171
 
3172
  # ══════════════════════════════════════════════════════════════════════════════
3173
- # GRADIO WEB APP : CrimeSceneSolver (refactored)
3174
- #
3175
- # Run AFTER Cell 2 (pipeline) and Cell 4 (gradio install).
3176
- # Agent Escalation Layer below this cell for the full experience.
3177
  # ══════════════════════════════════════════════════════════════════════════════
 
 
 
3178
 
3179
- import gradio as gr
3180
- import numpy as np
3181
- import json, os, traceback, smtplib, ssl, re, base64
3182
- from email.mime.text import MIMEText
3183
- from email.mime.multipart import MIMEMultipart
3184
- from PIL import Image
3185
-
3186
- # ─────────────────────────────────────────────────────────────────────────────
3187
- # SHARED CONSTANTS (used by both the web app and the agent layer)
3188
- # ─────────────────────────────────────────────────────────────────────────────
3189
- ADMIN_EMAIL = "garoldwerner2512@gmail.com"
3190
- SMTP_SERVER = "smtp.gmail.com"
3191
- SMTP_PORT = 465
3192
- SENDER_EMAIL = os.environ.get("SENDER_EMAIL", "")
3193
- SENDER_PASS = os.environ.get("SENDER_PASS", "")
3194
-
3195
- # Design tokens — pitch black / white / orange-saffron
3196
- C_BG = "#000000"
3197
- C_ORANGE = "#FF8C00"
3198
- C_RED = "#FF4500"
3199
- C_GREEN = "#4CAF50"
3200
-
3201
- # Shared HTML blocks
3202
- NAV_HTML = """
3203
- <div style="background:#000;border-bottom:1px solid #1a1a1a;
3204
- padding:16px 36px;display:flex;justify-content:space-between;align-items:center;">
3205
- <span style="font-family:'Roboto Mono',monospace;font-size:.92rem;
3206
- letter-spacing:.15em;color:#fff;text-transform:uppercase;
3207
- font-weight:700;-webkit-font-smoothing:antialiased;">
3208
- CrimeSceneSolver
3209
- </span>
3210
- <span style="font-family:'Roboto Mono',monospace;font-size:.6rem;
3211
- letter-spacing:.1em;color:#555;text-transform:uppercase;">
3212
- Scene Solver v3
3213
- </span>
3214
- </div>
3215
- """
3216
-
3217
- FOOTER_HTML = """
3218
- <div style="border-top:1px solid #0d0d0d;margin-top:16px;">
3219
- <div style="background:#0d0d0d;border-bottom:1px solid #1a1a1a;padding:14px 32px;">
3220
- <div style="max-width:860px;margin:0 auto;display:flex;align-items:flex-start;gap:12px;">
3221
- <svg style="flex-shrink:0;margin-top:2px;" width="14" height="14" viewBox="0 0 24 24"
3222
- fill="none" stroke="#FF8C00" stroke-width="2">
3223
- <circle cx="12" cy="12" r="10"/>
3224
- <line x1="12" y1="8" x2="12" y2="12"/>
3225
- <line x1="12" y1="16" x2="12.01" y2="16"/>
3226
- </svg>
3227
- <p style="font-family:'Roboto Mono',monospace;font-size:.62rem;
3228
- letter-spacing:.04em;color:#888;line-height:1.7;margin:0;font-weight:500;">
3229
- <span style="color:#FF8C00;font-weight:700;">DISCLAIMER &mdash; </span>
3230
- This report is produced by machine learning models and should be treated as an
3231
- investigative aid, not a definitive conclusion. Human expert review is strongly
3232
- recommended before any operational action is taken.
3233
- </p>
3234
- </div>
3235
- </div>
3236
- <div style="text-align:center;padding:18px;font-family:'Roboto Mono',monospace;
3237
- font-size:.6rem;letter-spacing:.1em;color:#444;text-transform:uppercase;">
3238
- CrimeSceneSolver &nbsp;&middot;&nbsp; Scene Solver v3 &nbsp;&middot;&nbsp;
3239
- Confidential &nbsp;&middot;&nbsp; ML outputs require expert review
3240
- </div>
3241
- </div>
3242
- """
3243
-
3244
-
3245
- def sec_label(text: str) -> str:
3246
- """Shared section-label helper used across the web app and agent layer."""
3247
- return (f"<div style='font-family:\"Roboto Mono\",monospace;font-size:.62rem;"
3248
- f"letter-spacing:.14em;text-transform:uppercase;color:#ddd;"
3249
- f"margin-bottom:10px;font-weight:800;-webkit-font-smoothing:antialiased;"
3250
- f"border-left:3px solid #FF8C00;padding-left:10px;'>{text}</div>")
3251
-
3252
-
3253
- def risk_color(r: float) -> str:
3254
- if r >= 0.70: return C_RED
3255
- if r >= 0.40: return C_ORANGE
3256
- return C_GREEN
3257
-
3258
-
3259
- # ─────────────────────────────────────────────────────────────────────────────
3260
- # EMAIL
3261
- # ─────────────────────────────────────────────────────────────────────────────
3262
- def send_contact_email(name: str, email: str, phone: str, message: str) -> bool:
3263
- if not SENDER_EMAIL or not SENDER_PASS:
3264
- with open("/tmp/css_contact.log", "a") as f:
3265
- f.write(f"[CONTACT] {name} | {email} | {phone}\n")
3266
- return False
3267
- try:
3268
- msg = MIMEMultipart("alternative")
3269
- msg["Subject"] = f"[CrimeSceneSolver] Contact: {name}"
3270
- msg["From"] = SENDER_EMAIL
3271
- msg["To"] = ADMIN_EMAIL
3272
- msg.attach(MIMEText(f"Name:{name}\nEmail:{email}\nPhone:{phone}\nMessage:\n{message}\n", "plain"))
3273
- ctx = ssl.create_default_context()
3274
- with smtplib.SMTP_SSL(SMTP_SERVER, SMTP_PORT, context=ctx) as srv:
3275
- srv.login(SENDER_EMAIL, SENDER_PASS)
3276
- srv.sendmail(SENDER_EMAIL, ADMIN_EMAIL, msg.as_string())
3277
- return True
3278
- except Exception:
3279
- return False
3280
-
3281
-
3282
- # ─────────────────────────────────────────────────────────────────────────────
3283
- # PIPELINE WRAPPER
3284
- # ─────────────────────────────────────────────────────────────────────────────
3285
- def run_pipeline(video_path: str) -> dict:
3286
- # pipeline instantiated lazily by the Space app
3287
- results = pipeline.analyze(
3288
- video_path=video_path, base_output_dir="/tmp/css_webapp",
3289
- enable_llava=True, enable_rl=True, enable_tracking=True,
3290
- )
3291
- s0 = results.get("stage0") or {}
3292
- s1 = results.get("stage1") or {}
3293
- s2 = results.get("stage2") or {}
3294
- s3 = results.get("stage3") or {}
3295
- s4 = results.get("stage4") or {}
3296
- s5 = results.get("stage5") or {}
3297
- s6 = results.get("stage6") or {}
3298
- s7 = results.get("stage7") or {}
3299
- s8 = results.get("stage8") or {}
3300
- fps = float(s0.get("fps", 25.0))
3301
-
3302
- def ts(fi): return frame_to_timestamp(int(fi), fps)
3303
-
3304
- risk_score = float(s7.get("final_risk", s6.get("fused_risk", 0.0)))
3305
- rl_status = s7.get("status", "")
3306
- _rl_map = {"CRITICAL":"CRITICAL","HIGH_RISK":"HIGH","MODERATE":"MODERATE","LOW_RISK":"LOW"}
3307
- risk_label = (_rl_map.get(rl_status, "") or
3308
- results.get("risk_label_final") or
3309
- s6.get("risk_level", "LOW"))
3310
- status = results.get("status", "UNKNOWN")
3311
- narrative = (s8 or {}).get("unified_narrative") or (s8 or {}).get("description") or ""
3312
-
3313
- report = {
3314
- "overall_status": status,
3315
- "final_risk_score": f"{risk_score:.1%}",
3316
- "risk_level": risk_label,
3317
- "dynamic_threshold": f"{results.get('threshold',0):.6f}",
3318
- "ae_max_error": f"{s0.get('max_error',0):.6f}",
3319
- "ae_avg_error": f"{s0.get('avg_error',0):.6f}",
3320
- "peak_timestamp": ts(s0.get("peak_frame_idx",0)),
3321
- "anomaly_window": f"{ts(s0.get('anomaly_start_frame',0))} \u2192 {ts(s0.get('anomaly_end_frame',0))}",
3322
- "stage1_tf_binary": f"{s1.get('classification','?')} ({s1.get('confidence',0):.1%})",
3323
- "stage2_anomaly_type": f"{s2.get('class','N/A')} ({s2.get('confidence',0):.1%})",
3324
- "stage2_top3": s2.get("top3", []),
3325
- "stage2_all_probs": s2.get("all_probabilities", {}),
3326
- "stage3_weapon": ("YES — " + str(s3.get("dominant_class","?"))
3327
- if s3.get("weapon_flag") else "None detected"),
3328
- "stage4_audio": f"{s4.get('status','N/A')} (score: {s4.get('audio_score',0):.4f})",
3329
- "stage5_motion": f"{s5.get('motion_intensity',0):.3f} | {', '.join(s5.get('behaviors',[]) or ['STATIC'])}",
3330
- "stage6_fusion_risk": f"{s6.get('fused_risk',0):.1%} [{s6.get('risk_level','?')}]",
3331
- "stage6_reasoning": s6.get("reasoning_trace", []),
3332
- "stage7_rl_final": f"{s7.get('final_risk',0):.1%} [{rl_status}]",
3333
- "stage7_interpretation": s7.get("interpretation",""),
3334
- "llava_narrative": narrative,
3335
- # raw values for charts
3336
- "_stage1_conf": float(s1.get("confidence", 0)),
3337
- "_stage2_conf": float(s2.get("confidence", 0)),
3338
- "_stage2_top3_raw": s2.get("top3", []),
3339
- "_stage2_all_probs": s2.get("all_probabilities", {}),
3340
- "_risk_score": risk_score,
3341
- "_fused_risk": float(s6.get("fused_risk", 0)),
3342
- "_audio_score": float(s4.get("audio_score", 0)),
3343
- "_motion": float(s5.get("motion_intensity", 0)),
3344
- }
3345
-
3346
- frames_l = s0.get("frames", [])
3347
- scores_l = s0.get("scores", [])
3348
- samp_idx = s0.get("sampled_indices", [])
3349
- top_frame = None
3350
- if frames_l and scores_l:
3351
- psi = int(s0.get("peak_score_idx", 0))
3352
- pfi = min(samp_idx[psi] if psi < len(samp_idx) else 0, len(frames_l) - 1)
3353
- top_frame = frames_l[pfi]
3354
-
3355
- def _serial(o):
3356
- if isinstance(o, np.ndarray): return o.tolist()
3357
- if isinstance(o, np.integer): return int(o)
3358
- if isinstance(o, np.floating): return float(o)
3359
- return str(o)
3360
-
3361
- jd = {
3362
- "status": status, "risk_score": round(risk_score, 4), "risk_level": risk_label,
3363
- "threshold": round(float(results.get("threshold", 0)), 6),
3364
- "stage0": {"max_error": round(float(s0.get("max_error",0)),6),
3365
- "avg_error": round(float(s0.get("avg_error",0)),6),
3366
- "peak_timestamp": ts(s0.get("peak_frame_idx",0)),
3367
- "anomaly_start": ts(s0.get("anomaly_start_frame",0)),
3368
- "anomaly_end": ts(s0.get("anomaly_end_frame",0))},
3369
- "stage1": {"classification": s1.get("classification","?"),
3370
- "confidence": round(float(s1.get("confidence",0)),4)},
3371
- "stage2": {"class": s2.get("class","N/A"),
3372
- "confidence": round(float(s2.get("confidence",0)),4),
3373
- "top3": s2.get("top3",[])},
3374
- "stage3": {"weapon_flag": bool(s3.get("weapon_flag",False)),
3375
- "dominant_class": s3.get("dominant_class","None"),
3376
- "weapon_count": int(s3.get("weapon_count",0))},
3377
- "stage4": {"status": s4.get("status","N/A"),
3378
- "audio_score": round(float(s4.get("audio_score",0)),4)},
3379
- "stage5": {"motion_intensity": round(float(s5.get("motion_intensity",0)),4),
3380
- "behaviors": s5.get("behaviors",[])},
3381
- "stage6": {"fused_risk": round(float(s6.get("fused_risk",0)),4),
3382
- "risk_level": s6.get("risk_level","?"),
3383
- "reasoning": s6.get("reasoning_trace",[])},
3384
- "stage7": {"final_risk": round(risk_score,4), "status": rl_status,
3385
- "interpretation": s7.get("interpretation","")},
3386
- "llava_narrative": narrative,
3387
- }
3388
-
3389
- pdf = (results.get("pdf_path_local") or results.get("pdf_path")
3390
- or "/tmp/forensic_report_v3.pdf")
3391
- if not os.path.exists(pdf): pdf = None
3392
-
3393
- graph_path = results.get("graph_path", "/tmp/deviation_graph_v3.png")
3394
- if not os.path.exists(graph_path): graph_path = None
3395
-
3396
- return {"risk_score": risk_score, "risk_label": risk_label, "status": status,
3397
- "report": report, "top_frame": top_frame, "json_data": jd,
3398
- "pdf_path": pdf, "graph_path": graph_path}
3399
-
3400
-
3401
- # ─────────────────────────────────────────────────────────────────────────────
3402
- # HTML HELPERS
3403
- # ─────────────────────────────────────────────────────────────────────────────
3404
- def img_to_data_uri(path: str) -> str:
3405
- """Encode an image file as a base64 data-URI for inline HTML embedding."""
3406
- if not path or not os.path.exists(path):
3407
- return ""
3408
- ext = os.path.splitext(path)[1].lower().lstrip(".")
3409
- mime = {"png":"image/png","jpg":"image/jpeg","jpeg":"image/jpeg"}.get(ext,"image/png")
3410
- with open(path, "rb") as f:
3411
- encoded = base64.b64encode(f.read()).decode("utf-8")
3412
- return f"data:{mime};base64,{encoded}"
3413
-
3414
-
3415
- def spinner_html() -> str:
3416
- return """
3417
- <div style="text-align:center;padding:18px 0;">
3418
- <div style="display:inline-block;width:40px;height:40px;
3419
- border:3px solid rgba(255,255,255,0.07);border-top-color:#FF8C00;
3420
- border-right-color:rgba(255,140,0,0.28);border-radius:50%;
3421
- animation:arc-spin .75s cubic-bezier(0.6,0.2,0.4,0.8) infinite;"></div>
3422
- <div style="font-family:'Roboto Mono',monospace;font-size:.64rem;
3423
- letter-spacing:.12em;color:#FF8C00;margin-top:11px;text-transform:uppercase;font-weight:600;">
3424
- Processing&hellip; this may take several minutes
3425
- </div>
3426
- </div>"""
3427
-
3428
-
3429
- def no_result_html() -> str:
3430
- return ("<div style='text-align:center;padding:60px 20px;color:#333;"
3431
- "font-family:Charter,serif;font-size:1rem;letter-spacing:.04em;'>"
3432
- "Upload footage and press ANALYSE to begin.</div>")
3433
-
3434
-
3435
- # ─────────────────────────────────────────────────────────────────────────────
3436
- # REPORT HTML BUILDER
3437
- # ─────────────────────────────────────────────────────────────────────────────
3438
- def build_report_html(report: dict, risk_score: float, risk_label: str,
3439
- status: str, graph_path: str = None) -> str:
3440
- col = risk_color(risk_score)
3441
-
3442
- def numfmt(v: str) -> str:
3443
- return re.sub(r'([\d]+\.?[\d]*%?)',
3444
- r"<span style='font-family:\"Times New Roman\",Times,serif;'>\1</span>", v)
3445
-
3446
- def row(label: str, value: str, mono: bool = False) -> str:
3447
- vd = numfmt(str(value))
3448
- vfont = "font-family:'Roboto Mono',monospace;font-size:.8rem;" if mono else "font-size:.82rem;"
3449
- return (f"<tr>"
3450
- f"<td style='padding:9px 14px;color:#aaa;font-size:.78rem;"
3451
- f"border-bottom:1px solid #222;white-space:nowrap;width:38%;'>{label}</td>"
3452
- f"<td style='padding:9px 14px;color:#eee;{vfont}border-bottom:1px solid #222;'>{vd}</td>"
3453
- f"</tr>")
3454
-
3455
- def section_hdr(title: str) -> str:
3456
- return (f"<div style='font-size:.65rem;letter-spacing:.16em;text-transform:uppercase;"
3457
- f"color:#FF8C00;padding:14px 0 6px 2px;border-bottom:2px solid #FF8C00;"
3458
- f"margin-bottom:0;font-family:\"Roboto Mono\",monospace;font-weight:800;'>{title}</div>")
3459
-
3460
- def section(title: str, rows_html: str, extra: str = "") -> str:
3461
- return (f"<div style='margin-bottom:26px;'>"
3462
- + section_hdr(title)
3463
- + f"<table style='width:100%;border-collapse:collapse;'>{rows_html}</table>"
3464
- + extra + "</div>")
3465
-
3466
- # Risk badge
3467
- is_crit = (status == "CRITICAL")
3468
- badge_text = f"{'ANOMALY DETECTED' if is_crit else 'NO ANOMALY DETECTED'} ({status})"
3469
- badge = (f"<div style='text-align:center;padding:28px 20px;margin-bottom:28px;"
3470
- f"background:{col};border-radius:4px;'>"
3471
- f"<div style='font-size:3.8rem;font-weight:700;color:#fff;"
3472
- f"font-family:\"Times New Roman\",Times,serif;line-height:1;'>"
3473
- f"{risk_score:.1%}</div>"
3474
- f"<div style='color:rgba(255,255,255,.85);font-size:.75rem;margin-top:10px;"
3475
- f"letter-spacing:.14em;text-transform:uppercase;font-family:\"Roboto Mono\",monospace;"
3476
- f"font-weight:600;'>{badge_text} &nbsp;|&nbsp; {risk_label} RISK</div>"
3477
- f"</div>")
3478
-
3479
- # Raw chart values
3480
- stage1_conf = float(report.get("_stage1_conf", 0))
3481
- stage2_conf = float(report.get("_stage2_conf", 0))
3482
- top3_raw = report.get("_stage2_top3_raw", []) or []
3483
- all_probs = report.get("_stage2_all_probs", {}) or {}
3484
- fused_risk = float(report.get("_fused_risk", risk_score))
3485
- audio_score = float(report.get("_audio_score", 0))
3486
- motion = float(report.get("_motion", 0))
3487
- PIE_COLORS = ["#c0392b","#2471a3","#d35400","#6c3483","#117a65","#1a5276","#7d6608"]
3488
-
3489
- # ── Donut pie (SVG) ──────────────────────────────────────────────────────
3490
- def donut_pie_svg(probs: dict) -> str:
3491
- if not probs: return ""
3492
- import math
3493
- items = sorted(probs.items(), key=lambda x: x[1], reverse=True)
3494
- labels = [i[0] for i in items]
3495
- sizes = [max(float(i[1]), 1e-6) for i in items]
3496
- total = sum(sizes)
3497
- cx = cy = R = 90; R = 70; r = 36
3498
- wedges = ""
3499
- start = -90
3500
- for idx, (label, val) in enumerate(zip(labels, sizes)):
3501
- sweep = (val / total) * 360
3502
- end = start + sweep
3503
- x1o, y1o = cx + R * math.cos(math.radians(start)), cy + R * math.sin(math.radians(start))
3504
- x2o, y2o = cx + R * math.cos(math.radians(end)), cy + R * math.sin(math.radians(end))
3505
- x1i, y1i = cx + r * math.cos(math.radians(end)), cy + r * math.sin(math.radians(end))
3506
- x2i, y2i = cx + r * math.cos(math.radians(start)), cy + r * math.sin(math.radians(start))
3507
- lg = 1 if sweep > 180 else 0
3508
- fill = PIE_COLORS[idx % len(PIE_COLORS)]
3509
- mid_a = start + sweep / 2
3510
- lx = cx + (R + r) / 2 * math.cos(math.radians(mid_a))
3511
- ly = cy + (R + r) / 2 * math.sin(math.radians(mid_a))
3512
- num_lbl = (f"<text x='{lx:.1f}' y='{ly:.1f}' text-anchor='middle' "
3513
- f"dominant-baseline='middle' fill='#fff' font-size='9' font-weight='bold'>{idx+1}</text>"
3514
- if sweep > 15 else "")
3515
- wedges += (f"<path d='M {x1o:.2f} {y1o:.2f} A {R} {R} 0 {lg} 1 {x2o:.2f} {y2o:.2f} "
3516
- f"L {x1i:.2f} {y1i:.2f} A {r} {r} 0 {lg} 0 {x2i:.2f} {y2i:.2f} Z' "
3517
- f"fill='{fill}' stroke='#000' stroke-width='1'/>{num_lbl}")
3518
- start = end
3519
- legend = "".join(
3520
- f"<div style='display:flex;align-items:center;gap:7px;margin-bottom:5px;'>"
3521
- f"<div style='width:11px;height:11px;min-width:11px;background:{PIE_COLORS[i%len(PIE_COLORS)]};border-radius:2px;'></div>"
3522
- f"<span style='font-family:\"Roboto Mono\",monospace;font-size:.62rem;color:#ccc;'>"
3523
- f"{i+1}. {cls}&nbsp;<span style='font-family:\"Times New Roman\",Times,serif;color:#fff;font-weight:700;'>"
3524
- f"({v/total*100:.1f}%)</span></span></div>"
3525
- for i, (cls, v) in enumerate(zip(labels, sizes))
3526
- )
3527
- svg = (f"<svg width='180' height='180' viewBox='0 0 180 180'>{wedges}"
3528
- f"<circle cx='{cx}' cy='{cy}' r='{r-2}' fill='#050505'/>"
3529
- f"<text x='{cx}' y='{cy}' text-anchor='middle' dominant-baseline='middle' "
3530
- f"fill='#666' font-size='8'>ANOMALY</text>"
3531
- f"<text x='{cx}' y='{cy+11}' text-anchor='middle' dominant-baseline='middle' "
3532
- f"fill='#666' font-size='8'>CLASSES</text></svg>")
3533
- return (f"<div style='display:flex;gap:16px;align-items:flex-start;padding:16px;"
3534
- f"background:#050505;border:1px solid #1e1e1e;border-radius:3px;margin-top:6px;'>"
3535
- f"<div style='flex-shrink:0;'>{svg}</div><div style='flex:1;'>{legend}</div></div>")
3536
-
3537
- # ── Horizontal bar chart ──────────────────────────────────────────────────
3538
- def conf_bar_chart(top3: list) -> str:
3539
- if not top3: return ""
3540
- BAR_COLORS = ["#c0392b","#d35400","#5d6d7e"]
3541
- rows = "".join(
3542
- f"<div style='margin-bottom:10px;'>"
3543
- f"<div style='display:flex;justify-content:space-between;align-items:center;margin-bottom:4px;'>"
3544
- f"<span style='font-family:\"Roboto Mono\",monospace;font-size:.68rem;color:#ccc;'>{cls}</span>"
3545
- f"<span style='font-family:\"Times New Roman\",Times,serif;color:#fff;font-weight:700;font-size:.82rem;margin-left:8px;'>{prob*100:.1f}%</span>"
3546
- f"</div><div style='background:#1a1a1a;border-radius:2px;height:10px;overflow:hidden;'>"
3547
- f"<div style='background:{BAR_COLORS[idx]};height:10px;width:{prob*100:.1f}%;border-radius:2px;'></div>"
3548
- f"</div></div>"
3549
- for idx, (cls, prob) in enumerate(top3[:3])
3550
- )
3551
- return (f"<div style='padding:14px 16px;background:#050505;border:1px solid #1e1e1e;"
3552
- f"border-radius:3px;margin-top:6px;'>"
3553
- f"<div style='font-family:\"Roboto Mono\",monospace;font-size:.6rem;"
3554
- f"letter-spacing:.14em;color:#FF8C00;text-transform:uppercase;margin-bottom:12px;'>"
3555
- f"Top-3 Anomaly Predictions</div>{rows}</div>")
3556
-
3557
- # ── Signal gauge bars ─────────────────────────────────────────────────────
3558
- def risk_gauge(val: float, label: str, color: str) -> str:
3559
- pct = min(100, int(val * 100))
3560
- return (f"<div style='margin-bottom:10px;'>"
3561
- f"<div style='display:flex;justify-content:space-between;align-items:center;margin-bottom:3px;'>"
3562
- f"<span style='font-family:\"Roboto Mono\",monospace;font-size:.65rem;color:#aaa;'>{label}</span>"
3563
- f"<span style='font-family:\"Times New Roman\",Times,serif;color:#fff;font-weight:700;font-size:.8rem;'>{val:.1%}</span>"
3564
- f"</div><div style='background:#1a1a1a;border-radius:2px;height:8px;overflow:hidden;'>"
3565
- f"<div style='background:{color};height:8px;width:{pct}%;border-radius:2px;'></div>"
3566
- f"</div></div>")
3567
-
3568
- # ── Small donut pies for Stage 1 / Stage 2 ───────────────────────────────
3569
- def small_donut(pct: float, color: str, label: str, size: int = 84) -> str:
3570
- import math
3571
- cx = cy = size // 2
3572
- R = cx - 8
3573
- r = R - 12
3574
- sweep = pct * 360
3575
- end_a = -90 + sweep
3576
- x1, y1 = cx + R*math.cos(math.radians(-90)), cy + R*math.sin(math.radians(-90))
3577
- x2, y2 = cx + R*math.cos(math.radians(end_a)), cy + R*math.sin(math.radians(end_a))
3578
- xi1, yi1 = cx + r*math.cos(math.radians(end_a)), cy + r*math.sin(math.radians(end_a))
3579
- xi2, yi2 = cx + r*math.cos(math.radians(-90)), cy + r*math.sin(math.radians(-90))
3580
- lg = 1 if sweep > 180 else 0
3581
- if pct >= 0.999:
3582
- arc = (f"<circle cx='{cx}' cy='{cy}' r='{R}' fill='none' stroke='{color}' stroke-width='12'/>"
3583
- f"<circle cx='{cx}' cy='{cy}' r='{r}' fill='#050505'/>")
3584
- else:
3585
- arc = (f"<circle cx='{cx}' cy='{cy}' r='{R}' fill='none' stroke='#1a1a1a' stroke-width='12'/>"
3586
- f"<path d='M {x1:.2f} {y1:.2f} A {R} {R} 0 {lg} 1 {x2:.2f} {y2:.2f} "
3587
- f"L {xi1:.2f} {yi1:.2f} A {r} {r} 0 {lg} 0 {xi2:.2f} {yi2:.2f} Z' fill='{color}'/>"
3588
- f"<circle cx='{cx}' cy='{cy}' r='{r}' fill='#050505'/>")
3589
- return (f"<div style='text-align:center;'>"
3590
- f"<svg width='{size}' height='{size}' viewBox='0 0 {size} {size}'>"
3591
- f"{arc}"
3592
- f"<text x='{cx}' y='{cy}' text-anchor='middle' dominant-baseline='middle' "
3593
- f"fill='#fff' font-family='Times New Roman,Times,serif' font-size='11' font-weight='700'>"
3594
- f"{pct:.0%}</text></svg>"
3595
- f"<div style='font-family:\"Roboto Mono\",monospace;font-size:.55rem;color:#666;margin-top:2px;'>"
3596
- f"{label}</div></div>")
3597
-
3598
- charts_cls = (
3599
- f"<div style='display:flex;gap:20px;padding:14px 16px;background:#050505;"
3600
- f"border:1px solid #1e1e1e;border-radius:3px;margin-top:6px;align-items:flex-start;flex-wrap:wrap;'>"
3601
- + small_donut(stage1_conf, '#FF8C00', 'TF BINARY · Stage 1')
3602
- + small_donut(stage2_conf, '#c0392b', 'ANOMALY · Stage 2')
3603
- + f"<div style='flex:1;min-width:200px;'>{conf_bar_chart(top3_raw)}</div>"
3604
- + (f"<div style='width:100%;margin-top:10px;'>{donut_pie_svg(all_probs)}</div>" if all_probs else "")
3605
- + "</div>"
3606
- )
3607
- charts_sensing = (
3608
- f"<div style='padding:14px 16px;background:#050505;border:1px solid #1e1e1e;"
3609
- f"border-radius:3px;margin-top:6px;'>"
3610
- f"<div style='font-family:\"Roboto Mono\",monospace;font-size:.6rem;letter-spacing:.14em;"
3611
- f"color:#FF8C00;text-transform:uppercase;margin-bottom:12px;'>Signal Intensities</div>"
3612
- + risk_gauge(fused_risk, "Fusion Risk (Stage 6)", risk_color(fused_risk))
3613
- + risk_gauge(audio_score, "Audio Score (Stage 4)", "#2471a3")
3614
- + risk_gauge(min(motion, 1.0), "Motion Intensity (Stage 5)", "#117a65")
3615
- + "</div>"
3616
- )
3617
-
3618
- # ── Table rows ────────────────────────────────────────────────────────────
3619
- ae_rows = (row("Overall Status", report.get("overall_status",""))
3620
- + row("Dynamic Threshold", report.get("dynamic_threshold",""), mono=True)
3621
- + row("AE Max Error", report.get("ae_max_error",""), mono=True)
3622
- + row("AE Avg Error", report.get("ae_avg_error",""), mono=True)
3623
- + row("Peak Timestamp", report.get("peak_timestamp",""), mono=True)
3624
- + row("Anomaly Window", report.get("anomaly_window",""), mono=True))
3625
-
3626
- cl_rows = (row("TF Binary (Stage 1)", report.get("stage1_tf_binary",""))
3627
- + row("Anomaly Type (Stage 2)", report.get("stage2_anomaly_type","")))
3628
- for i, (cls, prob) in enumerate(report.get("stage2_top3",[]), 1):
3629
- cl_rows += row(f" Top-{i}", f"{cls} ({prob:.2%})", mono=True)
3630
-
3631
- det_rows = (row("Weapon Detection (Stage 3)", report.get("stage3_weapon",""))
3632
- + row("Audio Status (Stage 4)", report.get("stage4_audio",""))
3633
- + row("Motion (Stage 5)", report.get("stage5_motion","")))
3634
-
3635
- fus_rows = row("Fusion Risk (Stage 6)", report.get("stage6_fusion_risk",""))
3636
- for i, line in enumerate(report.get("stage6_reasoning",[]), 1):
3637
- fus_rows += row(f" Step {i}", line)
3638
-
3639
- rl_rows = (row("RL3033 Final Risk (Stage 7)", report.get("stage7_rl_final",""), mono=True)
3640
- + row("Interpretation", report.get("stage7_interpretation","")))
3641
-
3642
- narr = report.get("llava_narrative","")
3643
- narr_html = (
3644
- f"<div style='margin-bottom:26px;'>"
3645
- + section_hdr("AI Visual Narrative (LLaVA — Stage 8)")
3646
- + f"<div style='color:#ccc;font-size:.85rem;line-height:1.78;padding:16px 18px;"
3647
- f"background:#050505;border:1px solid #1e1e1e;border-radius:3px;margin-top:6px;'>{narr}</div>"
3648
- + "</div>"
3649
- ) if narr else ""
3650
-
3651
- ae_graph_html = ""
3652
- if graph_path:
3653
- data_uri = img_to_data_uri(graph_path)
3654
- if data_uri:
3655
- ae_graph_html = (
3656
- f"<div style='margin-bottom:26px;'>"
3657
- + section_hdr("AE Deviation Graph — Temporal Reconstruction Error")
3658
- + f"<div style='padding:14px;background:#050505;border:1px solid #1e1e1e;"
3659
- f"border-radius:3px;margin-top:6px;'>"
3660
- f"<img src='{data_uri}' alt='AE Deviation Graph' "
3661
- f"style='width:100%;border-radius:2px;display:block;'/>"
3662
- f"<div style='font-family:\"Roboto Mono\",monospace;font-size:.58rem;color:#555;"
3663
- f"margin-top:8px;text-align:center;'>"
3664
- f"Red = reconstruction error &nbsp;&middot;&nbsp; Green dashed = dynamic threshold "
3665
- f"&nbsp;&middot;&nbsp; Star = peak anomaly frame"
3666
- f"</div></div></div>"
3667
- )
3668
-
3669
- return (
3670
- f"<div style='background:#0a0a0a;padding:28px 26px;border:1px solid #1e1e1e;border-radius:4px;'>"
3671
- + badge + ae_graph_html
3672
- + section("Autoencoder — Stage 0", ae_rows)
3673
- + section("Classification — Stages 1 & 2", cl_rows, charts_cls)
3674
- + section("Detection & Sensing — Stages 3-5", det_rows, charts_sensing)
3675
- + section("Fusion Layer — Stage 6", fus_rows)
3676
- + section("RL3033 Temporal Reasoning — Stage 7", rl_rows)
3677
- + narr_html
3678
- + "</div>"
3679
- )
3680
-
3681
-
3682
- def process_video(video):
3683
- if video is None:
3684
- return (no_result_html(), None, None, None)
3685
- try:
3686
- out = run_pipeline(video)
3687
- r = out["risk_score"]
3688
- report_html = build_report_html(out["report"], r, out["risk_label"],
3689
- out["status"], graph_path=out.get("graph_path"))
3690
- fi = Image.fromarray(out["top_frame"]) if out["top_frame"] is not None else None
3691
- return (report_html, fi, json.dumps(out["json_data"], indent=2, default=str), out["pdf_path"])
3692
- except Exception:
3693
- tb = traceback.format_exc()
3694
- return (f"<div style='color:#FF4500;font-family:\"Roboto Mono\",monospace;"
3695
- f"font-size:.8rem;padding:20px;background:#050505;'>Analysis failed:\n\n{tb}</div>",
3696
- None, None, None)
3697
-
3698
-
3699
- def handle_contact(name: str, email: str, phone: str, msg: str) -> str:
3700
- """Shared contact form handler — used in both the base app and agent layer tabs."""
3701
- if not name.strip() or not email.strip() or not msg.strip():
3702
- return ("<div style='font-family:\"Roboto Mono\",monospace;font-size:.72rem;"
3703
- "color:#FF4500;padding:10px 0;font-weight:600;'>"
3704
- "Name, email, and message are required.</div>")
3705
- if not re.match(r"^[^@]+@[^@]+\.[^@]+$", email.strip()):
3706
- return ("<div style='font-family:\"Roboto Mono\",monospace;font-size:.72rem;"
3707
- "color:#FF4500;padding:10px 0;font-weight:600;'>"
3708
- "Please enter a valid email address.</div>")
3709
- send_contact_email(name.strip(), email.strip(), phone.strip(), msg.strip())
3710
- return ("<div style='font-family:\"Roboto Mono\",monospace;font-size:.72rem;"
3711
- "color:#4CAF50;padding:10px 0;letter-spacing:.06em;font-weight:600;'>"
3712
- "&#10003; Message received. We will respond at the provided email.</div>")
3713
-
3714
-
3715
- # ─────────────────────────────────────────────────────────────────────────────
3716
- # CSS — pitch black / white / orange palette
3717
- # ─────────────────────────────────────────────────────────────────────────────
3718
- APP_CSS = """
3719
- @import url('https://fonts.googleapis.com/css2?family=Roboto+Mono:wght@300;400;500;600;700&display=swap');
3720
-
3721
- :root {
3722
- --bg: #000000;
3723
- --surface:#080808;
3724
- --border: #1e1e1e;
3725
- --white: #FFFFFF;
3726
- --dim: #888888;
3727
- --orange: #FF8C00;
3728
- --red: #FF4500;
3729
- --mono: 'Roboto Mono', monospace;
3730
- --serif: Charter, 'Bitstream Charter', Georgia, serif;
3731
- }
3732
-
3733
- *, *::before, *::after { box-sizing:border-box; margin:0; padding:0; }
3734
- html, body, .gradio-container, .gradio-container > * {
3735
- background:var(--bg) !important; color:var(--white) !important;
3736
- font-family:var(--serif) !important;
3737
- }
3738
- .gradio-container {
3739
- zoom: 1.3 !important;
3740
- }
3741
- * { -webkit-font-smoothing:antialiased; }
3742
-
3743
- ::-webkit-scrollbar { width:4px; height:4px; }
3744
- ::-webkit-scrollbar-track { background:var(--bg); }
3745
- ::-webkit-scrollbar-thumb { background:#333; border-radius:2px; }
3746
-
3747
- @keyframes arc-spin {
3748
- from { transform:rotate(0deg); } to { transform:rotate(360deg); }
3749
- }
3750
-
3751
- /* TABS */
3752
- .tab-nav {
3753
- background:var(--bg) !important; border-bottom:1px solid var(--border) !important;
3754
- padding:0 24px !important; display:flex !important; gap:0 !important;
3755
- align-items:center !important; justify-content:flex-start !important;
3756
- }
3757
- .tab-nav button {
3758
- font-family:var(--mono) !important; font-size:.72rem !important;
3759
- letter-spacing:.13em !important; color:#999 !important;
3760
- background:transparent !important; border:none !important;
3761
- border-bottom:2px solid transparent !important; padding:14px 0 !important;
3762
- min-width:160px !important; width:160px !important; text-align:center !important;
3763
- text-transform:uppercase !important; transition:color .18s, border-color .18s !important;
3764
- font-weight:600 !important; white-space:nowrap !important; flex:0 0 160px !important;
3765
- }
3766
- .tab-nav button:hover { color:#ddd !important; }
3767
- .tab-nav button.selected {
3768
- color:#FFFFFF !important; border-bottom-color:var(--orange) !important; font-weight:700 !important;
3769
- }
3770
-
3771
- /* GRADIO INTERNALS */
3772
- .gradio-container .block, .gradio-container .panel,
3773
- .gradio-container .form, .gradio-container fieldset,
3774
- .gradio-container .wrap, .gradio-container .container,
3775
- .gradio-container > div { background:var(--bg) !important; border:none !important; }
3776
-
3777
- /* INPUTS */
3778
- input, textarea, select {
3779
- background:#0d0d0d !important; border:1px solid #2a2a2a !important;
3780
- border-radius:3px !important; color:#f0f0f0 !important;
3781
- font-family:var(--serif) !important; font-size:.88rem !important;
3782
- padding:12px 14px !important; transition:border-color .18s !important; width:100% !important;
3783
- }
3784
- input:focus, textarea:focus {
3785
- border-color:var(--orange) !important; outline:none !important;
3786
- box-shadow:0 0 0 1px var(--orange) !important;
3787
- }
3788
- label, .label-wrap span {
3789
- font-family:var(--mono) !important; font-size:.65rem !important;
3790
- letter-spacing:.1em !important; text-transform:uppercase !important;
3791
- color:#ccc !important; font-weight:600 !important;
3792
- }
3793
-
3794
- /* BUTTONS */
3795
- .gr-button {
3796
- background:var(--white) !important; color:var(--bg) !important;
3797
- border-radius:3px !important; font-family:var(--mono) !important;
3798
- font-size:.72rem !important; letter-spacing:.12em !important; font-weight:700 !important;
3799
- }
3800
- .gr-button:hover { background:var(--orange) !important; }
3801
- #ana-btn, #ana-btn button {
3802
- width:auto !important; max-width:260px !important;
3803
- min-height:44px !important; height:44px !important; padding:0 30px !important;
3804
- }
3805
-
3806
- /* VIDEO UPLOAD */
3807
- #vid-up { min-height:150px !important; max-height:200px !important; }
3808
- #vid-up .upload-container, #vid-up .wrap {
3809
- background:#060606 !important; border:1px dashed #2a2a2a !important;
3810
- border-radius:3px !important; min-height:150px !important; max-height:200px !important;
3811
- }
3812
- #vid-up .wrap:hover { border-color:var(--orange) !important; }
3813
-
3814
- /* OUTPUT AREAS */
3815
- #report-html { background:var(--bg) !important; }
3816
- #kf-img, #kf-img img { border-radius:3px !important; border:1px solid var(--border) !important; width:100% !important; }
3817
- #json-out, #json-out textarea {
3818
- background:#060606 !important; color:#bbb !important;
3819
- font-family:var(--mono) !important; font-size:.75rem !important;
3820
- line-height:1.7 !important; border:1px solid var(--border) !important; border-radius:3px !important;
3821
- }
3822
- #pdf-out { border:1px solid var(--border) !important; border-radius:3px !important; }
3823
- #pdf-out .file-preview { background:#060606 !important; }
3824
-
3825
- .rule { border:none; border-top:1px solid #111; margin:28px 0; }
3826
- .contact-card {
3827
- background:#080808; border:1px solid #1e1e1e; border-radius:4px; padding:28px 28px 20px;
3828
- }
3829
- .contact-info-block {
3830
- background:#060606; border:1px solid #222; border-radius:3px; padding:18px 20px; margin-top:16px;
3831
- }
3832
- footer { display:none !important; }
3833
- """
3834
-
3835
- HOME_HTML = """
3836
- <div style="max-width:900px;margin:0 auto;padding:0 28px 72px;">
3837
-
3838
- <!-- HERO -->
3839
- <div style="padding:72px 0 56px;border-bottom:1px solid #111;">
3840
- <div style="font-family:'Roboto Mono',monospace;font-size:.62rem;letter-spacing:.18em;
3841
- color:#FF8C00;text-transform:uppercase;margin-bottom:16px;font-weight:700;">
3842
- Forensic AI Platform &nbsp;&middot;&nbsp; Scene Solver v3
3843
- </div>
3844
- <h1 style="font-family:Charter,Georgia,serif;font-size:3.6rem;font-weight:400;
3845
- color:#fff;line-height:1.08;margin-bottom:22px;letter-spacing:-.01em;">
3846
- Uncover the Fact.
3847
- </h1>
3848
- <p style="font-family:Charter,Georgia,serif;font-size:1rem;color:#666;
3849
- max-width:560px;line-height:1.78;margin-bottom:40px;">
3850
- CrimeSceneSolver is an end-to-end forensic intelligence platform.
3851
- Upload CCTV footage and a nine-stage AI pipeline analyses it, then
3852
- Sherl𝟬ck, the agentic escalation layer, briefs you by voice or text,
3853
- dispatches evidence to all channels, and generates a full PDF dossier.
3854
- </p>
3855
- <button onclick="(function(){
3856
- var b=document.querySelectorAll('.tab-nav button,[role=tab]');
3857
- for(var i=0;i<b.length;i++){var t=(b[i].textContent||'').trim().toLowerCase();
3858
- if(t.indexOf('analyse')!==-1){b[i].click();return;}}
3859
- if(b.length>1)b[1].click();
3860
- })()"
3861
- style="background:#fff;color:#000;font-family:'Roboto Mono',monospace;font-size:.74rem;
3862
- letter-spacing:.14em;text-transform:uppercase;padding:14px 36px;border-radius:3px;
3863
- border:none;cursor:pointer;font-weight:700;transition:background .2s;"
3864
- onmouseover="this.style.background='#FF8C00'" onmouseout="this.style.background='#fff'">
3865
- Analyse Footage &rarr;
3866
- </button>
3867
- </div>
3868
-
3869
- <!-- NINE-STAGE PIPELINE -->
3870
- <div style="padding:56px 0 48px;border-bottom:1px solid #111;">
3871
- <div style="font-family:'Roboto Mono',monospace;font-size:.62rem;letter-spacing:.16em;
3872
- color:#FF8C00;text-transform:uppercase;margin-bottom:28px;font-weight:700;">
3873
- Nine-Stage Forensic Pipeline
3874
- </div>
3875
- <div style="display:grid;grid-template-columns:repeat(3,1fr);gap:10px;">
3876
- <div style="padding:20px 18px;background:#080808;border:1px solid #1e1e1e;border-radius:3px;">
3877
- <div style="font-family:'Roboto Mono',monospace;font-size:.55rem;color:#FF8C00;margin-bottom:6px;font-weight:700;">STAGE 0</div>
3878
- <div style="font-family:Charter,Georgia,serif;font-size:.9rem;color:#fff;margin-bottom:6px;">Autoencoder (AE)</div>
3879
- <div style="font-family:Charter,Georgia,serif;font-size:.76rem;color:#555;line-height:1.6;">Primary anomaly signal. Reconstruction error over a dynamic threshold triggers the pipeline.</div>
3880
- </div>
3881
- <div style="padding:20px 18px;background:#080808;border:1px solid #1e1e1e;border-radius:3px;">
3882
- <div style="font-family:'Roboto Mono',monospace;font-size:.55rem;color:#FF8C00;margin-bottom:6px;font-weight:700;">STAGE 1</div>
3883
- <div style="font-family:Charter,Georgia,serif;font-size:.9rem;color:#fff;margin-bottom:6px;">TimeSformer Binary</div>
3884
- <div style="font-family:Charter,Georgia,serif;font-size:.76rem;color:#555;line-height:1.6;">Semantic gate: classifies the clip as Normal or Suspicious. Normal overrides the AE signal to SAFE.</div>
3885
- </div>
3886
- <div style="padding:20px 18px;background:#080808;border:1px solid #1e1e1e;border-radius:3px;">
3887
- <div style="font-family:'Roboto Mono',monospace;font-size:.55rem;color:#FF8C00;margin-bottom:6px;font-weight:700;">STAGE 2</div>
3888
- <div style="font-family:Charter,Georgia,serif;font-size:.9rem;color:#fff;margin-bottom:6px;">TimeSformer Multi-class</div>
3889
- <div style="font-family:Charter,Georgia,serif;font-size:.76rem;color:#555;line-height:1.6;">Fine-grained anomaly type across 7 categories using AE-weighted temporal sampling.</div>
3890
- </div>
3891
- <div style="padding:20px 18px;background:#080808;border:1px solid #1e1e1e;border-radius:3px;">
3892
- <div style="font-family:'Roboto Mono',monospace;font-size:.55rem;color:#FF8C00;margin-bottom:6px;font-weight:700;">STAGE 3</div>
3893
- <div style="font-family:Charter,Georgia,serif;font-size:.9rem;color:#fff;margin-bottom:6px;">YOLO Weapon Detection</div>
3894
- <div style="font-family:Charter,Georgia,serif;font-size:.76rem;color:#555;line-height:1.6;">Hard safety trigger. Melee and firearm detection within the anomaly window. Forces 0.90 risk floor.</div>
3895
- </div>
3896
- <div style="padding:20px 18px;background:#080808;border:1px solid #1e1e1e;border-radius:3px;">
3897
- <div style="font-family:'Roboto Mono',monospace;font-size:.55rem;color:#FF8C00;margin-bottom:6px;font-weight:700;">STAGE 4</div>
3898
- <div style="font-family:Charter,Georgia,serif;font-size:.9rem;color:#fff;margin-bottom:6px;">Audio Feature Extraction</div>
3899
- <div style="font-family:Charter,Georgia,serif;font-size:.76rem;color:#555;line-height:1.6;">ZCR, MFCC variance, spectral flux, RMS, and centroid scored against global thresholds.</div>
3900
- </div>
3901
- <div style="padding:20px 18px;background:#080808;border:1px solid #1e1e1e;border-radius:3px;">
3902
- <div style="font-family:'Roboto Mono',monospace;font-size:.55rem;color:#FF8C00;margin-bottom:6px;font-weight:700;">STAGE 5</div>
3903
- <div style="font-family:Charter,Georgia,serif;font-size:.9rem;color:#fff;margin-bottom:6px;">Tracking &amp; Motion</div>
3904
- <div style="font-family:Charter,Georgia,serif;font-size:.76rem;color:#555;line-height:1.6;">YOLO tracking with Farneback optical flow fallback. Detects FAST_MOVEMENT, CLUSTERING, STATIC.</div>
3905
- </div>
3906
- <div style="padding:20px 18px;background:#080808;border:1px solid #1e1e1e;border-radius:3px;">
3907
- <div style="font-family:'Roboto Mono',monospace;font-size:.55rem;color:#FF8C00;margin-bottom:6px;font-weight:700;">STAGE 6</div>
3908
- <div style="font-family:Charter,Georgia,serif;font-size:.9rem;color:#fff;margin-bottom:6px;">Fusion Layer</div>
3909
- <div style="font-family:Charter,Georgia,serif;font-size:.76rem;color:#555;line-height:1.6;">Explainable rule-based reasoning combining all stage outputs into a single risk score with a full trace.</div>
3910
- </div>
3911
- <div style="padding:20px 18px;background:#080808;border:1px solid #1e1e1e;border-radius:3px;">
3912
- <div style="font-family:'Roboto Mono',monospace;font-size:.55rem;color:#FF8C00;margin-bottom:6px;font-weight:700;">STAGE 7</div>
3913
- <div style="font-family:Charter,Georgia,serif;font-size:.9rem;color:#fff;margin-bottom:6px;">RL3033 Agent</div>
3914
- <div style="font-family:Charter,Georgia,serif;font-size:.76rem;color:#555;line-height:1.6;">PPO reinforcement learning agent. Final risk blends TFB, TFM, Fusion, and RL scores <span style="font-family:'Times New Roman',Times,serif;">(70/20/5/5%)</span>.</div>
3915
- </div>
3916
- <div style="padding:20px 18px;background:#080808;border:1px solid #1e1e1e;border-radius:3px;">
3917
- <div style="font-family:'Roboto Mono',monospace;font-size:.55rem;color:#FF8C00;margin-bottom:6px;font-weight:700;">STAGE 8</div>
3918
- <div style="font-family:Charter,Georgia,serif;font-size:.9rem;color:#fff;margin-bottom:6px;">LLaVA Narrative</div>
3919
- <div style="font-family:Charter,Georgia,serif;font-size:.76rem;color:#555;line-height:1.6;">Four keyframes described in natural language and synthesised into a coherent scene narrative.</div>
3920
- </div>
3921
- </div>
3922
- </div>
3923
-
3924
- <!-- SHERL0CK AGENT -->
3925
- <div style="padding:56px 0 48px;border-bottom:1px solid #111;">
3926
- <div style="font-family:'Roboto Mono',monospace;font-size:.62rem;letter-spacing:.16em;
3927
- color:#FF8C00;text-transform:uppercase;margin-bottom:28px;font-weight:700;">
3928
- Sherl𝟬ck — Agentic Escalation Layer
3929
- </div>
3930
- <div style="display:grid;grid-template-columns:1fr 1fr;gap:20px;">
3931
- <div style="padding:26px 22px;background:#080808;border:1px solid #1e1e1e;border-radius:3px;">
3932
- <div style="font-family:Charter,Georgia,serif;font-size:.95rem;color:#fff;margin-bottom:8px;">Voice Call Interface</div>
3933
- <div style="font-family:Charter,Georgia,serif;font-size:.8rem;color:#555;line-height:1.7;">
3934
- When a CRITICAL or HIGH risk is detected, Sherl𝟬ck initiates an incoming call.
3935
- Accept it to speak directly with the agent, Whisper transcribes your speech,
3936
- Cohere generates an analytical response, and XTTS v2 delivers it as a voice reply.
3937
- Server-side VAD handles turn detection automatically.
3938
- </div>
3939
- </div>
3940
- <div style="padding:26px 22px;background:#080808;border:1px solid #1e1e1e;border-radius:3px;">
3941
- <div style="font-family:Charter,Georgia,serif;font-size:.95rem;color:#fff;margin-bottom:8px;">Text Chat Interface</div>
3942
- <div style="font-family:Charter,Georgia,serif;font-size:.8rem;color:#555;line-height:1.7;">
3943
- A persistent text chat powered by Cohere command-a-05-2026 with the full pipeline
3944
- JSON pre-loaded as context. Ask about any stage output, request reasoning traces,
3945
- or say <em style="color:#FF8C00;font-style:normal;">"send report"</em> to dispatch evidence.
3946
- </div>
3947
- </div>
3948
- <div style="padding:26px 22px;background:#080808;border:1px solid #1e1e1e;border-radius:3px;">
3949
- <div style="font-family:Charter,Georgia,serif;font-size:.95rem;color:#fff;margin-bottom:8px;">Evidence Dispatch</div>
3950
- <div style="font-family:Charter,Georgia,serif;font-size:.8rem;color:#555;line-height:1.7;">
3951
- One click dispatches the full forensic dossier simultaneously to Telegram,
3952
- Discord, and Email, including the peak anomaly frame and the 11-page PDF report.
3953
- CRITICAL alerts auto-dispatch without user action.
3954
- </div>
3955
- </div>
3956
- <div style="padding:26px 22px;background:#080808;border:1px solid #1e1e1e;border-radius:3px;">
3957
- <div style="font-family:Charter,Georgia,serif;font-size:.95rem;color:#fff;margin-bottom:8px;">11-Page PDF Dossier</div>
3958
- <div style="font-family:Charter,Georgia,serif;font-size:.8rem;color:#555;line-height:1.7;">
3959
- Auto-generated PDF covering every stage: AE temporal graph, classification charts,
3960
- weapon detections with bounding boxes, audio feature breakdown, motion tracking,
3961
- fusion reasoning trace, RL3033 observation space, anomalous frames, and LLaVA narrative.
3962
- </div>
3963
- </div>
3964
- </div>
3965
- </div>
3966
-
3967
- <!-- WORKFLOW -->
3968
- <div style="padding:56px 0 48px;border-bottom:1px solid #111;">
3969
- <div style="font-family:'Roboto Mono',monospace;font-size:.62rem;letter-spacing:.16em;
3970
- color:#FF8C00;text-transform:uppercase;margin-bottom:28px;font-weight:700;">End-to-End Workflow</div>
3971
- <div style="display:grid;grid-template-columns:repeat(4,1fr);gap:24px;">
3972
- <div>
3973
- <div style="font-family:'Roboto Mono',monospace;font-size:.6rem;color:#333;margin-bottom:8px;font-weight:700;">01</div>
3974
- <div style="font-family:Charter,Georgia,serif;font-size:.92rem;color:#fff;margin-bottom:6px;">Upload Footage</div>
3975
- <div style="font-family:Charter,Georgia,serif;font-size:.78rem;color:#555;line-height:1.65;">Drop CCTV footage on the Analyse tab and press Analyse Footage.</div>
3976
- </div>
3977
- <div>
3978
- <div style="font-family:'Roboto Mono',monospace;font-size:.6rem;color:#333;margin-bottom:8px;font-weight:700;">02</div>
3979
- <div style="font-family:Charter,Georgia,serif;font-size:.92rem;color:#fff;margin-bottom:6px;">Pipeline Runs</div>
3980
- <div style="font-family:Charter,Georgia,serif;font-size:.78rem;color:#555;line-height:1.65;">All nine stages execute sequentially. Report, frame, JSON, and PDF are produced.</div>
3981
- </div>
3982
- <div>
3983
- <div style="font-family:'Roboto Mono',monospace;font-size:.6rem;color:#333;margin-bottom:8px;font-weight:700;">03</div>
3984
- <div style="font-family:Charter,Georgia,serif;font-size:.92rem;color:#fff;margin-bottom:6px;">Sherl0ck Calls</div>
3985
- <div style="font-family:Charter,Georgia,serif;font-size:.78rem;color:#555;line-height:1.65;">On HIGH or CRITICAL risk, an incoming call overlay appears. Accept to begin the briefing.</div>
3986
- </div>
3987
- <div>
3988
- <div style="font-family:'Roboto Mono',monospace;font-size:.6rem;color:#333;margin-bottom:8px;font-weight:700;">04</div>
3989
- <div style="font-family:Charter,Georgia,serif;font-size:.92rem;color:#fff;margin-bottom:6px;">Dispatch &amp; Close</div>
3990
- <div style="font-family:Charter,Georgia,serif;font-size:.78rem;color:#555;line-height:1.65;">Dispatch evidence to all channels, download the PDF dossier, and end the session.</div>
3991
- </div>
3992
- </div>
3993
- </div>
3994
-
3995
- <!-- PRICING -->
3996
- <div style="padding:56px 0 48px;border-bottom:1px solid #111;">
3997
- <div style="font-family:'Roboto Mono',monospace;font-size:.62rem;letter-spacing:.16em;
3998
- color:#FF8C00;text-transform:uppercase;margin-bottom:28px;font-weight:700;">Pricing</div>
3999
- <div style="display:grid;grid-template-columns:repeat(3,1fr);gap:12px;">
4000
- <div style="padding:26px 22px;background:#080808;border:1px solid #1e1e1e;border-radius:3px;">
4001
- <div style="font-family:'Roboto Mono',monospace;font-size:.62rem;color:#555;margin-bottom:12px;font-weight:700;">BASIC</div>
4002
- <div style="font-size:2rem;color:#fff;margin-bottom:14px;font-family:'Times New Roman',Times,serif;">Free</div>
4003
- <div style="font-family:Charter,Georgia,serif;font-size:.8rem;color:#555;line-height:2.1;">
4004
- 9-stage pipeline<br>5 analyses / month<br>PDF dossier<br>JSON output
4005
- </div>
4006
- </div>
4007
- <div style="padding:26px 22px;background:#080808;border:1px solid #FF8C00;border-radius:3px;">
4008
- <div style="font-family:'Roboto Mono',monospace;font-size:.62rem;color:#FF8C00;margin-bottom:12px;font-weight:700;">PRO</div>
4009
- <div style="font-size:2rem;color:#fff;margin-bottom:14px;font-family:'Times New Roman',Times,serif;">&#8377;49/mo</div>
4010
- <div style="font-family:Charter,Georgia,serif;font-size:.8rem;color:#888;line-height:2.1;">
4011
- Unlimited analyses<br>Sherl0ck voice &amp; text agent<br>Auto-dispatch (Telegram, Discord, Email)<br>LLaVA narratives<br>Priority support
4012
- </div>
4013
- </div>
4014
- <div style="padding:26px 22px;background:#080808;border:1px solid #1e1e1e;border-radius:3px;">
4015
- <div style="font-family:'Roboto Mono',monospace;font-size:.62rem;color:#555;margin-bottom:12px;font-weight:700;">ENTERPRISE</div>
4016
- <div style="font-size:2rem;color:#fff;margin-bottom:14px;font-family:'Times New Roman',Times,serif;">Custom</div>
4017
- <div style="font-family:Charter,Georgia,serif;font-size:.8rem;color:#555;line-height:2.1;">
4018
- Custom model fine-tuning<br>On-premise deployment<br>Dedicated SLA<br>API access
4019
- </div>
4020
- </div>
4021
- </div>
4022
- </div>
4023
-
4024
- <!-- TESTIMONIAL -->
4025
- <div style="padding:56px 0 24px;">
4026
- <div style="font-family:'Roboto Mono',monospace;font-size:.62rem;letter-spacing:.16em;
4027
- color:#FF8C00;text-transform:uppercase;margin-bottom:26px;font-weight:700;">From the Field</div>
4028
- <blockquote style="font-family:Charter,Georgia,serif;font-size:1.08rem;color:#aaa;line-height:1.76;
4029
- border-left:2px solid #FF8C00;padding-left:22px;font-style:italic;margin:0 0 14px;">
4030
- &ldquo;We closed a complex six-month investigation in under two weeks after deploying
4031
- SceneSolver. The nine-stage pipeline, Sherl0ck voice briefings, and automated
4032
- evidence dispatch changed how our forensics team operates entirely.&rdquo;
4033
- </blockquote>
4034
- <div style="font-family:'Roboto Mono',monospace;font-size:.66rem;color:#555;font-weight:500;">
4035
- &mdash; Detective Lara Monroe, Forensics Division
4036
- </div>
4037
- </div>
4038
-
4039
- </div>
4040
- """
4041
-
4042
-
4043
- # ─────────────────────────────────────────────────────────────────────────────
4044
- # BUILD APP
 
1
  # -*- coding: utf-8 -*-
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2
  """
3
+ scenesolver_core.py — Scene Solver v3 : Coherent Reasoning System
4
+ =================================================================
5
+ Faithful extraction of the nine-stage SceneSolver pipeline from the
6
+ authoritative Colab notebook (NewChecker.ipynb), packaged for the
7
+ CrimeSceneSolver Hugging Face Space.
8
 
 
 
 
 
 
9
  Stage 0 : AutoEncoder (AE) : PRIMARY anomaly signal (batched)
10
  Stage 1 : TimeSformer Binary : Semantic gate (anomaly/normal)
11
  Stage 2 : TimeSformer Multi-class : Fine-grained anomaly classification
 
18
 
19
  LOGIC-NORMAL: When Stage 1 = Normal, status is overridden to SAFE,
20
  Stages 2 and 3 are skipped, and LLaVA runs with neutral prompts.
21
+
22
+ Model assets (public repositories):
23
+ - samxkevin/scenesolver-assets (AE / TFB / TFM / RL / YOLO / audio)
24
+ - samxkevin/SceneDescriptionModel (LLaVA VLM)
25
+
26
+ DEPLOYMENT DIFFERENCES vs the notebook (compatibility only — no pipeline
27
+ logic changes):
28
+ 1. Checkpoint downloads are lazy (resolved on first use, cached) instead
29
+ of running inside the Config class body at import time.
30
+ 2. An ffmpeg binary shim (imageio-ffmpeg) is placed on PATH when the
31
+ system ffmpeg is missing (HF Gradio Spaces have no apt access).
32
+ 3. Stage 8 (LLaVA, ~14 GB) is released after each analysis so its
33
+ weights do not stay resident between runs on constrained hosts.
34
  """
35
 
36
+
37
  # ── Standard library ──────────────────────────────────────────────────────────
38
  import json
39
  import os
 
110
  # ══════════════════════════════════════════════════════════════════════════════
111
  # CONFIGURATION
112
  # ══════════════════════════════════════════════════════════════════════════════
113
+
114
+ # ── Deployment fix: guarantee an ffmpeg binary on PATH ────────────────────────
115
+ # Gradio SDK Spaces cannot apt-install; imageio-ffmpeg ships a static binary.
116
+ def _ensure_ffmpeg_on_path() -> None:
117
+ import tempfile as _tempfile
118
+ if shutil.which("ffmpeg"):
119
+ return
120
+ try:
121
+ import imageio_ffmpeg
122
+ exe = imageio_ffmpeg.get_ffmpeg_exe()
123
+ bindir = os.path.join(_tempfile.gettempdir(), "scenesolver_bin")
124
+ os.makedirs(bindir, exist_ok=True)
125
+ link = os.path.join(bindir, "ffmpeg")
126
+ if not os.path.exists(link):
127
+ try:
128
+ os.symlink(exe, link)
129
+ except OSError:
130
+ shutil.copy2(exe, link)
131
+ os.environ["PATH"] = bindir + os.pathsep + os.environ.get("PATH", "")
132
+ log.info(f"[FFmpeg] Using bundled binary: {exe}")
133
+ except Exception as e:
134
+ log.warning(f"[FFmpeg] Bundled binary unavailable: {e}")
135
+
136
+
137
+ _ensure_ffmpeg_on_path()
138
+
139
+
140
+ from huggingface_hub import hf_hub_download, snapshot_download
141
+ from functools import lru_cache
142
+
143
  HF_ASSET_REPO = "samxkevin/scenesolver-assets"
144
+ LLAVA_REPO = "samxkevin/SceneDescriptionModel"
145
+
146
+ # Deployment fix: the notebook resolved every checkpoint with hf_hub_download /
147
+ # snapshot_download inside the class body at import time. On a Space that
148
+ # blocks startup on multi-GB downloads, so each asset is now resolved lazily
149
+ # on first access and cached (identical paths, identical repositories).
150
+ _asset_cache: Dict[str, str] = {}
151
+ _asset_lock = threading.Lock()
152
+
153
+
154
+ def _hf_asset(filename: str) -> str:
155
+ with _asset_lock:
156
+ if filename not in _asset_cache:
157
+ log.info(f"[HF] Fetching asset: {HF_ASSET_REPO}/{filename}")
158
+ _asset_cache[filename] = hf_hub_download(
159
+ repo_id=HF_ASSET_REPO, filename=filename)
160
+ return _asset_cache[filename]
161
+
162
+
163
+ def _hf_llava_snapshot() -> str:
164
+ with _asset_lock:
165
+ if "__LLAVA__" not in _asset_cache:
166
+ log.info(f"[HF] Fetching VLM snapshot: {LLAVA_REPO} (~14 GB, first use only)")
167
+ _asset_cache["__LLAVA__"] = snapshot_download(repo_id=LLAVA_REPO)
168
+ return _asset_cache["__LLAVA__"]
169
+
170
+
171
+ class _LazyAssets(dict):
172
+ """dict-compatible mapping whose values download on first access."""
173
+
174
+ def __init__(self, resolvers: Dict[str, Any]):
175
+ super().__init__()
176
+ self._resolvers = resolvers
177
+ self._resolved: Dict[str, str] = {}
178
+
179
+ def __getitem__(self, key: str) -> str:
180
+ if key not in self._resolved:
181
+ self._resolved[key] = self._resolvers[key]()
182
+ return self._resolved[key]
183
+
184
+ def get(self, key: str, default: Any = None) -> Any:
185
+ if key not in self._resolvers:
186
+ return default
187
+ try:
188
+ return self[key]
189
+ except Exception as e:
190
+ log.warning(f"[HF] Asset '{key}' unavailable: {e}")
191
+ return default
192
+
193
+ def __contains__(self, key: object) -> bool:
194
+ return key in self._resolvers
195
+
196
+
197
+ AUDIOPATHS = _LazyAssets({
198
+ "CALLER_TUNE": lambda: _hf_asset("audio/phone.mp3"),
199
+ "XTTS_SPEAKER_WAV": lambda: _hf_asset("audio/DoctorVoice_22050.wav"),
200
+ })
201
+
202
 
203
  class Config:
204
+ # NOTE: `import spaces` (done first in app.py) keeps torch.cuda.is_available()
205
+ # True on ZeroGPU via CUDA emulation, so DEVICE resolves to cuda there and
206
+ # to cpu on CPU-only hosts — the notebook's device idiom is preserved.
207
  DEVICE = torch.device("cuda" if torch.cuda.is_available() else "cpu")
208
+ PATHS = _LazyAssets({
209
+ "AAE": lambda: _hf_asset("AAE/best_model.pth"),
210
+ "AE": lambda: _hf_asset("AE/best_autoencoder.pth"),
211
+ "RL_AGENT": lambda: _hf_asset("RL/best_model.zip"),
212
+ "TS_BINARY": lambda: _hf_asset("TFB/timesformer_best_binary.pth"),
213
+ "TS_MULTI": lambda: _hf_asset("TFM/best_model.pth"),
214
+ "YOLO": lambda: _hf_asset("YOLO/weapon_yolov8n.pt"),
215
+ "LLAVA": _hf_llava_snapshot,
216
+ })
217
+ DRIVE_ROOT = "/tmp/scenesolver"
218
+ log.info("[HF] SceneSolver asset registry ready (lazy download on first use).")
219
+
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
220
  SAFE_THRESHOLD_FLOOR = 0.030
221
  SAFE_THRESHOLD_LEGACY = 0.055
222
  AUDIO_ANOMALY_THRESHOLD = 0.1
 
1416
  self.model = None
1417
  return self
1418
 
1419
+ def unload(self):
1420
+ """Deployment fix: release the ~14 GB VLM after each analysis so its
1421
+ weights do not remain resident between runs on Space hardware."""
1422
+ if self.model is not None:
1423
+ del self.model
1424
+ self.model = None
1425
+ self.processor = None
1426
+ cleanup_memory()
1427
+ log.info("[Stage8-LLaVA] Model released.")
1428
+
1429
  def _build_prompt(self, role: str, frame_number: int, yolo_summary: str,
1430
  motion_behaviors: List[str], audio_status: str,
1431
  audio_score: float, binary_result: str) -> str:
 
2920
  self.alert_system = AlertSystem(self.config)
2921
  self.reporter = EnhancedReportGenerator()
2922
 
2923
+ def release_stage8(self):
2924
+ """Free Stage 8 (LLaVA) weights after an analysis (HF Spaces fix)."""
2925
+ try:
2926
+ self.stage8.unload()
2927
+ except Exception:
2928
+ pass
2929
+
2930
  def _make_output_dir(self, video_path: str, base: str) -> str:
2931
  name = os.path.splitext(os.path.basename(video_path))[0]
2932
  ts = datetime.now().strftime("%Y%m%d_%H%M%S")
 
3065
 
3066
  self._save_outputs(results, out)
3067
  self._print_summary(results)
3068
+ if enable_llava:
3069
+ self.release_stage8()
3070
  return results
3071
 
3072
  def _save_outputs(self, results: dict, out_dir: str):
 
3197
  return stream
3198
 
3199
 
3200
+
3201
  # ══════════════════════════════════════════════════════════════════════════════
3202
  # READY MESSAGE
3203
  # ══════════════════════════════════════════════════════════════════════════════
 
3214
  print(" results = pipeline.analyze('/path/to/video.mp4')")
3215
  print("=" * 65)
3216
 
 
 
 
 
 
 
 
 
 
3217
 
3218
  # ══════════════════════════════════════════════════════════════════════════════
3219
+ # SHARED PIPELINE SINGLETON (models are cached across analyses)
 
 
 
3220
  # ══════════════════════════════════════════════════════════════════════════════
3221
+ @lru_cache(maxsize=1)
3222
+ def get_pipeline() -> SceneSolverPipeline:
3223
+ return SceneSolverPipeline()
3224