# DRISHTI · ITMS — Concept Note
### Automated Photo Identification & Classification of Traffic Violations Using Computer Vision
Flipkart Gridlock 2.0 · Problem Statement 3 · Bengaluru (BTP / ASTraM-compatible)

## The problem
Cities generate huge volumes of traffic-camera footage daily. Manual inspection to find violations
is slow, inconsistent, and doesn't scale. Enforcement is reactive and there's no data-driven view of
where violations actually concentrate.

## Our solution
DRISHTI turns **any traffic camera clip** into structured, court-ready violation evidence — fully
automated. Upload a video (or connect a camera), pick the detection engine, and the system detects,
classifies, documents, and geo-maps violations, then routes them through an evidence-reliability gate
to a human-review queue before any challan is issued.

**Any-camera, not just expensive ITMS gantries.** The same pipeline runs on an existing CCTV clip, a
phone video, or a fixed junction camera — widening enforcement coverage cheaply.

## Violations detected (all 7 in the PS)
Helmet non-compliance · No seatbelt (car) · Triple riding (pose) · Red-light · Stop-line ·
Wrong-way · Illegal parking.

## How it maps to the 8 PS3 tasks
1. **Preprocessing** — auto blur-detection; night/rain/blur handling; **forensic super-resolution**
   (MFSR + Real-ESRGAN) to make far/blurry plates readable.
2. **Detection & classification** — YOLO26 + BoT-SORT; vehicles, riders, pedestrians + categories.
3. **Violation detection** — appearance (helmet/seatbelt/triple) + **geometry** (red-light via real
   HSV signal-colour detection, stop-line crossing, wrong-way, parking) using per-camera calibration.
4. **Classification + confidence** — per-violation type + **Evidence Reliability Score (ERS)**.
5. **ANPR** — plate detector + `fast-plate-ocr` (plate-specialised) + per-track voting.
6. **Evidence** — annotated frame (single box + label) + ESRGAN plate-zoom inset + metadata/timestamp.
7. **Analytics & reporting** — searchable records, officer review, Bengaluru hotspot map (Mappls).
8. **Performance** — measured P/R/F1/mAP + FPS; pipeline ~25–40 fps on a cloud T4.

## Evidence Reliability Score (ERS)
Each violation gets a 0–100 score (detection conf · OCR conf · multi-frame consensus · image quality
· margin). **≥85 → auto-fine, 60–84 → human review, <60 → discard.** Only high-confidence evidence is
auto-fined; the rest goes to an officer queue — decisions are final. This keeps the system trustworthy
and reduces wrong challans (the failure mode that erodes public trust).

## Architecture
- **Frontend** (Next.js, Vercel): command center, upload, review queue, live cameras, hotspot map.
- **Inference** (serverless cloud GPU, Modal): the full model pipeline, models cached on a Volume.
- **Detector**: YOLO26 + BoT-SORT — a realtime specialist (15 fps on a T4), chosen over
  open-vocab models (YOLOE/SAM3) which we measured as weaker on small plates and far slower.
- Secrets stay server-side; the heavy models run per-violation on crops to keep cost ≈ ₹0/video.

## Why cloud now, edge later
The multi-model stack needs a GPU — a free CPU host runs it at <1 fps. Processing is not
latency-critical (clips are uploaded, not live), so cloud GPU is the right call today. **Roadmap:**
distil the whole stack into a **single efficient model** trained on **Indian traffic data**,
for real-time on-edge/on-phone deployment at the most accurate results.

## What makes it stand out
Real working ANPR (not stubbed), real signal-colour red-light detection (competitors fake the signal
state), forensic plate super-resolution, an honest ERS evidence gate, per-camera calibration, and a
Mappls hotspot map — in one clean, deployable product.