docs/concept_note.md

DRISHTI · ITMS — Concept Note

Automated Photo Identification & Classification of Traffic Violations Using Computer Vision

Flipkart Gridlock 2.0 · Problem Statement 3 · Bengaluru (BTP / ASTraM-compatible)

The problem

Cities generate huge volumes of traffic-camera footage daily. Manual inspection to find violations is slow, inconsistent, and doesn't scale. Enforcement is reactive and there's no data-driven view of where violations actually concentrate.

Our solution

DRISHTI turns any traffic camera clip into structured, court-ready violation evidence — fully automated. Upload a video (or connect a camera), pick the detection engine, and the system detects, classifies, documents, and geo-maps violations, then routes them through an evidence-reliability gate to a human-review queue before any challan is issued.

Any-camera, not just expensive ITMS gantries. The same pipeline runs on an existing CCTV clip, a phone video, or a fixed junction camera — widening enforcement coverage cheaply.

Violations detected (all 7 in the PS)

Helmet non-compliance · No seatbelt (car) · Triple riding (pose) · Red-light · Stop-line · Wrong-way · Illegal parking.

How it maps to the 8 PS3 tasks

1. Preprocessing — auto blur-detection; night/rain/blur handling; forensic super-resolution (MFSR + Real-ESRGAN) to make far/blurry plates readable.
2. Detection & classification — YOLO26 + BoT-SORT; vehicles, riders, pedestrians + categories.
3. Violation detection — appearance (helmet/seatbelt/triple) + geometry (red-light via real HSV signal-colour detection, stop-line crossing, wrong-way, parking) using per-camera calibration.
4. Classification + confidence — per-violation type + Evidence Reliability Score (ERS).
5. ANPR — plate detector + fast-plate-ocr (plate-specialised) + per-track voting.
6. Evidence — annotated frame (single box + label) + ESRGAN plate-zoom inset + metadata/timestamp.
7. Analytics & reporting — searchable records, officer review, Bengaluru hotspot map (Mappls).
8. Performance — measured P/R/F1/mAP + FPS; pipeline ~25–40 fps on a cloud T4.

Evidence Reliability Score (ERS)

Each violation gets a 0–100 score (detection conf · OCR conf · multi-frame consensus · image quality · margin). ≥85 → auto-fine, 60–84 → human review, <60 → discard. Only high-confidence evidence is auto-fined; the rest goes to an officer queue — decisions are final. This keeps the system trustworthy and reduces wrong challans (the failure mode that erodes public trust).

Architecture

• Frontend (Next.js, Vercel): command center, upload, review queue, live cameras, hotspot map.
• Inference (serverless cloud GPU, Modal): the full model pipeline, models cached on a Volume.
• Detector: YOLO26 + BoT-SORT — a realtime specialist (15 fps on a T4), chosen over open-vocab models (YOLOE/SAM3) which we measured as weaker on small plates and far slower.
• Secrets stay server-side; the heavy models run per-violation on crops to keep cost ≈ ₹0/video.

Why cloud now, edge later

The multi-model stack needs a GPU — a free CPU host runs it at <1 fps. Processing is not latency-critical (clips are uploaded, not live), so cloud GPU is the right call today. Roadmap: distil the whole stack into a single efficient model trained on Indian traffic data, for real-time on-edge/on-phone deployment at the most accurate results.

What makes it stand out

Real working ANPR (not stubbed), real signal-colour red-light detection (competitors fake the signal state), forensic plate super-resolution, an honest ERS evidence gate, per-camera calibration, and a Mappls hotspot map — in one clean, deployable product.