README.md

---
license: mit
license_link: LICENSE
library_name: openvino
pipeline_tag: object-detection
tags:
  - openvino
  - intel
  - yolo
  - yolo26
  - heatmap
  - speed
  - traffic
  - tracking
  - edge-ai
  - metro
  - dlstreamer
language:
  - en
---

# Heatmap Generation

| Property | Value |
|---|---|
| **Category** | Object Detection + Speed Heatmap Aggregation |
| **Base Model** | [YOLO26](https://docs.ultralytics.com/models/yolo26/) (Ultralytics) |
| **Source Framework** | PyTorch (Ultralytics) |
| **Supported Precisions** | FP32, FP16, INT8 (mixed-precision) |
| **Inference Engine** | OpenVINO |
| **Hardware** | CPU, GPU, NPU |
| **Detected Class(es)** | All 80 COCO classes (heatmap colored by object speed) |

---

## Overview

Heatmap Generation is a Metro Analytics use case that detects objects across
video frames and colors each region of the scene by how fast traffic moves
through it. It is built on [YOLO26](https://docs.ultralytics.com/models/yolo26/), a
state-of-the-art real-time object detector, quantized to INT8 for efficient
inference on Intel hardware.

Each detection's per-frame displacement is used as a speed estimate, deposited
over the object's footprint and averaged per location with Gaussian smoothing
into a color-coded overlay.

The overlay uses the following color scheme:

- **Red** -- fast-moving traffic.
- **Yellow / green** -- medium speed.
- **Blue** -- slow-moving or stationary traffic.

Typical Metro deployments include:

- **Traffic Speed Mapping** -- highlight fast corridors and slow/congested lanes.
- **Congestion Detection** -- surface persistently slow (blue) areas for safety planning.
- **Pedestrian Flow Analysis** -- compare fast throughways against lingering areas.
- **Incident Spotting** -- flag unusually fast or stalled movement.

Available variants: `yolo26n`, `yolo26s`, `yolo26m`, `yolo26l`, `yolo26x`.
Smaller variants (`yolo26n`, `yolo26s`) are recommended for high-FPS edge
deployment; larger variants improve recall for small or distant objects.

---

## Prerequisites

- Python 3.11+
- [Install OpenVINO](https://docs.openvino.ai/2026/get-started/install-openvino.html) (latest version)
- [Install Intel DLStreamer](https://docs.openedgeplatform.intel.com/2026.0/edge-ai-libraries/dlstreamer/get_started/install/install_guide_ubuntu.html) (latest version)

Create and activate a Python virtual environment before running the scripts:

```bash
python3 -m venv .venv --system-site-packages
source .venv/bin/activate
```

> **Note:** The `--system-site-packages` flag is required so the virtual
> environment can access the system-installed OpenVINO and DLStreamer Python
> packages.

---

## Getting Started

### Download and Quantize Model

Run the provided script to download, export to OpenVINO IR, and optionally quantize:

```bash
chmod +x export_and_quantize.sh
./export_and_quantize.sh
```

This exports the default **yolo26n** model in **FP16** precision.

#### Optional: Select a Different Variant or Precision

```bash
./export_and_quantize.sh yolo26n FP32   # full-precision
./export_and_quantize.sh yolo26n INT8   # quantized
./export_and_quantize.sh yolo26s        # larger variant, default FP16
```

The script performs the following steps:

1. Installs dependencies (`openvino`, `ultralytics`; adds `nncf` for INT8).
2. Downloads a sample test image (`test.jpg`) and a sample test video (`test_video.mp4`).
3. Downloads the PyTorch weights and exports to OpenVINO IR.
4. *(INT8 only)* Quantizes the model using NNCF post-training quantization.

Output files:

- `yolo26n_openvino_model/` -- FP32 or FP16 OpenVINO IR model directory.
- `yolo26n_heatmap_int8.xml` / `.bin` -- INT8 quantized model *(only when `INT8` is selected)*.

#### Precision / Device Compatibility

| Precision | CPU | GPU | NPU |
|---|---|---|---|
| FP32 | Yes | Yes | No |
| FP16 | Yes | Yes | Yes |
| INT8 | Yes | Yes | Yes |

### OpenVINO Sample

The sample below runs YOLO26 inference on a video, estimates each object's
speed from its per-frame displacement, and writes a speed-colored heatmap
overlay (red = fast, blue = slow) to `output_openvino.mp4`.
It also saves the final speed heatmap as `heatmap.jpg`.
Change the `device` string to run on CPU, GPU, or NPU.

```python
import cv2
import numpy as np
import openvino as ov

CONF_THRESHOLD = 0.4
INPUT_SIZE = 640
HEATMAP_ALPHA = 0.55
MATCH_DIST = 80.0   # max px between frames to treat detections as the same object
MAX_SPEED = 20.0    # px/frame that maps to full red


def render_speed_heatmap(frame, speed_sum, count, alpha):
    """Color traffic by average speed: blue = slow, yellow = medium,
    red = fast. Only regions where vehicles were seen are tinted, so
    empty background keeps its original color."""
    avg = np.zeros_like(speed_sum)
    seen = count > 0
    avg[seen] = speed_sum[seen] / count[seen]
    avg = cv2.GaussianBlur(avg, (0, 0), sigmaX=15)
    presence = cv2.GaussianBlur(seen.astype(np.float32), (0, 0), sigmaX=15)
    norm = np.clip(avg / MAX_SPEED, 0, 1)  # 0 = slow (blue), 1 = fast (red)
    color = cv2.applyColorMap((norm * 255).astype(np.uint8), cv2.COLORMAP_JET)
    weight = (np.clip(presence, 0, 1) * alpha)[..., np.newaxis]
    overlay = frame.astype(np.float32) * (1 - weight) + color.astype(np.float32) * weight
    return overlay.astype(np.uint8), color


core = ov.Core()
model = core.read_model("yolo26n_openvino_model/yolo26n.xml")

# Change device to "GPU" or "NPU" to run on integrated GPU or NPU.
compiled = core.compile_model(model, "CPU")

cap = cv2.VideoCapture("test_video.mp4")
fps = cap.get(cv2.CAP_PROP_FPS) or 30.0
width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
writer = cv2.VideoWriter(
    "output_openvino.mp4", cv2.VideoWriter_fourcc(*"mp4v"), fps, (width, height))

speed_sum = np.zeros((height, width), dtype=np.float32)
count = np.zeros((height, width), dtype=np.float32)
prev_centroids = []
heatmap_color = None
frame_idx = 0
total_dets = 0

while True:
    ok, frame = cap.read()
    if not ok:
        break
    frame_idx += 1
    h0, w0 = frame.shape[:2]
    sx, sy = w0 / INPUT_SIZE, h0 / INPUT_SIZE

    blob = cv2.resize(frame, (INPUT_SIZE, INPUT_SIZE))
    blob = cv2.cvtColor(blob, cv2.COLOR_BGR2RGB).astype(np.float32) / 255.0
    blob = blob.transpose(2, 0, 1)[np.newaxis, ...]

    output = compiled([blob])[compiled.output(0)][0]
    dets = output[output[:, 4] >= CONF_THRESHOLD]
    total_dets += len(dets)

    cur_centroids = []
    for det in dets:
        x1, y1 = int(det[0] * sx), int(det[1] * sy)
        x2, y2 = int(det[2] * sx), int(det[3] * sy)
        x1, x2 = max(0, x1), min(width, x2)
        y1, y2 = max(0, y1), min(height, y2)
        cx, cy = (x1 + x2) / 2, (y1 + y2) / 2
        cur_centroids.append((cx, cy))

        # Speed = displacement from the nearest detection in the previous frame.
        speed = 0.0
        if prev_centroids:
            d = min(np.hypot(cx - px, cy - py) for px, py in prev_centroids)
            if d <= MATCH_DIST:
                speed = d
        speed_sum[y1:y2, x1:x2] += speed
        count[y1:y2, x1:x2] += 1.0
    prev_centroids = cur_centroids

    overlay, heatmap_color = render_speed_heatmap(frame, speed_sum, count, HEATMAP_ALPHA)
    cv2.putText(overlay, f"Detections: {len(dets)}", (10, 30),
                cv2.FONT_HERSHEY_SIMPLEX, 0.8, (255, 255, 255), 2)
    writer.write(overlay)

cap.release()
writer.release()

if heatmap_color is not None:
    cv2.imwrite("heatmap.jpg", heatmap_color)
    print("Saved: heatmap.jpg")

print(f"Processed {frame_idx} frames, {total_dets} total detections", flush=True)
```

**Device targets:**

- `"CPU"` -- default, works on all Intel platforms.
- `"GPU"` -- Intel integrated or discrete GPU.
- `"NPU"` -- Intel NPU (validate with `benchmark_app -d NPU`).

#### Expected Output

![OpenVINO expected output](expected_output_openvino.gif)

### DLStreamer Sample

The pipeline below runs the FP16 YOLO26 detector via `gvadetect`.
A buffer probe estimates each object's speed from its per-frame displacement
and overlays a speed-colored heatmap (red = fast, blue = slow) on each frame
before encoding to `output_dlstreamer.mp4`.

> **Notes on running this sample:**
>
> - Use the FP16 IR (`yolo26n_openvino_model/yolo26n.xml`). Class names are
>   read automatically from the model's embedded `metadata.yaml` by
>   DLStreamer 2026.0+ -- no external `labels-file` is required.
> - Export `PYTHONPATH` so the DLStreamer Python module is importable:
>
>   ```bash
>   source /opt/intel/openvino_2026/setupvars.sh
>   source /opt/intel/dlstreamer/scripts/setup_dls_env.sh
>   export PYTHONPATH=/opt/intel/dlstreamer/python:\
>   /opt/intel/dlstreamer/gstreamer/lib/python3/dist-packages:${PYTHONPATH:-}
>   ```

```python
import gi

gi.require_version("Gst", "1.0")
gi.require_version("GstAnalytics", "1.0")
from gi.repository import Gst, GLib, GstAnalytics

import numpy as np

Gst.init([])

# Import cv2 after Gst.init to avoid GStreamer re-initialization conflicts.
import cv2

INPUT_VIDEO = "test_video.mp4"
HEATMAP_ALPHA = 0.55
MATCH_DIST = 80.0   # max px between frames to treat detections as the same object
MAX_SPEED = 20.0    # px/frame that maps to full red


def render_speed_heatmap(frame, speed_sum, count, alpha):
    """Color traffic by average speed: blue = slow, yellow = medium,
    red = fast. Only regions where vehicles were seen are tinted, so
    empty background keeps its original color."""
    avg = np.zeros_like(speed_sum)
    seen = count > 0
    avg[seen] = speed_sum[seen] / count[seen]
    avg = cv2.GaussianBlur(avg, (0, 0), sigmaX=15)
    presence = cv2.GaussianBlur(seen.astype(np.float32), (0, 0), sigmaX=15)
    norm = np.clip(avg / MAX_SPEED, 0, 1)  # 0 = slow (blue), 1 = fast (red)
    color = cv2.applyColorMap((norm * 255).astype(np.uint8), cv2.COLORMAP_JET)
    weight = (np.clip(presence, 0, 1) * alpha)[..., np.newaxis]
    overlay = frame.astype(np.float32) * (1 - weight) + color.astype(np.float32) * weight
    return overlay.astype(np.uint8)


# For CPU: change device=GPU to device=CPU.
# For NPU: change device=GPU to device=NPU (batch-size=1, nireq=4 recommended).
pipeline_str = (
    f"filesrc location={INPUT_VIDEO} ! decodebin3 ! "
    "videoconvert ! video/x-raw,format=BGR ! "
    "gvadetect model=yolo26n_openvino_model/yolo26n.xml "
    "device=GPU "
    "threshold=0.4 ! queue ! "
    "appsink name=sink emit-signals=false sync=false"
)
pipeline = Gst.parse_launch(pipeline_str)
sink = pipeline.get_by_name("sink")
pipeline.set_state(Gst.State.PLAYING)

speed_sum = None
count = None
prev_centroids = []
writer = None
frame_idx = 0
total_dets = 0

while True:
    sample = sink.emit("pull-sample")
    if sample is None:
        break
    buf = sample.get_buffer()
    caps = sample.get_caps().get_structure(0)
    width = caps.get_value("width")
    height = caps.get_value("height")

    if speed_sum is None:
        speed_sum = np.zeros((height, width), dtype=np.float32)
        count = np.zeros((height, width), dtype=np.float32)

    ok, mapinfo = buf.map(Gst.MapFlags.READ)
    if not ok:
        continue
    frame = np.ndarray((height, width, 3), dtype=np.uint8,
                       buffer=mapinfo.data).copy()
    buf.unmap(mapinfo)
    frame_idx += 1

    rmeta = GstAnalytics.buffer_get_analytics_relation_meta(buf)
    cur_centroids = []
    det_count = 0
    if rmeta is not None:
        idx = 1
        while True:
            ok_od, od = rmeta.get_od_mtd(idx)
            if not ok_od:
                break
            _, x, y, w, h, _ = od.get_location()
            x1, y1 = max(0, int(x)), max(0, int(y))
            x2, y2 = min(width, int(x + w)), min(height, int(y + h))
            cx, cy = (x1 + x2) / 2, (y1 + y2) / 2
            cur_centroids.append((cx, cy))

            # Speed = displacement from the nearest detection last frame.
            speed = 0.0
            if prev_centroids:
                d = min(np.hypot(cx - px, cy - py) for px, py in prev_centroids)
                if d <= MATCH_DIST:
                    speed = d
            speed_sum[y1:y2, x1:x2] += speed
            count[y1:y2, x1:x2] += 1.0
            det_count += 1
            idx += 1
    prev_centroids = cur_centroids
    total_dets += det_count

    overlay = render_speed_heatmap(frame, speed_sum, count, HEATMAP_ALPHA)

    if writer is None:
        writer = cv2.VideoWriter(
            "output_dlstreamer.mp4", cv2.VideoWriter_fourcc(*"mp4v"),
            30.0, (width, height))
    writer.write(overlay)
    print(f"Frame {frame_idx}: detections={det_count}", flush=True)

pipeline.set_state(Gst.State.NULL)
if writer:
    writer.release()
print(f"Processed {frame_idx} frames, {total_dets} total detections", flush=True)
```

**Device targets:**

- `device=GPU` -- default in the sample code.
- `device=CPU` -- change `device=GPU` to `device=CPU`.
- `device=NPU` -- change `device=GPU` to `device=NPU`; use `batch-size=1` and `nireq=4` for best NPU utilization.

#### Expected Output

![DLStreamer expected output](expected_output_dlstreamer.gif)

---

## License

Licensed under the MIT License. See [LICENSE](LICENSE) for details.

## References

- [YOLO26 Documentation](https://docs.ultralytics.com/models/yolo26/)
- [Ultralytics Heatmap Guide](https://docs.ultralytics.com/guides/heatmaps/)
- [OpenCV Color Maps](https://docs.opencv.org/4.x/d3/d50/group__imgproc__colormap.html)
- [OpenVINO Documentation](https://docs.openvino.ai/)
- [Intel DLStreamer](https://docs.openedgeplatform.intel.com/2026.0/edge-ai-libraries/dlstreamer/index.html)