Upload face detector (YOLO11n/s), ONNX exports, scripts, model card

README.md

@@ -0,0 +1,85 @@
+---
+license: agpl-3.0
+library_name: ultralytics
+pipeline_tag: object-detection
+tags:
+  - face-detection
+  - yolo11
+  - object-detection
+  - onnx
+  - privacy
+---
+
+# face-model — single-class face detector for blurring faces in video
+
+YOLO11 (nano and small) fine-tuned from the COCO-pretrained Ultralytics weights as a one-class `face` detector on WIDER FACE,
+plus a video script that detects, tracks and pixelates (or blacks out) faces. Built for **recall** (privacy), not leaderboard accuracy.
+
+## Important licence notes (read before using commercially)
+
+- The code and weights are derived from **Ultralytics YOLO11, which is AGPL-3.0**. Using them in a product or network service means
+  complying with AGPL-3.0 or obtaining an Ultralytics enterprise licence.
+- The **training data, WIDER FACE, is listed as non-commercial** (CC BY-NC-ND 4.0 on its Hugging Face card; the original
+  WIDER FACE terms are for non-commercial research). Do not assume these weights are cleared for commercial use.
+- `code/face_id.py` (optional `--keep` mode) expects InsightFace `buffalo_l` ONNX models, which are **non-commercial research only**.
+  They are *not* included in this repo; download them yourself.
+
+## Files
+
+| File | Notes |
+|---|---|
+| `yolo11n/face_yolo11n.pt` | PyTorch weights, 5.4 MB |
+| `yolo11n/face_yolo11n_fp32.onnx`, `_fp16.onnx` | ONNX, fixed 640x640, batch 1 |
+| `yolo11n/face_yolo11n_int8.onnx` | INT8 (ONNX Runtime static QDQ). The Detect-head decode nodes are kept in float; plain INT8 returned zero detections |
+| `yolo11s/face_yolo11s.pt`, `_fp32.onnx`, `_fp16.onnx` | Larger, more accurate |
+| `code/` | `blur_video.py`, `train.py`, `convert_wider_to_yolo.py`, `eval_model.py`, `recall_at_conf.py`, `quantize_int8.py`, `face_id.py` |
+
+## Results (WIDER FACE val, 3,222 images / 39,112 faces, 640 px, all difficulty levels together)
+
+| Model | mAP50 | mAP50-95 |
+|---|---|---|
+| YOLO11n `.pt` | 0.684 | 0.362 |
+| YOLO11n ONNX FP32 | 0.683 | 0.362 |
+| YOLO11n ONNX FP16 | 0.682 | 0.361 |
+| YOLO11n ONNX INT8 | 0.667 | 0.348 |
+| YOLO11s `.pt` | 0.741 | 0.401 |
+| YOLO11s ONNX FP16 | 0.739 | 0.399 |
+
+**Recall at confidence 0.25 (IoU >= 0.5)**, i.e. the fraction of labelled faces that get a box:
+
+| Face height (px, images stored at <=640 px) | n=39,112 | YOLO11n FP16 | YOLO11n INT8 | YOLO11s FP16 |
+|---|---|---|---|---|
+| < 10 | 16,305 | 33.1% | 31.6% | 42.1% |
+| 10-20 | 10,521 | 72.2% | 70.7% | 79.1% |
+| 20-40 | 7,478 | 86.6% | 85.4% | 90.4% |
+| 40-80 | 3,167 | 93.7% | 92.9% | 95.3% |
+| > 80 | 1,641 | 95.6% | 95.0% | 97.3% |
+| **Overall** | | **61.4%** | **60.0%** | **67.9%** |
+
+WIDER FACE is dominated by tiny faces (median face height about 12 px). Performance on tiny, far-away faces is the main weakness;
+on faces >= 20 px the nano finds about 90%. **No detector guarantees every face is caught in every frame** — spot-check any
+video where missing a face matters.
+
+Not measured: speed on phones or other hardware. Any such numbers you may see elsewhere in this project were estimates.
+
+## Training
+
+- Start: COCO `yolo11n.pt` / `yolo11s.pt`, fine-tuned (not from scratch), single class `face`.
+- Data: WIDER FACE train, converted to YOLO format (`code/convert_wider_to_yolo.py`): invalid boxes dropped, small faces kept, images downscaled to <= 640 px long side.
+- 640 px, AMP. Nano: 40 epochs. Small: 40 epochs (resumed from a checkpoint at epoch 34 with the schedule shortened to end at 40).
+- Extra augmentation: random motion blur and JPEG compression to imitate video frames (`train.py`).
+
+## Use
+
+```bash
+pip install ultralytics onnxruntime-gpu lap
+python code/blur_video.py in.mp4 out.mp4 --model yolo11n/face_yolo11n_fp16.onnx -n 1            # pixelate
+python code/blur_video.py in.mp4 out.mp4 --model yolo11n/face_yolo11n_fp16.onnx --mode black     # solid black
+```
+
+Defaults: detect every 3 frames (`-n`) with ByteTrack in between, confidence 0.25, 15% box padding, 8 pixel blocks across the
+shorter side of each face. Use `-n 1` for the strictest coverage. Audio is preserved via ffmpeg. With an ONNX model use
+`--device 0` (GPU); `--device cpu` makes Ultralytics try to pip-install `onnxruntime`.
+
+`--keep ref.jpg [...]` leaves one person visible and blurs everyone else using face recognition; every other face stays blurred
+unless confirmed over several frames. It needs the InsightFace models mentioned above.

code/blur_video.py

@@ -0,0 +1,282 @@
+#!/usr/bin/env python3
+"""Detect, track and pixelate (or black out) faces in a video. Audio is preserved via ffmpeg.
+
+Detection runs every N frames; between detections ByteTrack's Kalman filter carries the boxes forward.
+Tuned for recall (privacy): low confidence threshold, box padding, raw detections are blurred immediately
+(without waiting for track confirmation) and recently lost tracks are held for a few frames.
+
+--keep: leave ONE person's face visible and blur everybody else. Give one or more reference photos of that
+person. Fail-safe by design: a face stays blurred unless its track was confirmed as the target by several
+consecutive face-recognition matches; a clear non-match, a track box jump, or going too long without a
+successful re-check puts the blur back.
+
+Examples:
+  python blur_video.py in.mp4 out.mp4 --model export/face_yolo11n_fp16.onnx -n 1
+  python blur_video.py in.mp4 out.mp4 --mode black
+  python blur_video.py in.mp4 out.mp4 -n 1 --keep me1.jpg me2.jpg
+"""
+import argparse
+import queue
+import shutil
+import subprocess
+import sys
+import threading
+import time
+from types import SimpleNamespace
+
+import cv2
+import numpy as np
+from ultralytics import YOLO
+from ultralytics.trackers.byte_tracker import BYTETracker
+
+
+def pad_box(x1, y1, x2, y2, pad, w, h):
+    bw, bh = x2 - x1, y2 - y1
+    x1, x2 = x1 - bw * pad, x2 + bw * pad
+    y1, y2 = y1 - bh * pad, y2 + bh * pad
+    return max(0, int(np.floor(x1))), max(0, int(np.floor(y1))), min(w, int(np.ceil(x2))), min(h, int(np.ceil(y2)))
+
+
+def iou(a, b):
+    ix = max(0.0, min(a[2], b[2]) - max(a[0], b[0]))
+    iy = max(0.0, min(a[3], b[3]) - max(a[1], b[1]))
+    inter = ix * iy
+    union = (a[2] - a[0]) * (a[3] - a[1]) + (b[2] - b[0]) * (b[3] - b[1]) - inter
+    return inter / union if union > 0 else 0.0
+
+
+def pixelate(frame, box, blocks):
+    """Downscale the region so its shorter side has `blocks` cells, then nearest-neighbour upscale."""
+    x1, y1, x2, y2 = box
+    if x2 - x1 < 2 or y2 - y1 < 2:
+        return
+    roi = frame[y1:y2, x1:x2]
+    h, w = roi.shape[:2]
+    cell = max(1.0, min(w, h) / blocks)
+    small = cv2.resize(roi, (max(1, round(w / cell)), max(1, round(h / cell))), interpolation=cv2.INTER_AREA)
+    frame[y1:y2, x1:x2] = cv2.resize(small, (w, h), interpolation=cv2.INTER_NEAREST)
+
+
+def blackout(frame, box):
+    x1, y1, x2, y2 = box
+    frame[y1:y2, x1:x2] = 0
+
+
+def open_ffmpeg_writer(src, dst, w, h, fps, crf):
+    cmd = [
+        "ffmpeg", "-y", "-loglevel", "error",
+        "-f", "rawvideo", "-pix_fmt", "bgr24", "-s", f"{w}x{h}", "-r", f"{fps}", "-i", "-",
+        "-i", src,
+        "-map", "0:v:0", "-map", "1:a?",
+        "-c:v", "libx264", "-preset", "fast", "-crf", str(crf), "-pix_fmt", "yuv420p",
+        "-c:a", "copy", "-shortest", "-movflags", "+faststart", dst,
+    ]
+    return subprocess.Popen(cmd, stdin=subprocess.PIPE)
+
+
+class KeepTracker:
+    """Decides which ByteTrack ids are the person to keep visible. Everything not confirmed stays blurred."""
+
+    def __init__(self, fid, refs, thr, min_size, hits_needed, recheck, expire, budget):
+        self.fid, self.refs, self.thr, self.min_size = fid, refs, thr, min_size
+        self.hits_needed, self.recheck, self.expire, self.budget = hits_needed, recheck, expire, budget
+        self.state = {}  # tid -> dict(hits, kept, last_ok, last_try, box)
+        self.seen, self.ever_kept = set(), set()
+
+    def is_kept(self, tid, idx):
+        s = self.state.get(tid)
+        if s is None or not s["kept"]:
+            return False
+        if idx - s["last_ok"] > self.expire:  # could not re-confirm for too long -> blur again
+            s["kept"], s["hits"] = False, 0
+            return False
+        return True
+
+    def update(self, frame, idx, tracked):
+        """tracked: list of (xyxy, tid) from a detection frame (clean, un-blurred frame)."""
+        todo = []
+        for box, tid in tracked:
+            self.seen.add(tid)
+            s = self.state.setdefault(tid, dict(hits=0, kept=False, last_ok=-10**9, last_try=-10**9, box=box))
+            if s["kept"] and iou(s["box"], box) < 0.3:  # box jumped: possible ID switch -> distrust
+                s["kept"], s["hits"] = False, 0
+            s["box"] = box
+            if not self.is_kept(tid, idx):
+                todo.append((0, box, tid))  # undecided first
+            elif idx - s["last_try"] >= self.recheck:
+                todo.append((1, box, tid))
+        todo.sort(key=lambda t: (t[0], self.state[t[2]]["last_try"]))
+        for _, box, tid in todo[:self.budget]:
+            s = self.state[tid]
+            s["last_try"] = idx
+            e = self.fid.embed_box(frame, box, min_size=self.min_size)
+            if e is None:
+                continue  # inconclusive: keep old state (a kept track lapses via `expire`)
+            if self.fid.similarity(e, self.refs) >= self.thr:
+                s["hits"] += 1
+                s["last_ok"] = idx
+                if s["hits"] >= self.hits_needed:
+                    s["kept"] = True
+                    self.ever_kept.add(tid)
+            else:  # clear non-match: blur immediately
+                s["hits"], s["kept"] = 0, False
+
+
+def main():
+    ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument("input")
+    ap.add_argument("output")
+    ap.add_argument("--model", default="runs/face_yolo11n/weights/best.pt", help=".pt or .onnx face detector")
+    ap.add_argument("-n", "--every", type=int, default=3, help="run detector every N frames (default 3)")
+    ap.add_argument("--conf", type=float, default=0.25, help="detection confidence threshold (default 0.25)")
+    ap.add_argument("--pad", type=float, default=0.15, help="pad each box by this fraction per side (default 0.15)")
+    ap.add_argument("--mode", choices=["pixelate", "black"], default="pixelate",
+                    help="pixelate faces (default) or fill the padded box with solid black")
+    ap.add_argument("--blocks", type=int, default=8,
+                    help="pixel blocks across a face's shorter side; LOWER = stronger pixelation (default 8)")
+    ap.add_argument("--hold", type=int, default=8, help="keep blurring a lost track's prediction this many frames")
+    ap.add_argument("--imgsz", type=int, default=640)
+    ap.add_argument("--device", default=None, help="e.g. 0 or cpu (default: auto)")
+    ap.add_argument("--crf", type=int, default=18, help="x264 quality (lower = better)")
+    k = ap.add_argument_group("keep one person visible (face recognition)")
+    k.add_argument("--keep", nargs="+", metavar="REF.jpg",
+                   help="reference photo(s) of the person to leave unblurred; everyone else is blurred")
+    k.add_argument("--keep-threshold", type=float, default=0.45,
+                   help="cosine similarity needed to count as the same person; HIGHER = stricter (default 0.45)")
+    k.add_argument("--keep-hits", type=int, default=2, help="consecutive matches before a track is unblurred")
+    k.add_argument("--keep-min-size", type=int, default=40,
+                   help="faces smaller than this many px are never recognised, so they stay blurred (default 40)")
+    k.add_argument("--keep-recheck", type=int, default=8, help="re-verify a kept track every this many frames")
+    k.add_argument("--keep-expire", type=int, default=24,
+                   help="blur a kept track again if it could not be re-confirmed for this many frames")
+    k.add_argument("--keep-budget", type=int, default=4, help="max recognitions per frame (speed cap)")
+    a = ap.parse_args()
+
+    if shutil.which("ffmpeg") is None:
+        sys.exit("ffmpeg not found on PATH")
+    if a.every < 1 or a.blocks < 1:
+        sys.exit("--every and --blocks must be >= 1")
+
+    keeper = None
+    if a.keep:
+        from face_id import FaceID
+        fid = FaceID("cpu" if a.device == "cpu" else "0")
+        refs = fid.reference_embeddings(a.keep)
+        print(f"keeping the person in {len(refs)} reference photo(s); threshold {a.keep_threshold}")
+        keeper = KeepTracker(fid, refs, a.keep_threshold, a.keep_min_size, a.keep_hits,
+                             a.keep_recheck, a.keep_expire, a.keep_budget)
+
+    cap = cv2.VideoCapture(a.input)
+    if not cap.isOpened():
+        sys.exit(f"cannot open {a.input}")
+    fps = cap.get(cv2.CAP_PROP_FPS) or 30.0
+    total = int(cap.get(cv2.CAP_PROP_FRAME_COUNT)) or 0
+    ok, frame = cap.read()
+    if not ok:
+        sys.exit("no frames in input")
+    H, W = frame.shape[:2]
+
+    model = YOLO(a.model, task="detect")
+    tracker = BYTETracker(SimpleNamespace(
+        track_high_thresh=a.conf, track_low_thresh=0.1, new_track_thresh=a.conf,
+        track_buffer=max(30, 4 * a.every), match_thresh=0.8, fuse_score=True))
+    writer = open_ffmpeg_writer(a.input, a.output, W, H, fps, a.crf)
+
+    # Decode and encode run in their own threads so they overlap with detection (frame order is preserved:
+    # one reader, one writer, FIFO queues). cv2/ffmpeg pipe I/O release the GIL.
+    rq, wq, write_failed = queue.Queue(maxsize=48), queue.Queue(maxsize=48), []
+    first_frame = frame
+
+    def reader():
+        rq.put(first_frame)
+        while True:
+            okr, f = cap.read()
+            if not okr:
+                rq.put(None)
+                return
+            rq.put(f)
+
+    def writer_loop():
+        try:
+            while True:
+                f = wq.get()
+                if f is None:
+                    return
+                writer.stdin.write(f.tobytes())
+        except (BrokenPipeError, OSError):
+            write_failed.append(True)
+            while wq.get() is not None:  # drain so the main thread never blocks on a full queue
+                pass
+
+    threading.Thread(target=reader, daemon=True).start()
+    wthread = threading.Thread(target=writer_loop)
+    wthread.start()
+
+    t0 = time.perf_counter()
+    idx = n_det = 0
+    while True:
+        frame = rq.get()
+        if frame is None:
+            break
+        items = []  # (xyxy, track_id or None)
+        if idx % a.every == 0:
+            r = model.predict(frame, conf=a.conf, imgsz=a.imgsz, device=a.device, verbose=False)[0]
+            det = r.boxes.cpu().numpy()
+            n_det += 1
+            tracked = tracker.update(det, frame)
+            items += [(b[:4], int(b[4])) for b in tracked]  # confirmed tracks
+            items += [(b, None) for b in det.xyxy]  # raw detections: don't wait for track confirmation
+            items += [(t.xyxy, t.track_id) for t in tracker.tracked_stracks if not t.is_activated]
+            items += [(t.xyxy, t.track_id) for t in tracker.lost_stracks if tracker.frame_id - t.end_frame <= a.hold]
+            if keeper:  # recognise on the clean frame, before anything is drawn
+                keeper.update(frame, idx, [(b[:4], int(b[4])) for b in tracked])
+        else:
+            # no detector this frame: advance Kalman state of every live track and use the prediction
+            tracker.frame_id += 1
+            pool = tracker.tracked_stracks + tracker.lost_stracks
+            if pool:
+                tracker.multi_predict(pool)
+            items += [(t.xyxy, t.track_id) for t in tracker.tracked_stracks]
+            items += [(t.xyxy, t.track_id) for t in tracker.lost_stracks if tracker.frame_id - t.end_frame <= a.hold]
+
+        if keeper:
+            kept_boxes = [np.asarray(b, dtype=float)[:4] for b, tid in items
+                          if tid is not None and keeper.is_kept(tid, idx)]
+            items = [(b, tid) for b, tid in items
+                     if not (tid is not None and keeper.is_kept(tid, idx))
+                     and not (tid is None and any(iou(np.asarray(b, dtype=float)[:4], kb) >= 0.5 for kb in kept_boxes))]
+
+        for b, _ in items:
+            box = pad_box(*np.asarray(b, dtype=float)[:4], a.pad, W, H)
+            if a.mode == "black":
+                blackout(frame, box)
+            else:
+                pixelate(frame, box, a.blocks)
+        wq.put(frame)
+
+        idx += 1
+        if idx % 100 == 0:
+            el = time.perf_counter() - t0
+            print(f"\r{idx}/{total or '?'} frames  {idx / el:.1f} FPS", end="", flush=True)
+
+    wq.put(None)
+    wthread.join()
+    cap.release()
+    try:
+        writer.stdin.close()
+    except (BrokenPipeError, OSError):
+        pass
+    rc = writer.wait()
+    if write_failed:
+        rc = rc or 1
+    el = time.perf_counter() - t0
+    print(f"\rdone: {idx} frames in {el:.1f}s = {idx / el:.1f} FPS processing "
+          f"(detector ran on {n_det} frames, every {a.every}) -> {a.output}")
+    if keeper:
+        print(f"keep: {len(keeper.ever_kept)} of {len(keeper.seen)} tracks were recognised as the target and left visible")
+    if rc != 0:
+        sys.exit(f"ffmpeg exited with code {rc}")
+
+
+if __name__ == "__main__":
+    main()

code/convert_wider_to_yolo.py

@@ -0,0 +1,91 @@
+#!/usr/bin/env python3
+"""Convert WIDER FACE (HF parquet, CUHK-CSE/wider_face @ refs/convert/parquet) to single-class YOLO format.
+
+- Skips boxes flagged `invalid` and degenerate boxes (<1px).
+- Keeps small faces (no min-size filter) since blurring needs high recall.
+- Drops images whose only boxes are invalid (they would teach the model to ignore real faces).
+- Downscales images so the long side is <= --max-side (matches training imgsz; boxes are normalized).
+
+Usage:
+  python convert_wider_to_yolo.py --raw raw/default --out dataset
+"""
+import argparse
+import io
+import os
+from concurrent.futures import ProcessPoolExecutor
+from pathlib import Path
+
+import pyarrow.parquet as pq
+from PIL import Image
+
+SPLITS = {"train": "train", "validation": "val"}
+
+
+def convert_row(args):
+    name, img_bytes, bboxes, invalid, out_img, out_lbl, max_side = args
+    valid = [b for b, inv in zip(bboxes, invalid) if not inv and b[2] >= 1 and b[3] >= 1]
+    if not valid:
+        return "dropped", 0, 0
+    im = Image.open(io.BytesIO(img_bytes)).convert("RGB")
+    w, h = im.size
+    scale = min(1.0, max_side / max(w, h))
+    if scale < 1.0:
+        im = im.resize((max(1, round(w * scale)), max(1, round(h * scale))), Image.LANCZOS)
+    stem = Path(name).stem
+    im.save(out_img / f"{stem}.jpg", quality=95)
+    lines = []
+    for x, y, bw, bh in valid:
+        x1, y1 = max(0.0, x), max(0.0, y)
+        x2, y2 = min(float(w), x + bw), min(float(h), y + bh)
+        if x2 - x1 < 1 or y2 - y1 < 1:
+            continue
+        lines.append(f"0 {(x1 + x2) / 2 / w:.6f} {(y1 + y2) / 2 / h:.6f} {(x2 - x1) / w:.6f} {(y2 - y1) / h:.6f}")
+    (out_lbl / f"{stem}.txt").write_text("\n".join(lines) + "\n")
+    return "ok", len(lines), len(bboxes) - len(valid)
+
+
+def rows(files, out_img, out_lbl, max_side):
+    for f in files:
+        pf = pq.ParquetFile(f)
+        for batch in pf.iter_batches(batch_size=64):
+            for r in batch.to_pylist():
+                yield (r["image"]["path"], r["image"]["bytes"], r["faces"]["bbox"],
+                       r["faces"]["invalid"], out_img, out_lbl, max_side)
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--raw", default="raw/default")
+    ap.add_argument("--out", default="dataset")
+    ap.add_argument("--max-side", type=int, default=640)
+    ap.add_argument("--limit", type=int, default=0, help="max images per split (0 = all; for testing)")
+    ap.add_argument("--workers", type=int, default=os.cpu_count())
+    a = ap.parse_args()
+
+    out = Path(a.out)
+    for src, dst in SPLITS.items():
+        files = sorted((Path(a.raw) / src).glob("*.parquet"))
+        out_img, out_lbl = out / "images" / dst, out / "labels" / dst
+        out_img.mkdir(parents=True, exist_ok=True)
+        out_lbl.mkdir(parents=True, exist_ok=True)
+        gen = rows(files, out_img, out_lbl, a.max_side)
+        if a.limit:
+            import itertools
+            gen = itertools.islice(gen, a.limit)
+        ok = dropped = faces = 0
+        with ProcessPoolExecutor(a.workers) as ex:
+            for status, n, _ in ex.map(convert_row, gen, chunksize=16):
+                if status == "ok":
+                    ok += 1
+                    faces += n
+                else:
+                    dropped += 1
+        print(f"{dst}: {ok} images, {faces} valid faces, {dropped} images dropped (no valid boxes)")
+
+    (out / "data.yaml").write_text(
+        f"path: {out.resolve()}\ntrain: images/train\nval: images/val\nnames:\n  0: face\n")
+    print("wrote", out / "data.yaml")
+
+
+if __name__ == "__main__":
+    main()

code/eval_model.py

@@ -0,0 +1,27 @@
+#!/usr/bin/env python3
+"""Evaluate one or more face-detector weights (.pt / .onnx) on the converted WIDER FACE val split.
+
+Usage: python eval_model.py export/face_yolo11n.pt export/face_yolo11n_fp32.onnx export/face_yolo11n_fp16.onnx
+"""
+import argparse
+
+from ultralytics import YOLO
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("models", nargs="+")
+    ap.add_argument("--data", default="dataset/data.yaml")
+    ap.add_argument("--imgsz", type=int, default=640)
+    ap.add_argument("--workers", type=int, default=4)
+    a = ap.parse_args()
+    for path in a.models:
+        r = YOLO(path, task="detect").val(data=a.data, imgsz=a.imgsz, batch=1, device=0,
+                                          workers=a.workers, plots=False, verbose=False)
+        print(f"RES {path}: mAP50={r.box.map50:.4f} mAP50-95={r.box.map:.4f} P={r.box.mp:.3f} R={r.box.mr:.3f} "
+              f"| {r.speed['preprocess']:.2f}+{r.speed['inference']:.2f}+{r.speed['postprocess']:.2f} ms/img",
+              flush=True)
+
+
+if __name__ == "__main__":
+    main()

code/face_id.py

@@ -0,0 +1,125 @@
+#!/usr/bin/env python3
+"""Face identity helper: SCRFD landmarks + ArcFace (w600k_r50) embeddings via plain onnxruntime.
+
+No `insightface` package needed (it would pull a CPU onnxruntime over onnxruntime-gpu); only the two ONNX files
+from InsightFace's buffalo_l pack in models/. NOTE: those pretrained models are licensed for non-commercial
+research use only.
+
+Used by blur_video.py --keep to leave one person's face visible while blurring everyone else.
+"""
+from pathlib import Path
+
+import cv2
+import numpy as np
+import onnxruntime as ort
+
+MODELS = Path(__file__).resolve().parent / "models"
+# ArcFace 112x112 alignment template (left eye, right eye, nose, left mouth, right mouth)
+ARC_TEMPLATE = np.array([[38.2946, 51.6963], [73.5318, 51.5014], [56.0252, 71.7366],
+                         [41.5493, 92.3655], [70.7299, 92.2041]], np.float32)
+
+
+def _session(path, device):
+    prov = ["CUDAExecutionProvider", "CPUExecutionProvider"] if device != "cpu" else ["CPUExecutionProvider"]
+    so = ort.SessionOptions()
+    so.log_severity_level = 3
+    return ort.InferenceSession(str(path), sess_options=so, providers=prov)
+
+
+def _nms(boxes, scores, thr=0.4):
+    idx = cv2.dnn.NMSBoxes([[float(b[0]), float(b[1]), float(b[2] - b[0]), float(b[3] - b[1])] for b in boxes],
+                           [float(s) for s in scores], 0.0, thr)
+    return np.array(idx, dtype=int).reshape(-1)
+
+
+class FaceID:
+    def __init__(self, device="cpu"):
+        self.det = _session(MODELS / "det_10g.onnx", device)
+        self.rec = _session(MODELS / "w600k_r50.onnx", device)
+        self.det_in = self.det.get_inputs()[0].name
+        self.rec_in = self.rec.get_inputs()[0].name
+
+    def detect(self, img, size=192, thr=0.4):
+        """SCRFD on `img` letterboxed (top-left) into a size x size canvas.
+        Returns boxes (N,4) xyxy, scores (N,), landmarks (N,5,2), all in `img` pixel coordinates."""
+        h, w = img.shape[:2]
+        s = size / max(h, w)
+        nw, nh = max(1, int(round(w * s))), max(1, int(round(h * s)))
+        canvas = np.zeros((size, size, 3), np.uint8)
+        canvas[:nh, :nw] = cv2.resize(img, (nw, nh))
+        blob = cv2.dnn.blobFromImage(canvas, 1 / 128.0, (size, size), (127.5, 127.5, 127.5), swapRB=True)
+        outs = self.det.run(None, {self.det_in: blob})
+        B, S, K = [], [], []
+        for i, stride in enumerate((8, 16, 32)):
+            sc, bb, kp = outs[i][:, 0], outs[i + 3] * stride, outs[i + 6] * stride
+            g = size // stride
+            ys, xs = np.mgrid[:g, :g]
+            ctr = np.repeat(np.stack([xs, ys], -1).reshape(-1, 2) * stride, 2, axis=0).astype(np.float32)
+            m = sc >= thr
+            if not m.any():
+                continue
+            c, bb, kp = ctr[m], bb[m], kp[m]
+            B.append(np.stack([c[:, 0] - bb[:, 0], c[:, 1] - bb[:, 1], c[:, 0] + bb[:, 2], c[:, 1] + bb[:, 3]], 1))
+            K.append(np.stack([c[:, 0:1] + kp[:, 0::2], c[:, 1:2] + kp[:, 1::2]], -1))
+            S.append(sc[m])
+        if not B:
+            return np.zeros((0, 4)), np.zeros(0), np.zeros((0, 5, 2))
+        B, S, K = np.concatenate(B) / s, np.concatenate(S), np.concatenate(K) / s
+        keep = _nms(B, S)
+        return B[keep], S[keep], K[keep]
+
+    def embed_aligned(self, img, lmk):
+        M, _ = cv2.estimateAffinePartial2D(lmk.astype(np.float32), ARC_TEMPLATE, method=cv2.LMEDS)
+        if M is None:
+            return None
+        face = cv2.warpAffine(img, M, (112, 112), borderValue=0)
+        blob = cv2.dnn.blobFromImage(face, 1 / 127.5, (112, 112), (127.5, 127.5, 127.5), swapRB=True)
+        e = self.rec.run(None, {self.rec_in: blob})[0][0]
+        return e / (np.linalg.norm(e) + 1e-9)
+
+    def embed_box(self, frame, box, expand=0.5, min_size=40):
+        """Embedding of the face inside `box` (xyxy) of a full frame, or None when it is too small / not found.
+        Crops the box expanded by `expand`, finds landmarks of the face nearest the box centre, aligns, embeds."""
+        H, W = frame.shape[:2]
+        x1, y1, x2, y2 = [float(v) for v in box[:4]]
+        bw, bh = x2 - x1, y2 - y1
+        if min(bw, bh) < min_size:
+            return None
+        cx1, cy1 = max(0, int(x1 - bw * expand)), max(0, int(y1 - bh * expand))
+        cx2, cy2 = min(W, int(x2 + bw * expand)), min(H, int(y2 + bh * expand))
+        crop = frame[cy1:cy2, cx1:cx2]
+        if crop.size == 0:
+            return None
+        B, S, K = self.detect(crop)
+        if len(B) == 0:
+            return None
+        tx, ty = (x1 + x2) / 2 - cx1, (y1 + y2) / 2 - cy1
+        d = [np.hypot((b[0] + b[2]) / 2 - tx, (b[1] + b[3]) / 2 - ty) / max(bw, 1) for b in B]
+        j = int(np.argmin(d))
+        if d[j] > 0.35:  # the found face is not the one in `box`
+            return None
+        return self.embed_aligned(crop, K[j])
+
+    def reference_embeddings(self, paths):
+        """One embedding per reference photo (largest face in each). Raises if a photo has no usable face."""
+        embs = []
+        for p in paths:
+            img = cv2.imread(str(p))
+            if img is None:
+                raise SystemExit(f"cannot read reference image {p}")
+            B, S, K = self.detect(img, size=640, thr=0.5)
+            if len(B) == 0:
+                raise SystemExit(f"no face found in reference image {p}")
+            if len(B) > 1:
+                print(f"warning: {len(B)} faces in {p}; using the largest")
+            j = int(np.argmax((B[:, 2] - B[:, 0]) * (B[:, 3] - B[:, 1])))
+            e = self.embed_aligned(img, K[j])
+            if e is None:
+                raise SystemExit(f"could not align face in {p}")
+            embs.append(e)
+        return np.stack(embs)
+
+    @staticmethod
+    def similarity(emb, refs):
+        """Best cosine similarity of `emb` against the reference embeddings."""
+        return float((refs @ emb).max())

code/quantize_int8.py

@@ -0,0 +1,71 @@
+#!/usr/bin/env python3
+"""Post-training static INT8 quantization (QDQ, per-channel weights) of the face detector ONNX.
+
+Usage: python quantize_int8.py export/face_yolo11n_fp32.onnx export/face_yolo11n_int8.onnx [--calib 300]
+"""
+import argparse
+import random
+from pathlib import Path
+
+import cv2
+import numpy as np
+import onnx
+from onnxruntime.quantization import CalibrationDataReader, CalibrationMethod, QuantFormat, QuantType, quantize_static
+
+
+def letterbox(img, size=640):
+    h, w = img.shape[:2]
+    s = min(size / h, size / w)
+    nh, nw = round(h * s), round(w * s)
+    img = cv2.resize(img, (nw, nh), interpolation=cv2.INTER_LINEAR)
+    canvas = np.full((size, size, 3), 114, np.uint8)
+    top, left = (size - nh) // 2, (size - nw) // 2
+    canvas[top:top + nh, left:left + nw] = img
+    return canvas
+
+
+class Reader(CalibrationDataReader):
+    def __init__(self, files, input_name):
+        self.it = iter(files)
+        self.name = input_name
+
+    def get_next(self):
+        f = next(self.it, None)
+        if f is None:
+            return None
+        img = letterbox(cv2.imread(str(f)))[:, :, ::-1].transpose(2, 0, 1)  # BGR->RGB, CHW
+        return {self.name: (img[None].astype(np.float32) / 255.0)}
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("src")
+    ap.add_argument("dst")
+    ap.add_argument("--calib", type=int, default=300)
+    ap.add_argument("--images", default="dataset/images/train")
+    a = ap.parse_args()
+
+    random.seed(0)
+    files = sorted(Path(a.images).glob("*.jpg"))
+    random.shuffle(files)
+    files = files[:a.calib]
+    g = onnx.load(a.src).graph
+    inp = g.input[0].name
+    # The Detect head's decode (DFL box decoding, class Sigmoid, final Concats) mixes box values up to 640 with
+    # 0-1 class scores in one tensor; INT8 would give them one shared scale and crush the scores. Keep it float.
+    exclude = [n.name for n in g.node
+               if n.name.startswith("/model.23/") and not n.name.startswith(("/model.23/cv2.", "/model.23/cv3."))]
+    print(f"keeping {len(exclude)} decode nodes in float")
+    quantize_static(a.src, a.dst, Reader(files, inp), quant_format=QuantFormat.QDQ, per_channel=True,
+                    weight_type=QuantType.QInt8, activation_type=QuantType.QUInt8,
+                    calibrate_method=CalibrationMethod.MinMax, nodes_to_exclude=exclude)
+    # keep ultralytics metadata (class names, stride, imgsz) so the model loads like the original
+    src, q = onnx.load(a.src), onnx.load(a.dst)
+    del q.metadata_props[:]
+    q.metadata_props.extend(src.metadata_props)
+    onnx.save(q, a.dst)
+    print(f"RES wrote {a.dst}: {Path(a.dst).stat().st_size / 1e6:.1f} MB (from {Path(a.src).stat().st_size / 1e6:.1f} MB)")
+
+
+if __name__ == "__main__":
+    main()

code/recall_at_conf.py

@@ -0,0 +1,56 @@
+#!/usr/bin/env python3
+"""Face recall at a fixed confidence threshold on WIDER val (what fraction of GT faces get a box with IoU>=0.5)."""
+import argparse
+from pathlib import Path
+
+import numpy as np
+from ultralytics import YOLO
+
+
+def iou(a, b):
+    x1 = np.maximum(a[:, None, 0], b[None, :, 0]); y1 = np.maximum(a[:, None, 1], b[None, :, 1])
+    x2 = np.minimum(a[:, None, 2], b[None, :, 2]); y2 = np.minimum(a[:, None, 3], b[None, :, 3])
+    inter = np.clip(x2 - x1, 0, None) * np.clip(y2 - y1, 0, None)
+    aa = (a[:, 2] - a[:, 0]) * (a[:, 3] - a[:, 1]); ab = (b[:, 2] - b[:, 0]) * (b[:, 3] - b[:, 1])
+    return inter / (aa[:, None] + ab[None, :] - inter + 1e-9)
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--model", default="export/face_yolo11n_fp16.onnx")
+    ap.add_argument("--conf", type=float, default=0.25)
+    ap.add_argument("--iou", type=float, default=0.5)
+    ap.add_argument("--val", default="dataset/images/val")
+    ap.add_argument("--device", default="0", help="0 for GPU, cpu for CPU")
+    ap.add_argument("--imgsz", type=int, default=640, help="inference size (.pt models only; ONNX exports are fixed)")
+    a = ap.parse_args()
+
+    model = YOLO(a.model, task="detect")
+    files = sorted(Path(a.val).glob("*.jpg"))
+    hits, sizes, n_pred, n_fp = [], [], 0, 0
+    for i in range(0, len(files), 1):  # ONNX export is fixed batch=1
+        batch = files[i:i + 1]
+        for f, r in zip(batch, model.predict([str(x) for x in batch], conf=a.conf, imgsz=a.imgsz, device=a.device, verbose=False)):
+            h, w = r.orig_shape
+            gt = np.array([[float(v) for v in l.split()[1:]] for l in
+                           Path(str(f).replace("images", "labels").replace(".jpg", ".txt")).read_text().split("\n") if l.strip()])
+            g = np.stack([(gt[:, 0] - gt[:, 2] / 2) * w, (gt[:, 1] - gt[:, 3] / 2) * h,
+                          (gt[:, 0] + gt[:, 2] / 2) * w, (gt[:, 1] + gt[:, 3] / 2) * h], 1)
+            p = r.boxes.xyxy.cpu().numpy()
+            n_pred += len(p)
+            if len(p):
+                m = iou(g, p)
+                hits += list(m.max(1) >= a.iou)
+                n_fp += int((m.max(0) < a.iou).sum())
+            else:
+                hits += [False] * len(g)
+            sizes += list(gt[:, 3] * h)  # face height in px at the image's stored (<=640) scale
+    hits, sizes = np.array(hits), np.array(sizes)
+    print(f"RES faces={len(hits)} overall recall@conf{a.conf}/IoU{a.iou} = {hits.mean():.1%}  (false-positive boxes: {n_fp}/{n_pred})")
+    for lo, hi in [(0, 10), (10, 20), (20, 40), (40, 80), (80, 1e9)]:
+        m = (sizes >= lo) & (sizes < hi)
+        print(f"RES   face height {lo:>3}-{int(min(hi, 999)):>3}px: {hits[m].mean():6.1%}  (n={m.sum()})")
+
+
+if __name__ == "__main__":
+    main()

code/train.py

@@ -0,0 +1,93 @@
+#!/usr/bin/env python3
+"""Fine-tune YOLO11n (COCO-pretrained) as a single-class face detector on converted WIDER FACE.
+
+Adds video-like degradation (motion blur + JPEG compression) on top of ultralytics' default augmentation,
+implemented with plain cv2 so no albumentations / second OpenCV build is needed.
+
+Usage:
+  python train.py                       # full run (40 epochs)
+  python train.py --fraction 0.02 --epochs 1 --name smoke   # quick pipeline check
+"""
+import argparse
+import random
+from pathlib import Path
+
+import cv2
+import numpy as np
+from ultralytics import YOLO
+from ultralytics.data import augment
+
+
+def _motion_blur(img, max_k=15):
+    k = random.choice(range(3, max_k + 1, 2))
+    kernel = np.zeros((k, k), np.float32)
+    kernel[k // 2, :] = 1.0
+    rot = cv2.getRotationMatrix2D((k / 2 - 0.5, k / 2 - 0.5), random.uniform(0, 180), 1.0)
+    kernel = cv2.warpAffine(kernel, rot, (k, k))
+    s = kernel.sum()
+    return img if s == 0 else cv2.filter2D(img, -1, kernel / s)
+
+
+def _jpeg(img, qmin=20, qmax=90):
+    ok, enc = cv2.imencode(".jpg", img, [cv2.IMWRITE_JPEG_QUALITY, random.randint(qmin, qmax)])
+    return cv2.imdecode(enc, cv2.IMREAD_COLOR) if ok else img
+
+
+def install_video_degradation(p_blur=0.3, p_jpeg=0.3):
+    """Wrap ultralytics' Albumentations step (runs after mosaic/affine, train only) with cv2 degradations."""
+    orig = augment.Albumentations.__call__
+
+    def call(self, labels):
+        img = labels["img"]
+        if random.random() < p_blur:
+            img = _motion_blur(img)
+        if random.random() < p_jpeg:
+            img = _jpeg(img)
+        labels["img"] = img
+        return orig(self, labels)
+
+    augment.Albumentations.__call__ = call
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--data", default="dataset/data.yaml")
+    ap.add_argument("--model", default="yolo11n.pt")
+    ap.add_argument("--epochs", type=int, default=40)
+    ap.add_argument("--batch", type=int, default=64)
+    ap.add_argument("--imgsz", type=int, default=640)
+    ap.add_argument("--workers", type=int, default=16)
+    ap.add_argument("--fraction", type=float, default=1.0, help="fraction of train set (smoke tests)")
+    ap.add_argument("--cache", default="disk", help="ram | disk | False")
+    ap.add_argument("--project", default=str(Path(__file__).resolve().parent / "runs"))
+    ap.add_argument("--name", default="face_yolo11n")
+    ap.add_argument("--no-degrade", action="store_true")
+    ap.add_argument("--resume", default=None,
+                    help="path to a stopped run's last.pt; continues it and finishes at --epochs "
+                         "(the LR schedule is recomputed for the new total)")
+    a = ap.parse_args()
+
+    if not a.no_degrade:
+        install_video_degradation()
+
+    if a.resume:
+        import torch
+        ck = torch.load(a.resume, map_location="cpu", weights_only=False)
+        ck["train_args"]["epochs"] = a.epochs
+        torch.save(ck, a.resume)
+        model = YOLO(a.resume)
+        model.train(resume=True)
+    else:
+        model = YOLO(a.model)
+        model.train(
+            data=a.data, epochs=a.epochs, imgsz=a.imgsz, batch=a.batch, workers=a.workers,
+            fraction=a.fraction, cache=False if a.cache == "False" else a.cache, amp=True,
+            close_mosaic=5, patience=0, project=a.project, name=a.name, exist_ok=True,
+            plots=False,
+        )
+    m = model.val(data=a.data, imgsz=a.imgsz, batch=a.batch)
+    print(f"RESULT mAP50={m.box.map50:.4f} mAP50-95={m.box.map:.4f} P={m.box.mp:.4f} R={m.box.mr:.4f}")
+
+
+if __name__ == "__main__":
+    main()

yolo11n/face_yolo11n.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:98b9fedf995fdc7bf20c6bbccde5d1ca46b3fb908843207be67794395d6d6a28
+size 5447834

yolo11n/face_yolo11n_fp16.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:958d99534290613fb7fb646467671a893b5ebc9941de385ceea039cc0612e827
+size 5355828

yolo11n/face_yolo11n_fp32.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9506b1f2b497cf0d3c39eb756ecacef82965d6d01e8037f3c2a91f6cb1b22297
+size 10604582

yolo11n/face_yolo11n_int8.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:48a6930873aee81ce40464b13de001aff90757e86f119d08cac642e9e4d80778
+size 3258276

yolo11s/face_yolo11s.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:09cca2067809392d575e947b6be2a7b4bc09123b078c60f5f41155afdae833c9
+size 19155034

yolo11s/face_yolo11s_fp16.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a2be67ca882dd4e694a407a59b1c9762508e3756b7077b8f919799b2ed492e72
+size 19017872

yolo11s/face_yolo11s_fp32.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1c4eac37a7a5efbe791225022799ac8980743bf0271204f43f782b327785c1eb
+size 37928280