UrbanAnonymizer

Face and License Plate Anonymization for Street-Level Images and Video

Mehmet Kerem Turkcan
Columbia University

UrbanAnonymizer detects faces and license plates in street-level imagery and masks them. A single two-class detector covers both object types. For video, an offline forward–backward tracker extends each mask across the frames in which the detector briefly loses the object, so a face that is detected in some frames is masked in all of them.

Highlights

• Accurate on external benchmarks. Face AP₅₀ of 80.6 on UFDD and 56.5 on PP4AV, and plate AP₅₀ of 82.5 on PP4AV, against 65.8, 25.7 and 62.1 for EgoBlur Gen2.
• Faces and plates in one model. D-FINE-X (HGNetv2-B5) at 1408 px input, trained on 119k images from five public datasets with class-verified partial labels.
• Temporally complete masking. Offline forward–backward tracking removes 93% of the frames in which a face drops out of the mask between detections on MOT17 and MOT20.
• Deployable. ONNX exports and a TensorRT 11 build path; 39 fps on 1080p video on one A100.

Explainer Video

Benchmarks

Every method is scored by the same evaluator (eval/evaluate_detector.py) on the same annotations: COCO AP and AP₅₀, with each baseline run through its publisher's preprocessing at a 0.001 score floor (0.05 for CenterFace). UFDD and PP4AV are external benchmarks. The held-out and CRPD test sets are disjoint from training but come from sources that also contribute training images.

Faces.

Method	Held-out AP50	Held-out AP	UFDD AP50	UFDD AP	PP4AV AP50	PP4AV AP
YuNet (OpenCV Zoo)	67.1	35.8	64.1	27.3	35.1	9.0
CenterFace (deface)	69.6	37.2	69.9	30.2	27.1	6.4
YOLOv8n-face	60.6	33.6	59.3	25.6	10.5	2.6
EgoBlur Gen1	64.4	37.5	64.6	27.7	26.3	7.1
EgoBlur Gen2	65.2	38.3	65.8	28.3	25.7	6.8
Ours, RT-DETRv2-X @1280	81.6	50.5	78.8	34.7	52.9	16.2
Ours, D-FINE-X @1408	83.4	52.3	80.6	36.0	56.5	17.7

License plates.

Method	Held-out AP50	Held-out AP	PP4AV AP50	PP4AV AP	CRPD AP50	CRPD AP
YOLO11x-plate	79.9	44.2	40.8	14.8	90.6	52.0
EgoBlur Gen1	89.3	65.0	60.3	30.2	96.0	77.0
EgoBlur Gen2	89.8	64.5	62.1	30.2	94.8	73.3
Ours, RT-DETRv2-X @1280	91.0	64.4	74.7	38.4	95.0	72.4
Ours, D-FINE-X @1408	92.9	68.0	82.5	43.7	98.0	84.1

Recall at the recommended thresholds (IoU 0.5; masked area is the mean fraction of each image covered by masks with 20% padding).

Method	Thresholds (face / plate)	UFDD face	PP4AV face	PP4AV plate	CRPD plate	UFDD masked area
EgoBlur Gen1	0.9 / 0.9	47.0	11.2	14.8	89.4	2.7%
EgoBlur Gen2	0.674 / 0.745	56.5	14.5	23.0	75.1	2.9%
Ours, D-FINE-X @1408	0.1 / 0.2	87.4	82.4	81.9	99.8	8.1%

Per-benchmark metric files, including threshold sweeps, size bins and mask coverage, are in results/metrics/; results/tables.md is generated from them.

Video: Forward–Backward Tracking

tracker/offline_tracker.py detects every frame at a low score floor, links detections into tracks forward and backward in time, and masks every frame of each confirmed track: its low-score detections, interpolated gaps, and a short extrapolation past its ends. Detections at the operating threshold are always masked, so the output never masks less than per-frame anonymization.

MOTChallenge provides pedestrian boxes but no face boxes, so we measure against the head region of each annotated pedestrian (visibility ≥ 0.5, height ≥ 100 px) on the MOT17 and MOT20 training sequences. A transient exposure is a frame in which a head is unmasked although it is masked within 1/3 s before and after.

	Per-frame	Forward–backward tracking
Head-frames masked	87.4%	95.8%
Transient exposures	4,540	303
Late onset / early release	1,531 / 1,640	428 / 530

--face-confirm 0.2 confirms tracks from stronger detections and masks less of the frame (5.5% instead of 7.2% on these sequences) while still removing 87% of transient exposures.

Available Checkpoints

Checkpoint	Architecture	Input	Thresholds (face / plate)	Use case
dfine_x_1408/model.pth	D-FINE-X, HGNetv2-B5	1408	0.1 / 0.2	Highest accuracy (recommended)
dfine_x_1408/model.onnx	same, ONNX opset 16, FP32	1408	0.1 / 0.2	ONNX Runtime
dfine_x_1408/model_amp_fp16.onnx	same, traced under FP16 autocast	1408	0.1 / 0.2	TensorRT 11, strongly typed
dfine_x_1408/tensorrt/model_trt11_fp16_b4.plan	TensorRT 11.3 engine, A100	1408	0.1 / 0.2	Fastest on A100 (39 fps at 1080p)
rtdetrv2_x_1280/model.pth	RT-DETRv2-X, ResNet-101	1280	0.1 / 0.1	Faster video (56 fps at 1080p with TensorRT)

Each checkpoint folder contains its resolved config.yml. Weights are EMA weights without optimizer state.

Quick Start

Install the dependencies:

pip install -r requirements.txt

Download the repository:

huggingface-cli download mehmetkeremturkcan/UrbanAnonymizer --local-dir UrbanAnonymizer
cd UrbanAnonymizer

Images

python scripts/anonymize.py input.jpg output.jpg \
    --config dfine_x_1408/config.yml --checkpoint dfine_x_1408/model.pth \
    --size 1408 --face-score 0.1 --plate-score 0.2

import sys
sys.path.insert(0, "scripts")
from PIL import Image
from anonymize import Detector, apply_masks

detector = Detector("dfine_x_1408/config.yml", "dfine_x_1408/model.pth", device="cuda:0", size=1408)
image = Image.open("input.jpg")
detections = [d for d in detector.predict(image, score_threshold=0.1)
              if d["score"] >= (0.1 if d["category_id"] == 0 else 0.2)]   # 0 = face, 1 = license plate
apply_masks(image, detections, mode="opaque", padding=0.2).save("output.jpg")

Video

Offline, with forward–backward tracking (recommended):

python tracker/offline_tracker.py input.mp4 output.mp4 \
    --config dfine_x_1408/config.yml --checkpoint dfine_x_1408/model.pth --size 1408

Streaming, one frame at a time, with FFmpeg encoding:

python scripts/anonymize.py input.mp4 output.mp4 \
    --config dfine_x_1408/config.yml --checkpoint dfine_x_1408/model.pth \
    --size 1408 --face-score 0.1 --plate-score 0.2 \
    --frame-batch-size 4 --preprocess-workers 4 --video-writer ffmpeg --crf 18

TensorRT

dfine_x_1408/tensorrt/model_trt11_fp16_b4.plan is a strongly typed TensorRT 11.3 engine (batch 1–4) built from model_amp_fp16.onnx and validated against PyTorch on an A100. Install the runtime into its own directory and run:

pip install --target trt11 -r requirements-tensorrt.txt
python scripts/anonymize_tensorrt.py input.mp4 output.mp4 \
    --engine dfine_x_1408/tensorrt/model_trt11_fp16_b4.plan \
    --engine-metadata dfine_x_1408/tensorrt/tensorrt_build_provenance.json --trt-runtime-dir trt11 \
    --config dfine_x_1408/config.yml --size 1408 --face-score 0.1 --plate-score 0.2 \
    --frame-batch-size 4 --preprocess-workers 4 --video-writer ffmpeg --crf 18

TensorRT engines are specific to the GPU architecture and TensorRT version. For other GPUs, rebuild with scripts/build_tensorrt_engine.py --onnx dfine_x_1408/model_amp_fp16.onnx --size 1408 --precision fp32 --strongly-typed and validate the engine before use.

Reproducing the Comparison

eval/ contains the complete pipeline behind the benchmark tables:

Script	Purpose
evaluate_detector.py	Runs our checkpoints (sharded) and scores any prediction files
run_egoblur.py	EgoBlur Gen1 and Gen2 with the official preprocessing and thresholds
run_baseline.py	CenterFace (deface), YuNet, YOLOv8n-face and YOLO11x-plate
download_baselines.sh	Baseline weights and the pinned EgoBlur code
compute_metrics.py	Scores every method on every benchmark, threshold-free and at the recommended thresholds
make_tables.py	Writes results/tables.md and results/tables.json
../tracker/eval_mot_temporal.py	Temporal exposure analysis on MOT17 and MOT20

The held-out and CRPD test sets are part of the dataset repository; UFDD and PP4AV are obtained from their publishers and converted to the same COCO format in data/benchmarks/.

hf download mehmetkeremturkcan/UrbanAnonymizer-Dataset --repo-type dataset --local-dir data
for shard in data/data/test*.tar; do tar -xf "$shard" -C data; done
eval/download_baselines.sh

# predictions, one file per method and benchmark (shown for the held-out test set)
python eval/evaluate_detector.py --annotations data/annotations/test.json --image-root data \
    --config dfine_x_1408/config.yml --checkpoint dfine_x_1408/model.pth --size 1408 \
    --skip-metrics --output-prefix predictions/dfine_x_1408_heldout_s0of1
python eval/run_egoblur.py --annotations data/annotations/test.json --image-root data --generation gen2 \
    --min-score 0.001 --skip-metrics --output-prefix predictions/egoblur_gen2_heldout_s0of1
python eval/run_baseline.py --method centerface --annotations data/annotations/test.json --image-root data \
    --min-score 0.05 --output-prefix predictions/centerface_heldout_s0of1

# metrics and tables
python eval/compute_metrics.py --config eval/benchmarks.json --output-dir results/metrics
python eval/make_tables.py --metrics-dir results/metrics --output results/tables.md

All scripts accept --shard-index and --num-shards for parallel runs.

Dataset

The models are trained on UrbanAnonymizer-Dataset: 119,384 images and 347,427 face and plate boxes from Open Images, WIDER FACE, DARK FACE, CRPD and BirdsEye-RU, plus verified negative images, with per-image lists of the classes whose labels are complete.

Citation

@misc{turkcan2026urbananonymizer,
  title        = {UrbanAnonymizer: Face and License Plate Anonymization for Street-Level Images and Video},
  author       = {Turkcan, Mehmet Kerem},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/mehmetkeremturkcan/UrbanAnonymizer}}
}

License

The weights are released under CC BY-NC 4.0 because several training sources restrict commercial use. The bundled D-FINE and RT-DETR sources in external/ keep their Apache 2.0 licenses. Explainer footage: MOT17 and MOT20 (CC BY-NC-SA 3.0) and Wikimedia Commons clips under CC0, CC BY 3.0 and CC BY-SA 4.0, credited in the video.