shalev396/face-recognition

Face Recognition

Who is in this photo, and do two photos show the same person? Every network in this repo was trained in this project: a face detector finds the face and its five landmarks, the face is rotated and scaled onto a fixed template, two face embedders (an EfficientNet-B2 that starts from ImageNet weights and a ResNet-18 trained from scratch, both with a CosFace margin loss) turn it into an embedding, and a small MLP head names one of the 42 people of LFW with at least 25 photos. Verification compares two embeddings with the cosine distance.

No pretrained face network is part of the model.

Model

Detector	FaceDetector, our single-shot CNN: 5 stages of depthwise-separable residual blocks (BatchNorm + PReLU), a feature pyramid at strides 8/16/32 with a context module, and 10 square anchors (16-384 px). For every anchor it predicts face / not face, a box and 5 landmarks (eyes, nose tip, mouth corners). The largest face is used; no face found -> center square of the photo
Alignment	similarity transform (rotation, scale, shift) of the 5 landmarks onto the standard 112 x 112 face template, scaled to 128 x 128
Embedders	an ensemble of two CNNs, each backbone -> Dropout(0.2) -> Linear(-> 512) -> BatchNorm, flip-averaged and L2-normalised: EfficientNet-B2 (torchvision ImageNet weights as the starting point, 128 px input) and ResNet-18 (random initialisation, 128 px). The face embedding is their concatenation / sqrt(2): 1024-d, L2-normalised
Head	Linear(1024, 256) -> ReLU -> Dropout(0.3) -> Linear(256, 42), softmax
Parameters	22,837,410 in the deployed pipeline: detector 2,700,598 + efficientnet_b2 embedder 8,423,426 + resnet18 embedder 11,440,192 + head 273,194
Input	an RGB photo (PIL image, path, bytes or base64). Photos larger than 1024 px are downscaled first
Output	predict: {person: probability} for all 42 people, best first · verify: {same_person, distance, threshold, faces_detected} · embed: 1024 floats
Files	face_detector.safetensors, model.safetensors + config.json (embedders, head, class names, crop size, verify threshold), model.py, handler.py

Face detector

The detector borrows the ideas of MTCNN (Zhang et al., 2016), not its code or its layers: each candidate is scored on three tasks at once (face / not face, box, 5 landmarks), PReLU activations, and online hard-negative mining (only the 3 hardest negatives per face count). Instead of MTCNN's image pyramid and three cascaded networks, one fully-convolutional network looks at the photo once, with a feature pyramid for small, medium and large faces. A cascaded second stage (a deeper 64 x 64 landmark/rescoring network, like MTCNN's O-Net) was also trained and tested: it made detection worse (WIDER AP 0.874 -> 0.842 on large faces) and did not improve the recognition accuracy, so it was dropped.

• Training data: 16,660 photos with 32,397 faces: every Open Images V7 validation + test photo with a human-drawn "Human face" box (Google; annotations CC BY 4.0, images CC BY 2.0) and 5,000 LFW photos (never the recognition test photos or the unseen people). The 5 landmark labels are a data file (training/labels/landmark_labels.json.gz on GitHub) made once with a reference detector (MTCNN, facenet-pytorch); no third-party code or weights are part of the training or of this repo.
• Recipe: 640 px random crops, zoom-outs and 2x2 mosaics, flips, colour jitter; AdamW (lr 2e-3, weight decay 5e-4), batch 16, 30 epochs, cosine schedule; the epoch with the best AP on 600 held-out Open Images photos is kept (epoch 21).
• Evaluation: AP at IoU 0.5 for large (>= 64 px), medium (>= 32 px) and small (>= 16 px) faces, compared with that reference detector (MTCNN, which was trained on WIDER FACE itself and CelebA). WIDER FACE val is all 3,226 photos, used for evaluation only (CC BY-NC-ND): it is home ground for MTCNN and new to ours; the held-out Open Images photos are home ground for ours (the same kind of photos and boxes it trained on).

	large	medium	small
ours, held-out Open Images	0.910	0.884	0.853
MTCNN (reference), held-out Open Images	0.738	0.720	0.705
ours, WIDER FACE val	0.897	0.825	0.655
MTCNN (reference), WIDER FACE val	0.813	0.763	0.640

Usage

from huggingface_hub import hf_hub_download, snapshot_download
import sys
path = snapshot_download("shalev396/face-recognition")
sys.path.insert(0, path)
import model
predictor = model.load(path, device="cpu")   # or "cuda"
image = hf_hub_download("shalev396/face-recognition", "examples/Vladimir_Putin.jpg", repo_type="space")
print(list(predictor.predict(image).items())[:3])   # [('Vladimir Putin', 0.9...), ...]
print(predictor.verify(image, image))                # {'same_person': True, 'distance': 0.0, ...}
print(len(predictor.embed(image)))                   # 1024

Requirements: torch, torchvision, pillow, numpy, safetensors, huggingface_hub.

• Space / free API: shalev396/face-recognition, POST /gradio_api/call/predict.
• Inference Endpoint: handler.py makes the repo deployable (Deploy -> Inference Endpoints). Body {"inputs": <base64 image>} -> probabilities; {"inputs": {"image_a": ..., "image_b": ...}} -> verification; "parameters": {"task": "embed"} -> the embedding.

Training

• Data: LFW (funneled): 13,233 photos (250 x 250) of 5,749 people. Every photo is aligned with the same code the Predictor runs, using the detector trained just before (it finds a face in 13,233 of 13,233 of them; LFW is labelled by the centered person, so training picks the most central large face).
• Splits (seeded, nothing overlaps):
  • the 42 people with >= 25 photos keep the original 75/25 split per person (seed 42), so the test split is the same 647 photos the earlier FaceNet version of this model was tested on. 15% of their training photos are the validation split used for every choice: 1,649 train / 292 validation / 647 test photos;
  • unseen people: 10% of the other people with at least 2 photos (163 people, 662 photos) are never trained on; they measure how well the embeddings work on faces the model has never seen;
  • the embedders train on the 42 people's training photos + every other person: 11,632 photos of 5,586 identities.
• Stage 1, embedders: CosFace (scale 30, margin 0.35) over all 5,586 identities, AdamW (lr 1e-3 for the projection and the CosFace weights; 0.3x for the ImageNet-initialised backbone), weight decay 5e-4, batch 128, 2 warm-up epochs then a cosine schedule, mixed precision on GPU. Augmentation on the GPU: zoom 70-100% of the area, shift, +-10 deg rotation, horizontal flip, brightness/contrast/saturation +-30%, 10% grayscale, 25% random erasing. EfficientNet-B2: 25 epochs; ResNet-18: 80 epochs. After every epoch the checkpoint is scored on the validation split (nearest-centroid accuracy) and the unseen people (verification AUC); the best epoch is kept (EfficientNet-B2: 21, ResNet-18: 49).
• Stage 2, head: the MLP on the frozen 1024-d embeddings, AdamW (lr 1e-3, weight decay 1e-4), batch 64, up to 200 epochs, the epoch with the lowest validation loss is kept (epoch 196).
• Verification threshold: picked on the validation pairs (best balanced accuracy): 0.675, stored in config.json.
• This checkpoint: trained on 2026-09-26 on an RTX 2080 Ti with training/ (191 min in total).

Full code: training/ · Colab.

Experiments

Every embedder below was trained with the same code, alignment, splits and seeds, then given the same MLP head. (These experiments ran before our detector existed, on faces aligned with MTCNN (facenet-pytorch); the final model was then retrained end to end with our detector.) They were compared on the validation split, then on the unseen people's verification AUC, then on size; the test column is shown for completeness and was not used to choose anything.

embedder (MLP head on top)	params	train acc.	val acc.	test acc.	unseen-people AUC
ResNet-18 scratch, 42 people only, unaligned	11.4M	1.0000	0.9281	0.9243	0.8381
ResNet-18 scratch, all LFW, unaligned	11.4M	1.0000	0.9760	0.9567	0.9399
ResNet-50 ImageNet init, unaligned	24.6M	1.0000	0.9863	0.9815	0.9404
ResNet-18 scratch, aligned 112 px	11.4M	1.0000	0.9932	0.9753	0.9475
ResNet-18 scratch, aligned 128 px	11.4M	1.0000	0.9932	0.9768	0.9614
ResNet-50 ImageNet init, aligned	24.6M	1.0000	0.9897	0.9892	0.9442
ResNet-50 ImageNet init, aligned, 15 epochs	24.6M	1.0000	0.9863	0.9861	0.9406
EfficientNet-B2 ImageNet init, aligned	8.4M	1.0000	0.9897	0.9845	0.9519
Ensemble: ResNet-50 ImageNet init + ResNet-18 scratch	36.0M	1.0000	0.9966	0.9938	0.9753
Ensemble: ResNet-50 ImageNet init + EfficientNet-B2 ImageNet init	33.0M	1.0000	0.9966	0.9938	0.9631
Ensemble: EfficientNet-B2 ImageNet init + ResNet-18 scratch (chosen)	19.9M	1.0000	0.9966	0.9954	0.9772

What the experiments showed:

• More identities beat more epochs. Trained only on the 42 people, a ResNet-18 memorises them (train 100%, test 92.4%, unseen AUC 0.84). Adding the other ~5,600 LFW people as extra classes lifts test to 95.7% and the unseen AUC to 0.94: the network has to learn what makes faces differ in general.
• Alignment is the biggest single gain. Rotating every face onto the 5-point template raised the same ResNet-18 from 95.7% to 97.5-97.7% test accuracy.
• ImageNet initialisation helps, but not on unseen faces. ResNet-50 and EfficientNet-B2 started from ImageNet weights reach 98.5-98.9% on the 42 people, but their unseen-people AUC (0.94-0.95) is no better than the from-scratch ResNet-18 (0.96), and the ResNet-50 peaks after ~9 epochs and then memorises the training identities.
• Ensembling different networks is what closes most of the gap: the errors of an ImageNet- initialised network and a from-scratch one are different, and concatenating their embeddings gives 99.4-99.5% test accuracy and 0.96-0.98 unseen AUC.

The chosen configuration was then retrained from scratch by the notebook (training/notebook.ipynb, the run that produced these weights), with our own detector doing the alignment; its numbers are in Evaluation below.

Evaluation

metric (test)	value
accuracy	0.9969
f1_macro	0.9961
top3_accuracy	0.9985
verify_roc_auc	0.9973
verify_balanced_accuracy	0.9916
verify_true_accept_rate	0.9860
verify_false_accept_rate	0.0028
unseen_verify_roc_auc	0.9750

Identification on the 647 test photos: 645 of 647 correct (accuracy 0.9969, macro F1 0.9961). Train / validation / test accuracy: 1.0000 / 0.9932 / 0.9969. The mistakes: Arnold Schwarzenegger predicted as David Beckham; George W Bush predicted as Hans Blix.

For comparison, the earlier version of this model (a frozen FaceNet InceptionResnetV1 pretrained on 3.3 million VGGFace2 photos, non-commercial licence) got 646 of these 647 photos right (0.9985). This model reaches 0.9969 with embedders that learnt faces from about 12,000 LFW photos only, and is better on verification (balanced accuracy 0.9916 vs 0.9882). Its weights are this project's own, released under the MIT licence.

Verification (verify: same person when the cosine distance is below the threshold 0.675):

pairs	ROC AUC	same-person pairs accepted	different-people pairs accepted	balanced accuracy
all test pairs of the 42 people (208,981 pairs)	0.9973	98.60%	0.28%	0.9916
all pairs of the unseen people (218,791 pairs)	0.9750	65.73%	0.33%	0.8270

On people the model has never seen, the embeddings still rank pairs well (AUC 0.9750), but at the threshold tuned on the 42 known people it rejects many same-person pairs: for new people, a looser threshold (the Verify tab's slider) works better.

License

MIT, Copyright (c) 2026 Shalev Ben Moshe (see LICENSE): all the code and all the weights in this repo (face_detector.safetensors, model.safetensors) were written and trained in this project.

Where the training started from and what it learned from (nothing of these is redistributed here):

• The EfficientNet-B2 embedder started from torchvision's ImageNet classification weights (torchvision is BSD-3-Clause; the ImageNet images are licensed for research), like the other ImageNet-initialised models in this lab. The detector and the ResNet-18 embedder started from random weights.
• Data: LFW photos of public figures collected from news articles, published by UMass Amherst for research; Open Images V7 face photos (Google; annotations CC BY 4.0, images CC BY 2.0). The detector's landmark labels were made by MTCNN (facenet-pytorch, MIT, Copyright (c) 2019 Timothy Esler), used only during training.

Sensitive use

Face recognition identifies people, so it can be used for surveillance and to track people without their knowledge. This model is a learning project. It is not meant for identifying private people, access control, law enforcement or any decision about a person.

• Only use it on photos of people who agreed to it (or on the public-figure LFW photos).
• The Space's Enroll tab keeps embeddings in the visitor's browser session only and writes nothing to disk.
• Do not store or share embeddings of people without their consent: an embedding is biometric data (for example under the GDPR).

Limitations

• Closed set of 42 people. Every face gets the closest of the 42 LFW identities; there is no "unknown" class. A low top probability is a hint, not a rejection rule.
• Small training set. The embedders saw about 12,000 photos of ~5,600 people, most of them with one or two photos; on faces unlike LFW (profile views, low light, children, masks) expect lower accuracy than on the LFW test photos.
• Easy benchmark. LFW photos are news photos, mostly frontal and well lit, and one person's photos often come from the same events, so train and test photos can be very similar.
• Skewed data. George W Bush has 530 photos, while 23 of the 42 people have fewer than 40. LFW is mostly white, male, adult public figures; errors across demographic groups were not measured.
• Verification threshold (0.675) was tuned on LFW validation pairs of the 42 known people. For other people and photos the distance distributions shift (see the unseen-people row).