File size: 2,913 Bytes
35ec68e | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 | ---
library_name: pytorch
tags:
- face-recognition
- privacy
---
# DecoyFace
Implementation of DecoyFace split face recognition privacy protection:
client-side Decoy-Guided Coherent Mixing (DGCM) and the server-side Authorized Canonicalization Module (ACM),
together with the scripts that build the Reconstruction-Sensitive Subspace Decomposition (RSSD) statistics and train the server backbone.
The client encoder `F_c` is AdaFace IR-50 (MS1MV2) stem + stage 1 (output 64x56x56);
the server backbone `F_s` is the rest of the IR-50.
## Files
| File | Content |
| --- | --- |
| `decoyface.py` | `DecoyProtector`: `protect(z_A, z_B)` (client, DGCM) and `server_recover(z_t)` (server, ACM) |
| `models.py` | IR-50, its split into `ClientEncoder` (F_c) / `ServerBackbone` (F_s), and the proxy inversion U-Net used by RSSD |
| `common.py` | DecoyFace hyper-parameters `CFG`, fixed channel permutation, checkpoint paths |
| `data.py` | MS1M identity-folder dataset |
| `build_rssd.py` | Trains the proxy inversion U-Net, estimates the reconstruction-sensitive subspace -> `checkpoints/rssd.pt` |
| `build_bank.py` | Builds the auxiliary complementary bank and `mu_rec` -> `checkpoints/bank.pt` |
| `train_defender.py` | Trains F_s + ArcFace head on canonicalized protected features -> `checkpoints/defender.pt` |
## Checkpoints
| File | Content |
| --- | --- |
| `checkpoints/adaface_ir50_ms1mv2.backbone.pth` | AdaFace IR-50 MS1MV2 backbone state dict (from https://github.com/mk-minchul/AdaFace) |
| `checkpoints/rssd.pt` | `P_rec`, `P_N` (64x64), `U_rec`, `evals`, `sigma` |
| `checkpoints/bank.pt` | `aux_bank` (5000x64x56x56, fp16), `mu_rec` (64x56x56), `perm` (64) |
`proxy_unet.pt` and `defender.pt` are not included.
## Usage
```python
import torch
from common import ADAFACE_CKPT, CKPT, CFG
from models import load_ir50, ClientEncoder
from decoyface import DecoyProtector
Fc = ClientEncoder(load_ir50(ADAFACE_CKPT, "cuda")).cuda().eval()
rssd = torch.load(CKPT / "rssd.pt", map_location="cuda")
bank = torch.load(CKPT / "bank.pt", map_location="cuda")
protector = DecoyProtector(rssd["P_rec"], rssd["P_N"], bank["mu_rec"],
bank["aux_bank"].float(), bank["perm"], CFG, "cuda")
# x, decoy: (B,3,112,112) RGB in [0,1], ArcFace-aligned; decoy is a different identity
with torch.no_grad():
z_t = protector.protect(Fc(x), Fc(decoy)) # transmitted by the client
z_can = protector.server_recover(z_t) # fed to F_s on the server
```
## Rebuilding
MS1M root: identity folders named by integer, each holding 112x112 aligned `.jpg` faces.
Identities used: proxy U-Net first 5000, auxiliary bank 50000-60000, defender first 30000 (`--num_ids`).
```bash
python build_rssd.py --ms1m_root /path/to/ms1m
python build_bank.py --ms1m_root /path/to/ms1m
python train_defender.py --ms1m_root /path/to/ms1m
```
Scripts log to Weights & Biases (project `decoyface-repro`).
|