| --- |
| library_name: pytorch |
| tags: |
| - face-recognition |
| - privacy |
| --- |
| |
| # DecoyFace |
|
|
| Implementation of DecoyFace split face recognition privacy protection: |
| client-side Decoy-Guided Coherent Mixing (DGCM) and the server-side Authorized Canonicalization Module (ACM), |
| together with the scripts that build the Reconstruction-Sensitive Subspace Decomposition (RSSD) statistics and train the server backbone. |
|
|
| The client encoder `F_c` is AdaFace IR-50 (MS1MV2) stem + stage 1 (output 64x56x56); |
| the server backbone `F_s` is the rest of the IR-50. |
|
|
| ## Files |
|
|
| | File | Content | |
| | --- | --- | |
| | `decoyface.py` | `DecoyProtector`: `protect(z_A, z_B)` (client, DGCM) and `server_recover(z_t)` (server, ACM) | |
| | `models.py` | IR-50, its split into `ClientEncoder` (F_c) / `ServerBackbone` (F_s), and the proxy inversion U-Net used by RSSD | |
| | `common.py` | DecoyFace hyper-parameters `CFG`, fixed channel permutation, checkpoint paths | |
| | `data.py` | MS1M identity-folder dataset | |
| | `build_rssd.py` | Trains the proxy inversion U-Net, estimates the reconstruction-sensitive subspace -> `checkpoints/rssd.pt` | |
| | `build_bank.py` | Builds the auxiliary complementary bank and `mu_rec` -> `checkpoints/bank.pt` | |
| | `train_defender.py` | Trains F_s + ArcFace head on canonicalized protected features -> `checkpoints/defender.pt` | |
| |
| ## Checkpoints |
| |
| | File | Content | |
| | --- | --- | |
| | `checkpoints/adaface_ir50_ms1mv2.backbone.pth` | AdaFace IR-50 MS1MV2 backbone state dict (from https://github.com/mk-minchul/AdaFace) | |
| | `checkpoints/rssd.pt` | `P_rec`, `P_N` (64x64), `U_rec`, `evals`, `sigma` | |
| | `checkpoints/bank.pt` | `aux_bank` (5000x64x56x56, fp16), `mu_rec` (64x56x56), `perm` (64) | |
|
|
| `proxy_unet.pt` and `defender.pt` are not included. |
|
|
| ## Usage |
|
|
| ```python |
| import torch |
| from common import ADAFACE_CKPT, CKPT, CFG |
| from models import load_ir50, ClientEncoder |
| from decoyface import DecoyProtector |
| |
| Fc = ClientEncoder(load_ir50(ADAFACE_CKPT, "cuda")).cuda().eval() |
| rssd = torch.load(CKPT / "rssd.pt", map_location="cuda") |
| bank = torch.load(CKPT / "bank.pt", map_location="cuda") |
| protector = DecoyProtector(rssd["P_rec"], rssd["P_N"], bank["mu_rec"], |
| bank["aux_bank"].float(), bank["perm"], CFG, "cuda") |
| |
| # x, decoy: (B,3,112,112) RGB in [0,1], ArcFace-aligned; decoy is a different identity |
| with torch.no_grad(): |
| z_t = protector.protect(Fc(x), Fc(decoy)) # transmitted by the client |
| z_can = protector.server_recover(z_t) # fed to F_s on the server |
| ``` |
|
|
| ## Rebuilding |
|
|
| MS1M root: identity folders named by integer, each holding 112x112 aligned `.jpg` faces. |
| Identities used: proxy U-Net first 5000, auxiliary bank 50000-60000, defender first 30000 (`--num_ids`). |
|
|
| ```bash |
| python build_rssd.py --ms1m_root /path/to/ms1m |
| python build_bank.py --ms1m_root /path/to/ms1m |
| python train_defender.py --ms1m_root /path/to/ms1m |
| ``` |
|
|
| Scripts log to Weights & Biases (project `decoyface-repro`). |
|
|
