v0.2-clip-lntune-sbi
Browse files
README.md
CHANGED
|
@@ -1,25 +1,26 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: other
|
| 3 |
-
license_name: research-only
|
| 4 |
-
license_link: https://github.com/ondyari/FaceForensics
|
| 5 |
-
tags: [deepfake-detection, face-forensics, clip]
|
| 6 |
-
---
|
| 7 |
-
|
| 8 |
-
# deepfake-detect models
|
| 9 |
-
|
| 10 |
-
Trained heads for [deepfake-detect](https://github.com/CodeRafay/deepfake-detect): small classifiers that run on top of frozen `openai/clip-vit-large-patch14` face embeddings. Each version lives in its own folder and is never overwritten.
|
| 11 |
-
|
| 12 |
-
| version | what changed | FF++ test AUC | Celeb-DF v2 AUC (unseen) | size |
|
| 13 |
-
|---|---|---|---|---|
|
| 14 |
-
| [v0.
|
| 15 |
-
|
| 16 |
-
|
| 17 |
-
|
| 18 |
-
|
| 19 |
-
|
| 20 |
-
deepfake-detect
|
| 21 |
-
deepfake-detect video.mp4
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
**
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: other
|
| 3 |
+
license_name: research-only
|
| 4 |
+
license_link: https://github.com/ondyari/FaceForensics
|
| 5 |
+
tags: [deepfake-detection, face-forensics, clip]
|
| 6 |
+
---
|
| 7 |
+
|
| 8 |
+
# deepfake-detect models
|
| 9 |
+
|
| 10 |
+
Trained heads for [deepfake-detect](https://github.com/CodeRafay/deepfake-detect): small classifiers that run on top of frozen `openai/clip-vit-large-patch14` face embeddings. Each version lives in its own folder and is never overwritten.
|
| 11 |
+
|
| 12 |
+
| version | what changed | FF++ test AUC | Celeb-DF v2 AUC (unseen) | size |
|
| 13 |
+
|---|---|---|---|---|
|
| 14 |
+
| [v0.2-clip-lntune-sbi](v0.2-clip-lntune-sbi) | **latest**: CLIP LayerNorms tuned + self-blended fakes + flip/rescale/JPEG augmentation (best of 5 epochs: 4) | 0.965 | **0.914** | 425 KB |
|
| 15 |
+
| [v0.1-clip-linear-probe](v0.1-clip-linear-probe) | baseline: frozen CLIP + logistic regression, 8 frames/video, FF++ c23 | 0.905 | 0.738 | 3.7 KB |
|
| 16 |
+
|
| 17 |
+
Every folder contains `head.safetensors`, the model, with its backbone, threshold and metrics stored inside the file, `head.metrics.json` with all numbers from the training run, and `training_run.ipynb`, the executed training notebook with its full log, so every result can be checked.
|
| 18 |
+
|
| 19 |
+
```bash
|
| 20 |
+
pip install git+https://github.com/CodeRafay/deepfake-detect
|
| 21 |
+
deepfake-detect video.mp4 # latest version
|
| 22 |
+
deepfake-detect video.mp4 --head v0.1-clip-linear-probe # a specific version
|
| 23 |
+
```
|
| 24 |
+
|
| 25 |
+
**License:** research and non-commercial use only. The heads are trained on FaceForensics++, whose terms apply.
|
| 26 |
+
**Use responsibly:** a score is evidence, not proof. Don't make decisions about people based on it alone.
|
v0.2-clip-lntune-sbi/head.metrics.json
ADDED
|
@@ -0,0 +1,38 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"val": {
|
| 3 |
+
"files": 699,
|
| 4 |
+
"auc": 0.9648223869154102,
|
| 5 |
+
"ap": 0.9911666051223886,
|
| 6 |
+
"balanced_acc": 0.9132890365448505,
|
| 7 |
+
"frame_auc": 0.9402297059763502
|
| 8 |
+
},
|
| 9 |
+
"test": {
|
| 10 |
+
"files": 700,
|
| 11 |
+
"auc": 0.9649617346938776,
|
| 12 |
+
"ap": 0.9915034903372775,
|
| 13 |
+
"balanced_acc": 0.9026785714285714,
|
| 14 |
+
"frame_auc": 0.9341166515166013
|
| 15 |
+
},
|
| 16 |
+
"celebdf": {
|
| 17 |
+
"files": 518,
|
| 18 |
+
"auc": 0.9136979510905486,
|
| 19 |
+
"ap": 0.9516875403534745,
|
| 20 |
+
"balanced_acc": 0.8367151354923992,
|
| 21 |
+
"frame_auc": 0.8426357884591469
|
| 22 |
+
},
|
| 23 |
+
"config": {
|
| 24 |
+
"data": "data",
|
| 25 |
+
"eval": "['celebdf=celebdf']",
|
| 26 |
+
"out": "head.safetensors",
|
| 27 |
+
"backbone": "openai/clip-vit-large-patch14",
|
| 28 |
+
"frames": "8",
|
| 29 |
+
"crops": "crops",
|
| 30 |
+
"fakes": "both",
|
| 31 |
+
"epochs": "5",
|
| 32 |
+
"epoch_size": "20000",
|
| 33 |
+
"batch": "32",
|
| 34 |
+
"lr": "0.0001",
|
| 35 |
+
"head_lr": "0.001",
|
| 36 |
+
"best_epoch": 4
|
| 37 |
+
}
|
| 38 |
+
}
|
v0.2-clip-lntune-sbi/head.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8fe0e51c7b0508b786da75c11c1d235df7a547d1ab9894c0c5b9c941034ca1a6
|
| 3 |
+
size 425444
|
v0.2-clip-lntune-sbi/training_run.ipynb
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
{"cells":[{"cell_type":"markdown","id":"5a19338d","metadata":{"papermill":{"duration":0.001439,"end_time":"2026-10-19T13:34:03.036187+00:00","exception":false,"start_time":"2026-10-19T13:34:03.034748+00:00","status":"completed"},"tags":[]},"source":["# Train deepfake-detect on Kaggle\n","\n","Trains **v0.2**: CLIP with only its LayerNorms tuned, plus a linear head, on **FaceForensics++ (c23)** face crops with self-blended fakes, using the official train/val/test splits. Then it tests on **Celeb-DF v2**, a dataset the model never sees during training. That second number is the honest one. (v0.1's frozen linear probe is `python -m deepfake_detect.train` with the same arguments.)\n","\n","**Before running**\n","1. Settings → Accelerator → **GPU T4 x2** (one GPU is used). Turn **Internet on**.\n","2. Both datasets need an access request to their authors ([FaceForensics++](https://github.com/ondyari/FaceForensics), [Celeb-DF](https://github.com/yuezunli/celeb-deepfakeforensics)). Their licences forbid redistribution, so upload your copy as a **private** Kaggle dataset and attach it with *Add Input*.\n","3. Attach both datasets with *Add Input*. The notebook finds them under `/kaggle/input/` by itself.\n","\n","Expected runtime: ~2–3 h: ~1 h extracting face crops on all CPU cores (cached in `crops/` for re-runs), then ~1 h of training and evaluation on the GPU."]},{"cell_type":"code","execution_count":1,"id":"2f0e722f","metadata":{"execution":{"iopub.execute_input":"2026-10-19T13:34:03.039771Z","iopub.status.busy":"2026-10-19T13:34:03.039164Z","iopub.status.idle":"2026-10-19T13:34:26.496145Z","shell.execute_reply":"2026-10-19T13:34:26.495319Z"},"papermill":{"duration":23.460743,"end_time":"2026-10-19T13:34:26.498093+00:00","exception":false,"start_time":"2026-10-19T13:34:03.03735+00:00","status":"completed"},"tags":[]},"outputs":[{"name":"stdout","output_type":"stream","text":[" Installing build dependencies ... \u001b[?25l\u001b[?25hdone\r\n"," Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\r\n"," Preparing metadata (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\r\n"," Building wheel for deepfake-detect (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\r\n","\u001b[32m.\u001b[0m\u001b[32m.\u001b[0m\u001b[32m.\u001b[0m\u001b[32m.\u001b[0m\u001b[32m.\u001b[0m\u001b[32m.\u001b[0m\u001b[32m.\u001b[0m\u001b[32m [100%]\u001b[0m\r\n","\u001b[32m\u001b[32m\u001b[1m7 passed\u001b[0m\u001b[32m in 8.45s\u001b[0m\u001b[0m\r\n"]}],"source":["REPO = \"https://github.com/CodeRafay/deepfake-detect\"\n","BRANCH = \"main\"\n","!rm -rf /kaggle/working/deepfake-detect\n","!git clone -q -b $BRANCH $REPO /kaggle/working/deepfake-detect\n","!pip install -q /kaggle/working/deepfake-detect pytest\n","!python -m pytest -q /kaggle/working/deepfake-detect/tests"]},{"cell_type":"code","execution_count":2,"id":"cb70e426","metadata":{"execution":{"iopub.execute_input":"2026-10-19T13:34:26.502569Z","iopub.status.busy":"2026-10-19T13:34:26.502275Z","iopub.status.idle":"2026-10-19T13:34:53.591009Z","shell.execute_reply":"2026-10-19T13:34:53.589996Z"},"papermill":{"duration":27.092783,"end_time":"2026-10-19T13:34:53.592449+00:00","exception":false,"start_time":"2026-10-19T13:34:26.499666+00:00","status":"completed"},"tags":[]},"outputs":[{"name":"stdout","output_type":"stream","text":["FF++: /kaggle/input/notebooks/rafayadeel/ffpp-c23-download/ffpp\n","Celeb-DF: /kaggle/input/datasets/rafayadeel/celeb-df-v2\n","/kaggle/working/data/train/real: 720 videos\n","/kaggle/working/data/train/fake: 2880 videos\n","/kaggle/working/data/val/real: 140 videos\n","/kaggle/working/data/val/fake: 560 videos\n","/kaggle/working/data/test/real: 140 videos\n","/kaggle/working/data/test/fake: 560 videos\n","/kaggle/working/celebdf/real: 178 videos\n","/kaggle/working/celebdf/fake: 340 videos\n"]}],"source":["import json, urllib.request\n","from pathlib import Path\n","\n","def find_input(name):\n"," \"\"\"Kaggle mounts inputs at varying depths (e.g. /kaggle/input/datasets/<user>/<name>/), so search for a marker.\"\"\"\n"," hits = sorted(Path(\"/kaggle/input\").rglob(name))\n"," if not hits:\n"," raise FileNotFoundError(f\"no '{name}' under /kaggle/input - is the dataset attached via Add Input?\")\n"," return hits[0].parent\n","\n","FFPP = find_input(\"original_sequences\") # folder holding original_sequences/ and manipulated_sequences/\n","CELEBDF = find_input(\"List_of_testing_videos.txt\") # folder holding Celeb-real/, Celeb-synthesis/, YouTube-real/\n","print(\"FF++:\", FFPP)\n","print(\"Celeb-DF:\", CELEBDF)\n","METHODS = [\"Deepfakes\", \"Face2Face\", \"FaceSwap\", \"NeuralTextures\"]\n","DATA, CDF = Path(\"/kaggle/working/data\"), Path(\"/kaggle/working/celebdf\")\n","\n","def link(src, dst):\n"," if src.exists() and not dst.exists():\n"," dst.parent.mkdir(parents=True, exist_ok=True)\n"," dst.symlink_to(src)\n","\n","# Official FF++ splits are by source-video pair, so no identity appears in two splits.\n","for split in [\"train\", \"val\", \"test\"]:\n"," url = f\"https://raw.githubusercontent.com/ondyari/FaceForensics/master/dataset/splits/{split}.json\"\n"," for a, b in json.load(urllib.request.urlopen(url)):\n"," for v in (a, b):\n"," link(FFPP / f\"original_sequences/youtube/c23/videos/{v}.mp4\", DATA / split / \"real\" / f\"{v}.mp4\")\n"," for m in METHODS:\n"," for x, y in ((a, b), (b, a)):\n"," link(FFPP / f\"manipulated_sequences/{m}/c23/videos/{x}_{y}.mp4\", DATA / split / \"fake\" / f\"{m}_{x}_{y}.mp4\")\n","\n","# Celeb-DF official test list: \"1 path\" = real, \"0 path\" = fake.\n","for line in (CELEBDF / \"List_of_testing_videos.txt\").read_text().splitlines():\n"," if line.strip():\n"," lab, rel = line.split()\n"," link(CELEBDF / rel, CDF / (\"real\" if lab == \"1\" else \"fake\") / rel.replace(\"/\", \"_\"))\n","\n","for d in [DATA / s / c for s in [\"train\", \"val\", \"test\"] for c in [\"real\", \"fake\"]] + [CDF / \"real\", CDF / \"fake\"]:\n"," print(f\"{d}: {len(list(d.glob('*')))} videos\")"]},{"cell_type":"code","execution_count":3,"id":"2d117b1f","metadata":{"execution":{"iopub.execute_input":"2026-10-19T13:34:53.596684Z","iopub.status.busy":"2026-10-19T13:34:53.595944Z","iopub.status.idle":"2026-10-19T17:04:37.471222Z","shell.execute_reply":"2026-10-19T17:04:37.470212Z"},"papermill":{"duration":12583.879243,"end_time":"2026-10-19T17:04:37.473125+00:00","exception":false,"start_time":"2026-10-19T13:34:53.593882+00:00","status":"completed"},"tags":[]},"outputs":[{"name":"stdout","output_type":"stream","text":["crops train: 100%|████████████████████████| 3600/3600 [1:31:50<00:00, 1.53s/it]\r\n","data/train: 28752 face crops from 3600 files\r\n","crops val: 100%|██████████████████████████████| 700/700 [16:38<00:00, 1.43s/it]\r\n","data/val: 5590 face crops from 700 files\r\n","crops test: 100%|█████████████████████████████| 700/700 [17:32<00:00, 1.50s/it]\r\n","data/test: 5599 face crops from 700 files\r\n","crops celebdf: 100%|██████████████████████████| 518/518 [01:42<00:00, 5.04it/s]\r\n","celebdf: 4139 face crops from 518 files\r\n","Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.\r\n","preprocessor_config.json: 100%|████████████████| 316/316 [00:00<00:00, 1.46MB/s]\r\n","config.json: 100%|█████████████████████████| 4.52k/4.52k [00:00<00:00, 11.2MB/s]\r\n","\r\n","model.safetensors: downloading bytes: ██▍ | 193MB, 14.1MB/s \r\n","model.safetensors: reconstructing file: 4%|▏ | 67.1MB / 1.71GB \u001b[A\r\n","model.safetensors: downloading bytes: ██████▏ | 477MB, 39.4MB/s \r\n","model.safetensors: downloading bytes: ███████▋ | 597MB, 50.0MB/s \r\n","model.safetensors: reconstructing file: 39%|██▎ | 663MB / 1.71GB, 49.1MB/s \u001b[A\r\n","model.safetensors: downloading bytes: ██████████ | 782MB, 64.4MB/s \r\n","model.safetensors: downloading bytes: ███████████ | 862MB, 72.6MB/s \r\n","model.safetensors: downloading bytes: █████████████▎ | 1.03GB, 85.2MB/s \r\n","model.safetensors: downloading bytes: ██████████████ | 1.09GB, 90.4MB/s \r\n","model.safetensors: downloading bytes: ██████████████▊ | 1.16GB, 94.0MB/s \r\n","model.safetensors: reconstructing file: 86%|█████▏| 1.48GB / 1.71GB, 119MB/s \u001b[A\r\n","model.safetensors: downloading bytes: ██████████████████████| 1.18GB, 96.6MB/s \r\n","model.safetensors: reconstructing file: 100%|██████| 1.71GB / 1.71GB, 143MB/s \r\n","Loading weights: 100%|██████████████████████| 392/392 [00:00<00:00, 2692.43it/s]\r\n","[transformers] \u001b[1mCLIPVisionModelWithProjection LOAD REPORT\u001b[0m from: openai/clip-vit-large-patch14\r\n","Key | Status | | \r\n","-------------------------------------------------------------+------------+--+-\r\n","text_model.encoder.layers.{0...11}.layer_norm1.weight | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","text_model.encoder.layers.{0...11}.mlp.fc1.weight | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","text_model.encoder.layers.{0...11}.self_attn.out_proj.weight | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","text_model.encoder.layers.{0...11}.self_attn.k_proj.weight | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","text_model.encoder.layers.{0...11}.mlp.fc2.weight | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","text_model.encoder.layers.{0...11}.mlp.fc1.bias | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","text_model.encoder.layers.{0...11}.layer_norm2.bias | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","text_model.encoder.layers.{0...11}.self_attn.q_proj.bias | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","text_model.encoder.layers.{0...11}.self_attn.v_proj.bias | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","text_model.encoder.layers.{0...11}.self_attn.k_proj.bias | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","text_model.encoder.layers.{0...11}.self_attn.v_proj.weight | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","text_model.encoder.layers.{0...11}.mlp.fc2.bias | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","text_model.encoder.layers.{0...11}.layer_norm2.weight | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","text_model.encoder.layers.{0...11}.layer_norm1.bias | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","text_model.encoder.layers.{0...11}.self_attn.out_proj.bias | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","text_model.embeddings.position_embedding.weight | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","text_model.encoder.layers.{0...11}.self_attn.q_proj.weight | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","text_model.embeddings.token_embedding.weight | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","text_model.final_layer_norm.weight | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","text_projection.weight | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","logit_scale | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","text_model.final_layer_norm.bias | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","\r\n","Notes:\r\n","- \u001b[38;5;208mUNEXPECTED:\u001b[0m\t\u001b[3mcan be ignored when loading from different task/architecture; not ok if you expect identical arch.\u001b[0m\r\n","training 102,400 CLIP parameters + 769 head\r\n","epoch 1/5: 100%|██████████████████| 625/625 [13:59<00:00, 1.34s/it, loss=0.373]\r\n","epoch 1: val AUC 0.9305\r\n","epoch 2/5: 100%|██████████████████| 625/625 [14:12<00:00, 1.36s/it, loss=0.205]\r\n","epoch 2: val AUC 0.9583\r\n","epoch 3/5: 100%|██████████████████| 625/625 [14:14<00:00, 1.37s/it, loss=0.476]\r\n","epoch 3: val AUC 0.9633\r\n","epoch 4/5: 100%|██████████████████| 625/625 [14:14<00:00, 1.37s/it, loss=0.471]\r\n","epoch 4: val AUC 0.9649\r\n","epoch 5/5: 100%|██████████████████| 625/625 [14:10<00:00, 1.36s/it, loss=0.163]\r\n","epoch 5: val AUC 0.9649\r\n","best epoch: 4\r\n","Loading weights: 100%|██████████████████████| 392/392 [00:00<00:00, 1346.27it/s]\r\n","[transformers] \u001b[1mCLIPVisionModelWithProjection LOAD REPORT\u001b[0m from: openai/clip-vit-large-patch14\r\n","Key | Status | | \r\n","-------------------------------------------------------------+------------+--+-\r\n","text_model.encoder.layers.{0...11}.layer_norm1.weight | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","text_model.encoder.layers.{0...11}.mlp.fc1.weight | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","text_model.encoder.layers.{0...11}.self_attn.out_proj.weight | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","text_model.encoder.layers.{0...11}.self_attn.k_proj.weight | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","text_model.encoder.layers.{0...11}.mlp.fc2.weight | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","text_model.encoder.layers.{0...11}.mlp.fc1.bias | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","text_model.encoder.layers.{0...11}.layer_norm2.bias | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","text_model.encoder.layers.{0...11}.self_attn.q_proj.bias | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","text_model.encoder.layers.{0...11}.self_attn.v_proj.bias | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","text_model.encoder.layers.{0...11}.self_attn.k_proj.bias | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","text_model.encoder.layers.{0...11}.self_attn.v_proj.weight | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","text_model.encoder.layers.{0...11}.mlp.fc2.bias | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","text_model.encoder.layers.{0...11}.layer_norm2.weight | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","text_model.encoder.layers.{0...11}.layer_norm1.bias | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","text_model.encoder.layers.{0...11}.self_attn.out_proj.bias | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","text_model.embeddings.position_embedding.weight | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","text_model.encoder.layers.{0...11}.self_attn.q_proj.weight | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","text_model.embeddings.token_embedding.weight | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","text_model.final_layer_norm.weight | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","text_projection.weight | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","logit_scale | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","text_model.final_layer_norm.bias | \u001b[38;5;208mUNEXPECTED\u001b[0m | | \r\n","\r\n","Notes:\r\n","- \u001b[38;5;208mUNEXPECTED:\u001b[0m\t\u001b[3mcan be ignored when loading from different task/architecture; not ok if you expect identical arch.\u001b[0m\r\n","\r\n","threshold = 0.5616 (chosen on val)\r\n","\r\n","| set | files | AUC | AP | balanced acc | frame AUC |\r\n","|---|---|---|---|---|---|\r\n","| val | 699 | 0.965 | 0.991 | 0.913 | 0.940 |\r\n","| test | 700 | 0.965 | 0.992 | 0.903 | 0.934 |\r\n","| celebdf | 518 | 0.914 | 0.952 | 0.837 | 0.843 |\r\n","\r\n","saved head.safetensors (425 KB)\r\n"]}],"source":["!cd /kaggle/working && python -m deepfake_detect.finetune --data data --eval celebdf=celebdf --out head.safetensors"]},{"cell_type":"markdown","id":"f7a8b028","metadata":{"papermill":{"duration":0.112861,"end_time":"2026-10-19T17:04:37.703273+00:00","exception":false,"start_time":"2026-10-19T17:04:37.590412+00:00","status":"completed"},"tags":[]},"source":["Paste the table above into the README's **Results** section. Report it as is, even if the Celeb-DF number is unflattering. Next, sanity-check a few test videos:"]},{"cell_type":"code","execution_count":4,"id":"89f96836","metadata":{"execution":{"iopub.execute_input":"2026-10-19T17:04:37.936153Z","iopub.status.busy":"2026-10-19T17:04:37.9357Z","iopub.status.idle":"2026-10-19T17:04:52.757511Z","shell.execute_reply":"2026-10-19T17:04:52.756537Z"},"papermill":{"duration":14.941857,"end_time":"2026-10-19T17:04:52.759075+00:00","exception":false,"start_time":"2026-10-19T17:04:37.817218+00:00","status":"completed"},"tags":[]},"outputs":[{"name":"stderr","output_type":"stream","text":["Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.\n"]},{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"f8ed6d2a51974928b04ecc8ccdbef2e5","version_major":2,"version_minor":0},"text/plain":["Loading weights: 0%| | 0/392 [00:00<?, ?it/s]"]},"metadata":{},"output_type":"display_data"},{"name":"stderr","output_type":"stream","text":["[transformers] \u001b[1mCLIPVisionModelWithProjection LOAD REPORT\u001b[0m from: openai/clip-vit-large-patch14\n","Key | Status | | \n","-------------------------------------------------------------+------------+--+-\n","text_model.encoder.layers.{0...11}.self_attn.v_proj.weight | UNEXPECTED | | \n","text_model.encoder.layers.{0...11}.layer_norm1.weight | UNEXPECTED | | \n","text_model.encoder.layers.{0...11}.layer_norm1.bias | UNEXPECTED | | \n","text_model.encoder.layers.{0...11}.self_attn.q_proj.bias | UNEXPECTED | | \n","text_model.encoder.layers.{0...11}.self_attn.k_proj.weight | UNEXPECTED | | \n","text_model.encoder.layers.{0...11}.layer_norm2.bias | UNEXPECTED | | \n","text_model.encoder.layers.{0...11}.mlp.fc1.bias | UNEXPECTED | | \n","text_model.encoder.layers.{0...11}.mlp.fc2.bias | UNEXPECTED | | \n","logit_scale | UNEXPECTED | | \n","text_model.encoder.layers.{0...11}.self_attn.out_proj.weight | UNEXPECTED | | \n","text_model.encoder.layers.{0...11}.mlp.fc2.weight | UNEXPECTED | | \n","text_model.encoder.layers.{0...11}.mlp.fc1.weight | UNEXPECTED | | \n","text_model.final_layer_norm.bias | UNEXPECTED | | \n","text_model.encoder.layers.{0...11}.layer_norm2.weight | UNEXPECTED | | \n","text_model.encoder.layers.{0...11}.self_attn.v_proj.bias | UNEXPECTED | | \n","text_model.encoder.layers.{0...11}.self_attn.out_proj.bias | UNEXPECTED | | \n","text_model.embeddings.token_embedding.weight | UNEXPECTED | | \n","text_model.encoder.layers.{0...11}.self_attn.k_proj.bias | UNEXPECTED | | \n","text_model.embeddings.position_embedding.weight | UNEXPECTED | | \n","text_model.encoder.layers.{0...11}.self_attn.q_proj.weight | UNEXPECTED | | \n","text_model.final_layer_norm.weight | UNEXPECTED | | \n","text_projection.weight | UNEXPECTED | | \n","\n","Notes:\n","- UNEXPECTED:\tcan be ignored when loading from different task/architecture; not ok if you expect identical arch.\n"]},{"name":"stdout","output_type":"stream","text":["real real 0.3237\n","real real 0.3543\n","real fake 0.6283\n","fake fake 0.9679\n","fake fake 0.9259\n","fake fake 0.9758\n"]}],"source":["from deepfake_detect import Detector\n","det = Detector(\"/kaggle/working/head.safetensors\")\n","for p in sorted((CDF / \"real\").glob(\"*\"))[:3] + sorted((CDF / \"fake\").glob(\"*\"))[:3]:\n"," r = det.predict(p)\n"," print(p.parent.name, r[\"label\"], r[\"score\"])"]},{"cell_type":"markdown","id":"5584d450","metadata":{"papermill":{"duration":0.113129,"end_time":"2026-10-19T17:04:52.985527+00:00","exception":false,"start_time":"2026-10-19T17:04:52.872398+00:00","status":"completed"},"tags":[]},"source":["## Publish this version to Hugging Face\n","One-time setup: create a **Write** token at huggingface.co → Settings → Access Tokens, then add it to this notebook under *Add-ons → Secrets* with the label `HF_TOKEN`. Never paste the token into a cell.\n","\n","Set `VERSION` **before** running the notebook: `v<N>-<what changed>`, e.g. `v0.2-clip-lntune-sbi`. The head and metrics are uploaded into that folder. Existing versions are never overwritten. Afterwards, download the files and this executed notebook, and commit them to `models/<VERSION>/` on GitHub (as `training_run.ipynb`), plus a row in `models/README.md`."]},{"cell_type":"code","execution_count":5,"id":"151e32b0","metadata":{"execution":{"iopub.execute_input":"2026-10-19T17:04:53.223754Z","iopub.status.busy":"2026-10-19T17:04:53.223262Z","iopub.status.idle":"2026-10-19T17:04:53.362918Z","shell.execute_reply":"2026-10-19T17:04:53.362005Z"},"papermill":{"duration":0.26323,"end_time":"2026-10-19T17:04:53.364555+00:00","exception":false,"start_time":"2026-10-19T17:04:53.101325+00:00","status":"completed"},"tags":[]},"outputs":[{"name":"stdout","output_type":"stream","text":["No HF_TOKEN secret attached: skipping upload. head.safetensors is in this notebook's output.\n"]}],"source":["VERSION = \"v0.2-clip-lntune-sbi\" # v<N>-<what changed>; never reuse a published name\n","HF_REPO = \"CodeRafay/deepfake-detect\" # must match MODELS_REPO in deepfake_detect/model.py\n","\n","from huggingface_hub import CommitOperationAdd, HfApi\n","from kaggle_secrets import UserSecretsClient\n","\n","try:\n"," token = UserSecretsClient().get_secret(\"HF_TOKEN\")\n","except Exception:\n"," token = None\n","\n","if \"CHANGE-ME\" in VERSION:\n"," print(\"VERSION not set: skipping upload. head.safetensors is in this notebook's output.\")\n","elif not token:\n"," print(\"No HF_TOKEN secret attached: skipping upload. head.safetensors is in this notebook's output.\")\n","else:\n"," api = HfApi(token=token)\n"," if api.file_exists(HF_REPO, f\"{VERSION}/head.safetensors\"):\n"," print(f\"{VERSION} already exists on Hugging Face and versions are never replaced. Pick a new VERSION.\")\n"," else:\n"," api.create_commit(HF_REPO, commit_message=f\"add {VERSION}\", operations=[\n"," CommitOperationAdd(f\"{VERSION}/{f}\", f\"/kaggle/working/{f}\") for f in [\"head.safetensors\", \"head.metrics.json\"]])\n"," print(f\"done: https://huggingface.co/{HF_REPO}/tree/main/{VERSION}\")"]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.13.15"},"papermill":{"default_parameters":{},"duration":12655.402698,"end_time":"2026-10-19T17:04:56.044272+00:00","environment_variables":{},"exception":null,"input_path":"__notebook__.ipynb","output_path":"__notebook__.ipynb","parameters":{},"start_time":"2026-10-19T13:34:00.641574+00:00","version":"2.7.0"},"widgets":{"application/vnd.jupyter.widget-state+json":{"state":{"1c36fb6c93b14c3787f762b810eed0ba":{"model_module":"@jupyter-widgets/controls","model_module_version":"2.0.0","model_name":"HTMLStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"2.0.0","_model_name":"HTMLStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"2.0.0","_view_name":"StyleView","background":null,"description_width":"","font_size":null,"text_color":null}},"1f2e423cfb29418a9fba881753dbfa78":{"model_module":"@jupyter-widgets/base","model_module_version":"2.0.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"2.0.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"2.0.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border_bottom":null,"border_left":null,"border_right":null,"border_top":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"6083107af700418a8c55d3614a6e0573":{"model_module":"@jupyter-widgets/base","model_module_version":"2.0.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"2.0.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"2.0.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border_bottom":null,"border_left":null,"border_right":null,"border_top":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"8af5adc44c39459098d3cc3bdca719ba":{"model_module":"@jupyter-widgets/controls","model_module_version":"2.0.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"2.0.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"2.0.0","_view_name":"HTMLView","description":"","description_allow_html":false,"layout":"IPY_MODEL_cf801e16aad040528e4ca05e55402047","placeholder":"","style":"IPY_MODEL_ac10e73693334a84bc733bb2ff430ac6","tabbable":null,"tooltip":null,"value":"Loading weights: 100%"}},"983cc805a2f14b9bbac434c1caf281fa":{"model_module":"@jupyter-widgets/controls","model_module_version":"2.0.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"2.0.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"2.0.0","_view_name":"ProgressView","bar_style":"success","description":"","description_allow_html":false,"layout":"IPY_MODEL_6083107af700418a8c55d3614a6e0573","max":392.0,"min":0.0,"orientation":"horizontal","style":"IPY_MODEL_c0b46b6f4819425da907a6ffdf23a17b","tabbable":null,"tooltip":null,"value":392.0}},"9ee6cae9f2264211968866adc74560a9":{"model_module":"@jupyter-widgets/controls","model_module_version":"2.0.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"2.0.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"2.0.0","_view_name":"HTMLView","description":"","description_allow_html":false,"layout":"IPY_MODEL_b4acbf8d0c3b4de39cca716f88f7cbb7","placeholder":"","style":"IPY_MODEL_1c36fb6c93b14c3787f762b810eed0ba","tabbable":null,"tooltip":null,"value":" 392/392 [00:00<00:00, 1122.08it/s]"}},"ac10e73693334a84bc733bb2ff430ac6":{"model_module":"@jupyter-widgets/controls","model_module_version":"2.0.0","model_name":"HTMLStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"2.0.0","_model_name":"HTMLStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"2.0.0","_view_name":"StyleView","background":null,"description_width":"","font_size":null,"text_color":null}},"b4acbf8d0c3b4de39cca716f88f7cbb7":{"model_module":"@jupyter-widgets/base","model_module_version":"2.0.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"2.0.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"2.0.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border_bottom":null,"border_left":null,"border_right":null,"border_top":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c0b46b6f4819425da907a6ffdf23a17b":{"model_module":"@jupyter-widgets/controls","model_module_version":"2.0.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"2.0.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"2.0.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"cf801e16aad040528e4ca05e55402047":{"model_module":"@jupyter-widgets/base","model_module_version":"2.0.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"2.0.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"2.0.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border_bottom":null,"border_left":null,"border_right":null,"border_top":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"f8ed6d2a51974928b04ecc8ccdbef2e5":{"model_module":"@jupyter-widgets/controls","model_module_version":"2.0.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"2.0.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"2.0.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_8af5adc44c39459098d3cc3bdca719ba","IPY_MODEL_983cc805a2f14b9bbac434c1caf281fa","IPY_MODEL_9ee6cae9f2264211968866adc74560a9"],"layout":"IPY_MODEL_1f2e423cfb29418a9fba881753dbfa78","tabbable":null,"tooltip":null}}},"version_major":2,"version_minor":0}},"kaggle":{"accelerator":"nvidiaTeslaT4","dataSources":[{"sourceType":"datasetVersion","sourceId":20818539},{"sourceType":"kernelVersion","sourceId":359024685}],"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat":4,"nbformat_minor":5}
|