File size: 2,062 Bytes
fb92c8e | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 | ---
license: other
license_name: polyform-noncommercial-1.0.0
license_link: https://polyformproject.org/licenses/noncommercial/1.0.0
tags:
- deepfake-detection
- ensemble
---
# DeepSafe Ensemble Artifacts
The meta-learners that turn 19 individual detector scores into one calibrated
verdict, for [DeepSafe](https://github.com/deepsafehq/deepsafe-bench).
Unlike the model code and weights in the other DeepSafe repositories, **these
are first-party**: trained by us, licensed PolyForm Noncommercial 1.0.0, same
as the project.
## Contents
| Modality | Meta-learner | Held-out AUC |
|---|---|---|
| Image | LightGBM over 7 models | 0.9466 |
| Audio | Random Forest over 3 models | 0.8290 |
| Video | XGBoost over 9 models | 0.6694 |
Each modality ships four files:
- `<modality>_meta_learner.pkl` — the trained model
- `<modality>_scaler.pkl` — feature scaling
- `<modality>_calibrator.pkl` — Platt calibration, so scores read as probabilities
- `<modality>_config.json` — feature order and fallback weights
Trained on the 15,499-sample medium evaluation tier. Without these, the
inference server produces per-model scores but no ensemble verdict.
## Usage
`setup.sh` fetches these automatically. Manually:
```python
from huggingface_hub import snapshot_download
snapshot_download("deepsafe/ensemble", local_dir="models/ensemble/artifacts")
```
## The numbers are the point
Held-out video AUC is **0.6694**. That is barely above chance on generators the
models were not trained for, and it is lower than the cross-validated figure
produced during training. We publish the held-out number because the gap
between the two is the finding. See
[BENCHMARK.md](https://github.com/deepsafehq/deepsafe-bench/blob/main/BENCHMARK.md).
## Security note
These are Python pickles, which execute code on load. Only load them from a
source you trust. DeepSafe's loader refuses any pickle outside its configured
artifacts directory, but that is a guardrail, not a guarantee. If you are
security-sensitive, retrain your own with `deepsafe fit --tier 1`.
|