Image Classification
Grok
numpy
raiw-source-classify
image-forensics
source-attribution
open-set-classification
ai-image-detection
ai-generated-content
provider-attribution
openai
gpt-image
dall-e
google
imagen
flux
midjourney
stable-diffusion
ideogram
firefly
seedream
Instructions to use wiltodelta/raiw-source-classify with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Grok
How to use wiltodelta/raiw-source-classify with Grok:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Source-classify freeze (GitHub 9df5c2d)
Browse files- README.md +41 -38
- metrics.json +60 -58
- multi-provider-source-v3.npz → multi-provider-source-v4.npz +2 -2
README.md
CHANGED
|
@@ -8,81 +8,84 @@ tags:
|
|
| 8 |
- open-set-classification
|
| 9 |
---
|
| 10 |
|
| 11 |
-
# RAIW multi-provider source-pipeline classifier (`multi-provider-source-
|
| 12 |
|
| 13 |
-
This repository contains the pickle-free (`allow_pickle=False`) derivative weights for the abstaining `remove-ai-watermarks` pixel source-pipeline classifier across **10 AI provider families** (`openai`, `google`, `bytedance`, `flux`, `stable_diffusion`, `meta`, `midjourney`, `ideogram`, `firefly`, `xai`) plus `unknown` (`classify_source` / `remove-ai-watermarks classify-source`).
|
| 14 |
|
| 15 |
This model is **not a SynthID detector** and does not decode or verify any cryptographic or invisible watermark payload (`watermark_truth="unknown"`). Its labels describe complete generation and export pixel pipelines, so renderer, geometry, encoding, post-processing, and watermark state may be confounded. `unknown` does not mean that SynthID is absent, and a provider label does not prove that SynthID is present.
|
| 16 |
|
| 17 |
## Model architecture
|
| 18 |
|
| 19 |
-
- **Input (`
|
| 20 |
-
- `
|
| 21 |
-
- `
|
| 22 |
-
- `
|
| 23 |
-
-
|
| 24 |
-
- `
|
| 25 |
-
- **Stage
|
| 26 |
-
-
|
| 27 |
-
-
|
| 28 |
-
- **
|
|
|
|
|
|
|
|
|
|
| 29 |
|
| 30 |
The artifact contains only learned parameters, normalization constants, tree arrays, labels, and calibrated thresholds. It does not contain training images, paths, image hashes, embeddings, or a training catalog.
|
| 31 |
|
| 32 |
-
## Evaluation (`
|
| 33 |
|
| 34 |
-
The operating point was selected on a disjoint `8,
|
| 35 |
|
| 36 |
-
- **Unified 10-provider router:** `99.
|
| 37 |
|
| 38 |
| Provider | Test count | Standalone ROC AUC | Router recall | Router precision | Stage-2 version accuracy |
|
| 39 |
|---|---:|---:|---:|---:|---:|
|
| 40 |
-
| `openai` | 1,478 | 0.
|
| 41 |
-
| `google` | 1,382 | 0.
|
| 42 |
-
| `bytedance` | 389 | 0.
|
| 43 |
-
| `flux` | 1,357 | 0.
|
| 44 |
-
| `stable_diffusion` | 955 | 0.
|
| 45 |
-
| `meta` | 94 | 0.
|
| 46 |
-
| `midjourney` | 123 |
|
| 47 |
-
| `ideogram` | 118 | 0.
|
| 48 |
-
| `firefly` | 103 | 0.
|
| 49 |
-
| `xai` | 48 | 0.
|
| 50 |
-
|
| 51 |
-
### Held-out real-image specificity by variety (`4,
|
| 52 |
|
| 53 |
| Real-image variety | Test count | False positives | False-positive rate | Specificity |
|
| 54 |
|---|---:|---:|---:|---:|
|
| 55 |
-
| `photo` | 2,183 |
|
| 56 |
-
| `photo_retouched` | 339 |
|
| 57 |
| `classical_inpaint` | 17 | 0 | 0.00% | 100.00% |
|
| 58 |
-
| `digital_art` |
|
| 59 |
-
| `game_art` | 174 |
|
| 60 |
-
| `vector` | 135 |
|
| 61 |
| `interface` | 180 | 0 | 0.00% | 100.00% |
|
| 62 |
-
| `diagram` | 98 |
|
| 63 |
| `chart_data_viz` | 159 | 0 | 0.00% | 100.00% |
|
| 64 |
| `technical_architecture` | 84 | 0 | 0.00% | 100.00% |
|
| 65 |
| `procedural_graphic` | 13 | 0 | 0.00% | 100.00% |
|
| 66 |
| `document` | 165 | 0 | 0.00% | 100.00% |
|
| 67 |
| `gov_scientific_doc` | 32 | 0 | 0.00% | 100.00% |
|
| 68 |
| `receipt_invoice` | 181 | 0 | 0.00% | 100.00% |
|
| 69 |
-
| `print_publishing` | 229 |
|
| 70 |
-
| `stress_recompressed` | 103 |
|
| 71 |
| `foreign_ai` | 6 | 0 | 0.00% | 100.00% |
|
| 72 |
|
| 73 |
### External blind multi-variety real-image verification (`selective-v8`, `2,267` untouched images)
|
| 74 |
|
| 75 |
-
Evaluated on the hash-disjoint external `selective-v8` blind real-image cohorts (`2,267` images never seen in train, calibration, or test): **2 / 2,267 false positives (`0.09%` overall FPR, `99.91%` specificity)**.
|
| 76 |
|
| 77 |
| Blind `selective-v8` cell | Evaluated | False positives | False-positive rate | Specificity |
|
| 78 |
|---|---:|---:|---:|---:|
|
| 79 |
| `photo` | 450 | 1 | 0.22% | 99.78% |
|
| 80 |
| `vector` | 332 | 0 | 0.00% | 100.00% |
|
| 81 |
-
| `game_art` | 230 |
|
| 82 |
| `interface` | 342 | 0 | 0.00% | 100.00% |
|
| 83 |
| `diagram` | 305 | 0 | 0.00% | 100.00% |
|
| 84 |
| `document` | 308 | 0 | 0.00% | 100.00% |
|
| 85 |
-
| `stress_recompressed` | 300 |
|
| 86 |
|
| 87 |
## Intended use
|
| 88 |
|
|
|
|
| 8 |
- open-set-classification
|
| 9 |
---
|
| 10 |
|
| 11 |
+
# RAIW multi-provider source-pipeline classifier (`multi-provider-source-v4`)
|
| 12 |
|
| 13 |
+
This repository contains the compact (`13.35 MB`), pickle-free (`allow_pickle=False`) derivative weights for the abstaining `remove-ai-watermarks` pixel source-pipeline classifier across **10 AI provider families** (`openai`, `google`, `bytedance`, `flux`, `stable_diffusion`, `meta`, `midjourney`, `ideogram`, `firefly`, `xai`) plus `unknown` (`classify_source` / `remove-ai-watermarks classify-source`).
|
| 14 |
|
| 15 |
This model is **not a SynthID detector** and does not decode or verify any cryptographic or invisible watermark payload (`watermark_truth="unknown"`). Its labels describe complete generation and export pixel pipelines, so renderer, geometry, encoding, post-processing, and watermark state may be confounded. `unknown` does not mean that SynthID is absent, and a provider label does not prove that SynthID is present.
|
| 16 |
|
| 17 |
## Model architecture
|
| 18 |
|
| 19 |
+
- **Input (`512-d`, `extract_512d_v4_features`):**
|
| 20 |
+
- `144-d` multi-scale spatial SRM residuals, directional kurtosis, and cross-channel Y/Co/Cg co-occurrence statistics.
|
| 21 |
+
- `128-d` phase-invariant modulo-8 / modulo-16 sublattice energy, VAE upsampler checkerboards, `8x8` block-DCT quantization combs, and 4:2:0 chroma block-boundary ratios.
|
| 22 |
+
- `128-d` phase-free multi-patch radial and azimuthal log-power spectra, high-frequency rolloff slopes, and resampling second-derivative spectral peaks.
|
| 23 |
+
- `80-d` camera-sensor physics (Bayer CFA demosaicing periodicity, local noise-level heteroscedasticity, highlight clipping, edge ringing, and document/text binarization cues).
|
| 24 |
+
- `32-d` explicit codec, aspect-ratio, scale, and social-recompression regime vector (`x[480:512]`).
|
| 25 |
+
- **Stage 1 (`132` unit channels + `68` 2D multiplicative interaction gates across `10` canonical providers):**
|
| 26 |
+
- Two-tower ensemble executed in pure NumPy: `5` multi-seed Regime-FiLM `LayerNorm` + `SiLU` residual MLP trunks (`11` provider/unknown logits + `45` subfamily logits) paired with `44` packed `HistGradientBoosting` specialist forests (global provider, subfamily, codec-regime, hard-domain veto, and pairwise cross-AI disambiguator heads).
|
| 27 |
+
- Smooth softplus hinge activations combined with `68` 2D multiplicative interaction gates (`provider x hard-domain veto`, `provider x pairwise disambiguator`, `MLP x HGB` agreement), standardized against a learned regime-conditioned null baseline `(z_raw - mu(r)) / sigma(r)` and routed via damped competitive margins ($\gamma = 0.18$).
|
| 28 |
+
- **Stage 2 (`9` stacked version heads, `644-d`):**
|
| 29 |
+
- Vectorized `HistGradientBoosting` tree ensembles taking the `512-d` descriptor concatenated with the `132` Stage-1 unit scores to predict the canonical model/export subfamily (`version_subfamily`).
|
| 30 |
+
- **Artifact:** `multi-provider-source-v4.npz` (`allow_pickle=False`, `13.35 MB`)
|
| 31 |
+
- **SHA-256:** `bc62ee22331c0137e76c53f5026e14d08cae12a3483fa69f3000316806c146a3`
|
| 32 |
|
| 33 |
The artifact contains only learned parameters, normalization constants, tree arrays, labels, and calibrated thresholds. It does not contain training images, paths, image hashes, embeddings, or a training catalog.
|
| 34 |
|
| 35 |
+
## Evaluation (`127,001` deduplicated feature views; `10,585` locked test)
|
| 36 |
|
| 37 |
+
The operating point was selected strictly on a disjoint calibration partition (`8,363` primary calibration views + `3,011` social-recompressed real negatives across 16 fine-grained real-image varieties and group-locked `parent_sha256` families). On the `10,585`-image locked test split (`4,538` multi-variety `unknown` negatives + `6,047` AI provider images across native, resized, and social-recompressed views):
|
| 38 |
|
| 39 |
+
- **Unified 10-provider router:** `99.76%` `unknown` specificity (`0.24%` test `unknown` FPR; `0.14%` photographic FPR and `0.34%` non-photographic structured FPR), `81.35%` exact provider recall (`85.23%` any-AI recall; `88.08%` native, `84.58%` resized, and `76.81%` recompressed exact recall), `99.79%` any-AI precision when firing (`95.44%` exact-provider accuracy among classified AI; `96.32%` overall Stage-2 version accuracy given exact provider), and `0.9974` binary ROC-AUC / `0.9982` binary PR-AUC (`0.9976` mean standalone provider ROC-AUC).
|
| 40 |
|
| 41 |
| Provider | Test count | Standalone ROC AUC | Router recall | Router precision | Stage-2 version accuracy |
|
| 42 |
|---|---:|---:|---:|---:|---:|
|
| 43 |
+
| `openai` | 1,478 | 0.9997 | 92.5% | 96.4% | 95.8% |
|
| 44 |
+
| `google` | 1,382 | 0.9993 | 85.5% | 96.3% | 97.3% |
|
| 45 |
+
| `bytedance` | 389 | 0.9853 | 69.9% | 98.9% | 97.8% |
|
| 46 |
+
| `flux` | 1,357 | 0.9992 | 80.6% | 94.2% | 96.2% |
|
| 47 |
+
| `stable_diffusion` | 955 | 0.9990 | 70.5% | 95.3% | 97.8% |
|
| 48 |
+
| `meta` | 94 | 0.9987 | 89.4% | 93.3% | - (`meta:imagine`) |
|
| 49 |
+
| `midjourney` | 123 | 1.0000 | 63.4% | 92.9% | 100.0% |
|
| 50 |
+
| `ideogram` | 118 | 0.9997 | 75.4% | 73.0% | 74.2% |
|
| 51 |
+
| `firefly` | 103 | 0.9959 | 58.3% | 96.8% | 100.0% |
|
| 52 |
+
| `xai` | 48 | 0.9996 | 43.8% | 100.0% | 71.4% |
|
| 53 |
+
|
| 54 |
+
### Held-out real-image specificity by variety (`4,538` test negatives across 16 real classes + foreign AI)
|
| 55 |
|
| 56 |
| Real-image variety | Test count | False positives | False-positive rate | Specificity |
|
| 57 |
|---|---:|---:|---:|---:|
|
| 58 |
+
| `photo` | 2,183 | 3 | 0.14% | 99.86% |
|
| 59 |
+
| `photo_retouched` | 339 | 1 | 0.29% | 99.71% |
|
| 60 |
| `classical_inpaint` | 17 | 0 | 0.00% | 100.00% |
|
| 61 |
+
| `digital_art` | 440 | 1 | 0.23% | 99.77% |
|
| 62 |
+
| `game_art` | 174 | 3 | 1.72% | 98.28% |
|
| 63 |
+
| `vector` | 135 | 1 | 0.74% | 99.26% |
|
| 64 |
| `interface` | 180 | 0 | 0.00% | 100.00% |
|
| 65 |
+
| `diagram` | 98 | 0 | 0.00% | 100.00% |
|
| 66 |
| `chart_data_viz` | 159 | 0 | 0.00% | 100.00% |
|
| 67 |
| `technical_architecture` | 84 | 0 | 0.00% | 100.00% |
|
| 68 |
| `procedural_graphic` | 13 | 0 | 0.00% | 100.00% |
|
| 69 |
| `document` | 165 | 0 | 0.00% | 100.00% |
|
| 70 |
| `gov_scientific_doc` | 32 | 0 | 0.00% | 100.00% |
|
| 71 |
| `receipt_invoice` | 181 | 0 | 0.00% | 100.00% |
|
| 72 |
+
| `print_publishing` | 229 | 0 | 0.00% | 100.00% |
|
| 73 |
+
| `stress_recompressed` | 103 | 2 | 1.94% | 98.06% |
|
| 74 |
| `foreign_ai` | 6 | 0 | 0.00% | 100.00% |
|
| 75 |
|
| 76 |
### External blind multi-variety real-image verification (`selective-v8`, `2,267` untouched images)
|
| 77 |
|
| 78 |
+
Evaluated on the hash-disjoint external `selective-v8` blind real-image cohorts (`2,267` images never seen in train, calibration, or test): **2 / 2,267 false positives (`0.09%` overall FPR, `99.91%` specificity)**, plus **0 / 60 false positives (`0.00%` FPR)** on external phone thermal receipts.
|
| 79 |
|
| 80 |
| Blind `selective-v8` cell | Evaluated | False positives | False-positive rate | Specificity |
|
| 81 |
|---|---:|---:|---:|---:|
|
| 82 |
| `photo` | 450 | 1 | 0.22% | 99.78% |
|
| 83 |
| `vector` | 332 | 0 | 0.00% | 100.00% |
|
| 84 |
+
| `game_art` | 230 | 0 | 0.00% | 100.00% |
|
| 85 |
| `interface` | 342 | 0 | 0.00% | 100.00% |
|
| 86 |
| `diagram` | 305 | 0 | 0.00% | 100.00% |
|
| 87 |
| `document` | 308 | 0 | 0.00% | 100.00% |
|
| 88 |
+
| `stress_recompressed` | 300 | 1 | 0.33% | 99.67% |
|
| 89 |
|
| 90 |
## Intended use
|
| 91 |
|
metrics.json
CHANGED
|
@@ -1,10 +1,10 @@
|
|
| 1 |
{
|
| 2 |
"claim": "abstaining 10-provider source-pipeline classification with version subfamily attribution; not SynthID detection",
|
| 3 |
-
"schema": "multi-provider-source-
|
| 4 |
-
"artifact_file": "multi-provider-source-
|
| 5 |
-
"artifact_sha256": "
|
| 6 |
-
"feature_dimension":
|
| 7 |
-
"stacked_stage2_dimension":
|
| 8 |
"classes": [
|
| 9 |
"openai",
|
| 10 |
"google",
|
|
@@ -19,61 +19,63 @@
|
|
| 19 |
"unknown"
|
| 20 |
],
|
| 21 |
"corpus_counts": {
|
| 22 |
-
"total":
|
| 23 |
-
"train":
|
| 24 |
-
"calibration":
|
| 25 |
-
"
|
|
|
|
| 26 |
},
|
| 27 |
"locked_internal_test": {
|
| 28 |
-
"n":
|
| 29 |
-
"binary_roc_auc": 0.
|
| 30 |
-
"binary_pr_auc": 0.
|
| 31 |
-
"unknown_specificity": 0.
|
| 32 |
-
"unknown_fpr": 0.
|
| 33 |
-
"photo_fpr": 0.
|
| 34 |
-
"structured_fpr": 0.
|
| 35 |
-
"aggregate_target_recall": 0.
|
|
|
|
| 36 |
"recall": {
|
| 37 |
-
"openai": 0.
|
| 38 |
-
"google": 0.
|
| 39 |
-
"bytedance": 0.
|
| 40 |
-
"flux": 0.
|
| 41 |
-
"stable_diffusion": 0.
|
| 42 |
-
"meta": 0.
|
| 43 |
-
"midjourney": 0.
|
| 44 |
-
"ideogram": 0.
|
| 45 |
-
"firefly": 0.
|
| 46 |
-
"xai": 0.
|
| 47 |
},
|
| 48 |
"precision": {
|
| 49 |
-
"openai": 0.
|
| 50 |
-
"google": 0.
|
| 51 |
-
"bytedance": 0.
|
| 52 |
-
"flux": 0.
|
| 53 |
-
"stable_diffusion": 0.
|
| 54 |
-
"meta":
|
| 55 |
-
"midjourney": 0.
|
| 56 |
-
"ideogram": 0.
|
| 57 |
-
"firefly": 0.
|
| 58 |
"xai": 1.0
|
| 59 |
},
|
| 60 |
"real_variety_specificity": {
|
| 61 |
-
"photo": 0.
|
| 62 |
-
"photo_retouched": 0.
|
| 63 |
"classical_inpaint": 1.0,
|
| 64 |
-
"digital_art": 0.
|
| 65 |
-
"game_art":
|
| 66 |
-
"vector":
|
| 67 |
"interface": 1.0,
|
| 68 |
-
"diagram":
|
| 69 |
"chart_data_viz": 1.0,
|
| 70 |
"technical_architecture": 1.0,
|
| 71 |
"procedural_graphic": 1.0,
|
| 72 |
"document": 1.0,
|
| 73 |
"gov_scientific_doc": 1.0,
|
| 74 |
"receipt_invoice": 1.0,
|
| 75 |
-
"print_publishing":
|
| 76 |
-
"stress_recompressed": 0.
|
| 77 |
"foreign_ai": 1.0
|
| 78 |
}
|
| 79 |
},
|
|
@@ -97,9 +99,9 @@
|
|
| 97 |
},
|
| 98 |
"game_art": {
|
| 99 |
"evaluated": 230,
|
| 100 |
-
"false_positives":
|
| 101 |
-
"fpr": 0.
|
| 102 |
-
"specificity":
|
| 103 |
},
|
| 104 |
"interface": {
|
| 105 |
"evaluated": 342,
|
|
@@ -121,21 +123,21 @@
|
|
| 121 |
},
|
| 122 |
"stress_recompressed": {
|
| 123 |
"evaluated": 300,
|
| 124 |
-
"false_positives":
|
| 125 |
-
"fpr": 0.
|
| 126 |
-
"specificity":
|
| 127 |
}
|
| 128 |
}
|
| 129 |
},
|
| 130 |
"stage2_version_accuracy": {
|
| 131 |
-
"openai": 0.
|
| 132 |
-
"google": 0.
|
| 133 |
-
"bytedance": 0.
|
| 134 |
-
"flux": 0.
|
| 135 |
-
"stable_diffusion": 0.
|
| 136 |
-
"midjourney":
|
| 137 |
-
"ideogram": 0.
|
| 138 |
"firefly": 1.0,
|
| 139 |
-
"xai": 0.
|
| 140 |
}
|
| 141 |
}
|
|
|
|
| 1 |
{
|
| 2 |
"claim": "abstaining 10-provider source-pipeline classification with version subfamily attribution; not SynthID detection",
|
| 3 |
+
"schema": "multi-provider-source-v4",
|
| 4 |
+
"artifact_file": "multi-provider-source-v4.npz",
|
| 5 |
+
"artifact_sha256": "bc62ee22331c0137e76c53f5026e14d08cae12a3483fa69f3000316806c146a3",
|
| 6 |
+
"feature_dimension": 512,
|
| 7 |
+
"stacked_stage2_dimension": 644,
|
| 8 |
"classes": [
|
| 9 |
"openai",
|
| 10 |
"google",
|
|
|
|
| 19 |
"unknown"
|
| 20 |
],
|
| 21 |
"corpus_counts": {
|
| 22 |
+
"total": 127001,
|
| 23 |
+
"train": 105042,
|
| 24 |
+
"calibration": 8363,
|
| 25 |
+
"calibration_recomp_neg": 3011,
|
| 26 |
+
"test": 10585
|
| 27 |
},
|
| 28 |
"locked_internal_test": {
|
| 29 |
+
"n": 10585,
|
| 30 |
+
"binary_roc_auc": 0.9974117831066663,
|
| 31 |
+
"binary_pr_auc": 0.9981946495110887,
|
| 32 |
+
"unknown_specificity": 0.997576024680476,
|
| 33 |
+
"unknown_fpr": 0.0024239753195240193,
|
| 34 |
+
"photo_fpr": 0.001374255611543747,
|
| 35 |
+
"structured_fpr": 0.003397027600849257,
|
| 36 |
+
"aggregate_target_recall": 0.8134612204398876,
|
| 37 |
+
"any_ai_recall": 0.8523234661815776,
|
| 38 |
"recall": {
|
| 39 |
+
"openai": 0.9248985115020297,
|
| 40 |
+
"google": 0.8545586107091172,
|
| 41 |
+
"bytedance": 0.699228791773779,
|
| 42 |
+
"flux": 0.8061901252763449,
|
| 43 |
+
"stable_diffusion": 0.7047120418848167,
|
| 44 |
+
"meta": 0.8936170212765957,
|
| 45 |
+
"midjourney": 0.6341463414634146,
|
| 46 |
+
"ideogram": 0.7542372881355932,
|
| 47 |
+
"firefly": 0.5825242718446602,
|
| 48 |
+
"xai": 0.4375
|
| 49 |
},
|
| 50 |
"precision": {
|
| 51 |
+
"openai": 0.964033850493653,
|
| 52 |
+
"google": 0.9632952691680261,
|
| 53 |
+
"bytedance": 0.9890909090909091,
|
| 54 |
+
"flux": 0.94229112833764,
|
| 55 |
+
"stable_diffusion": 0.953257790368272,
|
| 56 |
+
"meta": 0.9333333333333333,
|
| 57 |
+
"midjourney": 0.9285714285714286,
|
| 58 |
+
"ideogram": 0.7295081967213115,
|
| 59 |
+
"firefly": 0.967741935483871,
|
| 60 |
"xai": 1.0
|
| 61 |
},
|
| 62 |
"real_variety_specificity": {
|
| 63 |
+
"photo": 0.9986257443884563,
|
| 64 |
+
"photo_retouched": 0.9970501474926253,
|
| 65 |
"classical_inpaint": 1.0,
|
| 66 |
+
"digital_art": 0.9977272727272727,
|
| 67 |
+
"game_art": 0.9827586206896551,
|
| 68 |
+
"vector": 0.9925925925925926,
|
| 69 |
"interface": 1.0,
|
| 70 |
+
"diagram": 1.0,
|
| 71 |
"chart_data_viz": 1.0,
|
| 72 |
"technical_architecture": 1.0,
|
| 73 |
"procedural_graphic": 1.0,
|
| 74 |
"document": 1.0,
|
| 75 |
"gov_scientific_doc": 1.0,
|
| 76 |
"receipt_invoice": 1.0,
|
| 77 |
+
"print_publishing": 1.0,
|
| 78 |
+
"stress_recompressed": 0.9805825242718447,
|
| 79 |
"foreign_ai": 1.0
|
| 80 |
}
|
| 81 |
},
|
|
|
|
| 99 |
},
|
| 100 |
"game_art": {
|
| 101 |
"evaluated": 230,
|
| 102 |
+
"false_positives": 0,
|
| 103 |
+
"fpr": 0.0,
|
| 104 |
+
"specificity": 1.0
|
| 105 |
},
|
| 106 |
"interface": {
|
| 107 |
"evaluated": 342,
|
|
|
|
| 123 |
},
|
| 124 |
"stress_recompressed": {
|
| 125 |
"evaluated": 300,
|
| 126 |
+
"false_positives": 1,
|
| 127 |
+
"fpr": 0.0033333333333333335,
|
| 128 |
+
"specificity": 0.9966666666666667
|
| 129 |
}
|
| 130 |
}
|
| 131 |
},
|
| 132 |
"stage2_version_accuracy": {
|
| 133 |
+
"openai": 0.958302852962692,
|
| 134 |
+
"google": 0.9729043183742591,
|
| 135 |
+
"bytedance": 0.9779411764705882,
|
| 136 |
+
"flux": 0.9616087751371115,
|
| 137 |
+
"stable_diffusion": 0.9777117384843982,
|
| 138 |
+
"midjourney": 1.0,
|
| 139 |
+
"ideogram": 0.7415730337078652,
|
| 140 |
"firefly": 1.0,
|
| 141 |
+
"xai": 0.7142857142857143
|
| 142 |
}
|
| 143 |
}
|
multi-provider-source-v3.npz → multi-provider-source-v4.npz
RENAMED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:bc62ee22331c0137e76c53f5026e14d08cae12a3483fa69f3000316806c146a3
|
| 3 |
+
size 13996740
|