Model card: Tell (final)
App model: Tell v2 (final_model_v2/ = run 20261015_112338_V2_V2_full_s42), trained with synthetic edits,
swapped headlines, caption dropout and per stream auxiliary heads (see v2_report.md). 106.9 M parameters,
31.8 M trainable. The Phase II model (final_model/, run 20261014_204630_P2_T4_tell_sdrop_s42) is kept for
comparison: it scores higher on the original test set but relies on caption style (stress test 0.27 vs 0.61).
Intended use
- A screening aid for fact checkers and content moderators. It ranks image and headline posts by how likely they are manipulated or miscaptioned, and shows where in the image to look.
- Teaching and research on multimodal misinformation and image forensics.
Not intended for
- Automatically deleting, demoting or labelling content without a human decision.
- Judging whether a claim is true. Tell checks image edits and image and caption agreement, not facts.
- Legal or forensic evidence.
- Identifying, profiling or tracking people. No face recognition is used.
- Non English captions, Indian regional news, video, or AI generated images. These were not in training, so performance there is unknown.
Training data
Fakeddit (Reddit, mostly US, English, 2008 to 2019) with distant supervision labels, and CASIA v2.0 (splicing and copy move edits). See dataset_card.md.
Known limitations
- Labels are noisy. They come from the subreddit, not from per post human checks.
- Caption shortcuts (measured). On the controlled subset the model scores 0.89 with the caption and 0.40 without it, below an image only model (0.61). Edited posts in the training data carry joke captions, and the model learned that style. A short, jokey headline on a real photo can push it towards manipulated.
- Mismatch detection is weak. When captions are swapped between posts, the share of posts called false connection rises only from 0.34 to 0.37. Do not rely on Tell to catch a real photo with a false caption.
- Edit types. CASIA edits are classic splicing and copy move. Modern AI inpainting and generative edits look different and may be missed.
- Compression (measured). JPEG quality 30 lowers macro F1 from 0.846 to 0.821; the version without stream dropout loses 4 points. WhatsApp forwarding and screenshots will weaken the edit evidence.
- Heatmaps. They are approximate and can light up on high texture regions (foliage, text overlays).
Risks and mitigations
| Risk | Mitigation |
|---|---|
| Over trust in a "verdict" | The app shows the probability, which evidence drove it, and a "screening signal, not proof" notice on every result |
| False accusation of an honest post | Intended only with a human in the loop. The false positive rate is reported per class |
| Adversarial evasion (re-compress after editing) | Measured in Phase II (section 5 of phase2_report.md); stream dropout halves the loss but does not remove it |
| Privacy | The app binds to this computer only (127.0.0.1); nothing is logged or saved by Tell. Gradio keeps uploads in a temporary folder, which the app purges every 5 minutes (files older than 10 minutes). No public API, no analytics |
| Malicious files | Uploads limited to 15 MB and 40 megapixels; unreadable or disguised files are rejected with a message; only files from the upload folder can be opened; the model loads with weights_only=True, so a swapped checkpoint cannot run code |
| Misleading heatmaps | The edit map uses the model's absolute 0 to 1 score, so a clean photo is not stretched to look suspicious; the app warns when a very short headline is likely to bias the verdict |
Metrics of Tell v2 (the app model)
| Metric | Value |
|---|---|
| Original test set macro F1 / accuracy | 0.811 / 0.814 |
| F1 authentic / manipulated / false connection | 0.757 / 0.851 / 0.826 |
| Stress test macro F1 (held out photos, synthetic edits and swapped headlines) | 0.614 |
| Stress test: splices / copy moves / swapped headlines caught, real photos kept | 80% / 50% / 48% / 71% |
| Controlled subset, caption removed | 0.404 (unchanged from Phase II) |
| Robustness: JPEG q30 / downscale x2 / blur | 0.790 / 0.796 / 0.768 |
| Latency per post (T4 GPU, batch 1) | 17.1 ms median |
Known v2 limits: no gain on real Photoshop Battles edits without a caption; still reacts to very short captions; more suspicious of real photos than Phase II (more false alarms); less robust to heavy blur.
Metrics of the Phase II model (for comparison; test set, 3,446 posts, resolution controlled images)
| Metric | Value |
|---|---|
| Macro F1 (seed 42 / mean of 3 seeds) | 0.846 / 0.841 ยฑ 0.004 |
| Accuracy | 0.846 |
| F1 authentic / manipulated / false connection | 0.821 / 0.890 / 0.826 |
| Controlled subset macro F1 | 0.893 |
| Controlled subset, caption removed | 0.403 |
| Caption removed / image removed (input level) | 0.415 / 0.425 |
| Caption streams zeroed inside the model | 0.642 |
| Robustness: JPEG q30 / downscale x2 / blur / caption swap | 0.821 / 0.835 / 0.816 / 0.514 |
| Forensic branch on CASIA test: pixel F1 / IoU / image AUC | 0.285 / 0.235 / 0.927 |
| Latency per post (T4 GPU, batch 1): median / p90 | 16.6 ms / 21.4 ms |
Baseline for comparison (late fusion, 3 seeds): macro F1 0.835 ยฑ 0.001. Weakest groups: r/confusing_perspective (47.8% accuracy) and r/misleadingthumbnails (68.0%).
Files
| File | What it is |
|---|---|
best.pt |
PyTorch checkpoint (float32), for app/app.py and web/server.py |
config.json, tokenizer/ |
model settings and the DistilBERT tokenizer |
tell_fp16.onnx |
the same model as one ONNX graph with half precision weights (computation stays in float32), for the in-browser website. Inputs: image [1,3,224,224] (normalised), input_ids and attention_mask [1,48] int64. Outputs: probs (authentic, manipulated, false_connection), gates (image, text, forensic, consistency), edit_map [1,1,224,224], focus [1,49]. On 52 check cases (the 10 test kit cases and variations) it gave the same answers as the PyTorch model, with probabilities within 0.0031. |
- Downloads last month
- 35