Model card: Tell (final)

App model: Tell v2 (final_model_v2/ = run 20261015_112338_V2_V2_full_s42), trained with synthetic edits, swapped headlines, caption dropout and per stream auxiliary heads (see v2_report.md). 106.9 M parameters, 31.8 M trainable. The Phase II model (final_model/, run 20261014_204630_P2_T4_tell_sdrop_s42) is kept for comparison: it scores higher on the original test set but relies on caption style (stress test 0.27 vs 0.61).

Intended use

  • A screening aid for fact checkers and content moderators. It ranks image and headline posts by how likely they are manipulated or miscaptioned, and shows where in the image to look.
  • Teaching and research on multimodal misinformation and image forensics.

Not intended for

  • Automatically deleting, demoting or labelling content without a human decision.
  • Judging whether a claim is true. Tell checks image edits and image and caption agreement, not facts.
  • Legal or forensic evidence.
  • Identifying, profiling or tracking people. No face recognition is used.
  • Non English captions, Indian regional news, video, or AI generated images. These were not in training, so performance there is unknown.

Training data

Fakeddit (Reddit, mostly US, English, 2008 to 2019) with distant supervision labels, and CASIA v2.0 (splicing and copy move edits). See dataset_card.md.

Known limitations

  • Labels are noisy. They come from the subreddit, not from per post human checks.
  • Caption shortcuts (measured). On the controlled subset the model scores 0.89 with the caption and 0.40 without it, below an image only model (0.61). Edited posts in the training data carry joke captions, and the model learned that style. A short, jokey headline on a real photo can push it towards manipulated.
  • Mismatch detection is weak. When captions are swapped between posts, the share of posts called false connection rises only from 0.34 to 0.37. Do not rely on Tell to catch a real photo with a false caption.
  • Edit types. CASIA edits are classic splicing and copy move. Modern AI inpainting and generative edits look different and may be missed.
  • Compression (measured). JPEG quality 30 lowers macro F1 from 0.846 to 0.821; the version without stream dropout loses 4 points. WhatsApp forwarding and screenshots will weaken the edit evidence.
  • Heatmaps. They are approximate and can light up on high texture regions (foliage, text overlays).

Risks and mitigations

Risk Mitigation
Over trust in a "verdict" The app shows the probability, which evidence drove it, and a "screening signal, not proof" notice on every result
False accusation of an honest post Intended only with a human in the loop. The false positive rate is reported per class
Adversarial evasion (re-compress after editing) Measured in Phase II (section 5 of phase2_report.md); stream dropout halves the loss but does not remove it
Privacy The app binds to this computer only (127.0.0.1); nothing is logged or saved by Tell. Gradio keeps uploads in a temporary folder, which the app purges every 5 minutes (files older than 10 minutes). No public API, no analytics
Malicious files Uploads limited to 15 MB and 40 megapixels; unreadable or disguised files are rejected with a message; only files from the upload folder can be opened; the model loads with weights_only=True, so a swapped checkpoint cannot run code
Misleading heatmaps The edit map uses the model's absolute 0 to 1 score, so a clean photo is not stretched to look suspicious; the app warns when a very short headline is likely to bias the verdict

Metrics of Tell v2 (the app model)

Metric Value
Original test set macro F1 / accuracy 0.811 / 0.814
F1 authentic / manipulated / false connection 0.757 / 0.851 / 0.826
Stress test macro F1 (held out photos, synthetic edits and swapped headlines) 0.614
Stress test: splices / copy moves / swapped headlines caught, real photos kept 80% / 50% / 48% / 71%
Controlled subset, caption removed 0.404 (unchanged from Phase II)
Robustness: JPEG q30 / downscale x2 / blur 0.790 / 0.796 / 0.768
Latency per post (T4 GPU, batch 1) 17.1 ms median

Known v2 limits: no gain on real Photoshop Battles edits without a caption; still reacts to very short captions; more suspicious of real photos than Phase II (more false alarms); less robust to heavy blur.

Metrics of the Phase II model (for comparison; test set, 3,446 posts, resolution controlled images)

Metric Value
Macro F1 (seed 42 / mean of 3 seeds) 0.846 / 0.841 ยฑ 0.004
Accuracy 0.846
F1 authentic / manipulated / false connection 0.821 / 0.890 / 0.826
Controlled subset macro F1 0.893
Controlled subset, caption removed 0.403
Caption removed / image removed (input level) 0.415 / 0.425
Caption streams zeroed inside the model 0.642
Robustness: JPEG q30 / downscale x2 / blur / caption swap 0.821 / 0.835 / 0.816 / 0.514
Forensic branch on CASIA test: pixel F1 / IoU / image AUC 0.285 / 0.235 / 0.927
Latency per post (T4 GPU, batch 1): median / p90 16.6 ms / 21.4 ms

Baseline for comparison (late fusion, 3 seeds): macro F1 0.835 ยฑ 0.001. Weakest groups: r/confusing_perspective (47.8% accuracy) and r/misleadingthumbnails (68.0%).

Files

File What it is
best.pt PyTorch checkpoint (float32), for app/app.py and web/server.py
config.json, tokenizer/ model settings and the DistilBERT tokenizer
tell_fp16.onnx the same model as one ONNX graph with half precision weights (computation stays in float32), for the in-browser website. Inputs: image [1,3,224,224] (normalised), input_ids and attention_mask [1,48] int64. Outputs: probs (authentic, manipulated, false_connection), gates (image, text, forensic, consistency), edit_map [1,1,224,224], focus [1,49]. On 52 check cases (the 10 test kit cases and variations) it gave the same answers as the PyTorch model, with probabilities within 0.0031.
Downloads last month
35
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Space using maosama0001/tell-v2 1