# What are you reporting:
- [ ] Evaluation dataset(s) found in a pre-training corpus. (e.g. COPA found in ThePile)
- [x] Evaluation dataset(s) found in a pre-trained model. (e.g. FLAN T5 has been trained on ANLI)
Contaminated Evaluation Dataset(s):
- ibragim-bad/arc_easy
Contaminated Model:
- Mistral 7B
Approach:
- [ ] Data-based approach
- [x] Model-based approach
**Description of your method, 3-4 sentences. Evidence of data contamination:**
They perform a statistical test on log probs of the model, where they compare the log prob of the dataset under its original ordering to the log probability under random permutations. Specifically, they have a shared version where they test that the log-probability under the canonical ordering X is higher than the average log probability under a random permutation.
**Citation:**
Is there a paper that reports the data contamination or describes the method used to detect data contamination? Yes
**url**: https://arxiv.org/abs/231…
- Add model-based results for MedNLI, RadNLI for GPT-3.5 and GPT-4 (2a036b2f599b01e80c284e70284212c6477145f6)
- Add PR link (ee21990995aed9ca66afaa00541ead58248ddbb2)
- Replace name with HF dataset (8ff4d82c089e3e2f2582ecf8d6f75014f4b733e7)
Co-authored-by: Jenny Chim <j-chim@users.noreply.huggingface.co>
Add data from "An Open-Source Data Contamination Report for Large Language Models" (#5)
- Add data from "An Open-Source Data Contamination Report for Large Language Models" (6169ce28edf11369ba1957b2266cba0bb4da2024)
- Update contamination_report.csv (4831ab13739e53bf664a1b82467a6a2d4794ed7e)
- Fix merge error (2e138568a5af505805315c084367b0c4c8bcab27)
Co-authored-by: Vishaal Udandarao <vishaal27@users.noreply.huggingface.co>
- Add data from "Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus" (ad06fdcd383dcbf2bd4b407332cdd6e7440c47af)
Co-authored-by: Vishaal Udandarao <vishaal27@users.noreply.huggingface.co>
