## What are you reporting:
- [ ] Evaluation dataset(s) found in a pre-training corpus. (e.g. COPA found in ThePile)
- [x] Evaluation dataset(s) found in a pre-trained model. (e.g. FLAN T5 has been trained on ANLI)
**Evaluation dataset(s)**: Name(s) of the evaluation dataset(s). If available in the HuggingFace Hub please write the path (e.g. `uonlp/CulturaX`), otherwise provide a link to a paper, GitHub or dataset-card.
gsm8k
hendrycks/competition_math
**Contaminated model(s)**: Name of the model(s) (if any) that have been contaminated with the evaluation dataset. If available in the HuggingFace Hub please list the corresponding paths (e.g. `allenai/OLMo-7B`).
Contaminated model in gsm8k:
`Qwen/Qwen-1_8B`, `Qwen/Qwen-14B`
Contaminated model in hendrycks/competition_math:
`BAAI/Aquila2-34B`, `BAAI/Aquila2-7B`, `Qwen/Qwen-1_8B`, `Qwen/Qwen-7B`, `Qwen/Qwen-14B`, `THUDM/chatglm3-6b`, `internlm/internlm2-7b`, `internlm/internlm2-20b`
**Contaminated corpora**: Name of …
Add Reports Based on "Llemma: An Open Language Model For Mathematics" (#23)
- Add Reports Based on "Llemma: An Open Language Model For Mathematics" (c50904f0ee3707576647934fba99b878f0ce563d)
- Add PR number + Postprocessing (582a8ca7bb4a6bdd2b05cfe3c1b2d62df251c916)
Co-authored-by: Wei-Lin (William) Chen <wlchen@users.noreply.huggingface.co>
- Add Aquila model series which have gsm8k test set contamination (9cf7873ad5a119cc9111a21dd88b38c74911937f)
- Merge remote-tracking branch 'origin/main' into pr/21 (446d97b84641db8fd0ff52b17dad4f9224ab28f0)
- Add PR number + postprocessing (ecb29a1b2b34353c6a1d2fa7bf037237ecb91107)
Co-authored-by: Bhavish Pahwa <bpHigh@users.noreply.huggingface.co>
- GPT-3.5 Spider contamination based on https://arxiv.org/pdf/2402.08100 (9588afb347cd07a7645852f6c9b966c5222aa76e)
- convert arxiv pdf link to abs (f6e12a010342efd037641bd2cb947fca4b1a5374)
- Add percentage of validation data contaminated by including db ids on which gpt-3.5 achieves more than 75% DC-accuracy (2a68192028574cf246c6af3b1d053cae7fd82acc)
- Add PR number (326e3ca343d14319e6f0c0930fdd06b7507aaacb)
Co-authored-by: Bhavish Pahwa <bpHigh@users.noreply.huggingface.co>
- Superglue/RealNews Contamination based on "Noise-Robust De-Duplication at Scale" (ec46e45d761b5d6620031f28ccf4b581e65614bf)
- Merge branch 'main' of https://huggingface.co/spaces/CONDA-Workshop/Data-Contamination-Report into pr/15 (854619bb3c2f72243cbc255d23bb13f03240e7ae)
Co-authored-by: Emily Silcock <emilys@users.noreply.huggingface.co>
Mistral 7B Arc Easy Contamination based on "Proving Test Set Contamination in Black Box Language Models" (#14)
- Mistral 7B Arc Easy Contamination based on "Proving Test Set Contamination in Black Box Language Models" (c36b27e9f9aa53a35f3d20eda2f4dcd4af826441)
- Changing name to match HF id (b151ec135996a305afce6f866026b5afcc096b6a)
Co-authored-by: Ameya Prabhu <AmeyaPrabhu@users.noreply.huggingface.co>
Added Contamination Evidence from GPT4 Tech Report using String matching on GPT-4 (#11)
GPT-3.5Turbo HumanEval Contamination based on "Generalization or Memorization: Data Contamination and Trustworthy Evaluation for Large Language Models" (#16)
- GPT-3.5Turbo HumanEval Contamination based on "Generalization or Memorization: Data Contamination and Trustworthy Evaluation for Large Language Models" (bad32343b38cfc431017fcd4c4fb1f13edb266b3)
- small fixes (d7810fa0909e40b3b422e4b09d4600df29295671)
Co-authored-by: Kate Solonko <jupyter31@users.noreply.huggingface.co>
Added Contamination Evidence on MMLU of ChatGPT/GPT4 from "Investigating data contamination in modern benchmarks for large language models" (#10)
- Add model-based results for MedNLI, RadNLI for GPT-3.5 and GPT-4 (2a036b2f599b01e80c284e70284212c6477145f6)
- Add PR link (ee21990995aed9ca66afaa00541ead58248ddbb2)
- Replace name with HF dataset (8ff4d82c089e3e2f2582ecf8d6f75014f4b733e7)
Co-authored-by: Jenny Chim <j-chim@users.noreply.huggingface.co>
Add data from "An Open-Source Data Contamination Report for Large Language Models" (#5)
- Add data from "An Open-Source Data Contamination Report for Large Language Models" (6169ce28edf11369ba1957b2266cba0bb4da2024)
- Update contamination_report.csv (4831ab13739e53bf664a1b82467a6a2d4794ed7e)
- Fix merge error (2e138568a5af505805315c084367b0c4c8bcab27)
Co-authored-by: Vishaal Udandarao <vishaal27@users.noreply.huggingface.co>
- Add data from "Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus" (ad06fdcd383dcbf2bd4b407332cdd6e7440c47af)
Co-authored-by: Vishaal Udandarao <vishaal27@users.noreply.huggingface.co>
