- GPT-3.5 Spider contamination based on https://arxiv.org/pdf/2402.08100 (9588afb347cd07a7645852f6c9b966c5222aa76e)
- convert arxiv pdf link to abs (f6e12a010342efd037641bd2cb947fca4b1a5374)
- Add percentage of validation data contaminated by including db ids on which gpt-3.5 achieves more than 75% DC-accuracy (2a68192028574cf246c6af3b1d053cae7fd82acc)
- Add PR number (326e3ca343d14319e6f0c0930fdd06b7507aaacb)
Co-authored-by: Bhavish Pahwa <bpHigh@users.noreply.huggingface.co>
Add Aquila model series which have gsm8k test set contamination
## What are you reporting:
- [ ] Evaluation dataset(s) found in a pre-training corpus. (e.g. COPA found in ThePile)
- [x] Evaluation dataset(s) found in a pre-trained model. (e.g. FLAN T5 has been trained on ANLI)
**Evaluation dataset(s)**: Name(s) of the evaluation dataset(s). If available in the HuggingFace Hub please write the path (e.g. `uonlp/CulturaX`), otherwise provide a link to a paper, GitHub or dataset-card.
`gsm8k`
**Contaminated model(s)**: Name of the model(s) (if any) that have been contaminated with the evaluation dataset. If available in the HuggingFace Hub please list the corresponding paths (e.g. `allenai/OLMo-7B`).
`Aquila2-34B` , `AquilaChat2-34B`
## Briefly describe your method to detect data contamination
- [ ] Data-based approach
- [x] Model-based approach
[Official Release Readme](https://huggingface.co/BAAI/Aquila2-34B/blob/main/README.md) has information that the 34B parameter versions of Aquila2 series contain gsm8k test set data con…
- Superglue/RealNews Contamination based on "Noise-Robust De-Duplication at Scale" (ec46e45d761b5d6620031f28ccf4b581e65614bf)
- Merge branch 'main' of https://huggingface.co/spaces/CONDA-Workshop/Data-Contamination-Report into pr/15 (854619bb3c2f72243cbc255d23bb13f03240e7ae)
Co-authored-by: Emily Silcock <emilys@users.noreply.huggingface.co>
Mistral 7B Arc Easy Contamination based on "Proving Test Set Contamination in Black Box Language Models" (#14)
- Mistral 7B Arc Easy Contamination based on "Proving Test Set Contamination in Black Box Language Models" (c36b27e9f9aa53a35f3d20eda2f4dcd4af826441)
- Changing name to match HF id (b151ec135996a305afce6f866026b5afcc096b6a)
Co-authored-by: Ameya Prabhu <AmeyaPrabhu@users.noreply.huggingface.co>
Added Contamination Evidence from GPT4 Tech Report using String matching on GPT-4 (#11)
GPT-3.5Turbo HumanEval Contamination based on "Generalization or Memorization: Data Contamination and Trustworthy Evaluation for Large Language Models" (#16)
- GPT-3.5Turbo HumanEval Contamination based on "Generalization or Memorization: Data Contamination and Trustworthy Evaluation for Large Language Models" (bad32343b38cfc431017fcd4c4fb1f13edb266b3)
- small fixes (d7810fa0909e40b3b422e4b09d4600df29295671)
Co-authored-by: Kate Solonko <jupyter31@users.noreply.huggingface.co>
Added Contamination Evidence on MMLU of ChatGPT/GPT4 from "Investigating data contamination in modern benchmarks for large language models" (#10)
- Add model-based results for MedNLI, RadNLI for GPT-3.5 and GPT-4 (2a036b2f599b01e80c284e70284212c6477145f6)
- Add PR link (ee21990995aed9ca66afaa00541ead58248ddbb2)
- Replace name with HF dataset (8ff4d82c089e3e2f2582ecf8d6f75014f4b733e7)
Co-authored-by: Jenny Chim <j-chim@users.noreply.huggingface.co>
Add data from "An Open-Source Data Contamination Report for Large Language Models" (#5)
- Add data from "An Open-Source Data Contamination Report for Large Language Models" (6169ce28edf11369ba1957b2266cba0bb4da2024)
- Update contamination_report.csv (4831ab13739e53bf664a1b82467a6a2d4794ed7e)
- Fix merge error (2e138568a5af505805315c084367b0c4c8bcab27)
Co-authored-by: Vishaal Udandarao <vishaal27@users.noreply.huggingface.co>
- Add data from "Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus" (ad06fdcd383dcbf2bd4b407332cdd6e7440c47af)
Co-authored-by: Vishaal Udandarao <vishaal27@users.noreply.huggingface.co>
