Clarify which examples the overlap check covered

README.md

@@ -86,7 +86,7 @@ All data comes from [`HuggingFaceTB/smol-smoltalk`](https://huggingface.co/datas
 - `openhermes-50k` is a sample of [OpenHermes 2.5](https://huggingface.co/datasets/teknium/OpenHermes-2.5). Its card is tagged `Synthetic`, `GPT-4` and `Distillation`, describes the data as "primarily synthetically generated" and compiled from many sources, and names among them Airoboros 2.2, GPT-4-only subsets of ChatBot Arena and ShareGPT, GPT4-LLM, WizardLM Evol-Instruct, SlimOrca, Platypus, CamelAI and Glaive. The card text we read states no license for the compilation. It is about 31% of the fine-tuning tokens. Whether the terms of the services that produced such outputs restrict their use for training a model is a question we did not resolve and are not qualified to answer.
 - `self-oss-instruct` corresponds to the Self-OSS-Starcoder2-Instruct dataset named in the SmolTalk card, and `explore-instruct-rewrite` to the Explore-Instruct-Rewriting dataset named there. We did not check the models or the licenses behind them.
 - For `smol-contraints`, `smol-summarize-20k` and `smollm-rewrite-30k`, the SmolTalk card says the generation code is published but we did not verify which models generated the data.
-- The fine-tuning data was not decontaminated against the evaluation sets of this card. The overlap was measured afterwards, and nothing was removed. Method: the 139,505 examples of the fine-tuning training file (42.6 million tokens, decoded back to text; the replay text, which is pretraining data and about 8% of the fine-tuning tokens, is not part of this file and was not checked) and every string of every benchmark item (question, choices, context, passage) were lowercased and split into words. An item counts as overlapping if any run of 13 (or of 8) consecutive words of it also occurs in some training example. Items in which no string has that many words cannot be tested and are left out of the percentages. The result is below. A hit shows shared text. It does not show whether the question or the answer was also in the training data, and the checked strings include passages. Runs of 8 words also match ordinary phrases.
+- The fine-tuning data was not decontaminated against the evaluation sets of this card. The overlap was measured afterwards, and nothing was removed. Method: the 139,505 examples of the fine-tuning training file (42.6 million tokens, decoded back to text; it lacks 88 of the 522 identity conversations, which were added to the training mix later and contain no benchmark text; the replay text, which is pretraining data and about 8% of the fine-tuning tokens, is not part of this file and was not checked) and every string of every benchmark item (question, choices, context, passage) were lowercased and split into words. An item counts as overlapping if any run of 13 (or of 8) consecutive words of it also occurs in some training example. Items in which no string has that many words cannot be tested and are left out of the percentages. The result is below. A hit shows shared text. It does not show whether the question or the answer was also in the training data, and the checked strings include passages. Runs of 8 words also match ordinary phrases.
 
 | Benchmark | Items | 13 words: testable | 13 words: overlapping | 8 words: testable | 8 words: overlapping |
 |---|---|---|---|---|---|