# Dataset coverage audit Counts describe retained data, not demonstrated model capabilities. | Measure | Training | Held out | | --- | --- | --- | | Examples | 3396 | 264 | | Distinct CVE IDs mentioned | 1190 | 93 | | iOS 27 / 27.x advisory examples | 326 | 0 | | CISA vendor labels | 158 | 29 | | CISA vendor/product pairs | 264 | 36 | CVE totals count unique identifiers mentioned, not all publicly known vulnerabilities or independently verified facts. Vendor counts use the source's labels. ## Gaps and interpretation - No iOS 27 advisory examples are in the current held-out split; iOS 27 generalization remains unmeasured. - Apple examples primarily teach advisory interpretation, not kernel/WebKit source-code auditing. - Original MCQ answer positions are heavily skewed toward A; the separate benchmark reorders eligible choices to balance answer letters. - Exact group/CVE and lexical overlap checks cannot rule out semantic overlap or base-model pretraining contamination. Split checks: **0 shared groups / 0 shared CVE IDs**. These checks do not establish absence of all data leakage. ## iOS releases represented The version names the **release containing a fix**, not an affected-version range. These are advisory tasks, not source-code audit tasks. | iOS release | Training examples | Held-out examples | | --- | --- | --- | | 27.2 | 163 | 0 | | 27 | 163 | 0 | | 26.7.2 | 134 | 0 | | 26.7.1 | 1 | 0 | | 26.7 | 95 | 0 | | 26.6.1 | 35 | 0 | | 26.6 | 97 | 0 | | 26.5.2 | 38 | 0 | | 26.5 | 67 | 0 | | 26.4 | 44 | 0 | | 26.3 | 43 | 0 | | 26.2 | 0 | 41 | | 18.7.11 | 3 | 0 | | 18.7.10 | 131 | 0 | | 18.7.9 | 49 | 0 | | 18.7.7 | 26 | 0 | | 18.7.5 | 29 | 0 | | 18.7.3 | 2 | 0 | | 16.7.16 | 0 | 1 | | 16.7.15 | 1 | 0 | | 15.8.7 | 4 | 0 | ## Most frequent Apple component labels Top 15 by training-example count. All component counts are in [coverage.json](./coverage.json). | Advisory component | Training examples | Held-out examples | | --- | --- | --- | | WebKit | 201 | 11 | | Kernel | 150 | 1 | | ImageIO | 62 | 0 | | Model I/O | 35 | 0 | | SceneKit | 30 | 0 | | CoreMedia | 22 | 0 | | AVEVideoEncoder | 21 | 0 | | CoreAudio | 18 | 0 | | Wi-Fi | 16 | 0 | | Security | 15 | 1 | | WebRTC | 15 | 0 | | Audio | 11 | 0 | | mDNSResponder | 11 | 0 | | Accounts | 10 | 0 | | FontParser | 10 | 0 | ## Other vendor coverage Top 15 CISA vendor labels by training count. | Vendor label | Training examples | Held-out examples | | --- | --- | --- | | Microsoft | 67 | 10 | | Cisco | 27 | 0 | | Fortinet | 16 | 0 | | Linux | 15 | 1 | | Google | 14 | 1 | | Ivanti | 13 | 1 | | Citrix | 11 | 0 | | Adobe | 10 | 0 | | Synacor | 9 | 0 | | SonicWall | 9 | 1 | | Oracle | 7 | 1 | | D-Link | 6 | 0 | | Apache | 6 | 0 | | Broadcom | 5 | 1 | | Palo Alto Networks | 5 | 0 | ## Topic keywords Non-exclusive keyword matches in prompt plus answer. A row can match several topics; these are heuristic tags, not expert annotations. | Keyword topic | Training matches | Held-out matches | | --- | --- | --- | | iOS / Apple platform | 1178 | 47 | | Kernel / operating systems | 948 | 69 | | Memory safety / binary analysis | 664 | 25 | | Web / API security | 681 | 69 | | Network security | 502 | 48 | | Identity / authentication | 463 | 52 | | Cryptography | 203 | 11 | | Cloud / containers | 575 | 55 | | Malware / incident analysis | 276 | 26 | ## Original multiple-choice answer positions The original training data is unchanged. Eligible benchmark choices are reordered to balance answer letters independently in each evaluation suite. | Correct letter | Training | Held out | | --- | --- | --- | | A | 1167 | 113 | | B | 502 | 47 | | C | 123 | 17 | | D | 25 | 3 | ## Reproducibility [Machine-readable counts and corpus SHA-256 hashes](./coverage.json). Audit generated 2026-11-09T13:40:21.918181+00:00.