Buckets:
| language: | |
| - en | |
| license: apache-2.0 | |
| task_categories: | |
| - text-generation | |
| - text-classification | |
| tags: | |
| - cybersecurity | |
| - application-security | |
| - vulnerabilities | |
| - cve | |
| - cwe | |
| - instruction-tuning | |
| - jsonl | |
| size_categories: | |
| - 10K<n<100K | |
|  | |
| # Security Vulnerability Instruction Dataset | |
| An English-language dataset of security-focused assistant conversations about software vulnerabilities. Examples cover vulnerability explanations, CVE analysis, severity and impact, CWE references, and remediation guidance. It combines CVE-oriented examples with HackerOne report conversations and can support instruction tuning or experiments in security-topic classification and analysis. | |
| ## What the data contains | |
| Each example pairs a prompt and an assistant response with structured metadata. The prompt may ask for a summary, technical description, impact or severity assessment, or defensive remediation. The answer may cite CVE/CWE identifiers, CVSS scores, affected products, attack conditions, impact, or mitigation steps, depending on the example. Conversations include system instructions and are not all the same length. | |
| The manifest reports these input groups: | |
| | Source group | Input records | | |
| |---|---:| | |
| | CVE-oriented (`qwen_cve_input`) | 35,423 | | |
| | HackerOne (`h1_input`) | 4,125 | | |
| The JSONL metadata includes a normalized `training_source` (`qwen_cve` or `h1`), vulnerability/report `class`, optional CWE and CVE identifiers, CVSS/severity annotations, quality and completeness indicators, and conversation-level flags. These fields are useful for filtering and analysis, but they are not guaranteed to be complete or authoritative. In the current splits, the CVE-oriented source is the majority (35,423 of 39,548 records); results should not be interpreted as representative of all security reports or production traffic. | |
| ## Dataset splits | |
| | Split | Records | | |
| |---|---:| | |
| | Train | 35,594 | | |
| | Validation | 1,977 | | |
| | Test | 1,977 | | |
| | **Total** | **39,548** | | |
| The splits use a 90/5/5 ratio. They were grouped to reduce leakage across splits using shared CVE IDs, normalized HackerOne report IDs, and exact conversation hashes. The accompanying `manifest.json` records the split class counts and processing details. The split seed is `240517`; 282 exact duplicate rows were removed before splitting. | |
| ## Data format | |
| Each `.jsonl` line is a JSON object with: | |
| - `messages`: a chat conversation represented as role/content pairs (`system`, `user`, and `assistant`). | |
| - `metadata`: labels and auxiliary fields, including `class`, `cwe`, `cvss_score`, `severity`, `cve_ids`, quality/completeness indicators, and source identifiers. | |
| Example (abridged): | |
| ```json | |
| { | |
| "messages": [ | |
| {"role": "system", "content": "You explain web security weaknesses using CWE and CVSS terminology correctly."}, | |
| {"role": "user", "content": "Summarize CVE-2026-9160."}, | |
| {"role": "assistant", "content": "..."} | |
| ], | |
| "metadata": { | |
| "class": "ssti", | |
| "cwe": "CWE-1336", | |
| "cvss_score": 8.0, | |
| "severity": "high", | |
| "cve_ids": ["CVE-2026-9160"] | |
| } | |
| } | |
| ``` | |
| ## Labels | |
| There are 35 vulnerability/report classes, including `access_control`, `auth`, `business_logic`, `cmdi`, `deser`, `info_disclosure`, `pathtraver`, `sqli`, `ssrf`, `ssti`, `xss`, and `xxe`. The `invalid_report` label is also present. The largest classes are `xss` (5,623 total), `access_control` (5,621), `cmdi` (5,118), and `info_disclosure` (4,855). Many tail classes have fewer than 100 records overall, and some have only a handful. Class frequencies are imbalanced; consult `manifest.json` for exact counts by split. Use macro-averaged metrics and per-class results when evaluating classifiers. | |
| ## Benchmarking | |
| No model scores or benchmark results are included with this dataset. The following is a recommended evaluation protocol, not a claim of measured performance. | |
| ### Task A: vulnerability class prediction | |
| Given an example's `messages`, predict `metadata.class`. Use the provided train, validation, and test splits. For a generative model, ask for exactly one label from the 35 class names, normalize the output to a known label, and score the parsed prediction. For a classifier, train on the training split and select settings on validation only. | |
| Report accuracy, macro-F1, weighted-F1, and per-class precision/recall/F1. Macro-F1 is especially useful because class counts vary substantially. Include the per-class support counts and a confusion matrix; do not report accuracy alone. | |
| ### Task B: security answer generation | |
| Use each example's conversation as an instruction-tuning target, with the assistant message as the reference answer. Report the prompt template, model/version, decoding settings, and whether metadata was exposed to the model. Evaluate factuality and usefulness with a documented human rubric or an explicitly named automated method. Consider checking CVE/CWE identifiers and numeric CVSS values against a dated authoritative source. Text-overlap scores alone do not establish correctness because valid security explanations can be phrased differently. | |
| ### Split and leakage cautions | |
| - Keep the supplied test split untouched for final evaluation; tune prompts and hyperparameters on validation. | |
| - The split construction groups shared CVE IDs, normalized HackerOne report IDs, and exact conversation hashes, but it does not guarantee independence by product, vendor, vulnerability family, or source. | |
| - Audit repeated templates and near-duplicate answers if the benchmark requires strict example independence. | |
| - About 3,908 records have `has_future_cve: true` across the full dataset. For claims about performance on information available before a cutoff date, filter or separately report these examples and record the cutoff policy. | |
| - Report results by source (`training_source`) as well as overall so the dominant CVE-oriented portion does not obscure performance on HackerOne examples. | |
| ## Loading | |
| ```python | |
| from datasets import load_dataset | |
| dataset = load_dataset("your-username/your-dataset-name", data_files={ | |
| "train": "train.jsonl", | |
| "validation": "valid.jsonl", | |
| "test": "test.jsonl", | |
| }) | |
| ``` | |
| Replace the repository ID with the dataset's Hugging Face username and repository name. | |
| ## Data quality and use considerations | |
| - This is a curated/assembled dataset, not a guarantee that every statement is correct, current, or suitable as production security advice. Verify vulnerability details against authoritative advisories before relying on them. | |
| - Some metadata fields may be noisy or inconsistent. Treat `cvss_score`, `severity`, and `class` as dataset annotations, not authoritative ratings or ground truth about exploitability. | |
| - `has_future_cve` is a dataset flag, not a statement that every referenced detail was unavailable at a particular model's cutoff. Confirm publication dates before making temporal evaluation claims. | |
| - The dataset contains dual-use cybersecurity information and should be used for defensive, educational, or research purposes with appropriate safeguards. | |
| ## Sources and processing | |
| The manifest identifies two input source groups: `h1_input` (4,125 records) and `qwen_cve_input` (35,423 records). After exact deduplication, 39,548 records remained; 282 duplicate rows were removed. There are 13,403 leakage groups. See `manifest.json` for details. | |
| ## License | |
| This dataset is distributed under the Apache License 2.0. See [`LICENSE`](LICENSE) for the full text. | |
Xet Storage Details
- Size:
- 7.52 kB
- Xet hash:
- cf2bb29795217e5441ab702377fb4e860d8175be4fea1ee1274e2266e029986e
·
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.