xtools-at/precious-datasets/CyberSec-Reasoning - Storage Bucket

🛡️ CyberSec-Reasoning-Premium

A rigorously cleaned, English-only cybersecurity reasoning SFT dataset of 3,067 chain-of-thought traces, covering offensive, defensive, and CTF domains. Part of the Premium series, upholding the standards of fable-5-premium, fable-5-premium-v2, and fable-5.1-premium.

Priorities: Quality > Ease of Access > Quantity

📊 Dataset Overview

Property	Value
Total Traces	3,067
Train Split	2,606 (85.0%)
Validation Split	230 (7.5%)
Test Split	231 (7.5%)
Average Quality	0.74 (0.5–1.0 band)
Language	English only (CJK-filtered)
License	Apache-2.0 (per-source attribution below)
Created	2026-09-20

🔗 Sources

Source	Raw	Kept	License	Description
dpevzner (Cybersecurity_Reasoning_Dataset)	17,991	1,588	Apache-2.0	Canonical expert cyber reasoning corpus — 9,870 internal dups collapsed, 3 template formats (ChatML, User/Assistant, Instruction/Response) normalized to messages
echel0nn1881 (kimi-cyber-reasoning)	997	997	WTFPL	Kimi K3 CoT across 13 cybersecurity disciplines and 4 systems-engineering domains
verno-labs (financial-ai-ctf-dataset)	400	333	CC-BY-4.0	Adversarial prompt-injection CTF defense conversations (red vs blue)
davidquicast (vulnerability-intelligence-diagrammatic-reasoning)	107	107	Apache-2.0	CVE deep-dive reasoning: socratic, logical, diagrammatic, and abductive chains
Nanhang (ctf-dataset)	2,763	42	MIT	CTF task corpus — 95% Chinese, English-only rows kept
Excluded: Vyber07/cyber-security	—	—	Apache-2.0	Gated (auto) at build time — not redistributable until access is accepted
Excluded: expertdata-factory/cybersecurity-reasoning-cot-v1	—	—	rights-review-required	Source flags unverified redistribution rights — dropped for licensing hygiene

🧹 Quality Pipeline

1. Deduplication — Cross-source SHA-256 hashing on normalized messages JSON (9,870+ exact dups collapsed)
2. Language Filtering — CJK-ratio check; rows >5% Chinese dropped for English-only consistency
3. Template Normalization — ChatML / User-Assistant / Instruction-Response raw text parsed into clean messages
4. PII Scrubbing — Local paths, API keys (sk-, hf_, AKIA), emails, IPs replaced with [REDACTED_*]
5. Content Filtering — Refusal patterns and short (<300 char) records removed
6. Quality Scoring — Length, structure, and depth composite; sub-0.5 traces dropped (mean 0.74)

📈 Domain Coverage

pwn · crypto · reverse · heap · kernel · misc · SOC threat intel · software engineering · web · network · forensics · prompt-injection defense · CVE analysis · vulnerability triage

🎯 Usage

With Hugging Face Datasets

from datasets import load_dataset

ds = load_dataset("saidutta69/CyberSec-Reasoning-Premium", data_files="openai_chat/train.parquet", split="train")

With Axolotl

datasets:
  - path: saidutta69/CyberSec-Reasoning-Premium
    type: chat_template
    data_files: openai_chat/train.parquet
    split: train

🧰 Schema

Both formats (openai_chat/ and agent_traces/) carry identical rows: messages (JSON string), model, source, domain, session_id, quality_scores (JSON string).

🧾 Provenance

Built from the listed public sources through the premium cleaning pipeline; full build stats shipped in build_stats.json. Companion datasets: fable-5-premium · fable-5-premium-v2 · fable-5.1-premium.

⚠️ Intended use: defensive security research and model training. Educational/research framing throughout; no operational exploitation guidance is included beyond what sources already provide publicly.