𝕏Guard-Train: A Large-Scale Dataset for Multi-Turn LM Safety

A comprehensive multi-turn safety dataset for improving conversational AI defenses against sophisticated jailbreaking attacks.

Dataset Details

• Website: https://x-teaming.github.io/
• Repository: https://github.com/salman-lui/x-teaming
• Paper: https://huggingface.co/papers/2504.13203

Dataset Description

This dataset consists of 30,695 multi-turn conversations, with complete attack-refusal pairs that enable robust multi-turn safety training.

Dataset Sources

We constructed 𝕏Guard-Train by proportionately sampling 10,000 harmful behaviors from WildJailbreak's vanilla harmful category. For each harmful behavior, our planner generated between two to five distinct attack plans, resulting in diverse attack trajectories incorporating various personas, contexts, and conversation approaches. We executed these plans using the complete 𝕏-Teaming pipeline, with GPT-4o, Gemini 2.0 Flash, and Deepseek V3 as target models, and Qwen-2.5-32B-IT handling both attack execution and TextGrad optimization. The pipeline refined attacker queries when verification scores decreased and dynamically adjusted plans that failed to achieve their harmful targets. This process resulted in highly effective jailbreaking conversations with an average of 5.10 turns, where one turn represents an attacker prompt and target model response pair. For successful jailbreaks, we replaced harmful model responses with carefully crafted helpful refusals.

Uses

This dataset is intended to be used for fine-tuning LLMs to make them resistant against advanced multi-turn jailbreaking attacks without sacrificing performance.

Dataset Structure

The dataset is presented as a single file without train, validation, or test splits. The conversations are given in ShareGPT format, with an even number of messages, and each message alternating between "human" and "gpt" roles, like so:

[
  {
    "conversations": [
      {"from": "human", "value": "Hello world"},
      {"from": "gpt", "value": "Foobar"}
    ]
  },
  {
    "conversations": [
      // ...
    ]
  }
]