SFT datasets - a trl-lib Collection

trl-lib 's Collections

Papers implemented in TRL

Prompt datasets (for GRPO, RLOO, ...)

Preference datasets (for DPO, reward modeling)

Unpaired preference datasets (KTO)

Models from the docs

Comparing DPO with IPO and KTO

SFT datasets

updated 1 minute ago

Datasets that load straight into SFTTrainer. Formats: https://huggingface.co/docs/trl/dataset_formats

trl-lib/Capybara

Viewer • Updated Sep 19, 2024 • 16k • 7.1k • 28

Note Conversational
trl-lib/tldr

Viewer • Updated Jan 8, 2025 • 130k • 2.86k • 32

Note Prompt-completion, summarization
trl-lib/chatbot_arena_completions

Viewer • Updated Apr 25, 2025 • 33k • 71 • 4

Note Conversational, used for distillation
trl-lib/llava-instruct-mix

Viewer • Updated Aug 16, 2025 • 228k • 758 • 4

Note Conversational, vision
HuggingFaceH4/llava-instruct-mix-vsft

Viewer • Updated Apr 11, 2024 • 273k • 1.5k • 50

Note External. Conversational, vision
allenai/tulu-3-sft-mixture

Viewer • Updated Dec 2, 2024 • 939k • 27.1k • 284

Note External. Conversational
HuggingFaceH4/Multilingual-Thinking

Viewer • Updated Aug 7, 2025 • 1k • 7.2k • 121

Note External. Conversational, reasoning