Models from the docs - a trl-lib Collection

trl-lib 's Collections

Papers implemented in TRL

Prompt datasets (for GRPO, RLOO, ...)

Preference datasets (for DPO, reward modeling)

Unpaired preference datasets (KTO)

Models from the docs

Comparing DPO with IPO and KTO

Models from the docs

updated 19 minutes ago

Models trained in the TRL documentation examples: https://huggingface.co/docs/trl

trl-lib/Qwen3-4B-LoRA

Updated Jul 28, 2025 • 1

Note LoRA adapter, SFT and DPO docs
trl-lib/Qwen2-0.5B-KTO

Text Generation • 0.5B • Updated Oct 18, 2024 • 9

Note KTO docs
trl-lib/Qwen2-0.5B-ORPO

Text Generation • 0.5B • Updated Oct 11, 2024 • 46 • 2

Note ORPO docs
trl-lib/Qwen2-0.5B-OnlineDPO

Text Generation • 0.5B • Updated Oct 23, 2024 • 17 • 1

Note Online DPO docs
trl-lib/Qwen2-0.5B-Reward

Text Classification • 0.5B • Updated Sep 5, 2024 • 417 • 1

Note Reward model, Online DPO docs
trl-lib/qwen1.5-1.8b-sft

Text Generation • 2B • Updated Mar 13, 2024 • 41 • • 4

Note SFT model, KTO script
trl-lib/pythia-1b-deduped-tldr-sft

1B • Updated Aug 2, 2024 • 278

Note TL;DR SFT, Online DPO docs
trl-lib/pythia-1b-deduped-tldr-rm

Updated Aug 27, 2024 • 2.77k

Note TL;DR reward model, Online DPO docs
trl-lib/pythia-2.8b-deduped-tldr-sft

Updated Aug 2, 2024 • 11

Note TL;DR SFT, Online DPO docs
trl-lib/pythia-2.8b-deduped-tldr-rm

Updated Aug 2, 2024 • 5

Note TL;DR reward model, Online DPO docs
trl-lib/pythia-6.9b-deduped-tldr-sft

Updated Aug 2, 2024 • 6

Note TL;DR SFT, Online DPO docs
trl-lib/pythia-6.9b-deduped-tldr-rm

Updated Aug 2, 2024 • 4

Note TL;DR reward model, Online DPO docs