StackLLaMA - a trl-lib Collection

trl-lib 's Collections

Papers implemented in TRL

Prompt datasets (for GRPO, RLOO, ...)

Preference datasets (for DPO, reward modeling)

Unpaired preference datasets (KTO)

Models from the docs

Comparing DPO with IPO and KTO

StackLLaMA

updated 18 minutes ago

LLaMA trained on Stack Exchange with RLHF, then DPO. https://huggingface.co/blog/stackllama and https://huggingface.co/blog/dpo-trl

Build error

Agents

Featured

213

StackLLaMa

🦙

213

Note Demo
lvwerra/stack-exchange-paired

Updated Mar 13, 2023 • 1.35k • 153

Note Data
trl-archive/llama-7b-se-peft

Updated Apr 6, 2023 • 4

Note SFT adapter
trl-archive/llama-7b-se-rm-peft

Updated Apr 6, 2023 • 8

Note Reward model adapter
trl-lib/llama-7b-se-rl-peft

Updated Apr 14, 2023 • 103

Note RL (PPO) adapter
kashif/stack-llama-2

Text Generation • Updated Aug 8, 2023 • 65 • 15

Note Llama 2 trained with DPO, served by the demo