trl internal testing

trl-internal-testing's activity

• All
• Models
• Datasets
• Spaces
• Buckets
• Papers
• Collections
• Community
• Posts
• Articles

sergiopaniego 

posted an update about 13 hours ago

view post

Post

70

FineEnvs/multi-harness-rl

See translation

• 
• 1 reply

·

🔥

1

1

+

Reply

albertvillanova 

in trl-internal-testing/tiny-NemotronHForCausalLM-ultra 3 days ago

Upload NemotronHForCausalLM

#1 opened 4 days ago by

albertvillanova

albertvillanova 

in trl-internal-testing/tiny-NemotronHForCausalLM-super 3 days ago

Upload NemotronHForCausalLM

#1 opened 4 days ago by

albertvillanova

albertvillanova 

in trl-internal-testing/tiny-NemotronHForCausalLM-3.5-lightning 3 days ago

Upload NemotronHForCausalLM

#1 opened 4 days ago by

albertvillanova

albertvillanova 

in trl-internal-testing/tiny-NemotronHForCausalLM-nano 3 days ago

Upload NemotronHForCausalLM

#1 opened 4 days ago by

albertvillanova

albertvillanova 

in trl-internal-testing/tiny-Qwen2_5_VLForConditionalGeneration 4 days ago

Upload Qwen2_5_VLForConditionalGeneration

#20 opened 5 days ago by

albertvillanova

albertvillanova 

in trl-internal-testing/tiny-Qwen2VLForConditionalGeneration 4 days ago

Upload Qwen2VLForConditionalGeneration

#7 opened 5 days ago by

albertvillanova

albertvillanova 

in trl-internal-testing/tiny-GptOssForCausalLM 4 days ago

Upload GptOssForCausalLM

#4 opened 5 days ago by

albertvillanova

albertvillanova 

in trl-internal-testing/tiny-Lfm2VlForConditionalGeneration-2.5 5 days ago

Upload Lfm2VlForConditionalGeneration

#1 opened 5 days ago by

albertvillanova

albertvillanova 

in trl-internal-testing/tiny-DeepseekV3ForCausalLM 5 days ago

Upload DeepseekV3ForCausalLM

#4 opened 8 days ago by

albertvillanova

albertvillanova 

in trl-internal-testing/tiny-DeepseekV3ForCausalLM-0528 5 days ago

Upload DeepseekV3ForCausalLM

#5 opened 8 days ago by

albertvillanova

albertvillanova 

in trl-internal-testing/tiny-Lfm2ForCausalLM 8 days ago

Upload Lfm2ForCausalLM

#1 opened 8 days ago by

albertvillanova

albertvillanova 

in trl-internal-testing/tiny-Phi3ForCausalLM-3.5 10 days ago

Upload Phi3ForCausalLM

#2 opened 11 days ago by

albertvillanova

albertvillanova 

in trl-internal-testing/tiny-Phi3ForCausalLM-3 10 days ago

Upload Phi3ForCausalLM

#2 opened 11 days ago by

albertvillanova

albertvillanova 

in trl-internal-testing/tiny-OPTForCausalLM 10 days ago

Upload OPTForCausalLM

#3 opened 11 days ago by

albertvillanova

albertvillanova 

in trl-internal-testing/tiny-Gemma2ForCausalLM 10 days ago

Upload Gemma2ForCausalLM

#3 opened 11 days ago by

albertvillanova

albertvillanova 

in trl-internal-testing/tiny-GemmaForCausalLM 10 days ago

Upload GemmaForCausalLM

#3 opened 11 days ago by

albertvillanova

albertvillanova 

in trl-internal-testing/tiny-FalconMambaForCausalLM 10 days ago

Upload FalconMambaForCausalLM

#3 opened 11 days ago by

albertvillanova

albertvillanova 

in trl-internal-testing/tiny-Qwen2ForCausalLM-R1-Distill 11 days ago

Upload Qwen2ForCausalLM

#2 opened 11 days ago by

albertvillanova

sergiopaniego 

posted an update 11 days ago

view post

Post

347

ICYMI, Async GRPO in TRL now supports LoRA and we wrote a looong blog testing it

> the adapter is a few megabytes, so the weight sync is a file instead of an NCCL transfer
> 3 HF Jobs: 1 trainer and 2 vLLM replicas
> the adapter travels through an HF Storage Bucket mounted in all 3 at the same path
> a proxy in front of the replicas routes each rollout to the one already holding its KV prefix

https://huggingface.co/blog/asyncgrpo-lora-hfjobs

See translation

• 

🔥

2

2

+

Reply