It's working nicely. But, I got error when I tried qLoRA with load_in_4bit: true.
[rank0]: File "/home/xxx/.pyenv/versions/3.12.10/envs/trlgrpo/lib/python3.12/site-packages/peft/tuners/lora/bnb.py", line 373, in merge
[rank0]: self.get_base_layer().weight = bnb.nn.Params4bit(w_data.to("cpu"), **kwargs).to(weight.device)
[rank0]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]: TypeError: Params4bit.__new__() got an unexpected keyword argument 'ds_param_type'
Does it support qLoRA, or not yet? I found the following in the TRL/GRPO document, so I thought qLoRA is supported. Thanks a lot.
- Use LoRA on vision-language projection layers
- Enable 4-bit quantization to reduce memory usage