Instructions to use caseyeasd/claudefriend-lora with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use caseyeasd/claudefriend-lora with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-9B-Base") model = PeftModel.from_pretrained(base_model, "caseyeasd/claudefriend-lora") - Notebooks
- Google Colab
- Kaggle
README: stop at <|im_end|> when serving
#1
by maxsloef - opened
Same issue as in caseyeasd/claudefriend: Qwen/Qwen3.5-9B-Base's config ends generation at <|endoftext|> (248044), but this model ends its turns with <|im_end|> (248046). Served as in the README, it runs past the end of its turn and writes the next turn itself. A LoRA repo can't change the base model's generation config, so this adds a note to the README: pass stop_token_ids=[248046] to vLLM, or eos_token_id=[248046, 248044] to generate().
I checked stop_token_ids=[248046] on vLLM 0.18 with the merged model: replies end cleanly at the end of the turn.
caseyeasd changed pull request status to open
caseyeasd changed pull request status to merged