autoRiver/harc-ablation-adapters

HARC ablation LoRA adapters

This repository contains the ten public LoRA adapters from the HARC ablation experiment. Each subdirectory is an independent adapter and can be loaded with its corresponding base model.

Adapters

Directory	Base model	KL coefficient	Margin
qwen_kl10_m03	Qwen/Qwen2.5-7B-Instruct	10	0.3
qwen_kl10_m07	Qwen/Qwen2.5-7B-Instruct	10	0.7
qwen_kl5_m05	Qwen/Qwen2.5-7B-Instruct	5	0.5
qwen_kl1_m05	Qwen/Qwen2.5-7B-Instruct	1	0.5
qwen_kl20_m05	Qwen/Qwen2.5-7B-Instruct	20	0.5
llama_kl10_m03	meta-llama/Llama-3.1-8B-Instruct	10	0.3
llama_kl10_m07	meta-llama/Llama-3.1-8B-Instruct	10	0.7
llama_kl1_m05	meta-llama/Llama-3.1-8B-Instruct	1	0.5
llama_kl5_m05	meta-llama/Llama-3.1-8B-Instruct	5	0.5
llama_kl20_m05	meta-llama/Llama-3.1-8B-Instruct	20	0.5

All adapters use LoRA rank 32 and alpha 64. The training source was the harc-ablate-qwen-llama-0727 AMLT experiment in the sft4safety project.

Loading an adapter

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

base_model = "Qwen/Qwen2.5-7B-Instruct"
adapter_repo = "autoRiver/harc-ablation-adapters"
adapter_subfolder = "qwen_kl10_m03"

tokenizer = AutoTokenizer.from_pretrained(base_model)
model = AutoModelForCausalLM.from_pretrained(
    base_model,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
model = PeftModel.from_pretrained(
    model,
    adapter_repo,
    subfolder=adapter_subfolder,
)

For the llama_* adapters, use meta-llama/Llama-3.1-8B-Instruct as base_model.

The base model remains subject to its original Hugging Face access license.