CATCH-Llama-3.3-70B — LoRA adapter for Llama-3.3-70B-Instruct

A LoRA adapter trained with CATCH (adversarial honesty training) on top of meta-llama/Llama-3.3-70B-Instruct, revision 6f6073b423013f6a7d4d9f39144961bfbfbc386b.

This is run ul9466, checkpoint-85 (epoch 3).

Usage

With peft:

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.3-70B-Instruct",
    torch_dtype="bfloat16", device_map="auto",
)
model = PeftModel.from_pretrained(base, "kibiddd/CATCH-Llama-3.3-70B")
tok = AutoTokenizer.from_pretrained("meta-llama/Llama-3.3-70B-Instruct")

With vLLM:

from huggingface_hub import snapshot_download
from vllm import LLM, SamplingParams
from vllm.lora.request import LoRARequest

path = snapshot_download("kibiddd/CATCH-Llama-3.3-70B", allow_patterns="adapter_*")
llm = LLM(model="meta-llama/Llama-3.3-70B-Instruct", enable_lora=True, max_lora_rank=64,
          tensor_parallel_size=4, enforce_eager=True)
out = llm.generate("Hello", SamplingParams(temperature=0.7, max_tokens=512),
                   lora_request=LoRARequest("catch", 1, path))

Training

method	CATCH
objective	toward (honest target) + clipped away / unlikelihood (deceptive target) + utility, under an embedding-space attack
loss weights toward / away / utility	0.5 / 0.5 / 1
cutoffs toward / away	0.5 / -5
attack eps / step / steps	0.05 / 1e-4 / 5
honesty data	216 contrast pairs
utility data	Magpie-Llama-3.3 (generated by Llama-3.3-70B-Instruct)
adv / utility mix	0.5 / 0.5
learning rate	5e-5, cosine, warmup_ratio 0.1
batch (device x accum.)	2 x 8
checkpoint	step 85 (epoch 3) of a 30-epoch schedule; early stopping ended training at epoch 11
LoRA	r 64, alpha 16, dropout 0.1, 7 target modules
precision	bf16, 4-bit base (nf4, double-quant off)

Built with Llama. Use is subject to the Llama 3.3 Community License.