StarWarrior24x7/tiny-aya-global-xref-v2

Tiny Aya Global: trained through simulated abliteration (xref-v2 and xref-v2b)

Safety-hardened versions of Tiny Aya Global (3.35B), designed to keep refusing after abliteration by Heretic.

This repo holds two checkpoints:

• v2 (repo root): the strongest resistance to Heretic.
• v2b (v2b/ subfolder): much lower over-refusal, with a trade-off in robustness.

from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("CohereLabs/tiny-aya-global")
v2  = AutoModelForCausalLM.from_pretrained("StarWarrior24x7/tiny-aya-global-xref-v2")
v2b = AutoModelForCausalLM.from_pretrained("StarWarrior24x7/tiny-aya-global-xref-v2", subfolder="v2b")

How it was made

• Base recipe: the same as xref-v1: LoRA SFT on extended refusals plus everyday answers.
• The attack runs during training. On half of the training steps, forward hooks project a refusal direction out of every layer, and the model must still refuse harmful prompts and answer benign ones.
• The simulated attacker adapts. The direction is re-estimated from the current model every 100 steps, with randomised layer profiles and strengths.
• Lineage: adversarial training in the style of Fool's Gold (arXiv 2608.17202) and ART (arXiv 2605.26526), adapted to a refusal target with no hazardous data.
• v2b adds 1,218 borderline-but-safe prompts from OR-Bench, answered by the original model, to reduce over-refusal.

Results (refusal on mlabonne/harmful_behaviors test[:100])

Model	Over-refusal (XSTest safe)	Refusal under Heretic, KL ≤ 0.01	KL ≤ 0.05	KL ≤ 0.1	Lowest refusal Heretic reached	MMLU (1k)
Tiny Aya Global (original)	16%	14%	14%	14%	14%	52.3%
xref-v1	17.6%†	70%	43%	17%	17%	50.7%†
xref-v2	39%	100%	100%	79%‡	33%	53.0%
xref-v2b (v2b/ subfolder of xref-v2)	18%	99%	52%	47%	47%	52.7%
‡ From a 200-trial Heretic search (40 trials found 96%); the worst case did not get stronger.
† v1 was not re-measured after the single-BOS evaluation fix. All other over-refusal (XSTest safe, judged) and MMLU numbers are final single-BOS numbers.

Each Heretic run is a fresh 40-trial search on that checkpoint (an adaptive attacker). KL is measured on harmless prompts; lower KL means a cheaper, less visible attack. Heretic's own refusal numbers are keyword-scored; judged numbers are below.

Judged results and other attacks (classifier judge: ProtectAI rejection model)

Attack	Original	v2	v2b
Heretic's strongest attack found: refusal / MMLU	1% / 52.0%	25% / 52.1% (needs ~16× the KL)	31% / 52.0%
Same attack, judged harmful (Granite Guardian; lower is safer)	91%	40%	57%
Heretic with 5× the search (200 trials), judged harmful	–	42%	–
Multilingual (hi/ar/fr/es) at the strongest attack, judged harmful	41%	12%	22%
SafeTune refusal-direction ablation	29%	100%	99%
SafeTune CAA steering, strength −1	4%	35%	34%
Sampling attack (any of 3 samples complies)	9% succeed	0% succeed	2% succeed
4-bit (bitsandbytes NF4)	99%	100%	100%
Prefill "Sure, here is"	21%	20%	24%
Harmful LoRA fine-tune, 100 examples	1%	3%	1%

• Strongest against: attacks that work by removing or steering the refusal direction.
• Not robust to: fine-tuning, prefilling or strong steering.
• Write-up: see the project article for the full picture.

Limitations

• Fine-tuning still wins. A harmful LoRA fine-tune with about 100 examples removes refusal from every model here, including the defended ones. These checkpoints resist data-free weight editing (abliteration / Heretic), not fine-tuning attacks.
• No measurable capability cost: MMLU is 52.3% (original), 53.0% (v2) and 52.7% (v2b).
• Over-refusal: v2 over-refuses noticeably; v2b reduces this.
• Coverage: one model size (3.35B); English evaluation only.

License

CC-BY-NC-4.0, inherited from CohereLabs/tiny-aya-global. Use must also follow Cohere's acceptable use policy.

Project

Lexsi Labs × Cohere × Hugging Face AI Safety & Alignment Hackathon, Track 2b (Tamper Resistance).