wsp_nov11_qwen3vl4b_complementary โ€” Complementary WSP (Model B)

Research checkpoint, built as

W_B = W_best + ฮปยทฯ‰ยทฮ”_cand_01_comp, with ฮปยทฯ‰ = 1.0.

  • W_best is the frozen best single edit, utsavz/wsp_nov11_qwen3vl4b_single. It is preserved bit-exactly inside this model.
  • ฮ”_cand_01_comp is a selected component of a second, independently built candidate.
  • The base is Qwen/Qwen3-VL-4B-Instruct at revision ebb281ec70b05090aa6165b016eac8ec08e71b17.
  • No gradients, fine-tuning, SFT or RL were used.

No independent evaluation has been performed yet. All numbers below are development measurements on the VLGuard training set only. No VLGuard test data, FigStep, MM-SafetyBench, MMStar or other benchmark was used or run.

Modified tensors (all others bit-identical to the base)

tensor shape origin
model.language_model.layers.18.self_attn.o_proj.weight 2560ร—4096 anchor (identical to Model A): rank-1 keyโ†’refusal write
model.language_model.layers.22.mlp.down_proj.weight 2560ร—9728 complementary component: rank-1 refusal-write gain
model.language_model.layers.24.mlp.down_proj.weight 2560ร—9728 complementary component: rank-1 refusal-write gain
model.language_model.layers.28.mlp.down_proj.weight 2560ร—9728 complementary component: rank-1 refusal-write gain

The complementary component

Candidate cand_01 was built independently from the original weights. It applied W โ† W + 0.8ยทr_l r_lแต€ W to the MLP down_proj of layers 22โ€“28, which multiplies by 1.8 the component of each late MLP's output that lies along that layer's refusal direction r_l.

On W_best's own activations from the construction examples, the selection kept only the layers where the largest benign write along r_l is below the mean write of the harmful prompts W_best still answers. Those are L22, L24 and L28. In the other layers, the benign tails exceed the residual-failure signal. The coefficient 1.0 was chosen from {0.5, 1.0, 1.5} on a separate selection split.

Considered but not added:

  • cand_04 (a gated L21 neuron). Its detector reads the anchor's own write (cosine 0.70), and naive stacking interfered with the anchor.
  • cand_04_comp, an anchor-orthogonalised re-derivation of cand_04. It gained only 4 items, all of which cand_01_comp also gained.
  • cand_02 (an attention gain). Its fixes were a subset of cand_01's, at a higher utility cost.

Development measurements

Greedy decoding, 20 new tokens, keyword refusal detection. The ASR is provisional.

split model gen ASR unsafes safe_unsafes over-refusal (gen) TF harm refusal TF benign refusal ฮ”NLL
selection M original 42.5% 70% 15% 0/24 53.1% 0/128 โ€“
selection M single (A) 30.0% 55% 5% 0/24 72.7% 0/128 โˆ’0.005
selection M this model 12.5% 25% 0% 0/24 87.5% 2/128 +0.009
confirmation K original 42.5% 60% 25% 0/24 51.6% 0/128 โ€“
confirmation K single (A) 22.5% 35% 10% 0/24 72.7% 0/128 +0.035
confirmation K this model 10.0% 15% 5% 1/24 87.5% 0/128 +0.016

Important caveat: complementarity was not demonstrated

A development-only control simply scaled the anchor edit by 1.667 (this is not a valid Model B, because it alters W_best).

  • It matched this model's ASR on both splits: M 12.5% (score 0.863 vs 0.828) and K 10.0% (0.734 vs 0.733).
  • Nearly all of this model's gains over W_best are items the control also fixes: 19/19 on M and 18/19 on K.

The improvement over W_best is therefore best explained by added refusal strength along a shared refusal direction, not by covering distinct failure types. Other limitations:

  • The evaluation samples are small, and refusal is detected by keywords.
  • This model has one extra generated over-refusal on K: a benign question about a satirical meme.
  • VLGuard unsafes (a harmful image with innocuous text) remains the main residual failure.
  • Generalisation is untested.

Use

import torch
from transformers import Qwen3VLForConditionalGeneration, AutoProcessor
model = Qwen3VLForConditionalGeneration.from_pretrained("utsavz/wsp_nov11_qwen3vl4b_complementary", dtype=torch.bfloat16)
processor = AutoProcessor.from_pretrained("utsavz/wsp_nov11_qwen3vl4b_complementary")
Downloads last month
13
Safetensors
Model size
4B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for utsavz/wsp_nov11_qwen3vl4b_complementary

Finetuned
(566)
this model