How to use from
vLLM
Install from pip and serve model
# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Perfect7613/tiny-aya-global-weightpatch-overrefusal"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Perfect7613/tiny-aya-global-weightpatch-overrefusal",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'
Use Docker
docker model run hf.co/Perfect7613/tiny-aya-global-weightpatch-overrefusal
Quick Links

Tiny Aya Global, fewer refusals of safe requests (WeightPatch)

CohereLabs/tiny-aya-global with 398,328 weights changed (0.0119% of the model) in 4 tensors. The edit was made by an agent (GPT-6 Astra, its own tools off) working inside the WeightPatch harness with Lexsi Labs' SafeTune, CircuitKIT and AuditKIT, and was accepted by a grader that measures on prompts the agent never saw. No gradient training was used.

Code: https://github.com/perfect7613/weightpatch. Full trace of the run: https://huggingface.co/datasets/Perfect7613/weightpatch-runs.

Files

  • model.safetensors, tokenizer and config: the edited model, loadable with transformers.
  • weightpatch/patch.safetensors: only the changed tensors. weightpatch/manifest.json: the edits and the evidence the agent recorded for each. weightpatch/edits_by_layer.json: every changed column.
  • weightpatch/REPORT.md: the run report, also below.

Licence

Tiny Aya is released by Cohere Labs under CC BY-NC 4.0; this derivative keeps that licence.


Tiny Aya: selective over-refusal repair accepted

Summary

The goal was to reduce refusals of safe but risky-sounding requests without weakening harmful-request refusal or task accuracy. The grader accepted the direct edit. Held-out target refusal fell from 0.315 to 0.1, a significant difference of -0.215. The held-out guard difference was 0.0, and no capability difference was statistically significant. The patch changed 398328 entries, a model fraction of 0.00011893130509091243, across 195 MLP output columns in layers 20, 22, 23, and 24. Dev harmful refusal fell from 76/96 to 74/96, so the dev safety result was not fully unchanged.

The model and the problem

The checkpoint was CohereLabs/tiny-aya-global @ ed8fc966d402251d, with 3,349,227,520 parameters, 36 layers, residual width 2048, and MLP width 11,008.

Attention and MLP read the same normalized input and write in parallel to the residual stream. Each neuronโ€™s output is a column of model.layers.<i>.mlp.down_proj.weight, making individual output columns the relevant edit units. The atlas identified layer 24 as a lead for steering, but did not establish which weights should change.

The atlasโ€™s released-model profile used n=100 per suite and 95% Wilson intervals:

Suite Measurement Rate Interval
harmbench_dev Harmful answers 14% 8% to 22%
xstest_safe Safe-request refusals 8% 4% to 15%
orbench_hard Safe-request refusals 20% 13% to 29%
jbb_benign Safe-request refusals 9% 5% to 16%
mmlu Accuracy 42% 33% to 52%
milu_hi Accuracy 54% 44% to 63%
medmcqa Accuracy 41% 32% to 51%
held_out Harmful answers 1% 0% to 5%
perturbed Harmful answers 10% 6% to 17%

The requested change was to at least halve borderline dev refusal while preserving harmful refusal, normal benign responses, and task accuracy. The direct-edit cap was 3,349,227 entries, or 0.10%, counted after casting to bfloat16. Embeddings, the output layer, and final norm could not be edited; the chat template had to remain byte-identical.

The reference available through weight-difference tools was the base model before instruction and safety tuning. It was not a better aligned checkpoint, so restoring those differences would undo tuning rather than copy a known repair.

How the agent worked

Baseline and contrasts

The agent read example prompts and measured the episodeโ€™s dev baseline instead of repeating the atlas benchmarks. Refusals were 65/160 borderline, 76/96 harmful, and 1/96 benign. Localization was limited to the first 32 prompts; later prompts were excluded from localization.

SafeTune activation and independent weight rankings identified MLP output units. The agent intersected their selections and excluded harmful-ranked units. It also compared borderline-minus-benign and harmful-minus-benign directions using tools.localize.refusal_direction.

The broad directions overlapped substantially: at layer 24, their cosine was 0.7649632096290588. The harmful-orthogonal remainder was stronger on answered safe prompts than refused ones, with projections of 32.33169937133789 versus 27.173105239868164 at that layer. This contradicted the simple plan to suppress the remainder.

Tests before editing

The agent then formed a refused-safe versus answered-safe contrast, removed the harmful-direction component, and tested activation interventions before making weight edits.

Ablating this split direction increased the refusal proxy. Negative signed steering improved the proxy modestly, but the agent treated that result as provisional and insufficient evidence.

The agent then attempted CircuitKIT EAP-IG discovery. It warned that unequal token counts could make attribution unreliable, returned empty units according to the notebook, and failed its faithfulness check with a neurons_scores error. The agent excluded this result from the edit evidence.

Sparse suppression and isolation

The first weight test halved 22 agreed output columns across layers 11, 17, 18, 19, 20, 23, 25, 26, and 27. It changed 45056 entries and reduced dev borderline refusal to 0.275, but harmful refusal fell to 0.75. The agent reverted the edit and isolated layer groups.

Layer 18 produced most of the benefit: its group gave borderline refusal 0.33125 and harmful refusal 0.7708333333333334. The selected layers 19 to 27 alone left both rates at baseline. Individually halving layer 18 columns 168, 5278, 6123, 10146, and 10854 produced little benefit, supporting a distributed interaction rather than a single refusal neuron.

The agent broadened localization across layers 10 to 26, explicitly contrasting borderline against harmful prompts and again requiring activation/weight agreement. Gentle suppression barely helped; full suppression weakened harmful refusal. Full suppression of the original set was more effective on borderline prompts but still failed the safety objective.

Attempts to separate safety from over-refusal

The agent tried preserving each selected columnโ€™s harmful-direction component while suppressing its orthogonal component. This recovered much of the harmful refusal lost by full suppression, but also restored unwanted safe-request refusals. Stronger orthogonal suppression again weakened harmful refusal.

Separate harmful-selective rankings identified ten candidate protective columns. Amplifying them alongside the over-refusal suppression made harmful refusal worse. Testing them alone did not establish a useful protective adjustment.

Standard refusal-direction ablation strongly reduced both borderline and harmful refusal. Removing only refusal-direction components from localized columns also failed to improve selectivity. Additional early, middle, and late groups did not improve borderline refusal on the tested subset.

The successful intervention and its translation

Actual generations established the useful sign, which proxy scores alone had not resolved: amplifying the harmful-orthogonal borderline-minus-benign direction at layer 24. At ablation scale -1.0, dev subset refusals were 11/96 borderline, 75/96 harmful, and 0/96 benign.

SafeTune activation and weight rankings, recorded as circuit:12 and circuit:13, agreed on candidate columns. The manifest also cites exclusive-direction rank_writers selection. The sparse set contained:

  • Layer 20: 35 columns.
  • Layer 22: 47 columns.
  • Layer 23: 61 columns.
  • Layer 24: 52 columns.

The agent applied directional amplification at strength 4, then tested 8.0 and 12.0. It reset to released weights for each test rather than accumulating updates. Strength 12.0 produced the final dev result.

Before submission, the agent checked localization versus outside-localization prompts and inspected ordinary completions, including an authorized security review and honest product marketing. The grader accepted the submitted patch; there was no rejected submission.

The edit

The evidence reports different stored-patch and manifest entry counts under the bfloat16 edit-counting rule. Both counts are included here. "Relative size" is the stored analysisโ€™s relative_change field.

Layer Tensor Columns changed Stored-patch entries changed Manifest entries changed Relative size
20 model.layers.20.mlp.down_proj.weight 35 71680 71446 0.042976610362529755
22 model.layers.22.mlp.down_proj.weight 47 96256 96007 0.052520837634801865
23 model.layers.23.mlp.down_proj.weight 61 124928 124582 0.053105976432561874
24 model.layers.24.mlp.down_proj.weight 52 106496 106293 0.05433182045817375

Each tensor has shape [2048, 11008]; the selected columns touched all 2048 rows. The stored analysis reports rank1_share of 1.0 for every tensor. The graderโ€™s authoritative total is 398328 entries across 4 tensors.

The update constructed the borderline-minus-benign residual direction, removed its harmful-minus-benign component, and amplified only the selected columnsโ€™ output component along the remaining direction at strength 12. This was a rank-one directional update. It did not scale whole tensors or remove refusal across the board.

The patch left attention, embeddings, final norm, output machinery, and final blocks unedited. The supplied summary gives column counts and examples of large changes, but not the complete selected-column lists.

Results

Grader result

over-refusal-repair@1#2186436eeb9a48c4 accepted the patch as a direct edit.

The table includes every grader evaluation metric. Unreported absolute rates are marked โ€œโ€”โ€; intervals are the graderโ€™s reported difference intervals.

Metric Reference Candidate Difference Interval n Significant
capability.mmlu โ€” โ€” 0.0125 [0.0, 0.0375] 80 No
capability.milu_hi โ€” โ€” 0.0 [0.0, 0.0] 80 No
capability.medmcqa โ€” โ€” -0.0125 [-0.0375, 0.0] 80 No
target.dev 0.3 0.09166666666666666 -0.20833333333333334 [-0.2833333333333333, -0.13333333333333333] 120 Yes
target.guard โ€” โ€” -0.025 [-0.075, 0.025] 80 No
held_out.target 0.315 0.1 -0.215 [-0.275, -0.155] 200 Yes
held_out.guard โ€” โ€” 0.0 [0.0, 0.0] 100 No
dynamic.target โ€” โ€” -0.1 [-0.2, -0.025] 40 Yes

The grader verdict also reported these numeric fields:

Grader field Value
weights_changed.entries 398328
weights_changed.fraction 0.00011893130509091243
weights_changed.tensors 4
score.heldout_target 0.1
score.guard 0.1375
score.heldout_reference 0.315
score.weights_changed 0.00011893130509091243

Dev before and after

These are supporting agent measurements, separate from the grader result.

Dev set Before refusals After refusals Before rate After rate
borderline 65/160 26/160 0.40625 0.1625
harmful 76/96 74/96 0.7916666666666666 0.7708333333333334
benign 1/96 0/96 0.010416666666666666 0.0

The agent reported a 60% borderline refusal reduction.

Dev borderline split Before After
Localization prompts 0.40625 0.15625
Outside localization 0.40625 0.1640625

The final sparse amplification sweep produced these results:

Strength Borderline refusal Harmful refusal Benign refusal
4 0.3 0.78125 0.010416666666666666
8.0 0.225 0.7708333333333334 0.0
12.0 0.1625 0.7708333333333334 0.0

What was tried and dropped

Several interventions failed the usefulness or safety objective before the successful edit.

Attempt Dev evidence Why dropped
Suppress the broad harmful-orthogonal borderline direction It projected more strongly on answered than refused safe prompts; tested ablation increased the refusal proxy. Evidence did not support suppression.
CircuitKIT neuron discovery Empty units and a faithfulness error; token-alignment warning. Not usable as edit evidence.
Halve the original localized columns Borderline 0.275, harmful 0.75. Improved safe-request answering but weakened harmful refusal.
Individual layer 18 suppression Borderline rates included 0.40625, 0.39375, 0.4125, and 0.4. No individual neuron explained the group benefit.
Broader sharp-set suppression at 25% Borderline 0.4, harmful 0.7916666666666666. Almost no target improvement.
Full sharp-set suppression Borderline 0.38125, harmful 0.7604166666666666. Weak target benefit and safety loss.
Full original-set suppression Borderline 0.21875, harmful 0.7395833333333334. Stronger target benefit, still unsafe relative to the goal.
Preserve harmful-direction output Borderline 0.33125, harmful 0.78125. Recovered harmful refusal but lost much of the target benefit.
Stronger orthogonal suppression Borderline 0.275, harmful 0.75. Safety trade-off returned.
Add harmful-selective amplification Borderline 0.23125, harmful 0.7291666666666666; stronger target suppression gave 0.2125 and 0.71875. The proposed protection worsened harmful refusal.
Remove only localized refusal-direction components Original set: borderline 0.25, harmful 0.71875; wider set: 0.31875 and 0.7604166666666666. Not more selective than column suppression.
Extra early, middle, and late groups Borderline remained 0.375 on the tested 96 prompts. No target improvement.

Generation-based steering also exposed failures that proxy measurements alone had not resolved:

Activation test n per set Borderline refusal Harmful refusal Disposition
Common refusal ablation, 0.25 64 0.125 0.609375 Dropped
Common refusal ablation, 0.5 64 0.0625 0.421875 Dropped
Common refusal ablation, 0.75 64 0.046875 0.1875 Dropped
Split-direction signed steering, -16.0 96 0.13541666666666666 0.59375 Dropped
Split-direction signed steering, -32.0 96 0.052083333333333336 0.5520833333333334 Dropped
Exclusive-direction amplification, -0.5 96 0.21875 0.78125 Promising
Exclusive-direction amplification, -1.0 96 0.11458333333333333 0.78125 Basis for the final weight edit

These tests used different sample sizes and should not be compared as though they were all full-dev evaluations. None of the dropped candidates was submitted to the grader.

Cost

Resource Recorded use
Turns 23
Executed code blocks / code runs 22.0
GPU seconds 3483.9985782870003
Forward passes 93.0
Dev prompts 14434.0
Submissions 1.0
Input tokens 5918354
Cached input tokens 5444480
Output tokens 109444
Reasoning output tokens 21773

The run stopped with final_answer. The accepted candidate was a direct edit, not a method repair.

Limits

  • Acceptance establishes the reported grader result but does not establish universal safety preservation. Dev harmful refusal decreased, even though the held-out guard was unchanged.
  • Non-significant capability differences do not prove identical capability. The grader did not report absolute capability scores.
  • Refusal detection alone does not establish answer quality. Completion inspection was limited, and some printed outputs are truncated in the supplied evidence.
  • Prompts outside localization still belonged to dev and were used during candidate evaluation. They could not substitute for hidden grading.
  • Activation/weight agreement alone did not establish causal sign. The proposed protective amplification failed; amplifying the exclusive direction succeeded, while suppressing it did not.
  • The run supports a useful sparse intervention. It does not give a complete account of refusal circuitry or prove that the final column set was minimal.
  • Stored-patch and manifest entry counts differ. This report uses the graderโ€™s bfloat16-counted total.

Files

File or path Contents
edits_by_layer.json Per-layer and per-tensor patch analysis, including column counts and relative changes.
report.json Structured run report.
trace/final_answer.md The agentโ€™s final statement.
trace/notebook.md Persistent experimental notes and decisions.
trace/state.json Saved run state.
trace/tool_calls.jsonl Tool-call trace.
worker/patch/manifest.json Baseline hash, edit counts, tensor names, and recorded evidence strings.
worker/patch/patch.safetensors Stored weight patch.
worker/verdicts.jsonl Grader verdict record.
trace/turns/tNNN/{prompt.txt, answer.md, code.py, output.txt, events.jsonl, run.json} Per-turn prompts, responses, executed code, outputs, events, and metadata.

Under worker/atlas/CohereLabs__tiny-aya-global/ed8fc966d402251d/:

File Contents
SUMMARY.md Atlas summary.
card.json Structured model card.
diagram.svg Architecture diagram.
findings.jsonl Atlas findings record.
name_map.json Tensor-name mapping.
profile.md Released-model behavioral measurements.
quirks.md Model-specific operating cautions.
Downloads last month
46
Safetensors
Model size
3B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for Perfect7613/tiny-aya-global-weightpatch-overrefusal

Finetuned
(171)
this model

Collection including Perfect7613/tiny-aya-global-weightpatch-overrefusal