Text Generation
Transformers
Safetensors
cohere2
weightpatch
weight-editing
safetune
circuitkit
auditkit
lexsi
conversational
Instructions to use Perfect7613/tiny-aya-global-weightpatch-overrefusal with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Perfect7613/tiny-aya-global-weightpatch-overrefusal with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Perfect7613/tiny-aya-global-weightpatch-overrefusal") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Perfect7613/tiny-aya-global-weightpatch-overrefusal") model = AutoModelForCausalLM.from_pretrained("Perfect7613/tiny-aya-global-weightpatch-overrefusal", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Perfect7613/tiny-aya-global-weightpatch-overrefusal with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Perfect7613/tiny-aya-global-weightpatch-overrefusal" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Perfect7613/tiny-aya-global-weightpatch-overrefusal", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Perfect7613/tiny-aya-global-weightpatch-overrefusal
- SGLang
How to use Perfect7613/tiny-aya-global-weightpatch-overrefusal with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Perfect7613/tiny-aya-global-weightpatch-overrefusal" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Perfect7613/tiny-aya-global-weightpatch-overrefusal", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Perfect7613/tiny-aya-global-weightpatch-overrefusal" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Perfect7613/tiny-aya-global-weightpatch-overrefusal", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use Perfect7613/tiny-aya-global-weightpatch-overrefusal with Docker Model Runner:
docker model run hf.co/Perfect7613/tiny-aya-global-weightpatch-overrefusal
|
Download weightpatch/INDEPENDENT_VERIFICATION.md from Perfect7613/tiny-aya-global-weightpatch-overrefusal: direct link, hf CLI and curl.
- Browser
- Download file 3.33 kB
-
https://huggingface.co/Perfect7613/tiny-aya-global-weightpatch-overrefusal/resolve/main/weightpatch/INDEPENDENT_VERIFICATION.md
- Command line
-
hf download hf://Perfect7613/tiny-aya-global-weightpatch-overrefusal/weightpatch/INDEPENDENT_VERIFICATION.md
-
curl -L -o INDEPENDENT_VERIFICATION.md https://huggingface.co/Perfect7613/tiny-aya-global-weightpatch-overrefusal/resolve/main/weightpatch/INDEPENDENT_VERIFICATION.md
3.33 kB
| # Independent check of this model | |
| This file reports a check of the edited model that does not use the harness that made it. | |
| The harness's own grader accepted the edit; a grader that belongs to the same system is not | |
| independent evidence, so the two models were compared again with a separate script. | |
| ## How it was run | |
| - Script: `scripts/verify_published_model.py` in the code repository. It imports only `torch`, | |
| `transformers` and `datasets`. It does not import the harness, its grader, the agent, or the | |
| Lexsi kits. | |
| - Models: `CohereLabs/tiny-aya-global` (released) and this repository (edited), both downloaded | |
| from Hugging Face and loaded in bfloat16. | |
| - Both models get the same prompts through the chat template, with greedy decoding. | |
| - Multiple-choice accuracy is read from the next-token scores of the letters A to D. | |
| - A refusal is an answer whose first 300 characters contain one of a fixed list of phrases | |
| ("I'm sorry", "I can't", "I cannot assist", and so on). This is cruder than the grader's | |
| detector. It is the same for both models, so read the difference, not the absolute rates. | |
| - Intervals are 95% paired bootstrap intervals over prompts (4,000 resamples). | |
| - Hardware: one NVIDIA L40S. Run on 2026-11-16. | |
| ## Results | |
| | Measurement | Prompts | Released | Edited | Difference (95% interval) | Harness grader reported | | |
| |---|---:|---:|---:|---|---| | |
| | OR-Bench hard: safe requests refused | 600 | 33.5% | 14.0% | -19.5 points (-22.8 to -16.2) | 31.5% to 10.0% on 200 prompts | | |
| | MedMCQA validation: accuracy | 4,183 | 40.57% | 40.76% | +0.19 points (-0.12 to +0.50) | -1.25 points on 80, not significant | | |
| | MMLU test: accuracy | 2,000 | 54.50% | 54.50% | 0.00 points (-0.30 to +0.30) | +1.25 points on 80, not significant | | |
| | JailbreakBench harmful: requests refused | 100 | 86% | 84% | -2 points (-7 to +3) | harmful answers -2.5 points, not significant | | |
| Weights: the two models have the same tensors, and 398,328 entries differ, in 195 columns of | |
| four matrices. The harness reported the same count. | |
| | Tensor | Entries changed | Columns changed | | |
| |---|---:|---:| | |
| | `model.layers.20.mlp.down_proj.weight` | 71,446 | 35 | | |
| | `model.layers.22.mlp.down_proj.weight` | 96,007 | 47 | | |
| | `model.layers.23.mlp.down_proj.weight` | 124,582 | 61 | | |
| | `model.layers.24.mlp.down_proj.weight` | 106,293 | 52 | | |
| ## What this supports | |
| - The edited model refuses fewer safe requests. The drop is about the size the grader reported, | |
| on three times as many prompts. | |
| - Accuracy on MedMCQA and MMLU did not change. On MedMCQA, 44 of 4,183 answers differ between the | |
| two models; on MMLU, 10 of 2,000. This is not an improvement in accuracy and none is claimed. | |
| - The edited model refuses harmful requests about as often as the released model. With 100 | |
| prompts, a change of a few points in either direction cannot be ruled out. | |
| ## Limits | |
| - One run, one seed for the prompt samples, one refusal rule. | |
| - OR-Bench hard is the dataset the edit was developed on. The 600 prompts here were drawn from | |
| the whole set with a different seed, so some of them were seen during development. | |
| - The phrase list reads English. Refusals worded another way are missed for both models. | |
| - Harmfulness of the answers was not judged here, only whether the model refused. | |
| Full numbers: `independent_verification.json` in this folder. | |