Spaces:
Running
Running
|
Download README.md from coralexbadea/SafetyCard: direct link, hf CLI and curl.
- Browser
- Download file 6.58 kB
-
https://huggingface.co/spaces/coralexbadea/SafetyCard/resolve/main/README.md
- Command line
-
hf download hf://spaces/coralexbadea/SafetyCard/README.md
-
curl -L -o README.md https://huggingface.co/spaces/coralexbadea/SafetyCard/resolve/main/README.md
6.58 kB
| title: "SafetyCard: LLM Safety Scanner" | |
| emoji: 🛡️ | |
| colorFrom: green | |
| colorTo: blue | |
| sdk: static | |
| app_file: index.html | |
| pinned: true | |
| license: apache-2.0 | |
| short_description: Free safety report card for any Hugging Face LLM | |
| tags: | |
| - llm | |
| - ai-safety | |
| - model-security | |
| - safetensors | |
| - pickle | |
| - model-scanner | |
| - supply-chain-security | |
| - interpretability | |
| - benchmark | |
| - leaderboard | |
| - uncensored | |
| - abliteration | |
| # 🛡️ SafetyCard: a free safety report card for any Hugging Face LLM | |
| **Is this model safe to download? Is it really derived from the model it claims? Has its safety training been stripped out?** | |
| SafetyCard answers these questions with measurements, not guesswork. It's **free**, it's **open source**, and the deep checks run **on your own GPU**. | |
| Paste a model id. In seconds you get a clear, shareable report card: | |
| | Check | What it tells you | Runs where | | |
| |---|---|---| | |
| | ⚡ **File safety** (grade A–F) | Can loading this model run hidden code? Pickle files, `trust_remote_code`, declared base model, license | Instantly, in your browser on this free Space | | |
| | 🧬 **Lineage** | Is it really a fine-tune of its claimed base model, heavily modified, or not derived from it at all? | Your GPU, or a CPU for small models | | |
| | 🛡️ **Safety structure** | Does the model still internally separate harmful from harmless requests, compared with its parent? | Your GPU | | |
| ## Example: a real card | |
| Real output for an "uncensored" community fine-tune, compared with the official model it was built from: | |
| ``` | |
| 🟢 File safety: grade A Safetensors only, no custom code required. | |
| 🧬 Lineage vs Qwen/Qwen2.5-1.5B-Instruct: FINE-TUNED (relative weight change 0.012) | |
| 🛡️ Safety structure d′: 7.10 (range 6.59–7.36; chance level ≤ 0.32) | |
| Parent Qwen/Qwen2.5-1.5B-Instruct: 10.16 (range 8.80–11.00) | |
| WEAKER THAN PARENT: safety structure is clearly weaker than the parent's. | |
| ``` | |
| ## Quick start | |
| **Instant file scan (free, no install):** paste a model id on this Space and click *Scan files*. It runs in your browser through the Hub API, and no weights are downloaded. Tip: link straight to a result with `?model=org/name`. | |
| **Deep scan on your own GPU (free):** | |
| ```bash | |
| git clone https://huggingface.co/spaces/coralexbadea/SafetyCard && cd SafetyCard | |
| pip install -r requirements.txt | |
| python -m safetycard.cli scan Qwen/Qwen2.5-1.5B-Instruct --deep --out card.json | |
| ``` | |
| Prefer a web UI? Run `python app.py` for the full local interface with all three checks. | |
| ## 🏆 Leaderboard: add your model | |
| Every deep scan writes a `card.json`. Open a pull request on this Space that adds it to `results/`, and your model appears on the leaderboard as soon as it is merged. Each card records the exact model revision and SafetyCard version, so anyone can reproduce it. | |
| ## Built to be safe itself | |
| A scanner that downloads untrusted models must not become the attack. SafetyCard: | |
| - loads **safetensors only**: it never unpickles a file; | |
| - **never** sets `trust_remote_code`, so no repo code is ever executed; | |
| - generates **no text**: the deep scan reads hidden states only; | |
| - reads the file list through the Hub API for the instant scan, so it downloads nothing but `config.json`. | |
| ## How accurate is it? | |
| Every score comes with its own controls, so you can tell signal from noise: | |
| - **Held-out scoring.** The direction and the layer are chosen on one half of the prompts and scored on the other half. | |
| - **Permutation null.** The whole procedure is repeated on shuffled labels, giving the "chance level" printed on every card. | |
| - **Prompt resampling.** Each model is measured with 3 independent prompt samples, and the card shows the range. "Weaker" or "stronger than parent" is only reported when the ranges **do not overlap**. | |
| Calibration on real models (d′, 3 prompt samples each): | |
| | Model | Size | Safety structure d′ | vs parent | | |
| |---|---|---|---| | |
| | Qwen2.5 base (pretrained only) | 1.5B | 4.00 (3.57–4.74) | — | | |
| | Qwen2.5 Instruct (official) | 1.5B | 10.16 (8.80–11.00) | STRONGER than base | | |
| | Community "uncensored" fine-tune | 1.5B | 7.10 (6.59–7.36) | WEAKER than official Instruct | | |
| | Community abliterated model | 0.5B | 4.23 (3.76–4.81) | no clear difference (too small, see below) | | |
| Lineage calibration: official instruct-tuning measures 0.010–0.042 relative weight change from its base. Two separately pretrained models of the same architecture (Qwen2.5 vs Qwen2) measure 1.18, and a model against itself measures 0. | |
| ## FAQ | |
| **Is it safe to download a Hugging Face model?** | |
| Usually, if the weights are in safetensors format and the model needs no custom code. That's grade A. Pickle-format files (`.bin`, `.pt`, `.ckpt`) can run arbitrary code when loaded, and so can models that require `trust_remote_code`. SafetyCard flags both before you download anything. | |
| **How can I tell if a model was uncensored or abliterated?** | |
| Removing safety training tends to weaken the model's internal harmful-vs-harmless separation. SafetyCard measures that separation against the parent model and reports a difference only when it's clear. | |
| **Is this model really a fine-tune of the model it claims?** | |
| The lineage check compares weights tensor by tensor against the declared base. An unrelated model stands out immediately. | |
| **Does SafetyCard detect hidden backdoors?** | |
| Not in this version, and we won't claim it does. No public tool can currently guarantee a model is backdoor-free. SafetyCard reports the evidence it can measure reliably. | |
| **Is it free?** | |
| Yes. The instant scan runs on this free Space, and deep scans run on your own hardware. Apache-2.0 licensed. | |
| **Which models are supported?** | |
| Instant scan: any model repo on the Hub. Deep scan: decoder language models with safetensors weights that load in 🤗 Transformers without custom code. | |
| ## Limitations (please read) | |
| - **A clean card is evidence, not a guarantee.** It does not prove a model is safe or free of hidden behaviours. | |
| - **Below about 1B parameters, the safety-structure check is too noisy** to compare models reliably, and the card warns you when that applies. | |
| - **Safety-structure compares a model with its parent.** Without a known parent, the d′ value has no reference point. | |
| - **The prompt sets are public** (`mlabonne/harmful_behaviors`, `mlabonne/harmless_alpaca`). A model trained to fool exactly this test could do so. | |
| - **Leaderboard cards are self-reported.** Each one records the revision and version so it can be re-run. | |
| ## License | |
| Apache-2.0. Contributions and new checks are welcome; open a discussion or pull request. | |