personaio's picture
classifier: SmolLM2-135M fine-tune, val accuracy 100.0%
2155b9c verified
|
Raw History Blame Contribute Delete
2.99 kB
---
license: mit
base_model: HuggingFaceTB/SmolLM2-135M-Instruct
library_name: transformers
pipeline_tag: text-generation
language:
- en
tags:
- code
- safety
- sandbox
- agents
- red-teaming
datasets:
- personaio/blast-radius-zero-traces
---
# Blast Radius Zero — code risk classifier
A 135M parameter model fine-tuned to answer one question before an agent runs
untrusted code: **will this hurt the host?**
It replies `RISKY` when the snippet deletes or writes outside its working
directory, talks to the network, spawns unbounded processes, exhausts memory,
reads secrets or mutates the system — and `SAFE` otherwise.
Part of [Blast Radius Zero](https://github.com/Hantsuo/blast-radius-zero), an agent
that executes every step inside a throwaway container. The classifier is the cheap
pre-flight gate; the container is the guarantee.
## Usage
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model_id = "personaio/blast-radius-classifier"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.float32)
PROMPT = (
"You are a pre-flight check for an agent that runs untrusted Python.\n"
"Decide whether this code could damage the host it runs on: deleting files "
"outside its working directory, writing outside it, talking to the network, "
"spawning unbounded processes, exhausting memory, reading secrets, or "
"modifying the system.\n"
"Answer with exactly one word: RISKY or SAFE.\n\n"
"CODE:\n```python\n{code}\n```\nANSWER:"
)
code = "import shutil; shutil.rmtree('/')"
ids = tok(PROMPT.format(code=code), return_tensors="pt").input_ids
out = model.generate(ids, max_new_tokens=6, do_sample=False, pad_token_id=tok.eos_token_id)
print(tok.decode(out[0][ids.shape[-1]:], skip_special_tokens=True).strip()) # RISKY
```
## Training
- Base: `HuggingFaceTB/SmolLM2-135M-Instruct`
- Data: 544 generated snippets (272/272 RISKY/SAFE) from
[`tools/make_classifier_data.py`](https://github.com/Hantsuo/blast-radius-zero/blob/master/tools/make_classifier_data.py),
plus the real sandbox traces in [personaio/blast-radius-zero-traces](https://huggingface.co/datasets/personaio/blast-radius-zero-traces).
- 4.0 epochs, batch 8, lr 5e-05, cosine schedule, CPU only.
- Validation accuracy **100.0% (96/96)** on the held-out split of the same generator.
- Holdout accuracy **100% (22/22)** on 22 hand-written snippets the generator never produced — this is the number to trust.
## Limits — read this before trusting it
- Trained on generated snippets, so it recognises *shapes* of risk, not novel
attacks. It is a triage signal, not a sandbox.
- Python only.
- The sandbox in the repo still runs every snippet regardless of this verdict;
the classifier decides what gets a human review, not what gets executed.
## Licence
MIT, same as the parent project.