websec / README.md

Upload character-level XSS detector

139ba48 verified 2 days ago

7.95 kB

---
license: mit
pipeline_tag: text-classification
tags:
- security
- xss
- cross-site-scripting
- web-security
- character-level
- pytorch
- safetensors
---

# Character-level XSS detector

A small neural network (1,959,361 parameters) that scores how likely a
string is a cross-site scripting (XSS) payload. It reads raw characters after
undoing URL and HTML encoding, so `%3Cscript%3E`, `&lt;script&gt;` and `<ScRiPt>`
look the same to it. It was trained from random weights; no pretrained model or
tokenizer is involved.

Use it to triage inputs: request parameters, form fields, log lines, user-supplied
HTML. A high score means "looks like an injection payload". It does not prove that
a payload would execute, and a low score does not prove an input is safe. Keep
context-aware output encoding and a Content Security Policy in place regardless.

## Quick start

```bash
pip install torch numpy safetensors huggingface_hub
```

```python
from huggingface_hub import hf_hub_download
import importlib.util

path = hf_hub_download("hareramray325/websec", "xss_detector.py")
spec = importlib.util.spec_from_file_location("xss_detector", path)
xss_detector = importlib.util.module_from_spec(spec)
spec.loader.exec_module(xss_detector)

detector = xss_detector.XSSDetector.from_pretrained("hareramray325/websec")
detector.predict(["<img src=x onerror=alert(1)>", "<b>hello</b>"])
# [{'text': '<img src=x onerror=alert(1)>', 'score': ..., 'is_xss': True},
#  {'text': '<b>hello</b>', 'score': ..., 'is_xss': False}]
```

`xss_detector.py` is plain Python in this repository; read it before running it.
The weights are in `model.safetensors`, so loading them executes no code.
`detector.score(texts)` returns calibrated probabilities; `predict` applies the
threshold 0.5241, tuned on validation data for 0.1% false alarms.
Pick your own threshold if your traffic differs.

## How it works

1. **Normalise:** URL-decode and HTML-unescape up to 3 times, drop NUL bytes,
   collapse whitespace, lowercase.
2. **Window:** inputs longer than 1000 characters are split into windows with a
   stride of 500; the highest window score counts, so padding a payload with
   harmless text does not hide it.
3. **Network:** one token per character (code points 0-255 each have their own token;
   every other character shares one), convolutions over 1, 3, 5 and 7 characters, two
   stride-2 convolutions, 4 Transformer encoder blocks (width 192, 8 heads), and a
   head over max- and mean-pooled features.
4. **Calibrate:** logits are divided by a temperature of 1.606 fitted on
   validation data.

Trained for 12 epochs (epoch 10 kept) on an NVIDIA RTX 5050 laptop GPU in about
5 minutes, with AdamW, a cosine learning-rate schedule and an averaged copy of
the weights.

## Training data

| Source | Used for | License |
|---|---|---|
| [Mereani & Howe XSS dataset](https://github.com/fmereani/Cross-Site-Scripting-XSS) | XSS attack URLs and payloads; benign URLs, JavaScript and text | MIT |
| Generated benign samples | Everyday HTML, URLs, code and prose that a naive model mistakes for attacks | Created for this model |
| Benign copies of attack URLs | Same sites and parameter names with the payload replaced | Derived from Mereani |
| Generated parameter and form-field values | Ordinary values the agent scores, so the model stops flagging them | Created for this model |
| Generated request targets | Ordinary request targets as access logs record them | Created for this model |
| Generated log lines | Application, server, mail and build log lines | Created for this model |
| Generated non-English text | Text in other scripts and languages that must not read as an attack | Created for this model |
| Generated non-XSS attacks (benign here) | SQLi, traversal, command injection, SSRF and scanner probes, labelled benign for an XSS-only detector | Created for this model |
| XSS payloads as field values | Existing payloads placed into the agent's value shape | Derived from Mereani (MIT) and generated templates |
| XSS payloads in request targets | Existing payloads placed into one URL parameter | Derived from Mereani (MIT) and generated templates |
| XSS payloads in log lines | Existing payloads placed into a log line | Derived from Mereani (MIT) and generated templates |

After normalising, exact duplicates and texts with conflicting labels were removed,
leaving 96,822 rows, split 70/15/15 into training, validation and test.
During training, each epoch adds varied copies of training attacks (real actions
such as cookie theft or redirects in place of `alert(1)`, other event handlers,
payloads placed inside ordinary text, URLs or JSON), template-built attacks, and
benign look-alikes (prose and code that mention `onclick`, `<script>` or cookies).
Site names in URLs are swapped in both classes so that no domain predicts a label.
Validation and test rows are never varied.

Every training row comes from an MIT-licensed source or was generated for this model.

## Evaluation

Both columns use the same data and splits. The linear baseline is character TF-IDF
with logistic regression, trained on the same training rows, at its own
validation-tuned threshold.

| Measurement | This model | Linear baseline |
|---|---|---|
| Held-out attacks caught | 3,392 / 3,408 (99.5%) | 3,359 (98.6%) |
| Held-out false alarms | 1 / 11,116 (0.01%) | 8 (0.07%) |
| Held-out detection at exactly 0.1% false alarms | 99.7% | 98.9% |
| External attacks caught (SecLists) | 7,734 / 7,896 (97.9%) | 7,755 (98.2%) |
| External attacks matching no training-pool row | 7,295 / 7,453 (97.9%) | 7,323 (98.3%) |
| External benign false alarms (WikiText, OpenAssistant) | 5 / 6,993 | 0 |
| Hand-written challenge attacks caught | 60 / 60 | 60 |
| Hand-written challenge false alarms | 0 / 80 | 0 |

**External benchmark:** 7,896 payloads from nine
[SecLists](https://github.com/danielmiessler/SecLists/tree/master/Fuzzing/XSS) lists (MIT) and
6,993 paragraphs and messages from WikiText and OpenAssistant. Nothing was trained or
tuned on it. Many SecLists payloads also occur in the training sources, so the row
counting only lines that match no row of the training pool is the stricter one.

**Challenge set:** 140 hand-written strings. Its past mistakes guided the training-time
variation, so treat it as a regression check, not an independent test. Missed: none

## Limitations

- **Unfamiliar benign text raises false alarms.** Benign training text is mostly URLs,
  JavaScript, reviews and generated HTML. Measure the false-alarm rate on your own
  traffic and adjust the threshold before blocking anything.
- **Programming help text is the riskiest benign input.** 5 of 1,994 OpenAssistant
  messages were flagged (mostly answers that mix prose and code blocks), against
  0 of 4,999 Wikipedia paragraphs.
- **Injected markup counts as an attack.** `<iframe src=...>`, `<script src=...>` and
  `<form>` in an *input* are scored high, as the training data labels them. Do not
  run it over whole web pages.
- **Non-Latin text is coarse.** Every character above code point 255 is one token, so
  Cyrillic, Devanagari or Chinese text looks alike. One external false alarm was a
  Russian-language message.
- **A score is evidence, not proof.** It does not know where the input is written
  (HTML body, attribute, script, URL), which decides whether it can execute.
- **Exact deduplication only.** Related payload families can sit in both training
  and test data, so held-out numbers are optimistic. The external benchmark is the
  better estimate.

## Intended use

Defensive: triaging inputs and logs, flagging suspicious values for review, and
measuring how well a filter copes with encoded payloads. It is not a web application
firewall on its own, and it was not built or evaluated for generating payloads.

	---
	license: mit
	pipeline_tag: text-classification
	tags:
	- security
	- xss
	- cross-site-scripting
	- web-security
	- character-level
	- pytorch
	- safetensors
	---

	# Character-level XSS detector

	A small neural network (1,959,361 parameters) that scores how likely a
	string is a cross-site scripting (XSS) payload. It reads raw characters after
	undoing URL and HTML encoding, so `%3Cscript%3E`, `<script>` and `<ScRiPt>`
	look the same to it. It was trained from random weights; no pretrained model or
	tokenizer is involved.

	Use it to triage inputs: request parameters, form fields, log lines, user-supplied
	HTML. A high score means "looks like an injection payload". It does not prove that
	a payload would execute, and a low score does not prove an input is safe. Keep
	context-aware output encoding and a Content Security Policy in place regardless.

	## Quick start

	```bash
	pip install torch numpy safetensors huggingface_hub
	```

	```python
	from huggingface_hub import hf_hub_download
	import importlib.util

	path = hf_hub_download("hareramray325/websec", "xss_detector.py")
	spec = importlib.util.spec_from_file_location("xss_detector", path)
	xss_detector = importlib.util.module_from_spec(spec)
	spec.loader.exec_module(xss_detector)

	detector = xss_detector.XSSDetector.from_pretrained("hareramray325/websec")
	detector.predict(["<img src=x onerror=alert(1)>", "<b>hello</b>"])
	# [{'text': '<img src=x onerror=alert(1)>', 'score': ..., 'is_xss': True},
	# {'text': '<b>hello</b>', 'score': ..., 'is_xss': False}]
	```

	`xss_detector.py` is plain Python in this repository; read it before running it.
	The weights are in `model.safetensors`, so loading them executes no code.
	`detector.score(texts)` returns calibrated probabilities; `predict` applies the
	threshold 0.5241, tuned on validation data for 0.1% false alarms.
	Pick your own threshold if your traffic differs.

	## How it works

	1. Normalise: URL-decode and HTML-unescape up to 3 times, drop NUL bytes,
	collapse whitespace, lowercase.
	2. Window: inputs longer than 1000 characters are split into windows with a
	stride of 500; the highest window score counts, so padding a payload with
	harmless text does not hide it.
	3. Network: one token per character (code points 0-255 each have their own token;
	every other character shares one), convolutions over 1, 3, 5 and 7 characters, two
	stride-2 convolutions, 4 Transformer encoder blocks (width 192, 8 heads), and a
	head over max- and mean-pooled features.
	4. Calibrate: logits are divided by a temperature of 1.606 fitted on
	validation data.

	Trained for 12 epochs (epoch 10 kept) on an NVIDIA RTX 5050 laptop GPU in about
	5 minutes, with AdamW, a cosine learning-rate schedule and an averaged copy of
	the weights.

	## Training data

	\| Source \| Used for \| License \|
	\|---\|---\|---\|
	\| [Mereani & Howe XSS dataset](https://github.com/fmereani/Cross-Site-Scripting-XSS) \| XSS attack URLs and payloads; benign URLs, JavaScript and text \| MIT \|
	\| Generated benign samples \| Everyday HTML, URLs, code and prose that a naive model mistakes for attacks \| Created for this model \|
	\| Benign copies of attack URLs \| Same sites and parameter names with the payload replaced \| Derived from Mereani \|
	\| Generated parameter and form-field values \| Ordinary values the agent scores, so the model stops flagging them \| Created for this model \|
	\| Generated request targets \| Ordinary request targets as access logs record them \| Created for this model \|
	\| Generated log lines \| Application, server, mail and build log lines \| Created for this model \|
	\| Generated non-English text \| Text in other scripts and languages that must not read as an attack \| Created for this model \|
	\| Generated non-XSS attacks (benign here) \| SQLi, traversal, command injection, SSRF and scanner probes, labelled benign for an XSS-only detector \| Created for this model \|
	\| XSS payloads as field values \| Existing payloads placed into the agent's value shape \| Derived from Mereani (MIT) and generated templates \|
	\| XSS payloads in request targets \| Existing payloads placed into one URL parameter \| Derived from Mereani (MIT) and generated templates \|
	\| XSS payloads in log lines \| Existing payloads placed into a log line \| Derived from Mereani (MIT) and generated templates \|

	After normalising, exact duplicates and texts with conflicting labels were removed,
	leaving 96,822 rows, split 70/15/15 into training, validation and test.
	During training, each epoch adds varied copies of training attacks (real actions
	such as cookie theft or redirects in place of `alert(1)`, other event handlers,
	payloads placed inside ordinary text, URLs or JSON), template-built attacks, and
	benign look-alikes (prose and code that mention `onclick`, `<script>` or cookies).
	Site names in URLs are swapped in both classes so that no domain predicts a label.
	Validation and test rows are never varied.

	Every training row comes from an MIT-licensed source or was generated for this model.

	## Evaluation

	Both columns use the same data and splits. The linear baseline is character TF-IDF
	with logistic regression, trained on the same training rows, at its own
	validation-tuned threshold.

	\| Measurement \| This model \| Linear baseline \|
	\|---\|---\|---\|
	\| Held-out attacks caught \| 3,392 / 3,408 (99.5%) \| 3,359 (98.6%) \|
	\| Held-out false alarms \| 1 / 11,116 (0.01%) \| 8 (0.07%) \|
	\| Held-out detection at exactly 0.1% false alarms \| 99.7% \| 98.9% \|
	\| External attacks caught (SecLists) \| 7,734 / 7,896 (97.9%) \| 7,755 (98.2%) \|
	\| External attacks matching no training-pool row \| 7,295 / 7,453 (97.9%) \| 7,323 (98.3%) \|
	\| External benign false alarms (WikiText, OpenAssistant) \| 5 / 6,993 \| 0 \|
	\| Hand-written challenge attacks caught \| 60 / 60 \| 60 \|
	\| Hand-written challenge false alarms \| 0 / 80 \| 0 \|

	External benchmark: 7,896 payloads from nine
	[SecLists](https://github.com/danielmiessler/SecLists/tree/master/Fuzzing/XSS) lists (MIT) and
	6,993 paragraphs and messages from WikiText and OpenAssistant. Nothing was trained or
	tuned on it. Many SecLists payloads also occur in the training sources, so the row
	counting only lines that match no row of the training pool is the stricter one.

	Challenge set: 140 hand-written strings. Its past mistakes guided the training-time
	variation, so treat it as a regression check, not an independent test. Missed: none

	## Limitations

	- Unfamiliar benign text raises false alarms. Benign training text is mostly URLs,
	JavaScript, reviews and generated HTML. Measure the false-alarm rate on your own
	traffic and adjust the threshold before blocking anything.
	- Programming help text is the riskiest benign input. 5 of 1,994 OpenAssistant
	messages were flagged (mostly answers that mix prose and code blocks), against
	0 of 4,999 Wikipedia paragraphs.
	- Injected markup counts as an attack. `<iframe src=...>`, `<script src=...>` and
	`<form>` in an input are scored high, as the training data labels them. Do not
	run it over whole web pages.
	- Non-Latin text is coarse. Every character above code point 255 is one token, so
	Cyrillic, Devanagari or Chinese text looks alike. One external false alarm was a
	Russian-language message.
	- A score is evidence, not proof. It does not know where the input is written
	(HTML body, attribute, script, URL), which decides whether it can execute.
	- Exact deduplication only. Related payload families can sit in both training
	and test data, so held-out numbers are optimistic. The external benchmark is the
	better estimate.

	## Intended use

	Defensive: triaging inputs and logs, flagging suspicious values for review, and
	measuring how well a filter copes with encoded payloads. It is not a web application
	firewall on its own, and it was not built or evaluated for generating payloads.