Upload character-level XSS detector
Browse files- README.md +143 -0
- config.json +22 -0
- metrics.json +273 -0
- model.safetensors +3 -0
- xss_detector.py +152 -0
README.md
ADDED
|
@@ -0,0 +1,143 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: mit
|
| 3 |
+
pipeline_tag: text-classification
|
| 4 |
+
tags:
|
| 5 |
+
- security
|
| 6 |
+
- xss
|
| 7 |
+
- cross-site-scripting
|
| 8 |
+
- web-security
|
| 9 |
+
- character-level
|
| 10 |
+
- pytorch
|
| 11 |
+
- safetensors
|
| 12 |
+
---
|
| 13 |
+
|
| 14 |
+
# Character-level XSS detector
|
| 15 |
+
|
| 16 |
+
A small neural network (1,959,361 parameters) that scores how likely a
|
| 17 |
+
string is a cross-site scripting (XSS) payload. It reads raw characters after
|
| 18 |
+
undoing URL and HTML encoding, so `%3Cscript%3E`, `<script>` and `<ScRiPt>`
|
| 19 |
+
look the same to it. It was trained from random weights; no pretrained model or
|
| 20 |
+
tokenizer is involved.
|
| 21 |
+
|
| 22 |
+
Use it to triage inputs: request parameters, form fields, log lines, user-supplied
|
| 23 |
+
HTML. A high score means "looks like an injection payload". It does not prove that
|
| 24 |
+
a payload would execute, and a low score does not prove an input is safe. Keep
|
| 25 |
+
context-aware output encoding and a Content Security Policy in place regardless.
|
| 26 |
+
|
| 27 |
+
## Quick start
|
| 28 |
+
|
| 29 |
+
```bash
|
| 30 |
+
pip install torch numpy safetensors huggingface_hub
|
| 31 |
+
```
|
| 32 |
+
|
| 33 |
+
```python
|
| 34 |
+
from huggingface_hub import hf_hub_download
|
| 35 |
+
import importlib.util
|
| 36 |
+
|
| 37 |
+
path = hf_hub_download("hareramray325/websec", "xss_detector.py")
|
| 38 |
+
spec = importlib.util.spec_from_file_location("xss_detector", path)
|
| 39 |
+
xss_detector = importlib.util.module_from_spec(spec)
|
| 40 |
+
spec.loader.exec_module(xss_detector)
|
| 41 |
+
|
| 42 |
+
detector = xss_detector.XSSDetector.from_pretrained("hareramray325/websec")
|
| 43 |
+
detector.predict(["<img src=x onerror=alert(1)>", "<b>hello</b>"])
|
| 44 |
+
# [{'text': '<img src=x onerror=alert(1)>', 'score': ..., 'is_xss': True},
|
| 45 |
+
# {'text': '<b>hello</b>', 'score': ..., 'is_xss': False}]
|
| 46 |
+
```
|
| 47 |
+
|
| 48 |
+
`xss_detector.py` is plain Python in this repository; read it before running it.
|
| 49 |
+
The weights are in `model.safetensors`, so loading them executes no code.
|
| 50 |
+
`detector.score(texts)` returns calibrated probabilities; `predict` applies the
|
| 51 |
+
threshold 0.7278, tuned on validation data for 0.1% false alarms.
|
| 52 |
+
Pick your own threshold if your traffic differs.
|
| 53 |
+
|
| 54 |
+
## How it works
|
| 55 |
+
|
| 56 |
+
1. **Normalise:** URL-decode and HTML-unescape up to 3 times, drop NUL bytes,
|
| 57 |
+
collapse whitespace, lowercase.
|
| 58 |
+
2. **Window:** inputs longer than 1000 characters are split into windows with a
|
| 59 |
+
stride of 500; the highest window score counts, so padding a payload with
|
| 60 |
+
harmless text does not hide it.
|
| 61 |
+
3. **Network:** one token per character (code points 0-255 each have their own token;
|
| 62 |
+
every other character shares one), convolutions over 1, 3, 5 and 7 characters, two
|
| 63 |
+
stride-2 convolutions, 4 Transformer encoder blocks (width 192, 8 heads), and a
|
| 64 |
+
head over max- and mean-pooled features.
|
| 65 |
+
4. **Calibrate:** logits are divided by a temperature of 1.676 fitted on
|
| 66 |
+
validation data.
|
| 67 |
+
|
| 68 |
+
Trained for 12 epochs (epoch 9 kept) on an NVIDIA RTX 5050 laptop GPU in about
|
| 69 |
+
4 minutes, with AdamW, a cosine learning-rate schedule and an averaged copy of
|
| 70 |
+
the weights.
|
| 71 |
+
|
| 72 |
+
## Training data
|
| 73 |
+
|
| 74 |
+
| Source | Used for | License |
|
| 75 |
+
|---|---|---|
|
| 76 |
+
| [Mereani & Howe XSS dataset](https://github.com/fmereani/Cross-Site-Scripting-XSS) | XSS attack URLs and payloads; benign URLs, JavaScript and text | MIT |
|
| 77 |
+
| Generated benign samples | Everyday HTML, URLs, code and prose that a naive model mistakes for attacks | Created for this model |
|
| 78 |
+
| Benign copies of attack URLs | Same sites and parameter names with the payload replaced | Derived from Mereani |
|
| 79 |
+
|
| 80 |
+
After normalising, exact duplicates and texts with conflicting labels were removed,
|
| 81 |
+
leaving 68,258 rows, split 70/15/15 into training, validation and test.
|
| 82 |
+
During training, each epoch adds varied copies of training attacks (real actions
|
| 83 |
+
such as cookie theft or redirects in place of `alert(1)`, other event handlers,
|
| 84 |
+
payloads placed inside ordinary text, URLs or JSON), template-built attacks, and
|
| 85 |
+
benign look-alikes (prose and code that mention `onclick`, `<script>` or cookies).
|
| 86 |
+
Site names in URLs are swapped in both classes so that no domain predicts a label.
|
| 87 |
+
Validation and test rows are never varied.
|
| 88 |
+
|
| 89 |
+
The public [web-attacks-long](https://huggingface.co/datasets/shengqin/web-attacks-long) dataset
|
| 90 |
+
was deliberately left out because it states no license. Every training row comes from
|
| 91 |
+
an MIT-licensed source or was generated for this model.
|
| 92 |
+
|
| 93 |
+
## Evaluation
|
| 94 |
+
|
| 95 |
+
Both columns use the same data and splits. The linear baseline is character TF-IDF
|
| 96 |
+
with logistic regression, trained on the same training rows, at its own
|
| 97 |
+
validation-tuned threshold.
|
| 98 |
+
|
| 99 |
+
| Measurement | This model | Linear baseline |
|
| 100 |
+
|---|---|---|
|
| 101 |
+
| Held-out attacks caught | 2,131 / 2,155 (98.9%) | 2,126 (98.7%) |
|
| 102 |
+
| Held-out false alarms | 6 / 8,084 (0.07%) | 11 (0.14%) |
|
| 103 |
+
| Held-out detection at exactly 0.1% false alarms | 98.9% | 98.0% |
|
| 104 |
+
| External attacks caught (SecLists) | 7,712 / 7,896 (97.7%) | 7,604 (96.3%) |
|
| 105 |
+
| External attacks matching no training-pool row | 7,282 / 7,460 (97.6%) | 7,176 (96.2%) |
|
| 106 |
+
| External benign false alarms (WikiText, OpenAssistant) | 5 / 6,993 | 0 |
|
| 107 |
+
| Hand-written challenge attacks caught | 60 / 60 | 58 |
|
| 108 |
+
| Hand-written challenge false alarms | 0 / 80 | 0 |
|
| 109 |
+
|
| 110 |
+
**External benchmark:** 7,896 payloads from nine
|
| 111 |
+
[SecLists](https://github.com/danielmiessler/SecLists/tree/master/Fuzzing/XSS) lists (MIT) and
|
| 112 |
+
6,993 paragraphs and messages from WikiText and OpenAssistant. Nothing was trained or
|
| 113 |
+
tuned on it. Many SecLists payloads also occur in the training sources, so the row
|
| 114 |
+
counting only lines that match no row of the training pool is the stricter one.
|
| 115 |
+
|
| 116 |
+
**Challenge set:** 140 hand-written strings. Its past mistakes guided the training-time
|
| 117 |
+
variation, so treat it as a regression check, not an independent test. Missed: none
|
| 118 |
+
|
| 119 |
+
## Limitations
|
| 120 |
+
|
| 121 |
+
- **Unfamiliar benign text raises false alarms.** Benign training text is mostly URLs,
|
| 122 |
+
JavaScript, reviews and generated HTML. Measure the false-alarm rate on your own
|
| 123 |
+
traffic and adjust the threshold before blocking anything.
|
| 124 |
+
- **Programming help text is the riskiest benign input.** 5 of 1,994 OpenAssistant
|
| 125 |
+
messages were flagged (mostly answers that mix prose and code blocks), against
|
| 126 |
+
0 of 4,999 Wikipedia paragraphs.
|
| 127 |
+
- **Injected markup counts as an attack.** `<iframe src=...>`, `<script src=...>` and
|
| 128 |
+
`<form>` in an *input* are scored high, as the training data labels them. Do not
|
| 129 |
+
run it over whole web pages.
|
| 130 |
+
- **Non-Latin text is coarse.** Every character above code point 255 is one token, so
|
| 131 |
+
Cyrillic, Devanagari or Chinese text looks alike. One external false alarm was a
|
| 132 |
+
Russian-language message.
|
| 133 |
+
- **A score is evidence, not proof.** It does not know where the input is written
|
| 134 |
+
(HTML body, attribute, script, URL), which decides whether it can execute.
|
| 135 |
+
- **Exact deduplication only.** Related payload families can sit in both training
|
| 136 |
+
and test data, so held-out numbers are optimistic. The external benchmark is the
|
| 137 |
+
better estimate.
|
| 138 |
+
|
| 139 |
+
## Intended use
|
| 140 |
+
|
| 141 |
+
Defensive: triaging inputs and logs, flagging suspicious values for review, and
|
| 142 |
+
measuring how well a filter copes with encoded payloads. It is not a web application
|
| 143 |
+
firewall on its own, and it was not built or evaluated for generating payloads.
|
config.json
ADDED
|
@@ -0,0 +1,22 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"architecture": "xss_char_cnn_transformer",
|
| 3 |
+
"format_version": 1,
|
| 4 |
+
"tokenizer": "codepoints_v1",
|
| 5 |
+
"network": {
|
| 6 |
+
"max_length": 1000,
|
| 7 |
+
"embed_dim": 128,
|
| 8 |
+
"width": 192,
|
| 9 |
+
"layers": 4,
|
| 10 |
+
"heads": 8
|
| 11 |
+
},
|
| 12 |
+
"threshold": 0.7277562894695883,
|
| 13 |
+
"temperature": 1.6763814589384218,
|
| 14 |
+
"window": 1000,
|
| 15 |
+
"stride": 500,
|
| 16 |
+
"max_decode_rounds": 3,
|
| 17 |
+
"labels": {
|
| 18 |
+
"0": "benign",
|
| 19 |
+
"1": "xss"
|
| 20 |
+
},
|
| 21 |
+
"initialization": "random_weights"
|
| 22 |
+
}
|
metrics.json
ADDED
|
@@ -0,0 +1,273 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"model": {
|
| 3 |
+
"kind": "char_cnn_transformer",
|
| 4 |
+
"initialization": "random_weights",
|
| 5 |
+
"parameters": 1959361,
|
| 6 |
+
"config": {
|
| 7 |
+
"max_length": 1000,
|
| 8 |
+
"embed_dim": 128,
|
| 9 |
+
"width": 192,
|
| 10 |
+
"layers": 4,
|
| 11 |
+
"heads": 8,
|
| 12 |
+
"dropout": 0.1
|
| 13 |
+
},
|
| 14 |
+
"device": "cuda",
|
| 15 |
+
"epochs": 12,
|
| 16 |
+
"best_epoch": 9,
|
| 17 |
+
"batch_size": 128,
|
| 18 |
+
"learning_rate": 0.0006,
|
| 19 |
+
"seed": 42,
|
| 20 |
+
"temperature": 1.6764,
|
| 21 |
+
"training_seconds": 261.2,
|
| 22 |
+
"torch_version": "2.11.0+cu128"
|
| 23 |
+
},
|
| 24 |
+
"threshold": 0.7277562894695883,
|
| 25 |
+
"target_fpr": 0.001,
|
| 26 |
+
"rows": 68258,
|
| 27 |
+
"sources": [
|
| 28 |
+
"mereani",
|
| 29 |
+
"synthetic",
|
| 30 |
+
"twin_urls"
|
| 31 |
+
],
|
| 32 |
+
"cleaning": {
|
| 33 |
+
"input_rows": 82341,
|
| 34 |
+
"empty_rows": 0,
|
| 35 |
+
"conflicting_texts": 0,
|
| 36 |
+
"duplicate_rows": 0,
|
| 37 |
+
"output_rows": 82341,
|
| 38 |
+
"excluded_sources": [
|
| 39 |
+
"webattacks"
|
| 40 |
+
],
|
| 41 |
+
"excluded_source_rows": 14076,
|
| 42 |
+
"challenge_overlap_rows_removed": 7
|
| 43 |
+
},
|
| 44 |
+
"held_out_test": {
|
| 45 |
+
"samples": 10239,
|
| 46 |
+
"roc_auc": 0.99932,
|
| 47 |
+
"pr_auc": 0.99571,
|
| 48 |
+
"detection_at_1pct_false_alarms": 0.9963,
|
| 49 |
+
"detection_at_0.1pct_false_alarms": 0.9889,
|
| 50 |
+
"threshold": 0.7278,
|
| 51 |
+
"detection_rate": 0.9889,
|
| 52 |
+
"false_alarm_rate": 0.0007,
|
| 53 |
+
"precision": 0.9972,
|
| 54 |
+
"confusion": {
|
| 55 |
+
"tp": 2131,
|
| 56 |
+
"fp": 6,
|
| 57 |
+
"fn": 24,
|
| 58 |
+
"tn": 8078
|
| 59 |
+
},
|
| 60 |
+
"by_source": {
|
| 61 |
+
"mereani": {
|
| 62 |
+
"attacks": 2155,
|
| 63 |
+
"caught": 2131,
|
| 64 |
+
"benign": 4210,
|
| 65 |
+
"false_alarms": 4
|
| 66 |
+
},
|
| 67 |
+
"synthetic": {
|
| 68 |
+
"attacks": 0,
|
| 69 |
+
"caught": 0,
|
| 70 |
+
"benign": 1860,
|
| 71 |
+
"false_alarms": 0
|
| 72 |
+
},
|
| 73 |
+
"twin_urls": {
|
| 74 |
+
"attacks": 0,
|
| 75 |
+
"caught": 0,
|
| 76 |
+
"benign": 2014,
|
| 77 |
+
"false_alarms": 2
|
| 78 |
+
}
|
| 79 |
+
}
|
| 80 |
+
},
|
| 81 |
+
"challenge_set": {
|
| 82 |
+
"attacks": 60,
|
| 83 |
+
"caught": 60,
|
| 84 |
+
"missed": [],
|
| 85 |
+
"benign": 80,
|
| 86 |
+
"false_alarms": []
|
| 87 |
+
},
|
| 88 |
+
"external_benchmark": {
|
| 89 |
+
"attack_sources": {
|
| 90 |
+
"seclists_brutelogic": {
|
| 91 |
+
"attacks": 101,
|
| 92 |
+
"caught": 89,
|
| 93 |
+
"unseen_attacks": 99,
|
| 94 |
+
"unseen_caught": 87
|
| 95 |
+
},
|
| 96 |
+
"seclists_jhaddix": {
|
| 97 |
+
"attacks": 106,
|
| 98 |
+
"caught": 90,
|
| 99 |
+
"unseen_attacks": 64,
|
| 100 |
+
"unseen_caught": 49
|
| 101 |
+
},
|
| 102 |
+
"seclists_mario": {
|
| 103 |
+
"attacks": 137,
|
| 104 |
+
"caught": 119,
|
| 105 |
+
"unseen_attacks": 135,
|
| 106 |
+
"unseen_caught": 119
|
| 107 |
+
},
|
| 108 |
+
"seclists_ofjaaah": {
|
| 109 |
+
"attacks": 198,
|
| 110 |
+
"caught": 172,
|
| 111 |
+
"unseen_attacks": 197,
|
| 112 |
+
"unseen_caught": 171
|
| 113 |
+
},
|
| 114 |
+
"seclists_payloadbox": {
|
| 115 |
+
"attacks": 1213,
|
| 116 |
+
"caught": 1104,
|
| 117 |
+
"unseen_attacks": 880,
|
| 118 |
+
"unseen_caught": 774
|
| 119 |
+
},
|
| 120 |
+
"seclists_polyglots": {
|
| 121 |
+
"attacks": 13,
|
| 122 |
+
"caught": 13,
|
| 123 |
+
"unseen_attacks": 13,
|
| 124 |
+
"unseen_caught": 13
|
| 125 |
+
},
|
| 126 |
+
"seclists_portswigger": {
|
| 127 |
+
"attacks": 6042,
|
| 128 |
+
"caught": 6042,
|
| 129 |
+
"unseen_attacks": 6041,
|
| 130 |
+
"unseen_caught": 6041
|
| 131 |
+
},
|
| 132 |
+
"seclists_rsnake": {
|
| 133 |
+
"attacks": 66,
|
| 134 |
+
"caught": 63,
|
| 135 |
+
"unseen_attacks": 12,
|
| 136 |
+
"unseen_caught": 9
|
| 137 |
+
},
|
| 138 |
+
"seclists_somdev": {
|
| 139 |
+
"attacks": 20,
|
| 140 |
+
"caught": 20,
|
| 141 |
+
"unseen_attacks": 19,
|
| 142 |
+
"unseen_caught": 19
|
| 143 |
+
}
|
| 144 |
+
},
|
| 145 |
+
"benign_sources": {
|
| 146 |
+
"oasst": {
|
| 147 |
+
"benign": 1994,
|
| 148 |
+
"false_alarms": 5
|
| 149 |
+
},
|
| 150 |
+
"wikitext": {
|
| 151 |
+
"benign": 4999,
|
| 152 |
+
"false_alarms": 0
|
| 153 |
+
}
|
| 154 |
+
},
|
| 155 |
+
"attacks": 7896,
|
| 156 |
+
"caught": 7712,
|
| 157 |
+
"unseen_attacks": 7460,
|
| 158 |
+
"unseen_caught": 7282,
|
| 159 |
+
"benign": 6993,
|
| 160 |
+
"false_alarms": 5,
|
| 161 |
+
"false_alarm_examples": [
|
| 162 |
+
"to connect to an ftp server in python, you can use the built-in ftplib module. here is an example function that connects to an ftp server: ``` import ftplib def ftp_connect(host, username, password): ",
|
| 163 |
+
"try this instead: import torch from transformers import berttokenizer, bertmodel # load the bert tokenizer tokenizer = berttokenizer.from_pretrained('bert-base-uncased') # encode a text input input_id",
|
| 164 |
+
"to create a frame by frame animation in react.js, you can follow these steps: 1. create a new react component for your animation. you can use the usestate hook to keep track of the current frame index",
|
| 165 |
+
"\u044f \u0445\u043e\u0442\u0435\u043b \u0431\u044b \u0440\u0430\u0441\u0441\u043a\u0430\u0437\u0430\u0442\u044c \u043e \u0437\u0430\u043c\u0435\u0447\u0430\u0442\u0435\u043b\u044c\u043d\u043e\u0439 \u0438\u0433\u0440\u0435 \u043f\u043e\u0434 \u043d\u0430\u0437\u0432\u0430\u043d\u0438\u0435\u043c \"the witcher 3: \u0434\u0438\u043a\u0430\u044f \u043e\u0445\u043e\u0442\u0430\". \u044d\u0442\u043e \u0440\u043e\u043b\u0435\u0432\u0430\u044f \u0438\u0433\u0440\u0430 \u0441 \u043e\u0442\u043a\u0440\u044b\u0442\u044b\u043c \u043c\u0438\u0440\u043e\u043c, \u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u0430\u043d\u043d\u0430\u044f \u043f\u043e\u043b\u044c\u0441\u043a\u043e\u0439 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u0435\u0439 cd projekt red. \u0438\u0433\u0440\u043e\u043a \u0438\u0433\u0440\u0430\u0435\u0442 \u0437\u0430 \u0433\u043b\u0430\u0432\u043d\u043e\u0433\u043e \u0433\u0435\u0440\u043e",
|
| 166 |
+
"there are several libraries available to help you work with websockets in javascript, each with varying levels of abstraction. if you're looking for the simplest and highest level of abstraction, you "
|
| 167 |
+
]
|
| 168 |
+
},
|
| 169 |
+
"linear_baseline": {
|
| 170 |
+
"held_out_test": {
|
| 171 |
+
"samples": 10239,
|
| 172 |
+
"roc_auc": 0.99981,
|
| 173 |
+
"pr_auc": 0.99941,
|
| 174 |
+
"detection_at_1pct_false_alarms": 0.9968,
|
| 175 |
+
"detection_at_0.1pct_false_alarms": 0.9796,
|
| 176 |
+
"threshold": 0.6494,
|
| 177 |
+
"detection_rate": 0.9865,
|
| 178 |
+
"false_alarm_rate": 0.0014,
|
| 179 |
+
"precision": 0.9949,
|
| 180 |
+
"confusion": {
|
| 181 |
+
"tp": 2126,
|
| 182 |
+
"fp": 11,
|
| 183 |
+
"fn": 29,
|
| 184 |
+
"tn": 8073
|
| 185 |
+
}
|
| 186 |
+
},
|
| 187 |
+
"challenge_set": {
|
| 188 |
+
"attacks": 60,
|
| 189 |
+
"caught": 58,
|
| 190 |
+
"missed": [
|
| 191 |
+
"<a href=\"#\" onclick=\"fetch('/api/delete-account',{method:'POST'})\">Free gift</a>",
|
| 192 |
+
"<p onmouseenter=$.post('/settings/email',{email:'a@attacker.example'})>Great article</p>"
|
| 193 |
+
],
|
| 194 |
+
"benign": 80,
|
| 195 |
+
"false_alarms": []
|
| 196 |
+
},
|
| 197 |
+
"external_benchmark": {
|
| 198 |
+
"attack_sources": {
|
| 199 |
+
"seclists_brutelogic": {
|
| 200 |
+
"attacks": 101,
|
| 201 |
+
"caught": 89,
|
| 202 |
+
"unseen_attacks": 99,
|
| 203 |
+
"unseen_caught": 87
|
| 204 |
+
},
|
| 205 |
+
"seclists_jhaddix": {
|
| 206 |
+
"attacks": 106,
|
| 207 |
+
"caught": 99,
|
| 208 |
+
"unseen_attacks": 64,
|
| 209 |
+
"unseen_caught": 58
|
| 210 |
+
},
|
| 211 |
+
"seclists_mario": {
|
| 212 |
+
"attacks": 137,
|
| 213 |
+
"caught": 120,
|
| 214 |
+
"unseen_attacks": 135,
|
| 215 |
+
"unseen_caught": 119
|
| 216 |
+
},
|
| 217 |
+
"seclists_ofjaaah": {
|
| 218 |
+
"attacks": 198,
|
| 219 |
+
"caught": 168,
|
| 220 |
+
"unseen_attacks": 197,
|
| 221 |
+
"unseen_caught": 167
|
| 222 |
+
},
|
| 223 |
+
"seclists_payloadbox": {
|
| 224 |
+
"attacks": 1213,
|
| 225 |
+
"caught": 1152,
|
| 226 |
+
"unseen_attacks": 880,
|
| 227 |
+
"unseen_caught": 825
|
| 228 |
+
},
|
| 229 |
+
"seclists_polyglots": {
|
| 230 |
+
"attacks": 13,
|
| 231 |
+
"caught": 13,
|
| 232 |
+
"unseen_attacks": 13,
|
| 233 |
+
"unseen_caught": 13
|
| 234 |
+
},
|
| 235 |
+
"seclists_portswigger": {
|
| 236 |
+
"attacks": 6042,
|
| 237 |
+
"caught": 5885,
|
| 238 |
+
"unseen_attacks": 6041,
|
| 239 |
+
"unseen_caught": 5884
|
| 240 |
+
},
|
| 241 |
+
"seclists_rsnake": {
|
| 242 |
+
"attacks": 66,
|
| 243 |
+
"caught": 62,
|
| 244 |
+
"unseen_attacks": 12,
|
| 245 |
+
"unseen_caught": 8
|
| 246 |
+
},
|
| 247 |
+
"seclists_somdev": {
|
| 248 |
+
"attacks": 20,
|
| 249 |
+
"caught": 16,
|
| 250 |
+
"unseen_attacks": 19,
|
| 251 |
+
"unseen_caught": 15
|
| 252 |
+
}
|
| 253 |
+
},
|
| 254 |
+
"benign_sources": {
|
| 255 |
+
"oasst": {
|
| 256 |
+
"benign": 1994,
|
| 257 |
+
"false_alarms": 0
|
| 258 |
+
},
|
| 259 |
+
"wikitext": {
|
| 260 |
+
"benign": 4999,
|
| 261 |
+
"false_alarms": 0
|
| 262 |
+
}
|
| 263 |
+
},
|
| 264 |
+
"attacks": 7896,
|
| 265 |
+
"caught": 7604,
|
| 266 |
+
"unseen_attacks": 7460,
|
| 267 |
+
"unseen_caught": 7176,
|
| 268 |
+
"benign": 6993,
|
| 269 |
+
"false_alarms": 0,
|
| 270 |
+
"false_alarm_examples": []
|
| 271 |
+
}
|
| 272 |
+
}
|
| 273 |
+
}
|
model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f8b64f013d75878e2546d8fdd957abd8226e7add8a30693598aa018927060298
|
| 3 |
+
size 7843860
|
xss_detector.py
ADDED
|
@@ -0,0 +1,152 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Standalone loader for the character-level XSS detector.
|
| 2 |
+
|
| 3 |
+
Needs only torch, numpy and safetensors (plus huggingface_hub to download):
|
| 4 |
+
|
| 5 |
+
from xss_detector import XSSDetector
|
| 6 |
+
detector = XSSDetector.from_pretrained("<repo id or local folder>")
|
| 7 |
+
detector.predict(["<img src=x onerror=alert(1)>", "<b>hello</b>"])
|
| 8 |
+
|
| 9 |
+
The weights are a safetensors file, so loading them runs no code from the download.
|
| 10 |
+
"""
|
| 11 |
+
from __future__ import annotations
|
| 12 |
+
|
| 13 |
+
import html
|
| 14 |
+
import json
|
| 15 |
+
import math
|
| 16 |
+
import re
|
| 17 |
+
from pathlib import Path
|
| 18 |
+
from urllib.parse import unquote
|
| 19 |
+
|
| 20 |
+
import numpy as np
|
| 21 |
+
import torch
|
| 22 |
+
from safetensors.torch import load_file
|
| 23 |
+
from torch import nn
|
| 24 |
+
from torch.nn import functional as F
|
| 25 |
+
|
| 26 |
+
PAD, OTHER, VOCAB = 0, 257, 258 # 1..256 are code points 0..255; OTHER is every later code point
|
| 27 |
+
_WHITESPACE = re.compile(r"\s+")
|
| 28 |
+
|
| 29 |
+
|
| 30 |
+
def normalize(text: str, rounds: int = 3) -> str:
|
| 31 |
+
"""Undo URL and HTML encoding (up to `rounds` times), drop NUL bytes, squeeze whitespace, lowercase."""
|
| 32 |
+
text = str(text)
|
| 33 |
+
for _ in range(rounds):
|
| 34 |
+
decoded = html.unescape(unquote(text))
|
| 35 |
+
if decoded == text:
|
| 36 |
+
break
|
| 37 |
+
text = decoded
|
| 38 |
+
text = text.replace("\x00", "")
|
| 39 |
+
return _WHITESPACE.sub(" ", text).strip().lower()
|
| 40 |
+
|
| 41 |
+
|
| 42 |
+
def windows(text: str, window: int, stride: int) -> list[str]:
|
| 43 |
+
"""Overlapping windows of a normalised text. Each is scored; the highest score counts."""
|
| 44 |
+
if len(text) <= window:
|
| 45 |
+
return [text]
|
| 46 |
+
return [text[i : i + window] for i in range(0, len(text) - window + stride, stride)]
|
| 47 |
+
|
| 48 |
+
|
| 49 |
+
def encode(text: str, max_length: int) -> np.ndarray:
|
| 50 |
+
points = np.frombuffer((text[:max_length] or " ").encode("utf-32-le", "surrogatepass"), dtype=np.uint32)
|
| 51 |
+
return np.where(points < 256, points + 1, OTHER).astype(np.int64)
|
| 52 |
+
|
| 53 |
+
|
| 54 |
+
class XSSNet(nn.Module):
|
| 55 |
+
"""Character embeddings -> convolutions over 1/3/5/7 characters -> 2 stride-2 convolutions -> Transformer encoder -> pooled head."""
|
| 56 |
+
|
| 57 |
+
def __init__(self, max_length: int, embed_dim: int, width: int, layers: int, heads: int, dropout: float = 0.0):
|
| 58 |
+
super().__init__()
|
| 59 |
+
self.max_length = max_length
|
| 60 |
+
self.embed = nn.Embedding(VOCAB, embed_dim, padding_idx=PAD)
|
| 61 |
+
self.convs = nn.ModuleList(nn.Conv1d(embed_dim, width // 4, k, padding=k // 2) for k in (1, 3, 5, 7))
|
| 62 |
+
self.down = nn.ModuleList(nn.Conv1d(width, width, 3, stride=2, padding=1) for _ in range(2))
|
| 63 |
+
self.position = nn.Embedding((max_length + 3) // 4, width)
|
| 64 |
+
layer = nn.TransformerEncoderLayer(width, heads, 3 * width, dropout, activation="gelu", batch_first=True, norm_first=True)
|
| 65 |
+
self.encoder = nn.TransformerEncoder(layer, layers, enable_nested_tensor=False)
|
| 66 |
+
self.norm = nn.LayerNorm(width)
|
| 67 |
+
self.head = nn.Sequential(nn.Linear(2 * width, width), nn.GELU(), nn.Dropout(dropout), nn.Linear(width, 1))
|
| 68 |
+
|
| 69 |
+
def forward(self, tokens: torch.Tensor) -> torch.Tensor:
|
| 70 |
+
"""Logit per row; positive means XSS."""
|
| 71 |
+
mask = tokens != PAD
|
| 72 |
+
x = self.embed(tokens).transpose(1, 2)
|
| 73 |
+
x = torch.cat([F.gelu(conv(x)) for conv in self.convs], dim=1) * mask.unsqueeze(1)
|
| 74 |
+
for down in self.down:
|
| 75 |
+
mask = mask[:, ::2]
|
| 76 |
+
x = F.gelu(down(x)) * mask.unsqueeze(1)
|
| 77 |
+
x = x.transpose(1, 2)
|
| 78 |
+
x = x + self.position(torch.arange(x.size(1), device=x.device))
|
| 79 |
+
x = self.norm(self.encoder(x, src_key_padding_mask=~mask))
|
| 80 |
+
keep = mask.unsqueeze(-1)
|
| 81 |
+
highest = x.masked_fill(~keep, -1e4).amax(dim=1)
|
| 82 |
+
average = (x * keep).sum(dim=1) / keep.sum(dim=1)
|
| 83 |
+
return self.head(torch.cat([highest, average], dim=-1)).squeeze(-1)
|
| 84 |
+
|
| 85 |
+
|
| 86 |
+
class XSSDetector:
|
| 87 |
+
def __init__(self, folder: str | Path, device: str | None = None, batch_size: int = 256):
|
| 88 |
+
folder = Path(folder)
|
| 89 |
+
self.config = json.loads((folder / "config.json").read_text(encoding="utf-8"))
|
| 90 |
+
if self.config.get("architecture") != "xss_char_cnn_transformer" or self.config.get("tokenizer") != "codepoints_v1":
|
| 91 |
+
raise ValueError("config.json does not describe this detector")
|
| 92 |
+
net = self.config["network"]
|
| 93 |
+
self.device = device or ("cuda" if torch.cuda.is_available() else "cpu")
|
| 94 |
+
self.model = XSSNet(net["max_length"], net["embed_dim"], net["width"], net["layers"], net["heads"])
|
| 95 |
+
self.model.load_state_dict(load_file(str(folder / "model.safetensors")))
|
| 96 |
+
self.model.to(self.device).eval()
|
| 97 |
+
self.threshold = float(self.config["threshold"])
|
| 98 |
+
self.temperature = float(self.config["temperature"])
|
| 99 |
+
if not (math.isfinite(self.temperature) and self.temperature > 0 and 0 <= self.threshold <= 1):
|
| 100 |
+
raise ValueError("invalid threshold or temperature in config.json")
|
| 101 |
+
self.batch_size = batch_size
|
| 102 |
+
|
| 103 |
+
@classmethod
|
| 104 |
+
def from_pretrained(cls, repo_or_folder: str, revision: str | None = None, device: str | None = None) -> "XSSDetector":
|
| 105 |
+
"""A local folder, or a Hugging Face repo id (downloads only config.json and model.safetensors)."""
|
| 106 |
+
if Path(repo_or_folder).is_dir():
|
| 107 |
+
return cls(repo_or_folder, device)
|
| 108 |
+
from huggingface_hub import snapshot_download
|
| 109 |
+
|
| 110 |
+
folder = snapshot_download(repo_or_folder, revision=revision, allow_patterns=["config.json", "model.safetensors"])
|
| 111 |
+
return cls(folder, device)
|
| 112 |
+
|
| 113 |
+
@torch.inference_mode()
|
| 114 |
+
def _logits(self, chunks: list[str]) -> np.ndarray:
|
| 115 |
+
rows = [encode(chunk, self.model.max_length) for chunk in chunks]
|
| 116 |
+
order = np.argsort([len(row) for row in rows], kind="stable")
|
| 117 |
+
out = np.zeros(len(rows))
|
| 118 |
+
start = 0
|
| 119 |
+
while start < len(order):
|
| 120 |
+
longest = len(rows[order[min(start + self.batch_size, len(order)) - 1]])
|
| 121 |
+
picks = order[start : start + max(8, min(self.batch_size, self.batch_size * 250 // longest))]
|
| 122 |
+
tokens = np.zeros((len(picks), max(len(rows[i]) for i in picks)), dtype=np.int64)
|
| 123 |
+
for j, i in enumerate(picks):
|
| 124 |
+
tokens[j, : len(rows[i])] = rows[i]
|
| 125 |
+
out[picks] = self.model(torch.from_numpy(tokens).to(self.device)).double().cpu().numpy()
|
| 126 |
+
start += len(picks)
|
| 127 |
+
return out
|
| 128 |
+
|
| 129 |
+
def score(self, texts: list[str]) -> np.ndarray:
|
| 130 |
+
"""Calibrated probability of XSS for each text (the highest over its windows)."""
|
| 131 |
+
chunks, owners = [], []
|
| 132 |
+
for i, text in enumerate(texts):
|
| 133 |
+
for chunk in windows(normalize(text, self.config["max_decode_rounds"]), self.config["window"], self.config["stride"]):
|
| 134 |
+
chunks.append(chunk)
|
| 135 |
+
owners.append(i)
|
| 136 |
+
out = np.zeros(len(texts))
|
| 137 |
+
if chunks:
|
| 138 |
+
z = np.clip(self._logits(chunks) / self.temperature, -50, 50)
|
| 139 |
+
np.maximum.at(out, owners, 1 / (1 + np.exp(-z)))
|
| 140 |
+
return out
|
| 141 |
+
|
| 142 |
+
def predict(self, texts: list[str]) -> list[dict]:
|
| 143 |
+
"""[{"text", "score", "is_xss"}] using the threshold tuned for 0.1% false alarms on validation data."""
|
| 144 |
+
return [{"text": text, "score": round(float(s), 6), "is_xss": bool(s > self.threshold)} for text, s in zip(texts, self.score(texts))]
|
| 145 |
+
|
| 146 |
+
|
| 147 |
+
if __name__ == "__main__":
|
| 148 |
+
import sys
|
| 149 |
+
|
| 150 |
+
detector = XSSDetector.from_pretrained(sys.argv[1] if len(sys.argv) > 1 else ".")
|
| 151 |
+
for row in detector.predict(sys.argv[2:] or ["<img src=x onerror=alert(1)>", "<b>hello</b>"]):
|
| 152 |
+
print(f"{'XSS ' if row['is_xss'] else 'benign'} {row['score']:.3f} {row['text'][:100]}")
|