| --- |
| license: other |
| pipeline_tag: text-classification |
| tags: |
| - shell |
| - bash |
| - cybersecurity |
| - safety |
| - classifier |
| --- |
| |
| # Kestrel |
|
|
| Kestrel is a compact local classifier for context-free cyber and system risk in |
| individual Bash tool calls. |
|
|
| ## Release |
|
|
| Version 0.1.0 publishes the portable JSON model artifact used for the Kestrel |
| evaluation. This repository does not include training code, training data, or a |
| detailed training recipe. |
|
|
| | File | Description | |
| |---|---| |
| | `classifier.json` | Portable Kestrel v0.1.0 model artifact | |
| | `config.json` | Model manifest with the artifact filename, version, and SHA-256 | |
| | `SHA256SUMS` | Integrity checksums for the model artifact and manifest | |
|
|
| The model is custom and is not a Transformers checkpoint. Download the artifact |
| with `huggingface_hub`: |
|
|
| ```python |
| import hashlib |
| import json |
| from pathlib import Path |
| |
| from huggingface_hub import snapshot_download |
| |
| model_dir = Path(snapshot_download( |
| repo_id="kontext-security/Kestrel", |
| allow_patterns=["config.json", "classifier.json"], |
| )) |
| config = json.loads((model_dir / "config.json").read_text()) |
| model_path = str(model_dir / config["artifact"]["filename"]) |
| |
| if hashlib.sha256(Path(model_path).read_bytes()).hexdigest() != config["artifact"]["sha256"]: |
| raise ValueError("Kestrel artifact checksum mismatch") |
| ``` |
|
|
| `config.json` is a manifest for this custom JSON model. The example fetches both |
| files from the same repository revision and checks the artifact against the |
| manifest. Fetching the manifest also enables Hugging Face's standard |
| [download statistics](https://huggingface.co/docs/hub/models-download-stats). |
| Existing code and direct links that fetch only `classifier.json` remain valid, |
| but those requests are not included in the standard download count. |
|
|
| ## Evaluation |
|
|
| On the published ShellRisk-Bench v0.1 test split, Kestrel reaches 0.947 |
| precision, 0.922 recall, and 0.934 F1. The test split is a same-source, |
| in-distribution holdout; it is not evidence of transfer to a novel command |
| dialect. |
|
|
| See [ShellRisk-Bench](https://github.com/kontext-security/shellrisk-bench) for |
| the fixed split, evaluation methodology, prompts, scorer, aggregate results, |
| and per-example Kestrel verdicts. The dataset is available at |
| [kontext-security/ShellRisk-Bench](https://huggingface.co/datasets/kontext-security/ShellRisk-Bench). |
|
|
| ## Scope and limitations |
|
|
| Kestrel evaluates one submitted Bash command without user, task, or session |
| context. It does not infer intent or make a complete authorization decision. |
| It is intended as one signal layered underneath deterministic protections and |
| runtime authorization policy, not as the sole control for executing commands. |
|
|
| ## Try it |
|
|
| Kestrel is available as part of Kontext, where it evaluates the cyber risk of |
| agent tool calls locally. Learn more and try it at |
| [kontext.security](https://kontext.security). |
|
|
| ## Integrity |
|
|
| ```text |
| SHA-256 1df8b3e5f2bfc4e1fe95230ee9b3d37f63aaa461a8551982fcc0dd7103c8221b classifier.json |
| SHA-256 3eb80e6744370a4c34743b14698eaa326121aa36f8f8e2234b7149223d63190a config.json |
| ``` |
|
|
| ## License |
|
|
| The artifact is published with `license: other`. The public benchmark sources |
| retain their own upstream terms; review the |
| [dataset provenance](https://github.com/kontext-security/shellrisk-bench/blob/main/DATASETS.md) |
| before reuse or redistribution. |
|
|
