Initialize SkillJev-2B project card
Browse files
README.md
ADDED
|
@@ -0,0 +1,40 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
pipeline_tag: text-classification
|
| 3 |
+
base_model: AlexWortega/openjev
|
| 4 |
+
datasets:
|
| 5 |
+
- ProtectSkills/MaliciousSkillBench
|
| 6 |
+
tags:
|
| 7 |
+
- agent-security
|
| 8 |
+
- skilljev
|
| 9 |
+
- research
|
| 10 |
+
---
|
| 11 |
+
|
| 12 |
+
# SkillJev-2B
|
| 13 |
+
|
| 14 |
+
SkillJev-2B is a research project to adapt OpenJev into a small probabilistic scorer for malicious AI-agent skills.
|
| 15 |
+
|
| 16 |
+
**Status:** repository initialization. No fine-tuned weights, calibration parameters, or evaluation results have been uploaded. This repository is private for team collaboration.
|
| 17 |
+
|
| 18 |
+
## Planned model
|
| 19 |
+
|
| 20 |
+
- Base repository: `AlexWortega/openjev`
|
| 21 |
+
- Base subfolder: `qwen3.5-2b-nli-v5`
|
| 22 |
+
- Pinned base revision: `a20448012c213128955ca0c693e7c943865cab77`
|
| 23 |
+
- Adaptation: LoRA with a trainable classification head.
|
| 24 |
+
- Fixed hypothesis: `This AI agent skill is malicious.`
|
| 25 |
+
|
| 26 |
+
The Direct route reads only the SKILL.md text. The Spector route adds SkillSpector findings derived from that same text. Training labels and source metadata are not model inputs. Scores require validation-set calibration before being presented as calibrated probabilities.
|
| 27 |
+
|
| 28 |
+
## Data and evaluation
|
| 29 |
+
|
| 30 |
+
MaliciousSkillBench is the public data entry point. Use its official Source-Disjoint manifests and record the exact dataset revision in each experiment. Calibration and threshold selection use public validation data only. TrendAI's 500K corpus is reserved for final scoring after freezing the model and settings; it is not hosted here or used for training or tuning.
|
| 31 |
+
|
| 32 |
+
## Team checkpoints
|
| 33 |
+
|
| 34 |
+
Use `direct/<run-id>/` and, when implemented, `spector/<run-id>/` for shared checkpoints. Each checkpoint must include the adapter, the trained classification head, its configuration, and a run manifest with code/data/base revisions and input settings. Include calibration and threshold settings when available. Reference checkpoints by their exact Hub commit revision. Training-resume state may remain local.
|
| 35 |
+
|
| 36 |
+
## Scope and terms
|
| 37 |
+
|
| 38 |
+
This project scores static SKILL.md text; it does not certify a complete package as safe. Skill examples are untrusted data and must not be installed as coding-agent skills or executed.
|
| 39 |
+
|
| 40 |
+
OpenJev is labelled MIT by its upstream author. Third-party skill texts retain their upstream terms. The license for our trained artifacts will be set before public release.
|