| # Using SkillJev-2B |
|
|
| The repository root contains the complete merged SkillJev-Direct model. No PEFT attachment or separate base-model download is needed. The original LoRA adapter and trained head are retained in [`adapter/`](https://huggingface.co/SkillJev/SkillJev-2B/blob/v1/adapter/README.md). |
|
|
| The verified recipe uses Python 3.12, PyTorch 2.11.0, Transformers 5.17.0, PEFT 0.21.2 and huggingface_hub 1.33.0 on CUDA/BF16, batch size 1. PEFT is needed only for the adapter route. The scorer limits GPU allocation to 8 GiB. |
| |
| ```bash |
| pip install torch==2.11.0 transformers==5.17.0 huggingface_hub==1.33.0 |
| hf download SkillJev/SkillJev-2B score.py --revision v1 --local-dir skilljev |
| python skilljev/score.py path/to/SKILL.md |
| ``` |
| |
| The helper downloads the versioned full model and returns a JSON object with the calibrated malicious probability, threshold, review decision and truncation flag. It reads the supplied file as inert text. |
| |
| For direct model loading: |
| |
| ```python |
| import torch |
| from transformers import AutoModelForSequenceClassification, AutoTokenizer |
|
|
| tokenizer = AutoTokenizer.from_pretrained("SkillJev/SkillJev-2B", revision="v1") |
| model = AutoModelForSequenceClassification.from_pretrained( |
| "SkillJev/SkillJev-2B", revision="v1", |
| dtype=torch.bfloat16, attn_implementation="sdpa", trust_remote_code=False, |
| ) |
| ``` |
| |
| Apply the input policy and scoring settings from [`scoring_config.json`](https://huggingface.co/SkillJev/SkillJev-2B/blob/v1/scoring_config.json), as implemented in [`score.py`](https://huggingface.co/SkillJev/SkillJev-2B/blob/v1/score.py). The exact input is `Premise: {SKILL.md}\nHypothesis: This AI agent skill is malicious.` Long inputs retain the prefix plus the complete hypothesis within 8,192 tokens. No generation or chat template is used. |
|
|
| The native labels are contradiction=0, entailment=1 and neutral=2. Compute `z = entailment - logaddexp(contradiction, neutral)`, then `p = sigmoid(0.76127103215625*z + 0.3934838733671388)`. The frozen review gate is `z >= -2.515625536875148`; its equivalent calibrated probability threshold is `0.17922765383653141`. Native entailment softmax is not the calibrated probability. The unused neutral class is not a validated uncertainty estimate. |
|
|
| [`merge_verification.json`](https://huggingface.co/SkillJev/SkillJev-2B/blob/v1/merge_verification.json) records a fresh-process native reload and comparisons with the unmerged selected adapter across all validation and test records, using unchanged calibration and thresholds. A score below the gate does not certify the full skill package as safe. |
|
|
