Using SkillJev-2B
The repository root contains the complete merged SkillJev-Direct model. No PEFT attachment or separate base-model download is needed. The original LoRA adapter and trained head are retained in adapter/.
The verified recipe uses Python 3.12, PyTorch 2.11.0, Transformers 5.17.0, PEFT 0.21.2 and huggingface_hub 1.33.0 on CUDA/BF16, batch size 1. PEFT is needed only for the adapter route. The scorer limits GPU allocation to 8 GiB.
pip install torch==2.11.0 transformers==5.17.0 huggingface_hub==1.33.0
hf download SkillJev/SkillJev-2B score.py --revision v1 --local-dir skilljev
python skilljev/score.py path/to/SKILL.md
The helper downloads the versioned full model and returns a JSON object with the calibrated malicious probability, threshold, review decision and truncation flag. It reads the supplied file as inert text.
For direct model loading:
import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("SkillJev/SkillJev-2B", revision="v1")
model = AutoModelForSequenceClassification.from_pretrained(
"SkillJev/SkillJev-2B", revision="v1",
dtype=torch.bfloat16, attn_implementation="sdpa", trust_remote_code=False,
)
Apply the input policy and scoring settings from scoring_config.json, as implemented in score.py. The exact input is Premise: {SKILL.md}\nHypothesis: This AI agent skill is malicious. Long inputs retain the prefix plus the complete hypothesis within 8,192 tokens. No generation or chat template is used.
The native labels are contradiction=0, entailment=1 and neutral=2. Compute z = entailment - logaddexp(contradiction, neutral), then p = sigmoid(0.76127103215625*z + 0.3934838733671388). The frozen review gate is z >= -2.515625536875148; its equivalent calibrated probability threshold is 0.17922765383653141. Native entailment softmax is not the calibrated probability. The unused neutral class is not a validated uncertainty estimate.
merge_verification.json records a fresh-process native reload and comparisons with the unmerged selected adapter across all validation and test records, using unchanged calibration and thresholds. A score below the gate does not certify the full skill package as safe.