Text Classification
PEFT
Safetensors
English
npm
supply-chain-security
malware-detection
malicious-packages
security
triage
typed-decisions
lora
qlora
Instructions to use Takzen/jay-4b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use Takzen/jay-4b with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-4B") model = PeftModel.from_pretrained(base_model, "Takzen/jay-4b") - Notebooks
- Google Colab
- Kaggle
|
Download README.md from Takzen/jay-4b: direct link, hf CLI and curl.
- Browser
- Download file 9.88 kB
-
https://huggingface.co/Takzen/jay-4b/resolve/main/README.md
- Command line
-
hf download hf://Takzen/jay-4b/README.md
-
curl -L -o README.md https://huggingface.co/Takzen/jay-4b/resolve/main/README.md
9.88 kB
| license: apache-2.0 | |
| base_model: Qwen/Qwen3.5-4B | |
| library_name: peft | |
| language: | |
| - en | |
| pipeline_tag: text-classification | |
| tags: | |
| - npm | |
| - supply-chain-security | |
| - malware-detection | |
| - malicious-packages | |
| - security | |
| - triage | |
| - typed-decisions | |
| - lora | |
| - qlora | |
|  | |
| # Jay 4B (sójka) | |
| Jay 4B reads one npm package version, its package.json, file list and the code most likely to | |
| run, and answers two questions with a probability for every option: **is this version | |
| malicious?** and, if it is, **was the package built to attack, or is it a hijacked library?** | |
| It is a LoRA adapter on [Qwen/Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B) that runs in | |
| 4-bit on one consumer GPU. Each answer comes from the option-letter logits of one forward pass; | |
| no text is generated. Sibling of [Shrike 4B](https://huggingface.co/Takzen/shrike-4b), | |
| [Bullfinch 4B](https://huggingface.co/Takzen/bullfinch-4b) and [Serin 4B](https://huggingface.co/Takzen/serin-4b). | |
| **Why.** Malicious npm versions keep coming: DataDog's open dataset holds 25,460 versions built | |
| to attack and 1,448 hijacked releases published from 2024 to October 2026. Blocking install | |
| scripts (pnpm 10 does it by default) stops the attacks that run from a `preinstall` or | |
| `postinstall` hook, but in 2026 35% of the attack versions in that dataset run when the package | |
| is imported instead, with no hook at all. Jay reads the code, so it does not care how the code | |
| gets to run. It is a triage aid: it ranks what a person should look at first. | |
| ``` | |
| $ python jay_infer.py esbuild@0.28.2 | |
| esbuild@0.28.2 (7 files, install hooks: postinstall) | |
| verdict: benign p(malicious) = 0.017 | |
| code shown to the model: install.js, lib/main.js, bin/esbuild | |
| $ python jay_infer.py chai-as-repaired-5.32.9.tgz --registry-doc chai-as-repaired-5.32.9.json | |
| chai-as-repaired@5.32.9 (42 files, install hooks: none) | |
| verdict: malicious p(malicious) = 0.994 | |
| origin: attack p(hijacked) = 0.003 | |
| code shown to the model: index.js, lib/meta.js, lib/deprecations.js, lib/const.js, lib/time.js, file.js, lib/constants.js, lib/symbols.js, lib/transport-stream.js | |
| ``` | |
| esbuild fetches its platform binary from a `postinstall` script, the kind of benign hook an | |
| earlier pilot flagged (0.85 on esbuild's latest release). chai-as-repaired 5.32.9 is a 2026 | |
| test sample from the DataDog dataset, built to attack and without an install hook: its code | |
| runs when the package is imported. | |
| ## Results | |
| Test panel: 1,992 npm packages first seen in 2026 and never trained on (453 malicious | |
| versions, 1,622 benign), the same rows for every method. Flags at p(malicious) ≥ 0.5; | |
| GuardDog at its own risk labels. | |
| <!-- results:start --> | |
|  | |
| | Method | AUROC | Caught: built to attack | Caught: hijacked | False alarms: random npm | False alarms: with an install script | | |
| |---|---:|---:|---:|---:|---:| | |
| | Rule: has an install script | 0.736 | 62% | 57% | 2.8% | 99.5% | | |
| | GuardDog 3.2.0, any risk | 0.820 | 84% | 80% | 23% | 89% | | |
| | GuardDog 3.2.0, high risk (≥ 7.6) | 0.820 | 40% | 46% | 3.5% | 16% | | |
| | Qwen3.5-4B, untuned | 0.850 | 71% | 6.7% | 0.1% | 4.9% | | |
| | **Jay 4B** | **0.927** | **81%** | **40%** | **1.3%** | **3.3%** | | |
| <!-- results:end --> | |
| - **AUROC** +0.107 over GuardDog (95% interval 0.084 to 0.130) and +0.077 over the untuned | |
| base (0.056 to 0.097), paired bootstrap by package on the same rows. | |
| - **Origin.** Of the malicious versions, 94.3% are correctly told built to attack or hijacked | |
| (macro-F1 0.916; the untuned base 41%). | |
| - **Popular packages.** No false alarm on 197 versions of popular npm packages published in | |
| 2026 (names never seen in training; the install-script rule flags 12% of them), and none on | |
| 86 well-known packages at their latest release (esbuild, electron, bun and others). | |
| - **Calibration.** Temperature 1.231, fitted on a calibration split of 974 answers (ECE 1.3% to | |
| 0.8%). At a calibrated p ≥ 0.95 Jay still catches 70% of the malicious versions, with 4 | |
| false alarms in 1,622 benign versions (0.25%). | |
| - **Where it loses: hijacked libraries.** Jay catches 40% of them; GuardDog at any risk catches | |
| 80% and the install-script rule 57%, each with far more false alarms. A hijacked release is | |
| a real, often large library with injected code, and Jay sees about 6,000 characters of code. | |
| The first pilot caught 52% of hijacks but flagged 14% of benign packages with an install | |
| script, esbuild, electron and bun among them; this release gives up hijack recall for that. | |
| ## How to use | |
| [`jay_infer.py`](jay_infer.py) in this repo is self-contained (torch, transformers, peft, | |
| bitsandbytes, accelerate; a CUDA GPU): | |
| ``` | |
| python jay_infer.py left-pad@1.3.0 # fetched from registry.npmjs.org, integrity checked | |
| python jay_infer.py ./some-package-1.0.0.tgz # a local tarball | |
| python jay_infer.py express@5.1.0 --json # full distributions, files shown, install hooks | |
| ``` | |
| The tarball is read in memory: nothing is extracted, installed or run. The prompt is a chat | |
| message with the package as the state (package.json, the file list with sizes, then the | |
| install-hook targets, the entry points and the other code files by size, about 6,000 characters | |
| of code in all), the question and the lettered options with their definitions; the assistant | |
| turn is prefilled with `{"answer": "` and the decision is a softmax over the logits of the | |
| option letters, divided by the fitted temperature (in `CALIBRATION.json`). The origin question is | |
| asked only when p(malicious) ≥ 0.5. | |
| - **Speed.** One forward pass per question, up to ~3,600 tokens; about one question per second | |
| on a consumer GPU in WSL. | |
| - **Where it fits.** Score only what is new: versions that appear in a lockfile diff, keyed by | |
| name, version and integrity hash. A version whose hash differs from the public registry's | |
| deserves an alert of its own before any model reads it. | |
| ## Training | |
| - QLoRA: base frozen in 4-bit NF4, LoRA r 32, alpha 64, dropout 0.05 on the attention, | |
| Gated DeltaNet and MLP projections. | |
| - 7,000 draws planned (65% the verdict question, 35% the origin question with attack and | |
| hijacked balanced), batch 16, learning rate 1e-4 with cosine decay, prompts up to 3,584 | |
| tokens. Versions of popular npm packages (1,260 from 2023 to 2025, names kept out of the test | |
| set) are drawn twice as often as other benign versions. | |
| - The run was stopped at step 260 of 438; the adapter kept is step 250 (4,000 examples seen), | |
| the lowest NLL on the development panel (0.067). About 5 hours on one desktop GPU, | |
| evaluations included. | |
| - Every example is rendered with a random option order; each question has three phrasings. | |
| - Two runs, no paid compute. The first (6,000 draws, no popular packages) reached AUROC 0.946 | |
| but flagged 14% of benign packages with an install script; the second, this release, 3.3%. | |
| ## Data | |
| - **Malicious:** npm samples from DataDog's | |
| [malicious-software-packages-dataset](https://github.com/DataDog/malicious-software-packages-dataset) | |
| (Apache 2.0): `malicious_intent` = built to attack, `compromised_lib` = hijacked library. Read | |
| in memory from the password-protected zips; never extracted or run. | |
| - **Benign, three sets:** 15,000 random npm versions published 2024 to October 2026, drawn from | |
| the registry's replication feed (names that DataDog or OSV's malicious-packages list mark as | |
| malicious excluded); 1,797 random versions that ship an install script, so that "has a hook" | |
| cannot stand in for "malicious"; and 931 clean versions of the hijacked libraries, the last | |
| release before each attack. | |
| - Versions with no code that could run (an empty package, or code nothing loads) are dropped | |
| on both sides. | |
| - **Split by date:** packages first seen in 2026 are the test set; earlier ones are training, | |
| with two 5% slices (by package name) for development and calibration. A package never | |
| crosses splits; a hijacked library's clean version goes with its attack. Campaigns that | |
| publish the same code under many names are capped (one version per package and per code | |
| fingerprint in the test set). | |
| - The training data is not released: it contains live malware. | |
| - No outputs of commercial models or decision APIs were used. | |
| ## Limitations | |
| - **npm only.** PyPI is not covered yet. | |
| - **Hijacked libraries:** 40% caught on the test panel (see Results). | |
| - **It judges what it is shown:** about 6,000 characters of code, picked by install hooks, | |
| entry points and size. Code past that budget, inside binaries, or downloaded at run time is | |
| not seen. | |
| - **Label bias.** DataDog finds most of its samples with GuardDog, so the malicious side leans | |
| towards what GuardDog detects; malware that GuardDog misses is rare in training and test, and | |
| recall in the wild may be lower. Benign versions are random registry versions filtered | |
| against DataDog's and OSV's lists; a few may be malicious and unknown. | |
| - **One time slice.** The test set is January to October 2026; new techniques will drift. | |
| - **Beacons count as malicious:** bug bounty and dependency-confusion proofs of concept that | |
| report the host to a callback domain are labelled malicious, as in the dataset. | |
| - **Adversarial text** aimed at the model inside a package (comments claiming it is safe) was | |
| not tested. | |
| - It is a triage aid: p(malicious) < 0.5 does not mean safe, and every flag needs a person. | |
| ## Licence and attribution | |
| - The adapter and `jay_infer.py` are released under the Apache License 2.0. The base model is | |
| Qwen3.5-4B (Apache 2.0). | |
| - Malicious samples come from DataDog's malicious-software-packages-dataset (Apache 2.0). | |
| The GuardDog baseline is DataDog's [GuardDog](https://github.com/DataDog/guarddog) 3.2.0 (Apache 2.0). | |
| - npm is a trademark of npm, Inc. Jay is not affiliated with or endorsed by npm, GitHub or DataDog. | |