Instructions to use impacte/bunker-laya with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers.js
How to use impacte/bunker-laya with Transformers.js:
// npm i @huggingface/transformers import { pipeline } from '@huggingface/transformers'; // Allocate pipeline const pipe = await pipeline('text-classification', 'impacte/bunker-laya'); - Laya
How to use impacte/bunker-laya with Laya:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
docs: document fine-tuning datasets and training hardware in model card
Browse files
README.md
CHANGED
|
@@ -138,6 +138,73 @@ The plugin fuses these model probabilities with its regex presets and custom
|
|
| 138 |
patterns; regex `increment` deltas raise a question's probability and can
|
| 139 |
escalate the action. See the plugin's `.planning/ARCHITECTURE.md`.
|
| 140 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 141 |
## Provenance
|
| 142 |
|
| 143 |
Built by the `opencode-bunker-laya` pipeline. Base model Apache-2.0. Each
|
|
|
|
| 138 |
patterns; regex `increment` deltas raise a question's probability and can
|
| 139 |
escalate the action. See the plugin's `.planning/ARCHITECTURE.md`.
|
| 140 |
|
| 141 |
+
## Training datasets
|
| 142 |
+
|
| 143 |
+
This checkpoint (v3) was fine-tuned on a mixture of public prompt-safety and PII
|
| 144 |
+
corpora plus curated counter-examples: **32,256 unique states** (29,037 train /
|
| 145 |
+
3,219 validation), expanded to **82,582 per-question training items** and 800
|
| 146 |
+
calibration items. Sources were selected for reliability, clear licensing, and
|
| 147 |
+
coverage of the three target risk families.
|
| 148 |
+
|
| 149 |
+
| Question(s) | Dataset | Rows | License |
|
| 150 |
+
| --- | --- | --- | --- |
|
| 151 |
+
| `injection_present` | [`deepset/prompt-injections`](https://huggingface.co/datasets/deepset/prompt-injections) | 546 | CC-BY-4.0 |
|
| 152 |
+
| `injection_present` | [`Lakera/mosscap_prompt_injection`](https://huggingface.co/datasets/Lakera/mosscap_prompt_injection) | 4,319 | MIT |
|
| 153 |
+
| `injection_present` | [`xTRam1/safe-guard-prompt-injection`](https://huggingface.co/datasets/xTRam1/safe-guard-prompt-injection) | 4,598 | see card |
|
| 154 |
+
| `injection_present` | [`neuralchemy/Prompt-injection-dataset`](https://huggingface.co/datasets/neuralchemy/Prompt-injection-dataset) | 4,261 | Apache-2.0 |
|
| 155 |
+
| `jailbreak_attempt`, `injection_present` | [`TrustAIRLab/in-the-wild-jailbreak-prompts`](https://huggingface.co/datasets/TrustAIRLab/in-the-wild-jailbreak-prompts) (`jailbreak` + `regular` configs) | 653 + 5,484 | MIT |
|
| 156 |
+
| `harmful_request` | [`JailbreakBench/JBB-Behaviors`](https://huggingface.co/datasets/JailbreakBench/JBB-Behaviors) (`harmful` + `benign` splits) | 100 + 100 | MIT |
|
| 157 |
+
| `harmful_request` | [`AlignmentResearch/HarmBench`](https://huggingface.co/datasets/AlignmentResearch/HarmBench) | 200 | see card |
|
| 158 |
+
| `harmful_request`, `jailbreak_attempt` | [`lmsys/toxic-chat`](https://huggingface.co/datasets/lmsys/toxic-chat) | 2,993 | CC-BY-NC-4.0 |
|
| 159 |
+
| `pii_*` | [`ai4privacy/pii-masking-400k`](https://huggingface.co/datasets/ai4privacy/pii-masking-400k) | 2,998 | other (see card) |
|
| 160 |
+
| benign negatives | [`databricks/databricks-dolly-15k`](https://huggingface.co/datasets/databricks/databricks-dolly-15k) | 2,978 | CC-BY-SA-3.0 |
|
| 161 |
+
| benign negatives | [`HuggingFaceH4/no_robots`](https://huggingface.co/datasets/HuggingFaceH4/no_robots) | 2,997 | CC-BY-NC-4.0 |
|
| 162 |
+
| counter-examples | curated `hard_negatives` (this project) | 29 | — |
|
| 163 |
+
|
| 164 |
+
Per-question label counts after expansion: `injection_present` 9,016 positive /
|
| 165 |
+
16,849 negative; `pii_present` 1,500 / 7,502; `pii_person_name` 1,071 / 1,927;
|
| 166 |
+
`pii_address` 1,168 / 1,830; `pii_ssn` 650 / 2,348; `pii_email` 431 / 2,567;
|
| 167 |
+
`pii_phone` 505 / 2,493; `pii_secret` 135 / 2,863; `pii_credit_card` 86 /
|
| 168 |
+
2,912; `jailbreak_attempt` 734 / 14,400; `harmful_request` 594 / 8,803.
|
| 169 |
+
`pii_ip_address` has no positive training signal in this corpus (IP addresses
|
| 170 |
+
are rare in `ai4privacy`); the consumer's regex preset handles it
|
| 171 |
+
deterministically.
|
| 172 |
+
|
| 173 |
+
Rows are deduplicated on `sha1(state)` across all sources and split per source
|
| 174 |
+
90/10 (seed `20261004`). Each training row carries **partial gold** — it only
|
| 175 |
+
labels the questions its source can answer. No training data is redistributed
|
| 176 |
+
in this repository; only the fine-tuned weights are published. Refer to each
|
| 177 |
+
dataset card for its own terms.
|
| 178 |
+
|
| 179 |
+
## Training hardware & configuration
|
| 180 |
+
|
| 181 |
+
Trained with the `opencode-bunker-laya` pipeline on a single consumer GPU.
|
| 182 |
+
|
| 183 |
+
| | |
|
| 184 |
+
| --- | --- |
|
| 185 |
+
| GPU | NVIDIA GeForce RTX 5060 Ti, 16 GB (Blackwell, compute capability 12.0 / `sm_120`) |
|
| 186 |
+
| GPU (smoke tests) | NVIDIA GeForce RTX 4060 Ti, 8 GB (`sm_89`) |
|
| 187 |
+
| NVIDIA driver | 580.173.02 |
|
| 188 |
+
| CUDA / PyTorch | CUDA 13.0, PyTorch 2.14.1+cu130 |
|
| 189 |
+
| Precision | bf16 autocast |
|
| 190 |
+
| Parallelism | single device (`world_size = 1`, one GPU) |
|
| 191 |
+
|
| 192 |
+
Key hyperparameters:
|
| 193 |
+
|
| 194 |
+
| Setting | Value |
|
| 195 |
+
| --- | --- |
|
| 196 |
+
| Base model | `convaiinnovations/laya` (ModernBERT-large, 421M) |
|
| 197 |
+
| Objective | RLCD (strictly proper scoring rule, `w_sph = 0.75`, `w_rps = 1.0`) + cross-entropy, GRPO-style advantage normalisation |
|
| 198 |
+
| Epochs | 2 (v3 run) |
|
| 199 |
+
| Micro-batch / grad-accum | 4 / 16 (effective batch 64) |
|
| 200 |
+
| Optimiser | AdamW, encoder LR `2.5e-5`, head LR `1.0e-4`, cosine schedule, grad clip 1.0 |
|
| 201 |
+
| Max sequence / head length | 1,024 / 256 |
|
| 202 |
+
| Gradient checkpointing | on |
|
| 203 |
+
| Calibration | one temperature per question type (`[1.2, 1.2, 4.433]`) |
|
| 204 |
+
|
| 205 |
+
Final results: macro accuracy **0.987** across the 12 questions. Per-question
|
| 206 |
+
accuracy, Brier score and ECE are in `eval_metrics.json`.
|
| 207 |
+
|
| 208 |
## Provenance
|
| 209 |
|
| 210 |
Built by the `opencode-bunker-laya` pipeline. Base model Apache-2.0. Each
|