| --- |
| license: apache-2.0 |
| language: |
| - en |
| base_model: |
| - Qwen/Qwen3.5-0.8B |
| pipeline_tag: text-classification |
| library_name: peft |
| tags: |
| - security |
| - soc |
| - decision-model |
| - typed-decisions |
| - long-context |
| - tool-calling |
| - lora |
| - unsloth |
| --- |
| <div align="center"> |
| <img src="https://github.com/bloomifycafe/blossomsAI/blob/main/assets/logo.png?raw=true" alt="Logo"/> |
| </div> |
| </br> |
| <div align="center"> |
|
|
| # 🌟 Sepal-0.8B |
|
|
| </div> |
|
|
| ### A typed decision model for security operations (SOC) work |
|
|
| ## NOTE |
| - This is the full training run of Sepal-0.8B. It replaces the earlier test run on this repo (tag `full08b-test-run`, still available). |
| - The scores below are from our own development panels. Calibration on the held-out report half (ECE 0.031) is just above our 0.030 target. |
| - Larger sizes (2B, 4B, 9B) on the same recipe will follow. |
|
|
| ## 📋 Overview |
|
|
| - Sepal answers **typed questions** about a given state (an alert, a log window, a policy, a document) in one pass: yes/no (`noul`), choice among up to 255 options, and score levels. Every answer comes with calibrated probabilities, not free text. |
| - It is specialised for SOC decisions: alert verdicts, rule and policy matching, ATT&CK mapping, vulnerability priority, injection detection in attacker-controlled fields, response gating, and "insufficient evidence" abstention, with general decision data mixed in to keep broad skills. |
| - New in this version: **long log windows** (states up to about 16k tokens) with many custom yes/no criteria, and **tool and intent selection**. |
| - It is an adapter (LoRA rank 64 plus a decision head) on the post-trained Qwen3.5-0.8B. |
|
|
| ## 🔧 Method |
|
|
| - Trained with [Unsloth](https://github.com/unslothai/unsloth)'s decision trainer (Clef head, one prompt per record) using LoRA rank 64, alpha 128, on every linear layer of the language model; lr 1e-4, cosine with 10% warm-up, one epoch, 64 records a step. |
| - The recipe combines the three levers that won our screening round: the post-trained Qwen3.5-0.8B as the starting point, a tool and intent family, and long log windows up to true 16k states. |
| - Training corpus `full-v15`: 691,705 records, 449.4M tokens, 149 sources (security layer about 41% of tokens), including generated SOC data, long log windows with custom criteria, tool-selection data built from training splits only, public security datasets, and general decision datasets. Every training record was screened against our benchmark: zero exact or near copies, and no shared instances. |
| - 10,808 steps, 11.5 GPU-hours on one RTX PRO 6000 (96 GB), training inputs up to 24,576 tokens. |
| - One temperature per question type and option-count bucket, fitted on held-out datasets, is stored in `sepal.json`. |
|
|
| ## 📊 Scores |
|
|
| Development panels of our own benchmark (SOCDecide 0.7.0), our long-log and tool panels, and held-out general datasets. Test partitions were not read. |
|
|
| | Panel | This version | Earlier test run | |
| |---|---|---| |
| | Security decisions, macro skill (SEC-dev) | **0.822** | 0.785 | |
| | Security coverage at 5% error | **0.846** | 0.801 | |
| | Security coverage at 1% error | 0.558 | 0.589 | |
| | Long log windows with custom criteria, skill (LONG-LOG) | **0.590** | 0.191 | |
| | Long logs at 16k tokens, skill | **0.485** | not measured | |
| | "Insufficient evidence" detection on long logs, F1 | **0.92** | not measured | |
| | Tool and intent selection, macro skill (TOOL) | **0.926** | 0.545 | |
| | Unseen security sources, skill | **0.337** | 0.305 | |
| | General decisions, skill (GEN-dev) | 0.338 | 0.342 | |
| | Calibration error (ECE, held-out report half) | 0.031 | 0.019 | |
|
|
| Public [Decision Index](https://github.com/sileod/decision-index) 0.3 (our own run, HLE not included): coming soon. The earlier test run scored 16.63. |
|
|
| ## 📁 Files |
|
|
| - `adapter_model.safetensors`, `adapter_config.json`: the LoRA adapter for `Qwen/Qwen3.5-0.8B` (revision `2fc06364715b967f1860aea9cf38778875588b17`). |
| - `joint_head.safetensors`, `joint_head_config.json`, `unsloth_decision_config.json`: the decision head read by Unsloth's decision trainer. |
| - `sepal.json`: training settings, versions and the fitted temperatures. |
| - Tokenizer files from the base model. |
|
|
| ## 💫 Quantization |
|
|
| Coming Soon! |
|
|
| ## 🤝 Contributors |
|
|
| Developed with ❤️ by [BlossomAI](https://github.com/BlossomAI) |
|
|
| <p align="left"> |
| <img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/made with unsloth.png" width="200" /> |
| </p> |
|
|
| --- |
| <div align="center"> |
| <sub>Star ⭐️ this repo if you find it valuable!</sub> |
| </div> |
|
|
