| ---
|
| license: apache-2.0
|
| language:
|
| - en
|
| - zh
|
| library_name: pytorch
|
| tags:
|
| - network-security
|
| - traffic-classification
|
| - encrypted-traffic
|
| - open-set-recognition
|
| - mamba
|
| - gdn
|
| - pytorch
|
| metrics:
|
| - accuracy
|
| - f1
|
| - auroc
|
| ---
|
|
|
| # IntraSpanNet
|
|
|
| **English** | [简体中文](https://github.com/lupingQAQ/IntraSpanNet/blob/main/README.zh-CN.md)
|
|
|
| IntraSpanNet names what produced a flow from its first five packets: a security appliance, an endpoint security agent, or a piece of common software. It reports a product label, a type, a tier, and an open-set distance. Flows that do not match a known product are rejected as unknown.
|
|
|
|
|
| **DOI:** [10.5281/zenodo.23643146](https://zenodo.org/records/23643146)
|
|
|
| **Code and training guide:** [github.com/lupingQAQ/IntraSpanNet](https://github.com/lupingQAQ/IntraSpanNet) — [docs/TRAINING.md](https://github.com/lupingQAQ/IntraSpanNet/blob/main/docs/TRAINING.md)
|
|
|
| **Install:** `pip install intraspannet` runs on CPU; `pip install "intraspannet[gpu]"` adds the Triton kernel. Same weights either way. |
|
|
| ## Model details
|
|
|
| - Architecture: ByteEmbed, four Gated DeltaNet blocks, four task heads.
|
| - Input: 1600 bytes per flow (5 packets x 320 bytes, IPv4 addresses masked).
|
| - Parameters: 2.86M per arm; 11.44M for the four-arm ensemble.
|
| - Output classes: 18 products, 10 types, 3 tiers, 11 malicious classes.
|
| - Open-set method: three sub-prototypes per class; cosine distance; threshold 0.1041.
|
| - Ensemble: classifier arms A-S0, A-LDA, B-LDA, B-S0; gate arms A-LDA and B-LDA. |
| - Arm naming: the letter is the pre-training run (A or B), the suffix is the training variant (`s0` standard multi-task objective, `lda` logit adjustment by class prior, `hn` hard-negative weighting). Checkpoint files carry the same labels in lower case, for example `stage3_a_s0.pth` is arm A-S0.
|
| - License: Apache-2.0.
|
|
|
| ## Intended use
|
|
|
| - Asset discovery: identify security agents, appliances, and common services on hosts and internal networks.
|
| - Shadow IT and shadow security review: find products that are present but not in the asset inventory.
|
| - Research: byte-level traffic classification and open-set recognition.
|
|
|
| The model is intended for authorized monitoring on networks you own or administer. It is not a replacement for an IDS, EDR, or vulnerability scanner.
|
|
|
| ## Out-of-scope use
|
|
|
| - Identifying users, content, or application payloads.
|
| - Monitoring networks without authorization.
|
| - Mobile and IoT device classification. The model was not trained for those domains.
|
| - Real-time blocking or automated enforcement without human review.
|
|
|
| ## Training data
|
|
|
| - 40,283 labeled flows from lab captures of 18 products and services.
|
| - Public benchmark training splits for pretraining and the malicious head.
|
| - Contrastive pretraining uses 33,184 training-only flows. Validation and holdout flows are excluded.
|
| - No customer traffic is included in the released weights or repository.
|
|
|
| The product labels are:
|
|
|
| `windows_defender`, `clamav_freshclam_win`, `clamav_freshclam`, `velociraptor`, `velociraptor_win`, `osquery`, `wazuh_agent`, `wireguard`, `openvpn`, `stunnel`, `squid_proxy`, `ntopng`, `guacamole_mgmt`, `waf_modsecurity`, `suricata_update`, `web_http`, `ssh`, `ftp`.
|
|
|
| ## Evaluation
|
|
|
| | Metric | Test (503 flows) | All validation (1006 flows) |
|
| |---|---:|---:|
|
| | Product accuracy | 0.8787 | 0.8817 |
|
| | Tier accuracy | 0.9245 | 0.9334 |
|
| | Macro-F1 (supported classes) | 0.7621 | 0.8240 |
|
| | Gate AUROC | 0.9784 | 0.9820 (dev) |
|
|
|
| Additional results:
|
|
|
| - Unknown rejection: 100% on 18 unknown-product bins.
|
| - Acceptance: 8/10 products without timing rescue; 9/10 with the internal timing rule, which is not included in v0.1.0.
|
| - Throughput: 253 flows/s for the ensemble, 1,017 flows/s for one arm, on an RTX 4090.
|
|
|
| The test split is a flow-level half of the validation set, not a session-level holdout. Each class has one held-out session. See [docs/RESULTS.md](https://github.com/lupingQAQ/IntraSpanNet/blob/main/docs/RESULTS.md) for the full protocol.
|
|
|
| ## Limitations
|
|
|
| - `stunnel` has zero recall in the acceptance demo (0/30).
|
| - Flow-level test split; session-level generalization is not measured.
|
| - CPU inference uses a plain PyTorch port of the GDN block: correct, but about two seconds per flow per arm, so it is for inspection rather than a live probe. `pip install "intraspannet[gpu]"` enables the Triton path.
|
| - The model was trained on lab traffic. Production performance with different software versions, proxies, or NAT is unknown.
|
| - The malicious head is a secondary output and is not calibrated for production alerting.
|
|
|
| ## Ethical and legal considerations
|
|
|
| Traffic classification can reveal sensitive information about a network. Use the model only on networks you own or have written authorization to monitor. Do not upload captures to public demos. Follow local law and organizational policy.
|
|
|
| The model outputs a probability and a distance, not a fact. Treat low-confidence and unknown results as prompts for human review.
|
|
|
| ## Further training
|
|
|
| The supervised and reinforcement-learning objectives are part of the pip package, so these weights are a starting point rather than a fixed artifact.
|
|
|
| | Item | What it does |
|
| |---|---|
|
| | `SFTConfig` | Loss weights for the multi-task supervised stage |
|
| | `compute_sft_loss` | Product, type, tier, malicious and contrastive terms |
|
| | `RLConfig` | Reward weights for the REINFORCE stage |
|
| | `compute_rl_loss` | Correct product, correct type, honest rejection |
|
| | `Trainer` | One optimizer step per call |
|
| | `load_backbone` | Warm start across a changed label set, resizing the product head |
|
|
|
| ```python
|
| from intraspannet.model import DualHeadClassifier
|
| from intraspannet.train import SFTConfig, Trainer, load_backbone
|
|
|
| model = DualHeadClassifier(num_software=18)
|
| model, report = load_backbone("stage3_a_s0.pth", model)
|
| trainer = Trainer(model)
|
| stats = trainer.step_sft(batch, SFTConfig(lda=1.0), class_prior=prior)
|
| ```
|
|
|
| Two runnable examples live in the repository: `examples/train_sft.py` and `examples/train_rl.py`, both driven by a JSONL manifest of captures and product labels. The guide covers the batch contract, prototype and threshold recalibration, and the checkpoint format the CLI reloads:
|
|
|
| - [docs/TRAINING.md](https://github.com/lupingQAQ/IntraSpanNet/blob/main/docs/TRAINING.md) (English)
|
| - [docs/TRAINING.zh-CN.md](https://github.com/lupingQAQ/IntraSpanNet/blob/main/docs/TRAINING.zh-CN.md) (Chinese)
|
|
|
| Stage 1, the contrastive pretraining on lab captures, is not included because it depends on a data layout that cannot be redistributed.
|
|
|
| ## Files
|
|
|
| - `stage3_*.pth`: six classifier arms.
|
| - `prototypes_*_k3.pth`: prototype matrices.
|
| - `stage1_clean_*.pth`: contrastive encoders.
|
| - `student_d1.pth`: distilled student, experimental.
|
| - `config.json`: ensemble configuration and reference metrics.
|
| - `labels.json`: label table.
|
| - `SHA256SUMS`: checksums.
|
|
|
| ## Citation
|
|
|
| ```bibtex
|
| @software{luping2026intraspannet,
|
| author = {luping},
|
| title = {IntraSpanNet: Identifying security appliances, endpoint agents and common software from intranet flows},
|
| year = {2026},
|
| doi = {10.5281/zenodo.23643146},
|
| url = {https://github.com/lupingQAQ/IntraSpanNet},
|
| note = {Model: https://huggingface.co/luping123/IntraSpanNet}
|
| }
|
| ```
|
|
|
