---
language:
- en
- code
license: apache-2.0
tags:
- code-oracle
- tyranid
- modernbert
- onnx
- int8
- code-verification
- neuro-symbolic
- static-analysis
- warhammer-40k
base_model: answerdotai/ModernBERT-base
pipeline_tag: text-classification
library_name: onnx
---
# 🧬 Tyranid-BERT (164M INT8)
"The Hive Mind of Code Verification. It doesn't converse. It doesn't negotiate. It devours corrupted ASTs and purges regressions in 20 milliseconds."
---
## ⚙️ The Doctrine: Synaptic Code Verification
Generative code review agents waste seconds buffering chat tokens, offering apologies, and hallucinating justifications for broken patches.
**Tyranid-BERT rejects generative babble.**
Acting as the neural synapse of the **[Code Oracle](https://github.com/wahyuzero/code-oracle)** neuro-symbolic engine, Tyranid-BERT takes the **Linearized Subgraph DSL** extracted from Tree-sitter ASTs and Tarjan Strongly Connected Components (SCC), and evaluates patch safety with ruthless, mathematically calibrated precision.
Where **[Qwen-Servitor](https://huggingface.co/wxsys/qwen-servitor)** acts as the brute cybernetic thrall devouring massive raw pull requests (262k tokens), **Tyranid-BERT** is the hyper-specialized synaptic assassin: a **164M bidirectional encoder quantized to INT8 (151 MB)** executing multi-task code diagnosis in **< 20 ms on commodity CPUs**.
---
## 🔬 Synaptic Architecture & Multi-Task Heads
```
[ LINEARIZED AST SUBGRAPH DSL (< 400 TOKENS) ]
│
▼
┌────────────────────────────────────────────────────────┐
│ ModernBERT-base 164M Backbone │
│ • Bidirectional Unpadding FlashAttention │
│ • Rotary Position Embeddings (RoPE) │
│ • 8,192 Token Native Receptive Field │
│ • Zero Autoregressive Overhead (Single Forward Pass) │
└────────────────────────────┬───────────────────────────┘
│
[ [CLS] Latent Vector ]
│
┌───────────────────────┼───────────────────────┐
▼ ▼ ▼
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ 1. RISK HEAD │ │ 2. TAXONOMY │ │ 3. UNCERTAINTY │
│ Continuous MSE │ │ 5-Class BCE │ │ Heteroscedastic │
│ Calibrated [0-1]│ │ Multi-Label │ │ Log-Variance │
│ T = 1.5967 │ │ Defect Vectors │ │ Confidence Gate │
└─────────────────┘ └─────────────────┘ └─────────────────┘
```
### 1. Multi-Task Output Telemetry
In a single forward inference pass (< 20 ms), Tyranid-BERT outputs:
| Telemetry Channel | Representation | Semantics |
| :--- | :--- | :--- |
| **Calibrated Risk Score** | Continuous Float `[0.000 .. 1.000]` | Calibrated regression score reflecting patch danger level. |
| **Multi-Label Risk Taxonomy** | 5 Independent Sigmoid Probabilities | • `BreakingPublicAPI`
• `SecuritySurface`
• `ConcurrencyHazard`
• `PerformanceRegression`
• `SilentLogicDrift` |
| **Epistemic Uncertainty** | Bounded Heteroscedastic Variance | Model self-awareness: flags ambiguous mutations outside training distribution. |
---
## ⚡ Runtime Formats & Weights
This repository provides three production-grade model variants:
1. **`model_int8.onnx`** (151 MB — **Recommended**):
* Dynamic INT8 quantized ONNX graph (targeting Linear/MatMul layers).
* **Zero-PyTorch footprint**: runs with standalone `onnxruntime` (~50 MB wheel).
* Sub-20ms inference on modern CPU architectures.
2. **`model.onnx`** (598 MB):
* Full FP32 ONNX graph with dynamic axes (`batch_size`, `sequence_length`).
3. **`model.safetensors`** (598 MB):
* Pure PyTorch weights for fine-tuning or Python research workflows.
---
## 🛠️ Quickstart
### 1. Installation
```bash
pip install onnxruntime transformers
```
### 2. Fast ONNX Runtime Inference (< 20ms)
```python
import numpy as np
import onnxruntime as ort
from transformers import AutoTokenizer
MODEL_ID = "wxsys/tyranid-bert"
# Load tokenizer and INT8 ONNX session
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
session = ort.InferenceSession("model_int8.onnx", providers=["CPUExecutionProvider"])
subgraph_dsl = """[DIFF_TARGET] app/auth.py::verify_token (MODIFIED)
[METADATA] File: app/auth.py | OldLines: [10..15] | NewLines: [10..18]
[NODES]
N0: app/auth.py::verify_token [def verify_token(token: str)] (SEED, MODIFIED)
N1: app/routes.py::login_handler [def login_handler()]
[EDGES]
N1 -> N0 (CALLS)
[GATE]
STATUS: APPROVED (conf: 0.98)
CYCLES: 0
VIOLATIONS: NONE"""
# Tokenize (< 400 tokens)
inputs = tokenizer(subgraph_dsl, max_length=512, truncation=True, padding=True, return_tensors="np")
ort_inputs = {
"input_ids": inputs["input_ids"].astype(np.int64),
"attention_mask": inputs["attention_mask"].astype(np.int64),
}
# Single forward pass
risk_score, taxonomy_logits, uncertainty = session.run(None, ort_inputs)
print(f"Risk Score: {float(risk_score[0]):.4f}")
print(f"Taxonomy Vector: {taxonomy_logits[0]}")
```
### 3. Integrated Use in Code Oracle CLI
Tyranid-BERT is the official neural verifier for **Code Oracle**:
```bash
# Verify a patch with sub-20ms INT8 neural decision head
code-oracle verify src/service.ts --patch patch.diff -w .
```
---
## 🧬 Training Dataset: Golden Hybrid v3
Tyranid-BERT was fine-tuned on the **Golden Hybrid v3** dataset (~4,900 samples):
* **Real Git Reverts & Hotfixes**: Mined from production repositories across Go, Python, TypeScript, and Rust.
* **Surviving Mutants**: Hard negatives that strictly passed Stage 1 (AST Syntax) and Stage 2 (Tarjan SCC Cycles).
* **Balanced Stratification**: Exact 50% PASS / 50% REJECT distribution with zero data leakage.
---
## 📜 Citation & License
* **License:** Apache 2.0
* **Base Architecture:** [answerdotai/ModernBERT-base](https://huggingface.co/answerdotai/ModernBERT-base)
* **Author / Maintainer:** Wahyu Zero ([@wahyuzero](https://github.com/wahyuzero) / `wxsys`)
* **Project:** [Code Oracle](https://github.com/wahyuzero/code-oracle)