| --- |
| language: |
| - en |
| - code |
| license: apache-2.0 |
| tags: |
| - code-oracle |
| - tyranid |
| - modernbert |
| - onnx |
| - int8 |
| - code-verification |
| - neuro-symbolic |
| - static-analysis |
| - warhammer-40k |
| base_model: answerdotai/ModernBERT-base |
| pipeline_tag: text-classification |
| library_name: onnx |
| --- |
| |
| # 🧬 Tyranid-BERT (164M INT8) |
|
|
| <p align="center"> |
| <strong>"The Hive Mind of Code Verification. It doesn't converse. It doesn't negotiate. It devours corrupted ASTs and purges regressions in 20 milliseconds."</strong> |
| </p> |
|
|
| <p align="center"> |
| <a href="https://github.com/wahyuzero/code-oracle"><img src="https://img.shields.io/badge/Engine-Code_Oracle-purple.svg" alt="Code Oracle"></a> |
| <a href="https://huggingface.co/wxsys/qwen-servitor"><img src="https://img.shields.io/badge/Sister_Unit-Qwen--Servitor-darkred.svg" alt="Sister Model"></a> |
| <img src="https://img.shields.io/badge/Base_Spine-ModernBERT--base_164M-blue.svg" alt="Base Model"> |
| <img src="https://img.shields.io/badge/Quantization-Dynamic_INT8-success.svg" alt="INT8"> |
| <img src="https://img.shields.io/badge/Latency-%3C20ms_CPU-brightgreen.svg" alt="Latency"> |
| <img src="https://img.shields.io/badge/Generative_Babble-0%25-black.svg" alt="Zero Generation"> |
| <img src="https://img.shields.io/badge/License-Apache_2.0-red.svg" alt="License"> |
| </p> |
|
|
| --- |
|
|
| ## ⚙️ The Doctrine: Synaptic Code Verification |
|
|
| Generative code review agents waste seconds buffering chat tokens, offering apologies, and hallucinating justifications for broken patches. |
|
|
| **Tyranid-BERT rejects generative babble.** |
|
|
| Acting as the neural synapse of the **[Code Oracle](https://github.com/wahyuzero/code-oracle)** neuro-symbolic engine, Tyranid-BERT takes the **Linearized Subgraph DSL** extracted from Tree-sitter ASTs and Tarjan Strongly Connected Components (SCC), and evaluates patch safety with ruthless, mathematically calibrated precision. |
|
|
| Where **[Qwen-Servitor](https://huggingface.co/wxsys/qwen-servitor)** acts as the brute cybernetic thrall devouring massive raw pull requests (262k tokens), **Tyranid-BERT** is the hyper-specialized synaptic assassin: a **164M bidirectional encoder quantized to INT8 (151 MB)** executing multi-task code diagnosis in **< 20 ms on commodity CPUs**. |
|
|
| --- |
|
|
| ## 🔬 Synaptic Architecture & Multi-Task Heads |
|
|
| ``` |
| [ LINEARIZED AST SUBGRAPH DSL (< 400 TOKENS) ] |
| │ |
| ▼ |
| ┌────────────────────────────────────────────────────────┐ |
| │ ModernBERT-base 164M Backbone │ |
| │ • Bidirectional Unpadding FlashAttention │ |
| │ • Rotary Position Embeddings (RoPE) │ |
| │ • 8,192 Token Native Receptive Field │ |
| │ • Zero Autoregressive Overhead (Single Forward Pass) │ |
| └────────────────────────────┬───────────────────────────┘ |
| │ |
| [ [CLS] Latent Vector ] |
| │ |
| ┌───────────────────────┼───────────────────────┐ |
| ▼ ▼ ▼ |
| ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ |
| │ 1. RISK HEAD │ │ 2. TAXONOMY │ │ 3. UNCERTAINTY │ |
| │ Continuous MSE │ │ 5-Class BCE │ │ Heteroscedastic │ |
| │ Calibrated [0-1]│ │ Multi-Label │ │ Log-Variance │ |
| │ T = 1.5967 │ │ Defect Vectors │ │ Confidence Gate │ |
| └─────────────────┘ └─────────────────┘ └─────────────────┘ |
| ``` |
|
|
| ### 1. Multi-Task Output Telemetry |
| In a single forward inference pass (< 20 ms), Tyranid-BERT outputs: |
|
|
| | Telemetry Channel | Representation | Semantics | |
| | :--- | :--- | :--- | |
| | **Calibrated Risk Score** | Continuous Float `[0.000 .. 1.000]` | Calibrated regression score reflecting patch danger level. | |
| | **Multi-Label Risk Taxonomy** | 5 Independent Sigmoid Probabilities | • `BreakingPublicAPI`<br>• `SecuritySurface`<br>• `ConcurrencyHazard`<br>• `PerformanceRegression`<br>• `SilentLogicDrift` | |
| | **Epistemic Uncertainty** | Bounded Heteroscedastic Variance | Model self-awareness: flags ambiguous mutations outside training distribution. | |
|
|
| --- |
|
|
| ## ⚡ Runtime Formats & Weights |
|
|
| This repository provides three production-grade model variants: |
|
|
| 1. **`model_int8.onnx`** (151 MB — **Recommended**): |
| * Dynamic INT8 quantized ONNX graph (targeting Linear/MatMul layers). |
| * **Zero-PyTorch footprint**: runs with standalone `onnxruntime` (~50 MB wheel). |
| * Sub-20ms inference on modern CPU architectures. |
| 2. **`model.onnx`** (598 MB): |
| * Full FP32 ONNX graph with dynamic axes (`batch_size`, `sequence_length`). |
| 3. **`model.safetensors`** (598 MB): |
| * Pure PyTorch weights for fine-tuning or Python research workflows. |
| |
| --- |
| |
| ## 🛠️ Quickstart |
| |
| ### 1. Installation |
| |
| ```bash |
| pip install onnxruntime transformers |
| ``` |
| |
| ### 2. Fast ONNX Runtime Inference (< 20ms) |
| |
| ```python |
| import numpy as np |
| import onnxruntime as ort |
| from transformers import AutoTokenizer |
| |
| MODEL_ID = "wxsys/tyranid-bert" |
| |
| # Load tokenizer and INT8 ONNX session |
| tokenizer = AutoTokenizer.from_pretrained(MODEL_ID) |
| session = ort.InferenceSession("model_int8.onnx", providers=["CPUExecutionProvider"]) |
| |
| subgraph_dsl = """[DIFF_TARGET] app/auth.py::verify_token (MODIFIED) |
| [METADATA] File: app/auth.py | OldLines: [10..15] | NewLines: [10..18] |
| [NODES] |
| N0: app/auth.py::verify_token [def verify_token(token: str)] (SEED, MODIFIED) |
| N1: app/routes.py::login_handler [def login_handler()] |
| [EDGES] |
| N1 -> N0 (CALLS) |
| [GATE] |
| STATUS: APPROVED (conf: 0.98) |
| CYCLES: 0 |
| VIOLATIONS: NONE""" |
| |
| # Tokenize (< 400 tokens) |
| inputs = tokenizer(subgraph_dsl, max_length=512, truncation=True, padding=True, return_tensors="np") |
| ort_inputs = { |
| "input_ids": inputs["input_ids"].astype(np.int64), |
| "attention_mask": inputs["attention_mask"].astype(np.int64), |
| } |
| |
| # Single forward pass |
| risk_score, taxonomy_logits, uncertainty = session.run(None, ort_inputs) |
| |
| print(f"Risk Score: {float(risk_score[0]):.4f}") |
| print(f"Taxonomy Vector: {taxonomy_logits[0]}") |
| ``` |
| |
| ### 3. Integrated Use in Code Oracle CLI |
| |
| Tyranid-BERT is the official neural verifier for **Code Oracle**: |
| |
| ```bash |
| # Verify a patch with sub-20ms INT8 neural decision head |
| code-oracle verify src/service.ts --patch patch.diff -w . |
| ``` |
| |
| --- |
| |
| ## 🧬 Training Dataset: Golden Hybrid v3 |
| |
| Tyranid-BERT was fine-tuned on the **Golden Hybrid v3** dataset (~4,900 samples): |
| * **Real Git Reverts & Hotfixes**: Mined from production repositories across Go, Python, TypeScript, and Rust. |
| * **Surviving Mutants**: Hard negatives that strictly passed Stage 1 (AST Syntax) and Stage 2 (Tarjan SCC Cycles). |
| * **Balanced Stratification**: Exact 50% PASS / 50% REJECT distribution with zero data leakage. |
| |
| --- |
| |
| ## 📜 Citation & License |
| |
| * **License:** Apache 2.0 |
| * **Base Architecture:** [answerdotai/ModernBERT-base](https://huggingface.co/answerdotai/ModernBERT-base) |
| * **Author / Maintainer:** Wahyu Zero ([@wahyuzero](https://github.com/wahyuzero) / `wxsys`) |
| * **Project:** [Code Oracle](https://github.com/wahyuzero/code-oracle) |
| |
