Magnus — SYNAXIM .symb Format (INT4)

Powered by SYNAXIM — Symbiotic Native Axim Inference Machine
Framework-Free LLM Inference | Attention ≡ Memory | O(1) State

⚡ v0.2.1 — Compartment Gate Release. CPU + GPU unified inference. 103.12 tok/s on NVIDIA CUDA, 18.35 tok/s on Apple M3 Metal. Symbiotic Compartment Gate: head-aware load balancer with dynamic α and Δt.

---

🔥 Verified Benchmarks — TinyLlama 1.1B (INT4, 30 tokens)

Device	Backend	tok/s	Avg Latency	Speedup
NVIDIA CUDA	PyTorch GPU + Compartment Gate	103.12	9.7 ms	6.4×
Apple M3 MPS	PyTorch Metal	18.35	54.5 ms	1.1×
CPU (GPU box)	NumPy + BLAS	16.05	62.3 ms	1×
Apple M3 CPU	NumPy + Accelerate	12.49	80.0 ms	baseline
SYNAXIM × AXIM	AXIM NEON + BLAS	12.88	77.6 ms	baseline

Model: TinyLlama-1.1B-Chat-v1.0 converted to .symb INT4 (584 MB).
NVIDIA GPU: marimo cloud (9 GB VRAM). Apple M3: 16 GB MacBook Pro.
All results with Symbiotic Compartment Gate (head-aware load balancer).
Single-token autoregressive generation (not batch/prefill).

---

About Magnus

Magnus is a custom fine-tuned Mistral-7B-Instruct-v0.3 with axim-alignment — stripped to text-only for maximum inference speed through SYNAXIM's O(1) Symbiotic State Engine.

• No KV-Cache — O(1) fixed-size memory matrix (DualFluidM)
• No PyTorch required — pure NumPy core (+ PyTorch for GPU acceleration)
• Proprietary .symb format — INT4 bitpacked, 3.8× compression
• Dual Engine — same code runs on CPU, CUDA, and MPS
• Compartment Gate — head-aware load balancer, dynamic α and Δt
• Multimodal — text + image understanding (vision encoder + M-projector)

---

Quick Start

Option 1: TinyLlama (small, fast, test anywhere)

pip install git+https://github.com/GRRN-MAKER/SYNAXIM.git
pip install torch transformers safetensors

from grrn_inference import SymbioticConverter

# Convert TinyLlama to .symb INT4 (downloads from HuggingFace)
SymbioticConverter().convert(
    source="TinyLlama/TinyLlama-1.1B-Chat-v1.0",
    output_dir="./tinyllama-symb",
    quantize="int4", group_size=128)

# Run on GPU (auto-detects CUDA or MPS)
from grrn_inference.dual_engine import DualEngine

engine = DualEngine("./tinyllama-symb", device="auto")
logits = engine.forward(1)  # → 97 tok/s on CUDA, 18 tok/s on MPS

Option 2: Magnus 7B (full model)

pip install git+https://github.com/GRRN-MAKER/SYNAXIM.git
huggingface-cli download GRRNMAKE/Magnus-SYMB --local-dir ./magnus-symb

from grrn_inference import GRRNModel

model = GRRNModel.from_pretrained("./magnus-symb", device="cuda")  # or "cpu", "mps", "dual"
result = model.generate("The meaning of life is", max_tokens=50, temperature=0.7)
print(result.text)

Option 3: Benchmark in the browser (Marimo)

pip install marimo
git clone https://github.com/GRRN-MAKER/SYNAXIM.git && cd SYNAXIM
marimo run marimo_cuda_bench.py

The marimo notebook clones the repo, converts the model, and benchmarks CPU vs GPU — all in the browser.

---

Benchmark CLI

# Auto-detect best device
python3 bench_synaxim.py ./tinyllama-symb --device dual

# Force specific device
python3 bench_synaxim.py ./tinyllama-symb --device cpu
python3 bench_synaxim.py ./tinyllama-symb --device cuda
python3 bench_synaxim.py ./tinyllama-symb --device mps

---

Model Details

Property	Value
Name	Magnus
Modality	Text + Vision (v0.2.1)
Base Model	mistralai/Mistral-7B-Instruct-v0.3
Parameters	~7.24B
Hidden Size	4096
Layers	32
Attention	32Q / 8KV (GQA 4:1)
Vocabulary	32,768
RoPE θ	1,000,000
Format	.symb INT4 (group_size=128)
Size	~3.85 GB
Adapter	32-layer Symbiotic Gate (knowledge distillation)

---

Device Selection — Dual Engine

Same .symb files work on all backends:

# High-level API
model = GRRNModel.from_pretrained("./magnus-symb", device="cpu")    # CPU (NumPy + BLAS)
model = GRRNModel.from_pretrained("./magnus-symb", device="cuda")   # NVIDIA GPU
model = GRRNModel.from_pretrained("./magnus-symb", device="mps")    # Apple Metal GPU
model = GRRNModel.from_pretrained("./magnus-symb", device="dual")   # Auto-detect best

# Direct Dual Engine
from grrn_inference.dual_engine import DualEngine
engine = DualEngine("./model-symb", device="auto")

Backend	Speed	Hardware
cuda	🔥 103+ tok/s	NVIDIA GPU (PyTorch CUDA)
mps	⚡ 18+ tok/s	Apple Silicon (PyTorch Metal)
cpu	⚡ 12-16 tok/s	Any CPU (NumPy + BLAS)
dual	Auto-detect	Best available (multi-GPU if present)
parallel	Multi-GPU	2-8 NVIDIA GPUs

---

Architecture

[Token ID] → [Embedding] → [22-32 Layer Pipeline] → [Logits] → [Next Token]
                                    │
                 Per Layer: RMSNorm → Attn → RMSNorm → MLP → Residual
                                    │
                    Attention:
                      QKV + RoPE + DualFluidM update
                      output = q @ (U @ V^T)
                      O(D×r) state, r=256, fixed memory
                    
                    DualFluidM:
                      Structured track (phase-rotated, anti-saturation)
                      + Unstructured track (Euler adaptive, anti-stiffness)
                      Dual retrieval: Y = Q·(M_struct + α·M_unstruct)

---

Roadmap

Feature	Status
Text generation / chat / streaming	✅ Done
.symb INT4 format	✅ Done
CPU engine (NumPy + Numba)	✅ Done
GPU engine (Triton fused kernels)	✅ Done
Dual Engine (CPU + CUDA + MPS)	✅ Done (v0.2)
DualFluidM (structured + unstructured M)	✅ Done (v0.2)
103.12 tok/s on NVIDIA CUDA (Compartment Gate)	✅ Verified
18.35 tok/s on Apple M3 MPS (Compartment Gate)	✅ Verified
AXIM runtime integration	✅ Done
Symbiotic Gate adapter training	✅ Done
Magnus as default model	✅ Done
OpenAI-compatible API server	✅ Done
Marimo cloud GPU notebooks	✅ Done
Native FP16/FP32 (no INT4 required)	✅ Done (v0.2.1)
Vision / image support	✅ Done (v0.2.1)
Multi-GPU tensor parallelism	✅ Done (v0.2.1)

---

Links

• Engine: github.com/GRRN-MAKER/SYNAXIM
• AXIM Runtime: github.com/GRRN-MAKER/Aximcomp
• Model (original): GRRNMAKE/Magnus
• Model (.symb): GRRNMAKE/Magnus-SYMB
• Author: GRRNMAKER

---

SYNAXIM — Because inference should be a machine, not a framework.
Built by GRRNMAKER