GRRNMAKE/kimi3_symb

⚡ Kimi-K3 (.symb Universal Symbolic Edition) — Powered by AXIM OS

GRRNMAKE/kimi3_symb is the complete 93-layer (KimiK3ForConditionalGeneration, hidden_size = 7,168, 69 Kimi Delta Attention KDA layers + 24 Multi-Head Latent Attention MLA layers, 896 routed Latent-MoE experts + shared experts per layer, Attention/MLP Block Residuals) conversion of Kimi-K3 into the .symb Universal Symbolic Weight Format developed by Cognitus Lab.

This repository includes both the native Kimi-K3 (163,840 vocab) weights (embeddings.symb, lm_head.symb) and the MATT/AIM Layer-00 Transferred SYNAXIM (128,256 vocab) weights (embeddings_synaxim.symb, lm_head_synaxim.symb, synaxim_tokenizer.json), enabling unified cross-model tokenization and ultra-fast execution via AXIM OS.

---

🚀 Accelerate .symb Loading & Inference with AXIM OS (pip install axim-os)

.symb models are engineered to load and execute dramatically faster with AXIM OS — the Universal CUDA-Free Compute & Graphics Runtime Engine by Cognitus Lab.

Install AXIM OS from PyPI

pip install axim-os

• PyPI Package: https://pypi.org/project/axim-os/
• Native Engine: axim-os (libaxim_cpu.so / libaxim_gpu.so) performs hardware-vectorized SIMD/NEON + OpenMP .symb (INT4 / MXFP4 group-128 $\rightarrow$ BF16 / FP32) weight loading and in-place dequantization directly in host memory without CUDA weight-loader overhead.

---

📊 Verified Speed & GPU Performance (AXIM OS 1.1.3 + GPU)

Benchmarked on NVIDIA GH200 Grace Hopper Superchip (96 GB HBM3 GPU + 72-Core Grace ARM64 CPU running axim-os 1.1.3):

Metric	Standard Weight Loading	.symb + AXIM OS 1.1.3	Improvement
Architecture Scale	93 Layers (D = 7,168, 896 Latent-MoE Experts/Layer)	100% .symb (INT4/MXFP4 Group-128 + FP32 Norms/Routers)	Zero skipped weights
Per-Tensor AXIM OS Dequant Speed	35–60 ms (Python/CUDA dequant)	0.62 ms – 1.07 ms / tensor (3,795 tensors in 2.36s dequant time)	> 30x faster dequantization
93-Layer Warm Forward Step (3,795 .symb Tensors)	Multi-node GPU cluster required	4.1s total (2.36s AXIM OS NEON+OpenMP dequant across 72 cores)	Single-node execution
Active GPU VRAM Footprint	> 1,000 GB (raw multi-node weights)	Runs within a single 96 GB HBM3 GPU via AXIM OS streaming	> 90% GPU VRAM reduction

---

🌐 .symb Works Universally Across All Model Architectures

The .symb Universal Symbolic Weight Format by Cognitus Lab is proven across every frontier neural architecture:

• Ultra-Scale 93-Layer Hybrid KDA + MLA + Latent-MoE (Kimi-K3): GRRNMAKE/kimi3_symb
• 45-Layer Hybrid KDA + NoPE-MLA + 4-Stream Hyper-Connection MoE (GLM-5.3-Flash): GRRNMAKE/GLM5.3Flash_symb
• 64-Layer Hybrid Gated DeltaNet + Gated GQA + Vision (Qwen3.8-27B): GRRNMAKE/Qwen3.8-27Bsymb
• 32-Layer Dense & Dual-Fluid Reasoning Models (Chííké): Native .symb + axim-os execution

---

🧬 SYNAXIM (128,256 Vocab) Tokenizer Transfer Included

Alongside the native 163,840-vocab embeddings.symb and lm_head.symb, this repository provides embeddings_synaxim.symb and lm_head_synaxim.symb aligned to the GRRNMAKE/SYNAXIM (128,256 vocab) tokenizer via MATT/AIM Layer-00 Feature-Space Alignment:

Verified 93-Layer Coherence Output (.symb + SYNAXIM + axim-os 1.1.3)

• Prompt: "The capital of France is" $\rightarrow$ "The capital of France is Paris, and" (" Paris" Rank #1, logit 18.33)
• Prompt: "The largest planet in our solar system is" $\rightarrow$ "The largest planet in our solar system is Jupiter, which" (" Jupiter" Rank #1, logit 19.65)

---

💻 Quickstart: Loading .symb Tensors with axim-os

pip install axim-os torch numpy tokenizers

import struct
import ctypes
import numpy as np
import torch
import axim_os
from axim.orchestrator.native_loader import load_native

# Initialize AXIM OS native C/SIMD engine
axim_lib = load_native("cpu")
axim_lib.axim_cpu_symb_dequant_bf16.argtypes = [
    ctypes.POINTER(ctypes.c_uint8),
    ctypes.POINTER(ctypes.c_float),
    ctypes.POINTER(ctypes.c_uint16),
    ctypes.c_size_t,
    ctypes.c_size_t,
    ctypes.c_size_t,
    ctypes.c_int,
]
axim_lib.axim_cpu_symb_dequant_bf16.restype = None

def load_symb_with_axim_os(path: str, out_buf: torch.Tensor = None) -> torch.Tensor:
    """Load and dequantize any .symb tensor via AXIM OS."""
    with open(path, "rb") as f:
        data = f.read()
    ndim, = struct.unpack_from("<I", data, 0)
    shape = struct.unpack_from(f"<{ndim}I", data, 4)
    n, group_size = struct.unpack_from("<II", data, 4 + 4 * ndim)
    offset = 4 + 4 * ndim + 8

    if group_size == 1:
        arr = np.frombuffer(data, dtype=np.float32, count=n, offset=offset).copy().reshape(shape)
        return torch.from_numpy(arr)

    pad = (group_size - (n % group_size)) % group_size
    n_padded = n + pad
    n_groups = n_padded // group_size
    scales_np = np.frombuffer(data, dtype=np.float32, count=n_groups, offset=offset)
    offset += 4 * n_groups
    packed_bytes = (n_padded + 1) // 2
    packed_np = np.frombuffer(data, dtype=np.uint8, count=packed_bytes, offset=offset)
    is_mxfp4 = 1 if scales_np[0] > 50.0 else 0

    out_cpu = out_buf if (out_buf is not None and tuple(out_buf.shape) == tuple(shape)) else torch.empty(shape, dtype=torch.bfloat16)
    axim_lib.axim_cpu_symb_dequant_bf16(
        packed_np.ctypes.data_as(ctypes.POINTER(ctypes.c_uint8)),
        scales_np.ctypes.data_as(ctypes.POINTER(ctypes.c_float)),
        ctypes.cast(out_cpu.data_ptr(), ctypes.POINTER(ctypes.c_uint16)),
        n_groups, group_size, n, is_mxfp4
    )
    return out_cpu

---

📁 Repository Structure

GRRNMAKE/kimi3_symb/
├── LICENSE                        # Cognitus Public Symbolic License v1.0 (CPSL-1.0)
├── README.md                      # Documentation, AXIM OS benchmarks & usage guide
├── axim_symb_loader.py            # Standalone AXIM OS .symb loader utility
├── config.json                    # Kimi-K3 93-layer architectural configuration
├── configuration_kimi_k3.py       # Kimi-K3 configuration module
├── modeling_kimi_k3.py            # Kimi-K3 architecture reference
├── modeling_kimi_linear.py        # Kimi Linear / KDA attention module
├── synaxim_tokenizer.json         # Transferred SYNAXIM (128,256 vocab) fast tokenizer
├── tiktoken.model                 # Native Kimi-K3 (163,840 vocab) tokenizer model
├── tokenizer_config.json          # Tokenizer configuration
├── embeddings_synaxim.symb        # SYNAXIM [128256, 7168] MATT/AIM Layer-00 aligned embeddings
├── lm_head_synaxim.symb           # SYNAXIM [128256, 7168] calibrated LM head
├── embeddings.symb                # Native [163840, 7168] embeddings
├── lm_head.symb                   # Native [163840, 7168] LM head
├── final_norm.symb                # Final RMSNorm [7168]
├── output_attn_res_norm.symb      # Output block residual norm
├── output_attn_res_proj.symb      # Output block residual projection
├── layer_00/                      # Layer 00 Dense KDA + SwiGLU MLP (.symb)
└── layer_01/ .. layer_92/         # Layers 01..92 Hybrid KDA / MLA + Latent-MoE (.symb)
    ├── q_proj.symb, k_proj.symb, v_proj.symb / q_a_proj.symb, kv_a_proj_with_mqa.symb ...
    ├── router.symb, e_score_correction_bias.symb
    ├── routed_expert_down_proj.symb, routed_expert_up_proj.symb, routed_expert_norm.symb
    ├── shared_expert_{gate,up,down}.symb
    └── experts_b0..b3_{gate,up,down}.symb # 896 routed Latent-MoE experts in 4 x [224, ...] 3D .symb banks

---

📜 License — Cognitus Public Symbolic License v1.0 (CPSL-1.0)

Released under the Cognitus Public Symbolic License v1.0 (CPSL-1.0) (see LICENSE).

@misc{cognitus2026kimi3_symb,
  title  = {Kimi-K3 (.symb Universal Symbolic Edition) with SYNAXIM Tokenizer Transfer & AXIM OS Runtime},
  author = {Cognitus Lab and GRRNMAKE},
  year   = {2026},
  url    = {https://huggingface.co/GRRNMAKE/kimi3_symb}
}