KorshAI Banner

KorshAI-4B-v5 (GGUF)

KorshAI-4B-v5 is a proprietary, state-of-the-art 4-billion parameter compact foundation coding and systems reasoning model architected, trained, and verified by KorshLab and the Korsh Foundation.

Engineered specifically for low-level systems programming, high-reliability web architectures, and formal multi-language verification, KorshAI-4B-v5 delivers enterprise-grade code generation speed and compiler-grade precision on edge workstations, consumer GPUs, and local CPU servers.


๐ŸŒŸ Key Highlights

  • Proprietary Compact Architecture: Native 4B parameter dense architecture tailored for algorithmic reasoning, zero-simulation code synthesis, and deep compiler syntax fidelity.
  • Polyglot Systems Mastery: Specialized in low-level systems engineering (C11, C++20, Rust), type-safe application logic (TypeScript, Python 3.12+), backend services (PHP 8.2+), and modern web platforms.
  • Strict Multi-Tier Verification: Every synthetic training unit was vetted using the KorshLab multi-tier sandbox compiler gate with zero-tolerance flags (-Wall -Wextra -Werror -pedantic).
  • Optimal imatrix Quantization: Quantized with custom importance matrices (imatrix) computed against polyglot systems corpora, preserving fp16 perplexity with a lean memory footprint.
  • High-Throughput Edge Inference: Fully optimized for llama.cpp AVX2/AVX-512 CPU execution and CUDA/Metal acceleration, sustaining high generation velocity even on entry-level edge servers.

๐Ÿ“Š Benchmark & Empirical Evaluation (v5)

KorshAI-4B-v5 was evaluated under the rigorous KorshLab Evaluation Suite (20 challenging multi-domain tasks evaluated via automated deterministic test runners and compiler validation):

Category Evaluation Score Verification Standard
JSON Schema & Structured Output 75.0% Strict RFC 7159 JSONSchema validation & constraint adherence
Core Coding & Algorithms 68.0% Multi-language algorithmic problem solving & edge cases
Reasoning & Formal Logic 42.4% Chain-of-thought deductive steps & invariant analysis
C++ Systems & Memory Safety 38.0% G++ 12+ C++20 strict compilation (-Wall -Wextra -Werror)
Overall Composite Score 55.85 Champion Model in KorshAI 4B Lineage

๐Ÿ’ป Hardware Compatibility Matrix

1. Requirements

Platform / Tier Hardware Target RAM / VRAM Footprint Expected Speed
CPU Only (Workstation / Edge) 4-Core x86_64 AVX2 (e.g. Dell PowerEdge T40, Intel Core i5/i7, AMD Ryzen) 6 GB RAM (active footprint ~2.2 GB) 5.0 โ€“ 15.0 tokens/s (TG)
~30-50 tok/s (PP)
GPU Accelerated (Consumer Desktop) NVIDIA RTX 3060, 3070, 4060, 4070 (6-8 GB VRAM) 6 GB โ€“ 8 GB VRAM (active ~2.5 GB) 60.0 โ€“ 90.0 tokens/s (TG)
~900+ tok/s (PP)
Apple Silicon (macOS Metal) Apple M1 / M2 / M3 / M4 8 GB Unified Memory 40.0 โ€“ 60.0 tokens/s (TG)
Low-Power Edge (SBC) Raspberry Pi 5 / RK3588 (ARM64) 8 GB LPDDR4X 6.0 โ€“ 9.0 tokens/s (TG)

2. Supported Runtimes

  • llama.cpp (b3000+)
  • ollama (v0.1.30+)
  • LM Studio / Jan.ai / LocalAI / Text Generation WebUI
  • Python via llama-cpp-python

๐Ÿ“ฆ Provided Model Artifacts

File Format Quantization Size SHA-256 / MD5 Description
KorshAI-4B-v5-Q4_K_M.gguf GGUF Q4_K_M (imatrix) 2.69 GB (2,886,966,752 B) MD5: 2434b5a7afb933372d7bccf1466942d1 Primary recommended quantization for production inference.

๐Ÿš€ Quickstart & Usage

1. CLI Execution with llama.cpp

llama-cli -m KorshAI-4B-v5-Q4_K_M.gguf \
  -p "<|im_start|>system\nYou are KorshAI, an expert systems and software engineer.<|im_end|>\n<|im_start|>user\nWrite an idiomatic, zero-allocation circular ring buffer in C++20.<|im_end|>\n<|im_start|>assistant\n" \
  -c 4096 \
  -ngl 99 \
  --temp 0.2

2. Production OpenAI-Compatible HTTP Server

llama-server -m KorshAI-4B-v5-Q4_K_M.gguf \
  --host 0.0.0.0 \
  --port 8080 \
  -c 8192 \
  -t 4 -tb 4 \
  --flash-attn on \
  -ctk q4_0 -ctv q4_0

Query the server:

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "KorshAI-4B-v5",
    "messages": [
      {"role": "system", "content": "You are KorshAI, an expert polyglot engineer."},
      {"role": "user", "content": "Implement a thread-safe singleton pattern in modern C++20."}
    ],
    "temperature": 0.2
  }'

3. Ollama Integration

Create a file named Modelfile:

FROM ./KorshAI-4B-v5-Q4_K_M.gguf

TEMPLATE """<|im_start|>system
{{ .System }}<|im_end|>
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"""

SYSTEM """You are KorshAI, a world-class autonomous software engineering model created by KorshLab and korsh.org."""

PARAMETER stop "<|im_end|>"
PARAMETER stop "<|endoftext|>"
PARAMETER temperature 0.2

Create and run:

ollama create korshai:4b-v5 -f Modelfile
ollama run korshai:4b-v5

๐Ÿ›๏ธ About Korsh & KorshLab

KorshAI-4B-v5 is developed, benchmarked, and maintained by KorshLab in partnership with the Korsh Foundation.


๐Ÿ“„ License & Attribution

This model is licensed under the Apache License, Version 2.0. Copyright ยฉ 2026 KorshLab & Korsh Foundation. All rights reserved.

Downloads last month
56
GGUF
Model size
4B params
Architecture
qwen35
Hardware compatibility
Log In to add your hardware

4-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support