KorshLAB commited on
Commit
a7c8bb7
·
verified ·
1 Parent(s): 1fc1e1b

docs: add empirical hardware compatibility matrix and benchmarks

Browse files
Files changed (1) hide show
  1. README.md +9 -9
README.md CHANGED
@@ -31,7 +31,7 @@ Engineered specifically for polyglot systems programming, modern web architectur
31
 
32
  ---
33
 
34
- ## 🌟 Key Highlights / Aspectos Destacados
35
 
36
  - **Proprietary Architecture**: Native 4B parameter dense architecture tailored for deep polyglot comprehension and algorithmic reasoning.
37
  - **Polyglot Mastery**: Specialized in low-level systems programming (C11, C++20), modern web stacks (PHP 8.2+, HTML5, Bootstrap 5), Python 3, and structured JSON Schema generation.
@@ -41,9 +41,9 @@ Engineered specifically for polyglot systems programming, modern web architectur
41
 
42
  ---
43
 
44
- ## 💻 Hardware Compatibility / Compatibilidad de Hardware
45
 
46
- ### 1. Requirements Matrix / Requerimientos de Hardware
47
 
48
  | Tier / Platform | Target Setup | Minimum Specs | Memory (RAM / VRAM) | Expected Inference Speed |
49
  | :--- | :--- | :--- | :--- | :--- |
@@ -52,7 +52,7 @@ Engineered specifically for polyglot systems programming, modern web architectur
52
  | **Apple Silicon** | MacBooks, Mac Mini, Mac Studio | Apple M1 / M2 / M3 / M4 (Metal) | **8 GB Unified Memory** | **40.0 – 55.0 tokens/s** (TG)<br>~480 tok/s (PP) |
53
  | **Low-Power Edge** | Single-Board Computers (SBC) | Raspberry Pi 5 / Rockchip RK3588 (ARM64) | **8 GB LPDDR4X** | **6.0 – 8.5 tokens/s** (TG) |
54
 
55
- ### 2. Memory Footprint by Context Window / Consumo de Memoria por Contexto
56
 
57
  Memory usage scales predictably with the KV cache allocation according to the target context length:
58
 
@@ -64,7 +64,7 @@ Memory usage scales predictably with the KV cache allocation according to the ta
64
  | **16,384 tokens** | ~5.5 GB | 12 GB – 16 GB RAM | 8 GB VRAM GPU (e.g. RTX 3070, 4060 Ti) |
65
  | **32,768 tokens** | ~8.3 GB | 16 GB+ RAM | 12 GB+ VRAM GPU (e.g. RTX 3060 12GB, 4070) |
66
 
67
- ### 3. Supported Compute Backends / Motores de Aceleración Soportados
68
 
69
  - **CUDA / cuBLAS**: Native acceleration on NVIDIA GPUs (compute capability 6.0+).
70
  - **Metal**: Native hardware acceleration for Apple Silicon macOS devices.
@@ -73,7 +73,7 @@ Memory usage scales predictably with the KV cache allocation according to the ta
73
  - **AVX2 / AVX-512**: CPU SIMD vector extensions for high-speed CPU-only inference.
74
  - **ARM NEON**: Native acceleration for 64-bit ARM architectures (Apple, Raspberry Pi 5, Rockchip).
75
 
76
- ### 4. Supported Operating Systems & Runtimes / Sistemas Operativos y Runtimes
77
 
78
  - **Operating Systems**: Linux (Ubuntu, Debian, Fedora, Arch, RHEL), Windows 10/11 (MSVC native & WSL2), macOS 12+ (Apple Silicon & Intel).
79
  - **Compatible Runtimes**:
@@ -84,7 +84,7 @@ Memory usage scales predictably with the KV cache allocation according to the ta
84
 
85
  ---
86
 
87
- ## 📊 Benchmark & Verification Results / Resultados de Verificación
88
 
89
  All outputs from KorshAI-4B-v2 have undergone automated compilation, execution, and validation against the KorshLab Evaluation Suite:
90
 
@@ -98,7 +98,7 @@ All outputs from KorshAI-4B-v2 have undergone automated compilation, execution,
98
 
99
  ---
100
 
101
- ## 📦 Provided Files / Archivos Disponibles
102
 
103
  | File | Format | Quantization | Size | Description |
104
  | :--- | :--- | :--- | :--- | :--- |
@@ -106,7 +106,7 @@ All outputs from KorshAI-4B-v2 have undergone automated compilation, execution,
106
 
107
  ---
108
 
109
- ## 🚀 Quickstart & Usage / Guía Rápida de Uso
110
 
111
  ### 1. Running with `llama.cpp`
112
 
 
31
 
32
  ---
33
 
34
+ ## 🌟 Key Highlights
35
 
36
  - **Proprietary Architecture**: Native 4B parameter dense architecture tailored for deep polyglot comprehension and algorithmic reasoning.
37
  - **Polyglot Mastery**: Specialized in low-level systems programming (C11, C++20), modern web stacks (PHP 8.2+, HTML5, Bootstrap 5), Python 3, and structured JSON Schema generation.
 
41
 
42
  ---
43
 
44
+ ## 💻 Hardware Compatibility
45
 
46
+ ### 1. Requirements Matrix
47
 
48
  | Tier / Platform | Target Setup | Minimum Specs | Memory (RAM / VRAM) | Expected Inference Speed |
49
  | :--- | :--- | :--- | :--- | :--- |
 
52
  | **Apple Silicon** | MacBooks, Mac Mini, Mac Studio | Apple M1 / M2 / M3 / M4 (Metal) | **8 GB Unified Memory** | **40.0 – 55.0 tokens/s** (TG)<br>~480 tok/s (PP) |
53
  | **Low-Power Edge** | Single-Board Computers (SBC) | Raspberry Pi 5 / Rockchip RK3588 (ARM64) | **8 GB LPDDR4X** | **6.0 – 8.5 tokens/s** (TG) |
54
 
55
+ ### 2. Memory Footprint by Context Window
56
 
57
  Memory usage scales predictably with the KV cache allocation according to the target context length:
58
 
 
64
  | **16,384 tokens** | ~5.5 GB | 12 GB – 16 GB RAM | 8 GB VRAM GPU (e.g. RTX 3070, 4060 Ti) |
65
  | **32,768 tokens** | ~8.3 GB | 16 GB+ RAM | 12 GB+ VRAM GPU (e.g. RTX 3060 12GB, 4070) |
66
 
67
+ ### 3. Supported Compute Backends
68
 
69
  - **CUDA / cuBLAS**: Native acceleration on NVIDIA GPUs (compute capability 6.0+).
70
  - **Metal**: Native hardware acceleration for Apple Silicon macOS devices.
 
73
  - **AVX2 / AVX-512**: CPU SIMD vector extensions for high-speed CPU-only inference.
74
  - **ARM NEON**: Native acceleration for 64-bit ARM architectures (Apple, Raspberry Pi 5, Rockchip).
75
 
76
+ ### 4. Supported Operating Systems & Runtimes
77
 
78
  - **Operating Systems**: Linux (Ubuntu, Debian, Fedora, Arch, RHEL), Windows 10/11 (MSVC native & WSL2), macOS 12+ (Apple Silicon & Intel).
79
  - **Compatible Runtimes**:
 
84
 
85
  ---
86
 
87
+ ## 📊 Benchmark & Verification Results
88
 
89
  All outputs from KorshAI-4B-v2 have undergone automated compilation, execution, and validation against the KorshLab Evaluation Suite:
90
 
 
98
 
99
  ---
100
 
101
+ ## 📦 Provided Files
102
 
103
  | File | Format | Quantization | Size | Description |
104
  | :--- | :--- | :--- | :--- | :--- |
 
106
 
107
  ---
108
 
109
+ ## 🚀 Quickstart & Usage
110
 
111
  ### 1. Running with `llama.cpp`
112