kgrabko commited on
Commit
beb4e9c
·
verified ·
1 Parent(s): 5f535b3

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +71 -64
README.md CHANGED
@@ -1,18 +1,18 @@
1
  ---
2
- language:
3
  - en
4
- - zh
5
- - ja
6
- - ko
7
- - fr
8
- - es
9
- - pt
10
- - de
11
- - it
12
- - ru
13
- - ar
14
- - vi
15
- - th
16
  tags:
17
  - code
18
  - coding
@@ -23,54 +23,59 @@ tags:
23
  license: unknown
24
  ---
25
 
26
- # JiRack Coder Reasoing 8B INT4
27
 
28
- A fast and efficient coding assistant with a clean built-in web UI, powered by Qwen3.0-Coder-8B-Instruct base and optimized using Microsoft ONNX Runtime.
29
 
30
- - JiRack is cloud model and save money on cloud and can be used as expert model in RAG on cloud with ONNX JiRack java server as alternative.
31
- - Subscription 1$ per month per user in updated license if not company
32
 
33
  ## Quick Start
34
- Watch the JiRack Coder 8B in action:
35
- **DEMO**: [JiRack Coder Reasoing 8B Web UI](https://youtu.be/mq1DxIov7Bw)
36
 
 
 
 
37
 
38
  ### Run with Docker
39
 
40
- ---
41
- --Default CPU--
42
 
43
- - docker run -d \
 
44
  --name jirack_coder_reasoing_8b \
45
  -p 7869:7869 \
46
  --restart unless-stopped \
47
  cmsmanhattan/jirack_coder_8b_int4_qwenbase:latest
 
48
 
49
- --Multi CPU--
50
 
51
- - docker run -d \
 
52
  --name jirack_coder_reasoing_8b \
53
  -p 7869:7869 \
54
  --restart unless-stopped \
55
  --memory=20g \
56
  --cpus=12 \
57
  cmsmanhattan/jirack_coder_8b_int4_qwenbase:latest
 
58
 
59
- ---GPU--
60
- -- comming soon
61
 
62
- - docker run -d \
63
- --name jirack_coder_reasoing_8b \
64
- -p 7869:7869 \
65
- --gpus all \
66
- --restart unless-stopped \
67
- cmsmanhattan/jirack_coder_8b_int4_gpu_qwenbase:latest
 
 
68
 
69
- ---
70
 
 
71
  services:
72
-
73
-
74
  image: cmsmanhattan/jirack_coder_8b_int4_qwenbase:latest
75
  container_name: jirack_onnx_service
76
  ports:
@@ -88,15 +93,16 @@ services:
88
  deploy:
89
  resources:
90
  limits:
91
- memory: 16g
 
92
 
93
  ## Access the UI
94
 
95
  Once the container is running, open your browser and navigate to:
96
 
97
- **`http://localhost:7869`**
98
 
99
- This opens the **JiRack Coder UI** — a clean web interface designed for coding.
100
 
101
  ## Changing the Port
102
 
@@ -104,51 +110,52 @@ The listening port can be easily modified directly from the **Settings** panel w
104
 
105
  ## Licensing
106
 
107
- - The **JiRack Coder 8B model** is provided under a commercial license. It ia about 12$ for year per user .
108
- - All **JiRack UI clients** are provided under a commercial license.
109
  - However, the UI clients can be used for free when running together with the official JiRack Docker containers, as long as they are not redistributed separately.
110
 
 
111
 
112
- **JiRack Coder 32B** is available exclusively under a commercial enterprise license.
113
 
114
- For commercial licensing, cluster deployment, or enterprise use of the JiRack Coder 32B and JiRack Coder 14B , please contact us.
115
- - JiRack MS Windows 11 Desktop chat client with ollama API setup : https://huggingface.co/kgrabko/JiRackTernary_1b/resolve/main/jirack-chat.zip
116
- - Live email chat with model via support@cmsmanhattan.com
117
 
 
118
 
119
  ## Hardware Recommendations for AMD Systems
120
- It is more heavy then JiRack Coder 7B INT8
121
- ### Recommended Hardware for JiRack Coder Reasoing 8B INT8 . It is one dcoker container
122
 
123
- | Use Case | CPU | GPU (ROCm) | VRAM / RAM | Expected Speed | Recommendation |
124
- |-----------------------|----------------------------------|-----------------------------------|----------------|---------------------|--------------------|
125
- | **Recommended** | Ryzen 7 7700 / 9700X | RX 7900 XTX / 7900 XT | 24GB VRAM | 50-75 tokens/s | Best choice |
126
- | **High Performance** | Ryzen 9 7950X / 9950X | RX 7900 XTX | 24GB+ VRAM | 65-90 tokens/s | Excellent |
127
- | **Enterprise** | EPYC 7003/9004 series | MI300X or 2x RX 7900 XTX | 48GB+ VRAM | 90-140 tokens/s | For 32B model |
128
- | **Budget Option** | Ryzen 5 7600 / 9600X | RX 7800 XT (16GB) | 16GB VRAM | 35-50 tokens/s | Acceptable |
129
 
130
- ### Important Memory Notes
 
 
 
 
 
131
 
132
- Even though the 8B INT4 model itself takes approximately **5–6 GB**, we recommend **at least 24GB VRAM** for the following reasons:
133
 
134
- - KV-cache consumption during generation (especially with long context)
 
 
135
  - ONNX Runtime overhead and temporary buffers
136
- - System stability and to avoid Out of Memory errors
137
  - Room for larger context windows
138
 
139
  **Minimum recommended:** 24GB VRAM (RX 7900 series)
140
  **Ideal:** 24–32GB VRAM
141
 
142
- For pure CPU inference (no GPU), we recommend at least **64GB system RAM** (Ryzen 9 7950X/9950X).
143
-
144
- ---
145
- I will the default model in full FP32 precision for quantization, allowing us to find the optimal balance between model size and performance.
146
 
 
147
 
148
  ## 📧 Contact & Licensing
149
- For joint venture opportunities, hardware integration, or licensing inquiries:
150
- - **Email:** [grabko@cmsmanhattan.com](mailto:grabko@cmsmanhattan.com)
151
- - **Phone:** +1 (516) 777-0945
152
- - **Location:** New York, USA
153
 
 
154
 
 
 
 
 
1
  ---
2
+ language:
3
  - en
4
+ - zh
5
+ - ja
6
+ - ko
7
+ - fr
8
+ - es
9
+ - pt
10
+ - de
11
+ - it
12
+ - ru
13
+ - ar
14
+ - vi
15
+ - th
16
  tags:
17
  - code
18
  - coding
 
23
  license: unknown
24
  ---
25
 
26
+ # JiRack Coder Reasoning 8B INT4
27
 
28
+ A fast and efficient coding assistant with a clean built-in web UI, powered by **Qwen3.0-Coder-8B-Instruct** and optimized using Microsoft ONNX Runtime.
29
 
30
+ - JiRack is a cloud-ready model that helps save money on cloud infrastructure. It can be used as an expert model in RAG deployments, with the ONNX JiRack Java server as an alternative.
31
+ - Subscription: **$1 per month per user** (updated license for non-company use).
32
 
33
  ## Quick Start
 
 
34
 
35
+ Watch JiRack Coder Reasoning 8B in action:
36
+
37
+ **DEMO**: [JiRack Coder Reasoning 8B Web UI](https://youtu.be/mq1DxIov7Bw)
38
 
39
  ### Run with Docker
40
 
41
+ **Default CPU**
 
42
 
43
+ ```bash
44
+ docker run -d \
45
  --name jirack_coder_reasoing_8b \
46
  -p 7869:7869 \
47
  --restart unless-stopped \
48
  cmsmanhattan/jirack_coder_8b_int4_qwenbase:latest
49
+ ```
50
 
51
+ **Multi CPU**
52
 
53
+ ```bash
54
+ docker run -d \
55
  --name jirack_coder_reasoing_8b \
56
  -p 7869:7869 \
57
  --restart unless-stopped \
58
  --memory=20g \
59
  --cpus=12 \
60
  cmsmanhattan/jirack_coder_8b_int4_qwenbase:latest
61
+ ```
62
 
63
+ **GPU (Coming soon)**
 
64
 
65
+ ```bash
66
+ docker run -d \
67
+ --name jirack_coder_reasoing_8b \
68
+ -p 7869:7869 \
69
+ --gpus all \
70
+ --restart unless-stopped \
71
+ cmsmanhattan/jirack_coder_8b_int4_gpu_qwenbase:latest
72
+ ```
73
 
74
+ ### Docker Compose Example
75
 
76
+ ```yaml
77
  services:
78
+ jirack:
 
79
  image: cmsmanhattan/jirack_coder_8b_int4_qwenbase:latest
80
  container_name: jirack_onnx_service
81
  ports:
 
93
  deploy:
94
  resources:
95
  limits:
96
+ memory: 16g
97
+ ```
98
 
99
  ## Access the UI
100
 
101
  Once the container is running, open your browser and navigate to:
102
 
103
+ `http://localhost:7869`
104
 
105
+ This opens the JiRack Coder UI — a clean web interface designed for coding.
106
 
107
  ## Changing the Port
108
 
 
110
 
111
  ## Licensing
112
 
113
+ - The JiRack Coder Reasoning 8B model is provided under a commercial license ($12 per user per year).
114
+ - All JiRack UI clients are provided under a commercial license.
115
  - However, the UI clients can be used for free when running together with the official JiRack Docker containers, as long as they are not redistributed separately.
116
 
117
+ JiRack Coder 32B is available exclusively under a commercial enterprise license.
118
 
119
+ For commercial licensing, cluster deployment, or enterprise use, please contact us.
120
 
121
+ - **JiRack MS Windows 11 Desktop Client (with Ollama API):**
122
+ https://huggingface.co/kgrabko/JiRackTernary_1b/resolve/main/jirack-chat.zip
 
123
 
124
+ - **Live email chat with the model:** support@cmsmanhattan.com
125
 
126
  ## Hardware Recommendations for AMD Systems
 
 
127
 
128
+ Note: This model is heavier than JiRack Coder 7B INT8.
129
+
130
+ ### Recommended Hardware for JiRack Coder Reasoning 8B INT4 (single Docker container)
 
 
 
131
 
132
+ | Use Case | CPU | GPU (ROCm) | VRAM / RAM | Expected Speed | Recommendation |
133
+ |------------------|--------------------------|---------------------------|------------|------------------|----------------|
134
+ | Recommended | Ryzen 7 7700 / 9700X | RX 7900 XTX / 7900 XT | 24GB VRAM | 50-75 tokens/s | Best choice |
135
+ | High Performance | Ryzen 9 7950X / 9950X | RX 7900 XTX | 24GB+ VRAM | 65-90 tokens/s | Excellent |
136
+ | Enterprise | EPYC 7003/9004 series | MI300X or 2x RX 7900 XTX | 48GB+ VRAM | 90-140 tokens/s | For 32B model |
137
+ | Budget Option | Ryzen 5 7600 / 9600X | RX 7800 XT (16GB) | 16GB VRAM | 35-50 tokens/s | Acceptable |
138
 
139
+ ## Important Memory Notes
140
 
141
+ Even though the 8B INT4 model itself takes approximately 5–6 GB, we recommend at least 24GB VRAM for the following reasons:
142
+
143
+ - KV-cache consumption during generation, especially with long context
144
  - ONNX Runtime overhead and temporary buffers
145
+ - System stability and avoiding out-of-memory errors
146
  - Room for larger context windows
147
 
148
  **Minimum recommended:** 24GB VRAM (RX 7900 series)
149
  **Ideal:** 24–32GB VRAM
150
 
151
+ For pure CPU inference (no GPU), we recommend at least 64GB system RAM (Ryzen 9 7950X/9950X).
 
 
 
152
 
153
+ I added the default model in full FP32 precision. This serves as the base for quantization, allowing us to find the optimal balance between model size and performance.
154
 
155
  ## 📧 Contact & Licensing
 
 
 
 
156
 
157
+ For joint venture opportunities, hardware integration, or licensing inquiries:
158
 
159
+ - **Email:** grabko@cmsmanhattan.com
160
+ - **Phone:** +1 (516) 777-0945
161
+ - **Location:** New York, USA