CMSManhattan/JiRackUltra_1b

JiRack Ultra 1B (CPU)

A fast and efficient ~1.5B model optimized for CPU inference. The model was refactored with BitNet features and an updated tokenizer that includes new Routing, Tool call, and Robotics tags. Built on a redesigned DeepSeek R1 architecture with native ternary (BitNet-style) support and ready-to-run GGUF quantizations.

• JiRack is a cloud-ready model that helps save money on cloud infrastructure. It can be used as an expert model in RAG deployments, with the ONNX JiRack Java server as an alternative.
• Subscription: $1 per month per user (updated license for non-company use).
• Corp Subscription: $3 per month per user (updated license for company use).
• It works without subscription but send message about subscription

Available Variants

Tag	Quant	Size	Approx. RAM	Description
cmsmanhattan/jirack-ultra-1b-cpu:latest	Full	0.55 GB	~1.8 GB	Full ternary reference
cmsmanhattan/jirack-ultra-1b-cpu-q4:latest	Q4_K_M	0.38 GB	~1.4 GB	Recommended balance
cmsmanhattan/jirack-ultra-1b-cpu-q3:latest	Q3_K_M	0.31 GB	~1.2 GB	Good quality / size trade-off
cmsmanhattan/jirack-ultra-1b-cpu-q2:latest	Q2_K	0.24 GB	~1.0 GB	Maximum compression

Quick Start

Run with Docker

Default CPU (Q4 recommended)

docker run -d \
  --name jirack_ultra_1b \
  -p 7869:7869 \
  --cpus=16 \
  -e THREADS=16 \
  -e THREADS_BATCH=16 \ 
  --restart unless-stopped \
  cmsmanhattan/jirack-ultra-1b-cpu-q4:latest

Q3

docker run -d \
  --name jirack_ultra_1b \
  -p 7869:7869 \
  --cpus=16 \
  -e THREADS=16 \
  -e THREADS_BATCH=16 \ 
  --restart unless-stopped \
  cmsmanhattan/jirack-ultra-1b-cpu-q3:latest

Q2 (lowest memory)

docker run -d \
  --name jirack_ultra_1b \
  -p 7869:7869 \
  --cpus=16 \
  -e THREADS=16  \
  -e THREADS_BATCH=16 \ 
  --restart unless-stopped \
  cmsmanhattan/jirack-ultra-1b-cpu-q2:latest

Full precision

docker run -d \
  --name jirack_ultra_1b \
  -p 7869:7869 \
  --cpus=16  \
  -e THREADS=16 \
  -e THREADS_BATCH=16 \ 
  --restart unless-stopped \
  cmsmanhattan/jirack-ultra-1b-cpu:latest

Multi CPU

docker run -d \
  --name jirack_ultra_1b \
  -p 7869:7869 \
  --cpus=16 \
  -e THREADS=16 \
  -e THREADS_BATCH=16 \ 
  --restart unless-stopped \
  --memory=4g \
  --cpus=4 \
  cmsmanhattan/jirack-ultra-1b-cpu-q4:latest

Docker Compose Example

services:
  jirack:
    image: cmsmanhattan/jirack-ultra-1b-cpu-q4:latest
    container_name: jirack_ultra_1b
    ports:
      - "7869:7869"
    volumes:
      - .:/app
      - ./web:/app/web
    environment:
      - MAX_TOKENS=2048
      - TEMPERATURE=0.7
      - TOP_P=0.9
      - DEFAULT_STREAM=False
      - INTRA_THREADS=4
      - USE_ENV_ALLOCATOR=1
      - THREADS=16 
      - THREADS_BATCH=16 
    deploy:
      resources:
        limits:
          memory: 4g

Access the UI

Once the container is running, open your browser and navigate to: http://localhost:7869 This opens the JiRack UI — a clean web interface.

Changing the Port

The listening port can be easily modified directly from the Settings panel within the JiRack UI.

Licensing

• The JiRack Ultra 1B model is provided under a commercial license ($12 per user per year).
• All JiRack UI clients are provided under a commercial license.
• However, the UI clients can be used for free when running together with the official JiRack Docker containers, as long as they are not redistributed separately. For commercial licensing, cluster deployment, or enterprise use of JiRack models, please contact us.
• JiRack MS Windows 11 Desktop Client (with Ollama API): https://huggingface.co/kgrabko/JiRackTernary_1b/resolve/main/jirack-chat.zip
• Live email chat with the model: support@cmsmanhattan.com

Hardware Recommendations

Recommended Hardware for JiRack Ultra 1B (single Docker container)

Use Case	CPU	RAM	Recommended Quant	Expected Speed	Recommendation
Recommended	Ryzen 5 / Intel i5	4–8 GB	Q4_K_M	Excellent interactive	Best choice
High Performance	Ryzen 7 / Intel i7	8–16 GB	Full / Q4	Excellent	Excellent
Low Memory	Modern 4+ core CPU	2–4 GB	Q3_K_M or Q2_K	Usable	Acceptable
Edge / Minimal	Laptop / SBC CPU	2 GB	Q2_K	Acceptable	Budget option

Important Memory Notes

Even though the quantized 1B models are very small, we recommend the following for best experience:

• Q4_K_M: 2–4 GB system RAM minimum
• Q3_K_M / Q2_K: 1.5–3 GB system RAM
• Full precision: 3–4 GB+ system RAM recommended Reasons for extra headroom:
• KV-cache consumption during generation
• Runtime overhead and temporary buffers
• System stability and avoiding out-of-memory errors
• Room for larger context windows Minimum recommended (Q4): 2–3 GB system RAM Ideal: 4–8 GB system RAM I added the default model in full precision. This serves as the base for quantization, allowing us to find the optimal balance between model size and performance.

Architecture Notes

• Refactored with BitNet features: Native BitLinear ternary path (b1.58-style) with λ-warmup STE
• Updated tokenizer: Extended with new special tags for Routing, Tool call, and Robotics
• Base: Redesigned Llama-3.2-1B style (Hidden 2048, Intermediate 8192, 16 layers, GQA 32/8, vocab 128256)
• RoPE θ = 10000, RMSNorm ε = 1e-6
• Ready-to-run GGUF quantizations (Q2_K, Q3_K_M, Q4_K_M)

📧 Contact & Licensing

For joint venture opportunities, hardware integration, or licensing inquiries:

• Email: grabko@cmsmanhattan.com
• Phone: +1 (516) 777-0945
• Location: New York, USA