Instructions to use CMSManhattan/Llama3Ternary_2b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use CMSManhattan/Llama3Ternary_2b with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf CMSManhattan/Llama3Ternary_2b:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf CMSManhattan/Llama3Ternary_2b:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf CMSManhattan/Llama3Ternary_2b:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf CMSManhattan/Llama3Ternary_2b:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf CMSManhattan/Llama3Ternary_2b:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf CMSManhattan/Llama3Ternary_2b:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf CMSManhattan/Llama3Ternary_2b:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf CMSManhattan/Llama3Ternary_2b:Q4_K_M

Use Docker

docker model run hf.co/CMSManhattan/Llama3Ternary_2b:Q4_K_M

Llama new
LM Studio
Jan

vLLM

How to use CMSManhattan/Llama3Ternary_2b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "CMSManhattan/Llama3Ternary_2b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "CMSManhattan/Llama3Ternary_2b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/CMSManhattan/Llama3Ternary_2b:Q4_K_M

Ollama
How to use CMSManhattan/Llama3Ternary_2b with Ollama:
```
ollama run hf.co/CMSManhattan/Llama3Ternary_2b:Q4_K_M
```
Unsloth Desktop

How to use CMSManhattan/Llama3Ternary_2b with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf CMSManhattan/Llama3Ternary_2b:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "CMSManhattan/Llama3Ternary_2b:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use CMSManhattan/Llama3Ternary_2b with Docker Model Runner:
```
docker model run hf.co/CMSManhattan/Llama3Ternary_2b:Q4_K_M
```

Lemonade

How to use CMSManhattan/Llama3Ternary_2b with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull CMSManhattan/Llama3Ternary_2b:Q4_K_M

Run and chat with the model

lemonade run user.Llama3Ternary_2b-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use CMSManhattan/Llama3Ternary_2b with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf CMSManhattan/Llama3Ternary_2b:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default CMSManhattan/Llama3Ternary_2b:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use CMSManhattan/Llama3Ternary_2b with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf CMSManhattan/Llama3Ternary_2b:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "CMSManhattan/Llama3Ternary_2b:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

LLama3 JiRack Ternary 2B

JiRack Ternary 2B is a natively trained ternary (1.58-bit) pre-trained language model with built-in tool calling, created by Konstantin Grabko.

The tokenizer and vocabulary are derived from Meta Llama 3 and are subject to the Meta Llama 3 Community License.

Status: Proprietary. The model weights are not publicly available. Access is provided only through a written agreement with the licensor. See LICENSE.

Overview

Who we are: NVIDIA Inception Program Member
Architecture: Decoder-only transformer, natively trained with ternary weights (1.58-bit)
Tokenizer: CMSManhattan/LlamaRoboticsTokenizer and Meta Llama 3 compatible without resize model due to using reserved spaces in tokenizer from llama3
GGUF: These current GGUF files are built using the BitNet fork on llama.cpp.
Capabilities: Text generation, built-in tool calling
Training: Pre-training from scratch with ternary quantization
This model had a ternary pretrain and is fully ternary, making it suitable for fine-tuning with a large capacity during pretraining. Also, the weights were trained entirely in ternary mode Including both pre-training and SFT phases. The model supports tool calls and is highly effective for mobile and edge devices acting as a routing model, which can save a significant amount of money. It's also more efficient than the new Jev model because you can run it locally in a fully ternary state.This model was trained with Llama, RobotiX, and a routing tokenizer.
https://huggingface.co/CMSManhattan/LlamaRoboticsTokenizer

Live chat with the LLama3 JiRack Ternary 2B

chat with JiRack-Llama3-Ternary-2B-TQ2_0.gguf https://huggingface.co/spaces/CMSManhattan/Llama3Ternary_2b

📢 Private Enterprise Models - comming soon

Follow the JiRack page for access to private JiRack Ternary and Full Precision models (2B to 72B parameters).
Full Ternary Pipeline: Trained entirely in ternary mode across pre-training, SFT, and fine-tuning.
Edge & Mobile Ready: Extremely cost-efficient routing models with native tool-calling support.
GGUF Quantizations: Pre-built GGUF binaries using the BitNet fork on llama.cpp.

📊 Benchmark & Evaluation Stage

The JiRack model series (2B to 72B) is currently undergoing comprehensive benchmark evaluations across reasoning, tool-calling accuracy, and local edge efficiency.

Highlights

Architecture: Full Ternary Pre-train + Ternary SFT (Llama + RobotiX + Routing Tokenizer)
Quantization/Runtime: GGUF binaries compiled via llama.cpp (BitNet fork)
Target Use Cases: On-device local execution, low-latency routing, enterprise tool-calling

📬 Follow the JiRack page for private model access, benchmark leaderboards, and enterprise licensing details.

Tool-use results (internal test, 117 cases)

Precision 2B

Category	SFT	Patch v1	Patch v2	Patch v3
single	36/38	25/38	16/38	34/38
parallel	17/19	15/19	11/19	18/19
follow-up	18/20	19/20	19/20	18/20
no-tool	11/25	25/25	25/25	25/25
clarify	6/15	14/15	15/15	14/15
Total	88 (75%)	98 (84%)	86 (74%)	109 (93%)

Llama3 Ternary 2B

Category	SFT	Patch v2	Patch v3 (loss weight)	Patch v4
single	17/38	6/38	7/38	21/38
parallel	6/19	3/19	2/19	4/19
follow-up	15/20	14/20	11/20	11/20
no-tool	3/25	25/25	25/25	23/25
clarify	0/15	10/15	11/15	4/15
Total	41 (35%)	58 (50%)	56 (48%)	63 (54%)

MIT JiRack with Ternary Achiterure for QAT

JiRackUltra_1b is about 2m downloads last month
https://huggingface.co/CMSManhattan/JiRackUltra_1b

Demo

A public demonstration is available as a Hugging Face Space: [link to Space]

The demo is provided for evaluation only. Users interact through the demo interface and do not receive model weights, checkpoints, or API access.

Access and Licensing

This model is proprietary. You may not download, redistribute, host, or serve the weights, including full-precision, quantized, or distilled versions, without a written agreement with the licensor.

To discuss commercial licensing, embedding, hosted deployment, or partnership, contact:

Konstantin Grabko [contact email]

See LICENSE for the full terms.

Usage

Usage instructions are available to licensed partners under their agreement.

# Example usage is shown in the demo Space.
# Model weights are not publicly distributed.

JiRack Codding Agent IDE

It is Agent Coding IDE for JiRack Models to run via Ollama on home PC
It good choose for Agent Coding IDE such as Cursor , Windsurf IDE or Devin IDE etc but more safe that ask you to apply changes and review.
Web site https://www.jirack.com
Eclise Plugin site https://marketplace.eclipse.org/content/jirack-coding-agent
Jetbrain Plugin site https://plugins.jetbrains.com/plugin/35073-jirack-agent
VS Code Plugin site https://marketplace.visualstudio.com/items?itemName=JiRackautomationCodingAgentIDE.jirack-agent

JiRack Codding Agent IDE files

Eclise Final release version https://huggingface.co/CMSManhattan/JiRackDeltaNet_27b/resolve/main/jirack_ide_final.zip
Microsoft VS Code plugin https://huggingface.co/spaces/CMSManhattan/README/blob/main/jirack-agent-0.9.0.vsix
Jetbrain plugin https://huggingface.co/spaces/CMSManhattan/README/resolve/main/jirack-jetbrains-0.11.0.zip
JiRack DeskTop UI https://huggingface.co/spaces/CMSManhattan/README/resolve/main/windows_v11.zip

Attribution

Any permitted use must include:

JiRack Ternary 2B by Konstantin Grabko
Built with Llama

Third-Party Notices

The Meta Llama 3 Community License is available at https://llama.com/llama3/license/

Disclaimer

The model is provided "AS IS," without warranty of any kind. The licensor is not liable for any damages arising from its use.

Copyright

Downloads last month: 717

Safetensors

Model size

2B params

Tensor type

F16

Model tree for CMSManhattan/Llama3Ternary_2b

Quantizations

1 model